Speech denoising with re-synthesis

Abstract

When performing speech denoising, the goal of and algorithm is to remove the unwanted frequency, known as noise, while preserving the feature vectors of the signal. In this text, we present an algorithm for speech denoising that removes the noise by converting the speech signal to text and then regenerating the speech signal from the text. We also aim to perform voice adaptation and retention of the emotional components (prosodic parameters) of the original speech in order to preserve the speech feature vectors. In order to do this, we studied the techniques and performance of some automatic speech recognition (ASR) and text-to-speech (TTS) systems and chose the architecture that is best suited for our purpose. Based on the results, we decided to use Google Cloud Speech API for ASR and hybrid speech synthesis for this project. Keywords: speech, noise, ASR, TTS, automatic speech recognition, text-to-speech, hybrid speech synthesis.