Abstract: A direct speech-to-speech translation (S2ST) model includes an encoder configured to receive an input speech representation that to an utterance spoken by a source speaker in a first language and encode the input speech representation into a hidden feature representation. The S2ST model also includes an attention module configured to generate a context vector that attends to the hidden representation encoded by the encoder. The S2ST model also includes a decoder configured to receive the context vector generated by the attention module and predict a phoneme representation that corresponds to a translation of the utterance in a second different language. The S2ST model also includes a synthesizer configured to receive the context vector and the phoneme representation and generate a translated synthesized speech representation that corresponds to a translation of the utterance spoken in the different second language.

Abstract: A computing device is described that accepts, a telephone call, from another device, initiated by a caller. Prior to establishing a telephone user interface that receives spoken input from the user and outputs spoken audio from the caller, the computing device executes a call screening service that outputs an audio user interface, to the other device and as part of the telephone call. The audio user interface interrogates the caller for additional information including a purpose of the telephone call, which allows the user to have more context of the telephone call before deciding whether to accept the call or hang up. The computing device outputs a graphical user interface associated with telephone call. The graphical user interface includes an indication of the additional information obtained via the audio user interface that interrogates the caller.

Abstract: A computing device is described that accepts, a telephone call, from another device, initiated by a caller. Prior to establishing a telephone user interface that receives spoken input from the user and outputs spoken audio from the caller, the computing device executes a call screening service that outputs an audio user interface, to the other device and as part of the telephone call. The audio user interface interrogates the caller for additional information including a purpose of the telephone call, which allows the user to have more context of the telephone call before deciding whether to accept the call or hang up. The computing device outputs a graphical user interface associated with telephone call. The graphical user interface includes an indication of the additional information obtained via the audio user interface that interrogates the caller.

Abstract: A method and system is disclosed for speech synthesis of streaming text. At a text-to-speech (“ITS) system, a real-time streaming text string having a starting point and an ending point may be received, and a first sub-string comprising a first portion of the text string received from an initial point to a first trigger point may be accumulated. The initial point is no earlier than the starting point and is prior to the first trigger point, and the first trigger point is no further than the ending point. A punctuation model of the ITS system may be applied to the first sub-string to generate a pre-processed first sub-string comprising the first sub-string with added grammatical punctuation as determined by the punctuation model. TTS synthesis processing may be applied to at least the pre-processed first sub-string to generate first synthesized speech, and audio play out of the first synthesized speech produced.

Abstract: A direct speech-to-speech translation (S2ST) model includes an encoder configured to receive an input speech representation that to an utterance spoken by a source speaker in a first language and encode the input speech representation into a hidden feature representation. The S2ST model also includes an attention module configured to generate a context vector that attends to the hidden representation encoded by the encoder. The S2ST model also includes a decoder configured to receive the context vector generated by the attention module and predict a phoneme representation that corresponds to a translation of the utterance in a second different language. The S2ST model also includes a synthesizer configured to receive the context vector and the phoneme representation and generate a translated synthesized speech representation that corresponds to a translation of the utterance spoken in the different second language.

Abstract: A computing device is described that accepts, a telephone call, from another device, initiated by a caller. Prior to establishing a telephone user interface that receives spoken input from the user and outputs spoken audio from the caller, the computing device executes a call screening service that outputs an audio user interface, to the other device and as part of the telephone call. The audio user interface interrogates the caller for additional information including a purpose of the telephone call, which allows the user to have more context of the telephone call before deciding whether to accept the call or hang up. The computing device outputs a graphical user interface associated with telephone call. The graphical user interface includes an indication of the additional information obtained via the audio user interface that interrogates the caller.