Abstract: A data processing system implements causing a user interface of a client device to render pre-produced audio content; receiving via a an audio receiver of the client device a user query raised during a portion of the pre-produced audio content being rendered on the user interface; causing the user interface to pause the pre-produced audio content in response to receiving the user query; automatically generating a text transcript of the user query using speech recognition; calling a generative model to generate a contextual text answer to the user query based on the text transcript, the portion of the pre-produced audio content, and at least rendered portions of the pre-produced audio content; converting the contextual text answer into an audio answer using voice synthesis; causing the user interface to render the audio answer on the client device; and causing the user interface to resume the pre-produced audio content.

Abstract: Techniques for causing an LLM to generate semantically related phrase variations for an identified phrase are disclosed. An LLM that is generally pre-trained on an arbitrary corpus of language training data is accessed. Seed data is fed as input to the LLM. The seed data includes multiple phrases that are semantically related and that describe a command. When any one of the phrases is received as utterance input, the utterance input triggers execution of the command. The LLM generates multiple phrase variations based on the phrases, where each phrase variation is semantically related to the other phrases. When any one of the phrase variations is received as new utterance input, the new utterance input also triggers execution of the command. The phrases and phrase variations are then stored together in a data storage.

Abstract: Techniques for causing an LLM to generate semantically related phrase variations for an identified phrase are disclosed. An LLM that is generally pre-trained on an arbitrary corpus of language training data is accessed. Seed data is fed as input to the LLM. The seed data includes multiple phrases that are semantically related and that describe a command. When any one of the phrases is received as utterance input, the utterance input triggers execution of the command. The LLM generates multiple phrase variations based on the phrases, where each phrase variation is semantically related to the other phrases. When any one of the phrase variations is received as new utterance input, the new utterance input also triggers execution of the command. The phrases and phrase variations are then stored together in a data storage.

Abstract: Techniques for performing contextualized intent and slot extraction using a large language model (LLM) are disclosed. The LLM is generally pre-trained on an arbitrary corpus of language training data. A prompt is provided to the LLM. This prompt includes a limited number of prompt phrases. The prompt phrases share a semantic relationship with one another. A spoken utterance is recorded and then converted to text, resulting in generation of a transcription. The transcription is provided to the LLM. The LLM extracts, from the transcription, an extracted intent and an extracted slot. The extracted intent is determined to be related to a prompt-described intent that was included in the prompt. The prompt is supplemented by adding the extracted intent and the extracted slot to the prompt, resulting in the extracted intent being identified as sharing the semantic relationship with the other prompt phrases in the prompt.

Abstract: Techniques for performing contextualized intent and slot extraction using a large language model (LLM) are disclosed. The LLM is generally pre-trained on an arbitrary corpus of language training data. A prompt is provided to the LLM. This prompt includes a limited number of prompt phrases. The prompt phrases share a semantic relationship with one another. A spoken utterance is recorded and then converted to text, resulting in generation of a transcription. The transcription is provided to the LLM. The LLM extracts, from the transcription, an extracted intent and an extracted slot. The extracted intent is determined to be related to a prompt-described intent that was included in the prompt. The prompt is supplemented by adding the extracted intent and the extracted slot to the prompt, resulting in the extracted intent being identified as sharing the semantic relationship with the other prompt phrases in the prompt.

Abstract: Techniques for facilitating voice based dictation of programming code within a context of an IDE are disclosed. Programming code is fed to a text-to-speech (TTS) model. The TTS model generates an audio file associated with the code. The audio file is then fed to a speech-to-text (STT) model. The STT model generates a transcription file associated with the audio file. Each respective line of code included in the programming code is mapped to a corresponding line of code included in the transcription file, resulting in generation of a list of phrase pairings. These phrase pairings represent relationships between actual code and how that actual code sounds if read out loud. An LLM then ingests the list of phrase pairings. The LLM identifies correlations between programming vocabulary that has specific meaning within the context of the IDE and how that programming vocabulary sounds if read out loud.

Abstract: A large language model is used to detect the intent of a developer-spoken utterance. The large language model is pre-trained on natural language text and source code. A prompt to the large language model is augmented with a few-shot examples of pairs of an utterance and intent in order to guide the model to predict an intent for a given utterance. The few-shot examples are extracted from known utterance-intent pairs. The pairs closest to the developer-spoken utterance are incorporated into the prompt as the few-shot examples.

Abstract: Techniques for facilitating voice based dictation of programming code within a context of an IDE are disclosed. Programming code is fed to a text-to-speech (TTS) model. The TTS model generates an audio file associated with the code. The audio file is then fed to a speech-to-text (STT) model. The STT model generates a transcription file associated with the audio file. Each respective line of code included in the programming code is mapped to a corresponding line of code included in the transcription file, resulting in generation of a list of phrase pairings. These phrase pairings represent relationships between actual code and how that actual code sounds if read out loud. An LLM then ingests the list of phrase pairings. The LLM identifies correlations between programming vocabulary that has specific meaning within the context of the IDE and how that programming vocabulary sounds if read out loud.

Abstract: Techniques for causing an LLM to generate semantically related phrase variations for an identified phrase are disclosed. An LLM that is generally pre-trained on an arbitrary corpus of language training data is accessed. Seed data is fed as input to the LLM. The seed data includes multiple phrases that are semantically related and that describe a command. When any one of the phrases is received as utterance input, the utterance input triggers execution of the command. The LLM generates multiple phrase variations based on the phrases, where each phrase variation is semantically related to the other phrases. When any one of the phrase variations is received as new utterance input, the new utterance input also triggers execution of the command. The phrases and phrase variations are then stored together in a data storage.

Abstract: Techniques for performing contextualized intent and slot extraction using a large language model (LLM) are disclosed. The LLM is generally pre-trained on an arbitrary corpus of language training data. A prompt is provided to the LLM. This prompt includes a limited number of prompt phrases. The prompt phrases share a semantic relationship with one another. A spoken utterance is recorded and then converted to text, resulting in generation of a transcription. The transcription is provided to the LLM. The LLM extracts, from the transcription, an extracted intent and an extracted slot. The extracted intent is determined to be related to a prompt-described intent that was included in the prompt. The prompt is supplemented by adding the extracted intent and the extracted slot to the prompt, resulting in the extracted intent being identified as sharing the semantic relationship with the other prompt phrases in the prompt.

Abstract: Techniques for facilitating voice based dictation of programming code within a context of an IDE are disclosed. Programming code is fed to a text-to-speech (TTS) model. The TTS model generates an audio file associated with the code. The audio file is then fed to a speech-to-text (STT) model. The STT model generates a transcription file associated with the audio file. Each respective line of code included in the programming code is mapped to a corresponding line of code included in the transcription file, resulting in generation of a list of phrase pairings. These phrase pairings represent relationships between actual code and how that actual code sounds if read out loud. An LLM then ingests the list of phrase pairings. The LLM identifies correlations between programming vocabulary that has specific meaning within the context of the IDE and how that programming vocabulary sounds if read out loud.