Abstract: A technique sends a first prompt to a language model that specifies selector information. The selector information provides a summary of a group of functions that are capable of being invoked. The language model responds by choosing one or more functions from the group of functions. The technique then sends a second prompt to the language model that specifies more detailed information regarding just the function(s) that have been identified by the language model. The language model responds by providing invocation information for each of the functions, such as properly formatted API messages. The technique then invokes the function(s) based on the invocation information. The technique reduces the size of each prompt sent to the language model, which makes efficient use of resources and improves the quality of the language model's output results.

Abstract: A technique partitions a user's original query into plural smaller component queries, each of which has a common part and an instance-specific part. The technique distributes the component queries to plural processor instances of a processor. The plural processor instances transform the respective component queries into query-component responses by acting in parallel, independent of each other. The technique generates a final response based on the query-component responses, e.g., by assembling the component-query responses into the final response. The technique reduces latency because the processor instances work on parts of the user's original query at the same time, rather than as a single stream of consecutive tokens. The plural processor instances have access to a shared cache memory, and utilize relevant data that has been computed in response to previous queries.

Abstract: A technique partitions a user's original query into plural smaller component queries, each of which has a common part and an instance-specific part. The technique distributes the component queries to plural processor instances of a processor. The plural processor instances transform the respective component queries into query-component responses by acting in parallel, independent of each other. The technique generates a final response based on the query-component responses, e.g., by assembling the component-query responses into the final response. The technique reduces latency because the processor instances work on parts of the user's original query at the same time, rather than as a single stream of consecutive tokens. The plural processor instances have access to a shared cache memory, and utilize relevant data that has been computed in response to previous queries.

Abstract: Systems and methods are provided for smart audio segmentation using look-ahead based acousto-linguistic features. For example, systems and methods are provided for obtaining audio, processing the audio, identifying a potential segmentation boundary within the audio, and determining whether to generate a segment break at the potential segmentation boundary. One or more look-ahead words occurring after the potential segmentation boundary are identified, wherein an acoustic segmentation score and a language segmentation score associated with the potential segmentation boundary and the one or more look-ahead words are generated. Systems then either refrain from generating a segment break at the potential segmentation boundary or generate the segment break at the potential segmentation boundary based on the acoustic and/or language segmentation score at least meeting or exceeding a segmentation score threshold.

Abstract: Techniques are provided for early processing of a part of a user input to produce a response to the entire or final user input. While the user input is being received, a partial user input, which is a part of the final user input, is processed to produce a response. The response is a candidate response for the final user input. After the final user input is received, and if the partial user input is determined to match or be equivalent to the final user input, the first response, which is already available, is provided to one or more output devices for presentation. If the final user input is determined to differ from the partial user input, the final user input is processed to produce a second response to the final user input, and the second response is provided for presentation. In some instances, multiple partial user inputs are received and processed.

Abstract: A technique for interacting with a machine-trained language model uses dynamic prompt management. The technique includes: receiving an input query; accessing a state data store that provides candidate context information; partitioning the candidate context information into plural parts; selecting targeted context information from the candidate context information by determining a semantic relevance of the input query to each of the plural parts by performing vector-based analysis; creating prompt information that includes the input query and the targeted context information; submitting the prompt information to the machine-trained language model; and receiving a response from the machine-trained language model based on the prompt information. The technique has the overall effect of reducing the number of content units submitted to the language model, which, in turn, reduces the amount of resources and time required by the language model to process the input query.

Abstract: A technique for interacting with a machine-trained language model uses dynamic prompt management. The technique includes: receiving an input query and creating prompt information that expresses the input query and targeted context information. The targeted context information is selected from candidate context information. Further, a part of the prompt information is formed by compressing source information by reducing a number of content units in the source information (where the source information includes the input query and/or the candidate context information). The method further includes submitting the prompt information to the machine-trained language model, and receiving a response from the machine-trained language model based on the prompt information. The technique has the overall effect of reducing the number of content units submitted to the language model, which, in turn, reduces the amount of resources and time required by the language model to process the input query.

Abstract: Some disclosed embodiments are directed to obtaining a decoded audio data including a spoken language utterance recognized in audio data and identifying a disfluency in the decoded audio data. Upon determining that correcting the disfluency would improve a readability score of the decoded audio data, the system generates a particular correction to correct the disfluency and applies the particular correction to the decoded audio data. Then, an updated decoded audio data is generated which reflects the particular correction. The updated decoded audio data has improved readability over the decoded audio data.

Abstract: Systems are configured to obtain streaming audio data comprising language utterances, continuously decode the streaming audio data in order to generate decoded streaming audio data and determine whether a linguistic boundary exists within an initial segment of decoded streaming audio data. When a linguistic boundary is determined to exist, the systems apply a punctuation at the linguistic boundary and output a first portion of the initial segment of the streaming audio data ending at the linguistic boundary while refraining from outputting a second portion of the initial segment which is located temporally subsequent to the first portion of the initial segment. Systems are also configured to delay the output until predetermined punctuation validation processes have been performed.

Abstract: A data processing system implements obtaining a first textual content, segmenting the first textual content into a plurality of first segments, and providing each segment of the plurality of first segments to a first natural language processing (NLP) model to obtain a set of first readability scores for the plurality of first segments. The first NLP model is configured to analyze a textual input and to output a readability score representing a measurement of readability of the textual input. The system further implements aggregating the set of first segment readability scores to determine a first readability score for the first textual content, and perform at least one of causing the first readability score to be presented to a user or performing one or more actions on the first textual content based on the readability score.

Abstract: Systems generate segments of spoken language utterances based on different sets of segmentation boundaries. The systems are also configured to generate one or more formatted segments by assigning a punctuation tags at segmentation boundaries and to generate one or more final sentences from the one or more segments.

Abstract: A data processing system implements obtaining a first textual content, segmenting the first textual content into a plurality of first segments, and providing each segment of the plurality of first segments to a first natural language processing (NLP) model to obtain a set of first readability scores for the plurality of first segments. The first NLP model is configured to analyze a textual input and to output a readability score representing a measurement of readability of the textual input. The system further implements aggregating the set of first segment readability scores to determine a first readability score for the first textual content, and perform at least one of causing the first readability score to be presented to a user or performing one or more actions on the first textual content based on the readability score.

Abstract: Techniques are provided for early processing of a part of a user input to produce a response to the entire or final user input. While the user input is being received, a partial user input, which is a part of the final user input, is processed to produce a response. The response is a candidate response for the final user input. After the final user input is received, and if the partial user input is determined to match or be equivalent to the final user input, the first response, which is already available, is provided to one or more output devices for presentation. If the final user input is determined to differ from the partial user input, the final user input is processed to produce a second response to the final user input, and the second response is provided for presentation. In some instances, multiple partial user inputs are received and processed.