Abstract: Multi-stage fine-tuning for customizing automatic speech recognition with contextual adapters is performed. Entities to customize automatic speech recognition are received and used in an automatic speech recognition pipeline with a fine-tuned version of a speech recognition machine learning model and a fine-tuned version of a contextual adapter. The speech recognition machine learning model and the contextual adapter are fine-tuned in stages that include an initial stage that fine-tunes the speech recognition machine learning model using training data that includes multiple domains and subsequent tuning stages that freeze the speech recognition machine learning model while tuning the contextual adapter and tune the speech recognition machine learning model again.

Abstract: Techniques for augmenting automated speech recognition neural networks with scalable vocabularies are described. A cluster is selected from a plurality of clusters of similar sounding words based on a score, the score representing a similarity between an embedding of the cluster and an audio embedding of an utterance generated with an automated speech recognition encoder neural network. A bias factor is calculated based on a similarity between an embedding of a word in the selected cluster and the audio embedding. The audio embedding of the utterance is biased by the bias factor.

Abstract: A medical audio summarization service receives a medical conversation and an indication of a user preferred summarization style selected from a plurality of available summarization styles to generate a medical summary that conforms to the user preferred summarization style. A transcript is generated via a medical audio transcription service, and the transcript is used by a natural language processing engine (including a large language model) to generate the medical summary. The large language model is trained to be used to generate medical summaries that conform to respective ones of a plurality of user preferred summarization styles. The large language model is trained using training data comprising previously generated summaries and summary interaction metadata generated from user edits and/or feedback.

Abstract: Bias may be removed from automatic speech recognition model predictions using internal language model estimates. Audio data may be received for speech recognition. The audio data may be processed both through an automatic speech recognition model to produce original word token predictions and masked in different portions of the audio data to produce other word token predictions for the masked audio. A comparison of the original word token predictions and the other word token predictions may provide an estimate of an internal language model for the automatic speech recognition model. This estimate can be used to modify the original word token predictions to remove the lexical bias and produce a speech prediction.

Abstract: Pairs of text collections are obtained. An individual pair comprises (a) a source text collection which includes a first group of text sequences and (b) an annotated analysis result of the source text collection, comprising a second group of text sequences and a set of evidence mappings generated by an evidence mapping model. An evidence mapping indicates, for a particular text sequence of the second group, another text sequence of the first group which provides evidence for the particular text sequence. A quality metric of the model is obtained using an automated evaluation methodology in which a question is generated from the particular text sequence, and an analysis of a pair of answers (including 10 an answer generated using an evidence mapping) to the question is performed. The quality metric is provided via a programmatic interface.

Abstract: A medical audio summarization service receives a medical conversation and an indication of a user preferred summarization style selected from a plurality of available summarization styles to generate a medical summary that conforms to the user preferred summarization style. A transcript is generated via a medical audio transcription service, and the transcript is used by a natural language processing engine (including a large language model) to generate the medical summary. The large language model is trained to be used to generate medical summaries that conform to respective ones of a plurality of user preferred summarization styles. The large language model is trained using training data comprising previously generated summaries and summary interaction metadata generated from user edits and/or feedback.