Abstract: The system receives a voice query at an audio interface and converts the voice query to text. The system can determine pronunciation information during conversion and generate metadata the indicates a pronunciation of one or more words of the query, include phonetic information in the text query, or both. A query includes one or more entities, which may be more accurately identified based on pronunciation. The system searches for information, content, or both among one or more databases based on the generated text query, pronunciation information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, and retrieves the identified information to provide to the user.

Abstract: Systems and methods are described herein for enabling, on a local device, a voice control system that limits the amount of data needed to be transmitted to a remote server. A data structure is built at the local device to support a local speech-to-text model by receiving a query and transmitting, to a remote server over a communication network, a request for a speech-to-text transcription of the query. The transcription is received from the remote server and stored in the data structure at the local device in association with an audio clip of the query. Metadata describing the query is used to train the local speech-to-text model to recognize future instances of the query.

Abstract: Systems and methods are described herein for enabling, on a local device, a voice control system that limits the amount of data needed to be transmitted to a remote server. A data structure to support a local speech-to-text model is built at the local device from stored transcriptions of previous queries and known commands, and associates actions with each transcription. The transcription of the particular query is generated using the local speech-to- text model and is used to identify an associated action to perform.

Abstract: The system identifies one or more entities or content items among a plurality of stored information. The system generates an audio file based on a first text string that represents the entity or content item. Based on the first text string and at least one speech criterion, the system generating, using a speech-to-text module a second text string based on the audio file. The system then compares the text strings and stores the second text string if it is not identical to the first text string. The system generates metadata that includes results from text-speech-text conversions to forecast possible misidentifications when responding to voice queries during search operations. The metadata includes alternative representations of the entity.

Abstract: Insertion of supplemental content into a virtual environment is automated using a machine learning model. The machine learning model is trained to calculate a confidence value that a candidate virtual object fits into a virtual environment based on an input that includes a candidate virtual object, a list of persistent virtual objects, and a list of temporary virtual objects. The machine learning model is trained using the persistent and temporary objects displayed in the current virtual environment until it predicts that a selected virtual object fits into the current virtual environment. The trained machine learning model is then used to select a virtual object comprising supplemental content to be inserted as a new virtual object in the virtual environment.

Abstract: Insertion of supplemental content into a virtual environment is automated using a machine learning model. The machine learning model is trained to calculate a confidence value that a candidate virtual object fits into a virtual environment based on an input that includes a candidate virtual object, a list of persistent virtual objects, and a list of temporary virtual objects. The machine learning model is trained using the persistent and temporary objects displayed in the current virtual environment until it predicts that a selected virtual object fits into the current virtual environment. The trained machine learning model is then used to select a virtual object comprising supplemental content to be inserted as a new virtual object in the virtual environment.

Abstract: Insertion of supplemental content into a virtual environment is automated using a machine learning model. The machine learning model is trained to calculate a confidence value that a candidate virtual object fits into a virtual environment based on an input that includes a candidate virtual object, a list of persistent virtual objects, and a list of temporary virtual objects. The machine learning model is trained using the persistent and temporary objects displayed in the current virtual environment until it predicts that a selected virtual object fits into the current virtual environment. The trained machine learning model is then used to select a virtual object comprising supplemental content to be inserted as a new virtual object in the virtual environment.

Abstract: The system receives a voice query at an audio interface and converts the voice query to text. The system can determine pronunciation information during conversion and generate metadata that indicates a pronunciation of one or more words of the query, include phonetic information in the text query, or both. A query includes one or more entities that may be more accurately identified based on pronunciation. The system searches for information, content, or both among one or more databases based on the generated text query, pronunciation information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, and retrieves the identified information to provide to the user.

Abstract: The system receives a voice query at an audio interface and converts the voice query to text. The system can determine pronunciation information during conversion and generate metadata the indicates a pronunciation of one or more words of the query, include phonetic information in the text query, or both. A query includes one or more entities, which may be more accurately identified based on pronunciation. The system searches for information, content, or both among one or more databases based on the generated text query, pronunciation information, user profile information, search histories or trends, and optionally other information. The system identifies one or more entities or content items that match the text query, and retrieves the identified information to provide to the user.

Abstract: The system identifies one or more entities or content items among a plurality of stored information. The system generates an audio file based on a first text string that represents the entity or content item. Based on the first text string and at least one speech criterion, the system generating, using a speech-to-text module a second text string based on the audio file. The system then compares the text strings and stores the second text string if it is not identical to the first text string. The system generates metadata that includes results from text-speech-text conversions to forecast possible misidentifications when responding to voice queries during search operations.