Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request associated with a task from a user device; submitting the request to each of a plurality of distinct data providers; receiving a plurality of suggested dialog responses from two or more of the data providers; scoring the one or more suggested dialog responses based on one or more scoring factors; determining a particular dialog response to provide to the user based on the scoring; and providing the determined dialog response to the user device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request associated with a task from a user device; submitting the request to each of a plurality of distinct data providers; receiving a plurality of suggested dialog responses from two or more of the data providers; scoring the one or more suggested dialog responses based on one or more scoring factors; determining a particular dialog response to provide to the user based on the scoring; and providing the determined dialog response to the user device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.

Abstract: The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may receive audio data corresponding to an utterance spoken by a user, and may obtain structured data representative of the received audio data. The communications device may provide structured data to the executed application through a programmatic interface, and the executed application may perform the one or more operations in accordance with the structured data. The communications device may generate data indicative of an output of the one or more operations performed by the executed application, and may present at least a portion of the generated output data to a user through a corresponding interface.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request to perform a task from a user device; obtaining a dialog corresponding to the task; providing multiple protocol buffers to the user device; receiving a voice input and one or more annotated protocol buffers from the user device, the one or more annotated protocol buffers identifying corresponding non-verbal responses to content in the protocol buffers; and using the received protocol buffers to update a state of the dialog and to interpret the voice input.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for asynchronous execution of client requests. In some implementations, data indicating a user request to a digital assistant is received. An action corresponding to the user request is determined. It is determined that the action is classified as an action to be performed asynchronously to the user request. A confirmation message is sent, for output, and the action is performed asynchronously to the user request.

Abstract: The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may obtaining component data identifying a plurality of components of a voice-user interface from a computing system maintained by a voice-service provider, and may execute an application linked to a corresponding one of the components of the voice-user interface. The communications device may generate the native GUI based on an output of the executed application, and may generate an interface element representative of the corresponding one of the components of the voice-user interface. The communications device may present the generated interface element within the native GUI, which may embed the corresponding component of the voice-user interface into the native GUI.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request associated with a task from a user device; submitting the request to each of a plurality of distinct data providers; receiving a plurality of suggested dialog responses from two or more of the data providers; scoring the one or more suggested dialog responses based on one or more scoring factors; determining a particular dialog response to provide to the user based on the scoring; and providing the determined dialog response to the user device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request associated with a task from a user device; submitting the request to each of a plurality of distinct data providers; receiving a plurality of suggested dialog responses from two or more of the data providers; scoring the one or more suggested dialog responses based on one or more scoring factors; determining a particular dialog response to provide to the user based on the scoring; and providing the determined dialog response to the user device.

Abstract: The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may obtaining component data identifying a plurality of components of a voice-user interface from a computing system maintained by a voice-service provider, and may execute an application linked to a corresponding one of the components of the voice-user interface. The communications device may generate the native GUI based on an output of the executed application, and may generate an interface element representative of the corresponding one of the components of the voice-user interface. The communications device may present the generated interface element within the native GUI, which may embed the corresponding component of the voice-user interface into the native GUI.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using frames for performing tasks. One of the methods includes receiving a first request to perform a task, the first request comprising user speech identifying the task; generating a frame associated with the task, wherein the frame comprises one or more types of values necessary to perform the task, and wherein each type of value can be satisfied by a respective value; receiving a second request to provide information related to a question, the second request comprising user speech identifying the question; providing information identifying the question to a search engine, and receiving a response identifying one or more terms; determining that at least one term can satisfy a type of value necessary to perform the task; and storing the at least one term in the frame.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for asynchronous execution of client requests. In some implementations, data indicating a user request to a digital assistant is received. An action corresponding to the user request is determined. It is determined that the action is classified as an action to be performed asynchronously to the user request. A confirmation message is sent, for output, and the action is performed asynchronously to the user request.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request to perform a task from a user device; obtaining a dialog corresponding to the task; providing multiple protocol buffers to the user device; receiving a voice input and one or more annotated protocol buffers from the user device, the one or more annotated protocol buffers identifying corresponding non-verbal responses to content in the protocol buffers; and using the received protocol buffers to update a state of the dialog and to interpret the voice input.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for asynchronous execution of client requests. In some implementations, data indicating a user request to a digital assistant is received. An action corresponding to the user request is determined. It is determined that the action is classified as an action to be performed asynchronously to the user request. A confirmation message is sent, for output, and the action is performed asynchronously to the user request.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request to perform a task from a user device; obtaining a dialog corresponding to the task; providing multiple protocol buffers to the user device; receiving a voice input and one or more annotated protocol buffers from the user device, the one or more annotated protocol buffers identifying corresponding non-verbal responses to content in the protocol buffers; and using the received protocol buffers to update a state of the dialog and to interpret the voice input.

Abstract: Methods, systems, and apparatus, including computer programs stored on a computer-readable storage medium, for asynchronous execution of client requests. In some implementations, data indicating a user request to a digital assistant is received. An action corresponding to the user request is determined. It is determined that the action is classified as an action to be performed asynchronously to the user request. A confirmation message is sent, for output, and the action is performed asynchronously to the user request.

Abstract: The disclosed embodiments include computerized methods, systems, and devices, including computer programs encoded on a computer storage medium, for integrating voice-based interaction and control into a native graphical user interface (GUI) of an executed application. For example, a communications device may obtaining component data identifying a plurality of components of a voice-user interface from a computing system maintained by a voice-service provider, and may execute an application linked to a corresponding one of the components of the voice-user interface. The communications device may generate the native GUI based on an output of the executed application, and may generate an interface element representative of the corresponding one of the components of the voice-user interface. The communications device may present the generated interface element within the native GUI, which may embed the corresponding component of the voice-user interface into the native GUI.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for managing dialogs. In one aspect, a method includes receiving a request associated with a task from a user device; submitting the request to each of a plurality of distinct data providers; receiving a plurality of suggested dialog responses from two or more of the data providers; scoring the one or more suggested dialog responses based on one or more scoring factors; determining a particular dialog response to provide to the user based on the scoring; and providing the determined dialog response to the user device.