Abstract: A method includes receiving a spoken utterance that includes a plurality of words, and generating, using a neural network-based utterance classifier comprising a stack of multiple Long-Short Term Memory (LSTM) layers, a respective textual representation for each word of the of the plurality of words of the spoken utterance. The neural network-based utterance classifier trained on negative training examples of spoken utterances not directed toward an automated assistant server. The method further including determining, using the respective textual representation generated for each word of the plurality of words of the spoken utterance, that the spoken utterance is one of directed toward the automated assistant server or not directed toward the automated assistant server, and when the spoken utterance is directed toward the automated assistant server, generating instructions that cause the automated assistant server to generate a response to the spoken utterance.

Abstract: A method includes receiving a spoken utterance that includes a plurality of words, and generating, using a neural network-based utterance classifier comprising a stack of multiple Long-Short Term Memory (LSTM) layers, a respective textual representation for each word of the of the plurality of words of the spoken utterance. The neural network-based utterance classifier trained on negative training examples of spoken utterances not directed toward an automated assistant server. The method further including determining, using the respective textual representation generated for each word of the plurality of words of the spoken utterance, that the spoken utterance is one of directed toward the automated assistant server or not directed toward the automated assistant server, and when the spoken utterance is directed toward the automated assistant server, generating instructions that cause the automated assistant server to generate a response to the spoken utterance.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for classification using neural networks. One method includes receiving audio data corresponding to an utterance. Obtaining a transcription of the utterance. Generating a representation of the audio data. Generating a representation of the transcription of the utterance. Providing (i) the representation of the audio data and (ii) the representation of the transcription of the utterance to a classifier that, based on a given representation of the audio data and a given representation of the transcription of the utterance, is trained to output an indication of whether the utterance associated with the given representation is likely directed to an automated assistance or is likely not directed to an automated assistant.

Abstract: A method includes receiving a spoken utterance that includes a plurality of words, and generating, using a neural network-based utterance classifier comprising a stack of multiple Long-Short Term Memory (LSTM) layers, a respective textual representation for each word of the of the plurality of words of the spoken utterance. The neural network-based utterance classifier trained on negative training examples of spoken utterances not directed toward an automated assistant server. The method further including determining, using the respective textual representation generated for each word of the plurality of words of the spoken utterance, that the spoken utterance is one of directed toward the automated assistant server or not directed toward the automated assistant server, and when the spoken utterance is directed toward the automated assistant server, generating instructions that cause the automated assistant server to generate a response to the spoken utterance.

Abstract: A method includes receiving a spoken utterance that includes a plurality of words, and generating, using a neural network-based utterance classifier comprising a stack of multiple Long-Short Term Memory (LSTM) layers, a respective textual representation for each word of the of the plurality of words of the spoken utterance. The neural network-based utterance classifier trained on negative training examples of spoken utterances not directed toward an automated assistant server. The method further including determining, using the respective textual representation generated for each word of the plurality of words of the spoken utterance, that the spoken utterance is one of directed toward the automated assistant server or not directed toward the automated assistant server, and when the spoken utterance is directed toward the automated assistant server, generating instructions that cause the automated assistant server to generate a response to the spoken utterance.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: A method includes receiving a spoken utterance that includes a plurality of words, and generating, using a neural network-based utterance classifier comprising a stack of multiple Long-Short Term Memory (LSTM) layers, a respective textual representation for each word of the of the plurality of words of the spoken utterance. The neural network-based utterance classifier trained on negative training examples of spoken utterances not directed toward an automated assistant server. The method further including determining, using the respective textual representation generated for each word of the plurality of words of the spoken utterance, that the spoken utterance is one of directed toward the automated assistant server or not directed toward the automated assistant server, and when the spoken utterance is directed toward the automated assistant server, generating instructions that cause the automated assistant server to generate a response to the spoken utterance.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for classification using neural networks. One method includes receiving audio data corresponding to an utterance. Obtaining a transcription of the utterance. Generating a representation of the audio data. Generating a representation of the transcription of the utterance. Providing (i) the representation of the audio data and (ii) the representation of the transcription of the utterance to a classifier that, based on a given representation of the audio data and a given representation of the transcription of the utterance, is trained to output an indication of whether the utterance associated with the given representation is likely directed to an automated assistance or is likely not directed to an automated assistant.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for classification using neural networks. One method includes receiving audio data corresponding to an utterance. Obtaining a transcription of the utterance. Generating a representation of the audio data. Generating a representation of the transcription of the utterance. Providing (i) the representation of the audio data and (ii) the representation of the transcription of the utterance to a classifier that, based on a given representation of the audio data and a given representation of the transcription of the utterance, is trained to output an indication of whether the utterance associated with the given representation is likely directed to an automated assistance or is likely not directed to an automated assistant.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media for classification using neural networks. One method includes receiving audio data corresponding to an utterance. Obtaining a transcription of the utterance. Generating a representation of the audio data. Generating a representation of the transcription of the utterance. Providing (i) the representation of the audio data and (ii) the representation of the transcription of the utterance to a classifier that, based on a given representation of the audio data and a given representation of the transcription of the utterance, is trained to output an indication of whether the utterance associated with the given representation is likely directed to an automated assistance or is likely not directed to an automated assistant.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.

Abstract: A method of processing messages executes at a computing device having one or more processors and memory. The memory stores one or more programs configured for execution by the one or more processors. A first message for a user is analyzed for a structured content element. When found, content from a data source distinct from the message is obtained by executing an associated action. A first message display state is formed for the message comprising a message notification and the content. A messaging application user interface, comprising an electronic message list with a plurality of objects, is updated. Responsive to selection of a first object of the plurality of objects, the first object representing the first message, the user is enabled to toggle the first message display state between first and second display states, the second display state differing from the first by providing direct access to the first message.