Abstract: Systems and methods of the present disclosure are directed to a computer-implemented method for multimodal search refinement. The method includes obtaining a visual search query from a user comprising one or more query images. The method includes providing a search interface for display to the user, the search interface comprising one or more result images responsive to the one or more query images and an interface element indicative of a request to the user to refine the visual search query. The method includes obtaining, from the user, textual data comprising a refinement to the visual search query. The method includes appending, by the computing system, the textual data to the visual search query to obtain a multimodal search query.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Methods and apparatus related to determining reply content for a reply to an electronic communication. Some implementations are directed generally toward analyzing a corpus of electronic communications to determine relationships between one or more original message features of “original” messages of electronic communications and reply content that is included in “reply” messages of those electronic communications. Some implementations are directed generally toward providing reply text to include in a reply to a communication based on determined relationships between one or more message features of the communication and the reply text.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: Methods and apparatus related to determining reply content for a reply to an electronic communication. Some implementations are directed generally toward analyzing a corpus of electronic communications to determine relationships between one or more original message features of “original” messages of electronic communications and reply content that is included in “reply” messages of those electronic communications. Some implementations are directed generally toward providing reply text to include in a reply to a communication based on determined relationships between one or more message features of the communication and the reply text.

Abstract: Methods and apparatus related to determining reply content for a reply to an electronic communication. Some implementations are directed generally toward analyzing a corpus of electronic communications to determine relationships between one or more original message features of “original” messages of electronic communications and reply content that is included in “reply” messages of those electronic communications. Some implementations are directed generally toward providing reply text to include in a reply to a communication based on determined relationships between one or more message features of the communication and the reply text.

Abstract: Implementations set forth herein relate to an automated assistant that can control a camera according to one or more conditions specified by a user. A condition can be satisfied when, for example, the automated assistant detects a particular environment feature is apparent. In this way, the user can rely on the automated assistant to identify and capture certain moments without necessarily requiring the user to constantly monitor a viewing window of the camera. In some implementations, a condition for the automated assistant to capture media data can be based on application data and/or other contextual data that is associated with the automated assistant. For instance, a relationship between content in a camera viewing window and other content of an application interface can be a condition upon which the automated assistant captures certain media data using a camera.

Abstract: A system may use a machine-learned model to determine whether to classify a sequence of one or more words within a first document that is being edited as a candidate hyperlink based at least in part on context associated with the first document. In response to classifying the sequence of one or more words as the candidate hyperlink, the system may use the machine-learned model and based at least in part on the sequence of one or more words and the context to determine one or more candidate document to be hyperlinked from the sequence of one or more words. In response to receiving an indication of a second document being selected out of the one or more candidate documents, the system may modify the first document to associate the sequence of one or more words with a hyperlink to the second document.

Abstract: An example method includes receiving, by a computing system, an identification of a recipient of an electronic message being composed from a message account associated with a user; predicting, by the computing system and based on text contained in previous electronic messages sent from the message account, text for a body of the electronic message; and outputting, for display, the predicted text for optional selection and insertion into the body of the electronic message.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for generating and applying outgoing communication templates. In various implementations a corpus of outgoing communications sent by a user may be grouped into a plurality of clusters based on one or more attributes of a context of the user. One or more segments of each outgoing communication of a particular cluster may be classified as fixed in response to a determination that a count of occurrences of the one or more segments across the particular cluster satisfies a criterion. One or more remaining segments of each communication of the particular cluster may or may not be classified as transient. Based on sequences of classified segments associated with each communication of the particular cluster, an outgoing communication template may be generated to automatically populate at least a portion of a draft outgoing communication being prepared by the user.

Abstract: Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.

Abstract: Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.

Abstract: A system may use a machine-learned model to determine whether to classify a sequence of one or more words within a first document that is being edited as a candidate hyperlink based at least in part on context associated with the first document. In response to classifying the sequence of one or more words as the candidate hyperlink, the system may use the machine-learned model and based at least in part on the sequence of one or more words and the context to determine one or more candidate document to be hyperlinked from the sequence of one or more words. In response to receiving an indication of a second document being selected out of the one or more candidate documents, the system may modify the first document to associate the sequence of one or more words with a hyperlink to the second document.

Abstract: Methods and apparatus related to determining reply content for a reply to an electronic communication. Some implementations are directed generally toward analyzing a corpus of electronic communications to determine relationships between one or more original message features of “original” messages of electronic communications and reply content that is included in “reply” messages of those electronic communications. Some implementations are directed generally toward providing reply text to include in a reply to a communication based on determined relationships between one or more message features of the communication and the reply text.

Abstract: Methods and apparatus related to determining reply content for a reply to an electronic communication. Some implementations are directed generally toward analyzing a corpus of electronic communications to determine relationships between one or more original message features of “original” messages of electronic communications and reply content that is included in “reply” messages of those electronic communications. Some implementations are directed generally toward providing reply text to include in a reply to a communication based on determined relationships between one or more message features of the communication and the reply text.

Abstract: Systems and methods for classifying messages are provided. Each message in a plurality of messages is classified, thereby independently identifying a message category in a set of message categories for each respective message in the plurality. The plurality of messages is delivered to a plurality of recipients with a designation of the message category of each respective message in the first plurality of messages. A plurality of recipient initiated message interaction events for messages in the first plurality of messages over a predetermined period of time is collected from the plurality of recipients. A message categorization dataset is then constructed from (i) the first plurality of messages, (ii) the designation of the message category of each respective message in the subset of the first plurality of messages, and (iii) the plurality of recipient initiated message interaction events. This message categorization dataset is used to train or evaluate a message classifier.