Abstract: Processor(s) of a client device of a user can receive a telephone call that is initiated by an additional user, and, in response to receiving the telephone call, identify an entity that is associated with the additional user, and determine, based on the entity that is associated with the additional user, whether to (1) fully automate the telephone call, or (2) partially automate the telephone call. In fully automating the telephone call, the processor(s) can cause a chatbot to engage in a corresponding conversation with the additional user and without prompting the user for any input. In partially automating the telephone call, the processor(s) can cause the chatbot to engage in a corresponding conversation with the additional user but with prompting the user for input(s) via suggestion chip(s). In some implementations, the processor(s) can further determine whether to (3) refrain from automating the telephone call entirely.

Abstract: Implementations are described herein for identifying potentially false information in generative model output by performing entailment evaluation of generative model output. In various implementations, data indicative of a query may be processed to generate generative model output. Textual fragments may be extracted from the generative model output, and a subset of the textual fragments may be classified as being suitable for textual entailment analysis. Textual entailment analysis may be performed on each textual fragment of the subset, including formulating a search query based on the textual fragment, retrieving document(s) responsive to the search query, and processing the textual fragment and the document(s) using entailment machine learning model(s) to generate prediction(s) of whether the at least one document corroborates or contradicts the textual fragment. When natural language (NL) responsive to the query is rendered at a client device, annotation(s) may be rendered to express the prediction(s).

Abstract: Generating expanded responses that guide continuance of a human-to computer dialog that is facilitated by a client device and that is between at least one user and an automated assistant. The expanded responses are generated by the automated assistant in response to user interface input provided by the user via the client device, and are caused to be rendered to the user via the client device, as a response, by the automated assistant, to the user interface input of the user. An expanded response is generated based on at least one entity of interest determined based on the user interface input, and is generated to incorporate content related to one or more additional entities that are related to the entity of interest, but that are not explicitly referenced by the user interface input.

Abstract: Processor(s) of a client device of a user can receive a telephone call that is initiated by an additional user, and, in response to receiving the telephone call, identify an entity that is associated with the additional user, and determine, based on the entity that is associated with the additional user, whether to (1) fully automate the telephone call, or (2) partially automate the telephone call. In fully automating the telephone call, the processor(s) can cause a chatbot to engage in a corresponding conversation with the additional user and without prompting the user for any input. In partially automating the telephone call, the processor(s) can cause the chatbot to engage in a corresponding conversation with the additional user but with prompting the user for input(s) via suggestion chip(s). In some implementations, the processor(s) can further determine whether to (3) refrain from automating the telephone call entirely.

Abstract: The present disclosure provides systems and methods for improving detection and location determination accuracy of abnormalities during a gastroenterological procedure. One example method includes obtaining a video data stream generated by an endoscopic device during a gastroenterological procedure for a patient. The method includes generating a three-dimensional model of at least a portion of an anatomical structure viewed by the endoscopic device based at least in part on the video data stream. The method includes obtaining location data associated with one or more detected abnormalities based on localization data generated from the video data stream of the endoscopic device. The method includes generating a visual presentation of the three-dimensional model and the location data associated with the one or more detected abnormalities; and providing the visual presentation of the three-dimensional model and the location data associated with the detected abnormality for use in diagnosis of the patient.

Abstract: Implementations are provided for automatically mining corpus(es) of electronic video files for video clips that contain spoken utterances that are suitable usage examples to accompany or compliment dictionary definitions. These video clips may then be associated with target n-grams in a searchable database, such as a database underlying an online dictionary. In various implementations, a set of candidate video clips in which a target n-gram is uttered in a target context may be identified from a corpus of electronic video files. For each candidate video clip of the set, pre-existing manual subtitles associated with the candidate video clip may be compared to text generated based on speech recognition processing of an audio portion of the candidate video clip. Based at least in part on the comparing, a measure of suitability as a dictionary usage example may be calculated for the candidate video clip.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for audio-visual speech separation. A method includes: receiving, by a user device, a first indication of one or more first speakers visible in a current view recorded by a camera of the user device, in response, generating a respective isolated speech signal for each of the one or more first speakers that isolates speech of the first speaker in the current view and sending the isolated speech signals for each of the one or more first speakers to a listening device operatively coupled to the user device, receiving, by the user device, a second indication of one or more second speakers visible in the current view recorded by the camera of the user device, and in response generating and sending a respective isolated speech signal for each of the one or more second speakers to the listening device.

Abstract: Generating expanded responses that guide continuance of a human-to computer dialog that is facilitated by a client device and that is between at least one user and an automated assistant. The expanded responses are generated by the automated assistant in response to user interface input provided by the user via the client device, and are caused to be rendered to the user via the client device, as a response, by the automated assistant, to the user interface input of the user. An expanded response is generated based on at least one entity of interest determined based on the user interface input, and is generated to incorporate content related to one or more additional entities that are related to the entity of interest, but that are not explicitly referenced by the user interface input.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for audio-visual speech separation. A method includes: receiving, by a user device, a first indication of one or more first speakers visible in a current view recorded by a camera of the user device, in response, generating a respective isolated speech signal for each of the one or more first speakers that isolates speech of the first speaker in the current view and sending the isolated speech signals for each of the one or more first speakers to a listening device operatively coupled to the user device, receiving, by the user device, a second indication of one or more second speakers visible in the current view recorded by the camera of the user device, and in response generating and sending a respective isolated speech signal for each of the one or more second speakers to the listening device.

Abstract: Systems and methods for media aggregation are disclosed herein. The system includes a media system that can transform media items into one aggregated media item. A synchronization component synchronizes media items with respect to time. The synchronized media items can be analyzed and transformed into an aggregated media item for storage and/or display. In one implementation, the aggregated media item is capable of being displayed in multiple ways to create an enhanced and customizable viewing and/or listening experience.

Abstract: Processor(s) of a client device of a user can receive a telephone call that is initiated by an additional user, and, in response to receiving the telephone call, identify an entity that is associated with the additional user, and determine, based on the entity that is associated with the additional user, whether to (1) fully automate the telephone call, or (2) partially automate the telephone call. In fully automating the telephone call, the processor(s) can cause a chatbot to engage in a corresponding conversation with the additional user and without prompting the user for any input. In partially automating the telephone call, the processor(s) can cause the chatbot to engage in a corresponding conversation with the additional user but with prompting the user for input(s) via suggestion chip(s). In some implementations, the processor(s) can further determine whether to (3) refrain from automating the telephone call entirely.

Abstract: Implementations receive, via a client device, user input to initiate a telephone call with an entity, and, in response to receiving the user input to initiate the telephone call with the entity and prior to initiating the telephone call with the entity: obtain pre-call information that is stored in association with the entity, and cause the pre-call information that is stored in association with the entity to be provided for presentation to the user via the client device. The pre-call information may include any information that would be provided for presentation to a user subsequent to initiation of the telephone call with the entity. Further, implementations determine, based on user consumption of the pre-call information, whether to (1) proceed with initiating the telephone call with the entity, or (2) refrain from initiating the telephone call with the entity, and cause the client device to implement the appropriate action.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to synthetic call status updates. In some implementations, a method includes determining, by a task manager module, that a triggering event has occurred to provide a current status of a user call request. The method may then determine, by the task manager module, the current status of the user call request. A representation of the current status of the user call request is generated. Then, the generated representation of the current status of the user call request is provided to the user.

Abstract: Generating expanded responses that guide continuance of a human-to computer dialog that is facilitated by a client device and that is between at least one user and an automated assistant. The expanded responses are generated by the automated assistant in response to user interface input provided by the user via the client device, and are caused to be rendered to the user via the client device, as a response, by the automated assistant, to the user interface input of the user. An expanded response is generated based on at least one entity of interest determined based on the user interface input, and is generated to incorporate content related to one or more additional entities that are related to the entity of interest, but that are not explicitly referenced by the user interface input.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to synthetic call status updates. In some implementations, a method includes determining, by a task manager module, that a triggering event has occurred to provide a current status of a user call request. The method may then determine, by the task manager module, the current status of the user call request. A representation of the current status of the user call request is generated. Then, the generated representation of the current status of the user call request is provided to the user.

Abstract: The present disclosure is directed towards systems and methods that leverage machine-learned models to decrease the rate at which abnormal sites are missed during a gastroenterological procedure. In particular, the system and methods of the present disclosure can use machine-learning techniques to determine the coverage rate achieved during a gastroenterological procedure. Measuring the coverage rate of the gastroenterological procedure can allow medical professionals to be alerted when the coverage output is deficient and thus allow an additional coverage to be achieved and as a result increase in the detection rate for abnormal sites (e.g., adenoma, polyp, lesion, tumor, etc.) during the gastroenterological procedure.

Abstract: A computing device is described that accepts, a telephone call, from another device, initiated by a caller. Prior to establishing a telephone user interface that receives spoken input from the user and outputs spoken audio from the caller, the computing device executes a call screening service that outputs an audio user interface, to the other device and as part of the telephone call. The audio user interface interrogates the caller for additional information including a purpose of the telephone call, which allows the user to have more context of the telephone call before deciding whether to accept the call or hang up. The computing device outputs a graphical user interface associated with telephone call. The graphical user interface includes an indication of the additional information obtained via the audio user interface that interrogates the caller.

Abstract: Generating expanded responses that guide continuance of a human-to computer dialog that is facilitated by a client device and that is between at least one user and an automated assistant. The expanded responses are generated by the automated assistant in response to user interface input provided by the user via the client device, and are caused to be rendered to the user via the client device, as a response, by the automated assistant, to the user interface input of the user. An expanded response is generated based on at least one entity of interest determined based on the user interface input, and is generated to incorporate content related to one or more additional entities that are related to the entity of interest, but that are not explicitly referenced by the user interface input.

Abstract: A computing device is described that accepts, a telephone call, from another device, initiated by a caller. Prior to establishing a telephone user interface that receives spoken input from the user and outputs spoken audio from the caller, the computing device executes a call screening service that outputs an audio user interface, to the other device and as part of the telephone call. The audio user interface interrogates the caller for additional information including a purpose of the telephone call, which allows the user to have more context of the telephone call before deciding whether to accept the call or hang up. The computing device outputs a graphical user interface associated with telephone call. The graphical user interface includes an indication of the additional information obtained via the audio user interface that interrogates the caller.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving, at a mobile computing device that is associated with a called user, a call from a calling computing device that is associated with a calling user; in response to receiving the call, determining, by the mobile computing device, that data associated with the called user indicates that the called user will not respond to the call; in response to determining that the called user will not respond to the call, inferring, by the mobile computing device, an informational need of the calling user; and automatically providing, from the mobile computing device to the calling computing device, information associated with the called user and that satisfies the inferred informational need of the calling user.