Abstract: A display system varies a size of a field of view area of a display for augmented reality (AR) applications based on at least one of ambient light in the environment and content displayed at the display and varying a brightness level of the field of view area such that the brightness level within the field of view area is inversely proportional to the field of view area. Based on an amount of ambient light detected in the environment of the display system, the display system adjusts the size of the area of the field of view of the display in inverse proportion to the amount of detected ambient light. As the size of the field of view area decreases, the display system increases the brightness level of the display within the field of view such that the brightness level is approximately inversely proportional to the field of view area.

Abstract: A method is implemented in a computing device having an operating system configured to capture screenshot images in response to entries of a pre-defined user input. In the method, an application executes in a current state or mode in which an application screen is shown on the display device, detects an entry of the pre-defined user input by a user, and identifies one or more features that (i) are supported by the application, (ii) are at least partially determined by the current state or mode, and (iii) can use at least one type of information displayed by the application screen to perform respective function(s). The application generates a selectable link for each identified feature, and presents, on the display device, the generated selectable link(s), such that a selection by the user of any one of the generated selectable link(s) causes the application to perform the respective set of functions.

Abstract: Implementations are directed to using an automated assistant to initiate an assisted call on behalf of a given user. The assistant can, during the assisted call, receiving a request, from an additional user on the assisted call, for information that is not known to the assistant. In response, the assistant can render a prompt for the information and, while awaiting responsive input from the given user, continue the assisted call using already resolved value(s) for the assisted call. If responsive input is received within a threshold duration of time, synthesized speech, corresponding to the responsive input, is rendered as part of the assisted call. Implementations are additionally or alternatively directed to using the automated assistant to provide, during an ongoing call between a given user and an additional user, output that is based on a value requested by the additional user during the ongoing call.

Abstract: Mitigating the reality gap through training and utilization of at least one difference model. The difference model can be utilized to generate, for each of a plurality of instances of simulated state data generated by a robotic simulator, a corresponding instance of modified simulated state data. The difference model is trained so that a generated modified instance of simulated state data is closer to “real world data” than is a corresponding initial instance of simulated state data. Accordingly, the difference model can be utilized to mitigate the reality gap through modification of initially generated simulated state data, to make it more accurately reflect what would occur in a real environment. Moreover, the difference representation from the difference model can be used as input to the control policy to adapt the control learned from simulator to the real environment.

Abstract: Systems and methods for determining whether to combine responses from multiple automated assistants. An automated assistant may be invoked by a user utterance, followed by a query, which is provided to a plurality of automated assistants. A first response is received from a first automated assistant and a second response is received from a second automated assistant. Based on similarity between the responses, a primary automated assistant determines whether to combine the responses into a combined response. Once the combined response has been generated, one or more actions are performed in response to the combined response.

Abstract: Mechanisms for removing objectionable and/or inappropriate content from media content items are provided. In some embodiments, the method comprises: receiving a first media content item and a dictionary, wherein the first media content item includes an audio component and a video component; identifying a plurality of scenes and a plurality of scene breaks associated with the first media content item; transcribing the audio component of the first media content item to produce transcribed audio; comparing the transcribed audio to entries in the dictionary and storing matches between the transcribed audio and the entries; and generating a second media content item by removing at least a portion of at least one of the audio component and the video component based on the matches.

Abstract: The present disclosure is generally related to a data processing system to selectively invoke applications for execution. A data processing system can receive an input audio signal and can parse the input audio signal to identify a command. The data processing system can identify a first functionality of a first digital assistant application hosted on the data processing system in the vehicle and a second functionality of a second digital assistant application accessible via a client device. The data processing system can determine that one of the first functionality or the second functionality supports the command. The data processing system can select one of the first digital assistant application or the second digital assistant application based on the determination. The data processing system invoke one of the first digital assistant application or the second digital assistant application based on the selection.

Abstract: A head-mounted device may present a first user interface element and a second user interface element within a user interface that is included in a display of the head-mounted device. The head-mounted device may, in response to movement of the head-mounted device, move the first user interface element and the second user interface element within the display, the movement of the first user interface element decreasing a boundary distance below a threshold distance, the boundary distance being based on a distance between the first user interface element and a boundary associated with the user interface. The head-mounted device may decrease a size of the first user interface element while the boundary distance is decreasing and is below the threshold distance.

Abstract: The present disclosure provides systems and methods that generating health diagnostic information from an audio recording. A computing system can include a machine-learned health model comprising that includes a sound model trained to receive data descriptive of a patient audio recording and output sound description data. The computing system can include a diagnostic model trained to receive the sound description data and output a diagnostic score. The computing system can include at least one tangible, non-transitory computer-readable medium that stores instructions that, when executed, cause the processor to perform operations. The operations can include obtaining the patient audio recording; inputting data descriptive of the patient audio recording into the sound model; receiving, as an output of the sound model, the sound description data; inputting the sound description data into the diagnostic model; and receiving, as an output of the diagnostic model, the diagnostic score.

Abstract: Techniques are described herein for automatically synthesizing programs that include one or more functions in a spreadsheet programming language. A method includes: receiving a first example including input provided in a first cell in a spreadsheet; automatically synthesizing a plurality of candidate programs including a first set of candidate programs consistent with the first example, wherein each candidate program in the first set of candidate programs comprises at least one function in a spreadsheet programming language and, when the candidate program is executed, the candidate program generates output that matches the first example; ranking the plurality of candidate programs; and storing a highest-ranked program of the plurality of candidate programs in association with the first cell in the spreadsheet.

Abstract: Implementations set forth herein allow a user to access a first application in a foreground of a graphical interface, and simultaneously employ an automated assistant to respond to notifications arising from a second application. The user can provide an input, such as a spoken utterance, while viewing the first application in the foreground in order to respond to notifications from the second application without performing certain intervening steps that can arise under certain circumstances. Such intervening steps can include providing a user confirmation, which can be bypassed, and/or time-limited according to a timer, which can be displayed in response to the user providing a responsive input directed at the notification. A period for the timer can be set according to one or more characteristics that are associated with the notification, the user, and/or any other information that can be associated with the user receiving the notification.

Abstract: A method for performing cyber-security analysis includes generating a semantic graph in which each object is represented as a node, and each event associated with an object is represented as an edge. A cyber-threat related alert, with an associated alert type, is received from a source. A first object from the plurality of objects is modified based on the alert. A plurality of threat scores, each associated with an object, are calculated, substantially concurrently, based on the alert type. Subsequently, a plurality of modified threat scores are determined for each object, based on: (1) the threat score for that object, (2) a connectivity of that object to each of the remaining objects within the semantic graph; and (3) the threat score for each remaining object from the plurality of objects. A subgraph of the semantic graph is identified based on normalized versions of the modified threat scores.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing content items based on a location of a user's attention in a map interface. In one aspect, a method includes receiving data specifying one or more map attention spots in a viewport of a map interface presented at a user device. Each map attention spot is a location on a map of the map interface corresponding to a specified amount of user activity. Content items are identified for presentation with the map. For each content item and map attention spot, a distance between a presentation location for the content item on the map and a location of the identified map attention spot on the map is determined. A rank score for the content item based at least on a respective content item score for the content item and each determined distance for the content item.

Abstract: A system includes an optical reflector to reflect the light, the optical reflector having a rotor, a first stator, and a second stator. The system further includes a controller in communication with the optical reflector. The controller is to drive the optical reflector by applying a first actuation voltage to the first stator, and a second actuation voltage to the second stator. Further, the controller is to apply an excitation voltage to the first stator. Furthermore, the controller is to determine a relationship between a first capacitance between the rotor and the first stator, and a second capacitance between the rotor and the second stator. Based on the relationship, the controller is to determine a position attribute of the optical reflector.

Abstract: A server system obtains a transcription of content from a video and applies a machine learning resource to identify entities which are most likely to be searched for by a user viewing the video, based on the transcription of the content. The server system generates entity cards for each of the entities, each of the entity cards including descriptive content relating to a respective entity among the entities. The server system provides a user interface, to be displayed on a respective display of one or more user computing devices, for playing the video on a first portion of the user interface. When the video is played and a first entity among the entities is mentioned in the video, the user interface displays a first entity card on a second portion of the user interface, the first entity card including descriptive content relating to the first entity.

Abstract: Implementations described herein relate to providing suggestions, via a display modality, for completing a spoken utterance for an automated assistant, in order to reduce a frequency and/or a length of time that the user will participate in a current and/or subsequent dialog session with the automated assistant. A user request can be compiled from content of an ongoing spoken utterance and content of any selected suggestion elements. When a currently compiled portion of the user request (from content of a selected suggestion(s) and an incomplete spoken utterance) is capable of being performed via the automated assistant, any actions corresponding to the currently compiled portion of the user request can be performed via the automated assistant. Furthermore, any further content resulting from performance of the actions, along with any discernible context, can be used for providing further suggestions.

Abstract: Interface and mode selection for digital action execution is provided. For example, a system loads a script library embedded in an electronic resource. The system determines a historic level of engagement between a client computing device and one or more digital assistants. The system selects, based on a first property of the client computing device and the historic level of engagement, a type of digital interface. The system generates, based on the type of digital interface, a digital interface with the call-to-action. The system determines, responsive to an instruction to execute the call-to-action, a mode of execution. The system selects a digital assistant and a second client device to execute the call-to-action. The system transmits the call-to-action to the second client device for execution.

Abstract: Methods and apparatus related to determining non-textual reply content for a reply to an electronic communication and providing the non-textual reply content for inclusion in the reply. Some of those implementations are directed to determining, based on an electronic communication sent to a user, one or more electronic documents that are responsive to the electronic communication, and providing one or more of those electronic documents for inclusion in a reply by the user to the electronic communication. For example, the electronic documents may be automatically attached to the reply and/or link(s) to the electronic documents automatically provided in the reply.