Abstract: The systems and methods described herein can generate a voice-based interface to increase the accuracy of translations. The voice-based interface can result in fewer input audio signals being transmitted between devices of a network. A method includes: receiving a first input audio signal; generating, based on the first input audio signal, a first translation string in a second language and a second translation string in a first language; determining a first translation score based on a likelihood that the first input audio signal includes an utterance in the first language and a second translation score based on a likelihood that the first input audio signal includes an utterance in the second language; selecting the first translation string based on the first translation score and the second translation score; generating an output signal from the first translation string; and transmitting the output signal to the client device.

Abstract: A system and method provides a route and turn-by-turn directions based on estimates of current and future traffic along the route. A client device may request turn-by-turn directions between an initial and a final location. A server may identify a plurality of routes between the locations. Each route of the plurality of routes may be divided into route segments. For each route segment of a particular route, the server may estimate a travel time. The travel time may be based on estimated vehicle volume data generated from information received from other users vehicle Based on the estimated travel time for each route segment of a particular route, the server may estimate a total travel time for the particular route. The server may repeat this estimate for each of the plurality of routes between the locations and select the route with the lowest estimated travel time. Based on the selected route, the server may generate turn-by-turn directions and transmit the directions to the client device for display.

Abstract: Implementations of the present disclosure include actions of receiving a search query, identifying potential search results responsive to the search query, the potential search results corresponding to digital content stored in one or more computer-readable storage media, determining that the potential search results include user-generated content that is generated using one or more computer-implemented social services, receiving data associated with the user-generated content, the data including one or more scores, determining, based on the one or more scores, that the user-generated content is to be provided as a search result, generating search results, the search results including web-based search results and at least a portion of the user-generated content, and transmitting the search results to a client computing device for display to the searching user.

Abstract: Systems, methods, and apparatus for using a multimodal response in the dynamic generation of client device output that is tailored to a current modality of a client device is disclosed herein. Multimodal client devices can engage in a variety of interactions across the multimodal spectrum including voice only interactions, voice forward interactions, multimodal interactions, visual forward interactions, visual only interactions etc. A multimodal response can include a core message to be rendered for all interaction types as well as one or more modality dependent components to provide a user with additional information.

Abstract: The present disclosure provides projection neural networks and example applications thereof. In particular, the present disclosure provides a number of different architectures for projection neural networks, including two example architectures which can be referred to as: Self-Governing Neural Networks (SGNNs) and Projection Sequence Networks (ProSeqoNets). Each projection neural network can include one or more projection layers that project an input into a different space. For example, each projection layer can use a set of projection functions to project the input into a bit-space, thereby greatly reducing the dimensionality of the input and enabling computation with lower resource usage. As such, the projection neural networks provided herein are highly useful for on-device inference in resource-constrained devices. For example, the provided SGNN and ProSeqoNet architectures are particularly beneficial for on-device inference such as, for example, solving natural language understanding tasks on-device.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating suggested search queries. One method includes receiving, during a search session, a request for a suggested search query; in response to receiving the request for the suggested search query, identifying an entity that is associated with an item of media content; generating a suggested search query based on the identified entity; and providing data that causes the generated suggested search query to be presented in a user interface.

Abstract: Implementations set forth herein relate to intervening notifications provided by an application for mitigating computationally wasteful application launching behavior that is exhibited by some users. A state of a module of a target application can be identified by emulating user inputs previously provided by the user to the target application. In this way, the state of the module can be determined without visibly launching the target application. When the state of the module is determined to satisfy criteria for providing a notification to the user, the application can render a notification for the user. The application can provide intervening notifications for a variety of different target applications in order to reduce a frequency at which the user launches and closes applications to check for variations in target application content.

Abstract: Implementations relate to generating a proficiency measure, and utilizing the proficiency measure to adapt one or more automated assistant functionalities. The generated proficiency measure is for a particular class of automated assistant actions, and is specific to an assistant device and/or is specific to a particular user. A generated proficiency measure for a class can reflect a degree of proficiency, of a user and/or of an assistant device, for that class. Various automated assistant functionalities can be adapted, for a particular class, responsive to determining the proficiency measure satisfies a threshold, or fails to satisfy the threshold (or an alternate threshold). The adaptation(s) can make automated assistant processing more efficient and/or improve (e.g., shorten the duration of) user-assistant interaction(s).

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, relating to synthetic call status updates. In some implementations, a method includes determining, by a task manager module, that a triggering event has occurred to provide a current status of a user call request. The method may then determine, by the task manager module, the current status of the user call request. A representation of the current status of the user call request is generated. Then, the generated representation of the current status of the user call request is provided to the user.

Abstract: A method of vertical planar surface detection includes identifying, at an electronic device, a plurality of feature points of an environment proximate the electronic device based on imagery captured at the electronic device and based on received non-visual pose information obtained from non-visual sensors. The electronic device queries the plurality of feature points to identify one or more planar surfaces in the imagery captured at the electronic device. Further, the electronic device computes a plurality of oriented point vectors associated with the one or more planar surfaces. A vertical planar surface is detected based at least in part on one or more of the plurality of oriented point vectors being aligned in a direction perpendicular to gravity within a predetermined threshold.

Abstract: Implementations can receive, at a computing device, audio data corresponding to a spoken utterance of a user, process the audio data to generate, for one or more parts of the spoken utterance, a plurality of speech hypotheses, select a given one of the speech hypotheses, cause the given one of the speech hypotheses to be incorporated as a portion of a transcription associated with the software application, and store the plurality of speech hypotheses. In some implementations, the plurality of speech hypotheses can be loaded at an additional computing device when the transcription is accessed at the additional computing device. In additional or alternative implementations, the plurality of speech hypotheses can be loaded into memory of the computing device when the software application is reactivated and/or when a subsequent dialog session associated with the transcription is initiated.

Abstract: The present disclosure relates to generating and adjusting translated audio from a video-based source. The method includes receiving video data and corresponding audio data in a first language; generating a translated preliminary transcript in a second language; aligning timing windows of portions of the translated preliminary transcript with corresponding segments of the audio data; determining portions of the translated aligned transcript in the second language that exceed a timing window range of the corresponding segments of the audio data in the first language to generate flagged transcript portions; transmitting the original transcript, the translated aligned transcript, and the first speech dub to a first device, the generated flagged transcript portions included in the original transcript and the translated aligned transcript; receiving, from the first device, a modified original transcript; and generating, based on the modified original transcript, a second speech dub in the second language.

Abstract: To present augmented reality features without localizing a user, a client device receives a request for presenting augmented reality features in a camera view of a computing device of the user. Prior to localizing the user, the client device obtains sensor data indicative of a pose of the user, and determines the pose of the user based on the sensor data with a confidence level that exceeds a confidence threshold which indicates a low accuracy state. Then the client device presents one or more augmented reality features in the camera view in accordance with the determined pose of the user while in the low accuracy state.

Abstract: To reduce the effects of traffic jams, a traffic reduction system identifies road segments having more a threshold amount of traffic. The traffic reduction system also receives an indication of a current location of a client device operating in a vehicle and compares the current location to the locations of the identified road segments to determine whether the vehicle is on or approaching the road segment. If the vehicle is on or approaching the road segment, the traffic reduction system determines a target speed for the vehicle to maintain an equal distance between the vehicle and the vehicle in front of the vehicle and the vehicle and the vehicle behind the vehicle. The traffic reduction system then provides the target speed to the client device for display on a user interface. By maintaining equal distances between vehicles in front of and behind each other, the amount of traffic dissipates.

Abstract: Methods, systems, and computer-readable media are provided for collective reconciliation. In some implementations, a query is received, wherein the query is associated at least in part with a type of entity. One or more search results are caused to be generated based at least in part on the query. Previously generated data is retrieved associated with at least one search result of the one or more of search results, the data comprising one or more entity references in the at least one search result corresponding to the type of entity. The one or more entity references are ranked, and an entity result is selected from the one or more entity references based at least in part on the ranking. An answer to the query is provided based at least in part on the entity result.

Abstract: A system of multi-modal transmission of packetized data in a voice activated data packet based computer network environment is provided. A natural language processor component can parse an input audio signal to identify a request and a trigger keyword. Based on the input audio signal, a direct action application programming interface can generate a first action data structure, and a content selector component can select a content item. An interface management component can identify first and second candidate interfaces, and respective resource utilization values. The interface management component can select, based on the resource utilization values, the first candidate interface to present the content item. The interface management component can provide the first action data structure to the client computing device for rendering as audio output, and can transmit the content item converted for a first modality to deliver the content item for rendering from the selected interface.

Abstract: The present disclosure is directed to an automated neural architecture search approach for designing new neural network architectures such as, for example, resource-constrained mobile CNN models. In particular, the present disclosure provides systems and methods to perform neural architecture search using a novel factorized hierarchical search space that permits layer diversity throughout the network, thereby striking the right balance between flexibility and search space size. The resulting neural architectures are able to be run relatively faster and using relatively fewer computing resources (e.g., less processing power, less memory usage, less power consumption, etc.), all while remaining competitive with or even exceeding the performance (e.g., accuracy) of current state-of-the-art mobile-optimized models.