Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: A user can submit a visual query that includes one or more images. Various processing techniques such as optical character recognition (OCR) techniques can be used to recognize text (e.g. in the image, surrounding image(s), etc.) and/or various object detection techniques (e.g., machine-learned object detection models, etc.) may be used to detect objects (e.g., products, landmarks, animals, humans, etc.) within or related to the visual query. Content related to the detected text or object(s) can be identified and potentially provided to a user as search results or a proactive content feed. As such, aspects of the present disclosure enable the visual search system to more intelligently process a visual query to provide improved search results and content feeds, including those search results which are more personalized and/or consider contextual signals to account for implicit characteristics of the visual query and/or user's search intent.

Abstract: A method for a soft acceptance of a hotword receives audio data characterizing a soft hotword event detected by a hotword detector in streaming audio captured by a user device. The method also processes the audio data to determine that the audio data corresponds to a query specifying an action to perform on the user device. Without triggering performance of the action on the user device or the other device, the method provides a notification for output from the user device where the notification prompts a user associated with the user device to provide an affirmative input indication in order to trigger performance of the action on the user device or the other device and, when the user fails to provide the affirmative input indication, instructs the user device or the other device to not perform the action specified by the query.

Abstract: A user can submit a visual query that includes one or more images. Various processing techniques such as optical character recognition (OCR) techniques can be used to recognize text (e.g. in the image, surrounding image(s), etc.) and/or various object detection techniques (e.g., machine-learned object detection models, etc.) may be used to detect objects (e.g., products, landmarks, animals, humans, etc.) within or related to the visual query. Content related to the detected text or object(s) can be identified and potentially provided to a user as search results or a proactive content feed. As such, aspects of the present disclosure enable the visual search system to more intelligently process a visual query to provide improved search results and content feeds, including those search results which are more personalized and/or consider contextual signals to account for implicit characteristics of the visual query and/or user's search intent.

Abstract: A user can submit a visual query that includes one or more images. Various processing techniques such as optical character recognition (OCR) techniques can be used to recognize text (e.g. in the image, surrounding image(s), etc.) and/or various object detection techniques (e.g., machine-learned object detection models, etc.) may be used to detect objects (e.g., products, landmarks, animals, humans, etc.) within or related to the visual query. Content related to the detected text or object(s) can be identified and potentially provided to a user as search results or a proactive content feed. As such, aspects of the present disclosure enable the visual search system to more intelligently process a visual query to provide improved search results and content feeds, including those search results which are more personalized and/or consider contextual signals to account for implicit characteristics of the visual query and/or user's search intent.

Abstract: A method for a soft acceptance of a hotword receives audio data characterizing a soft hotword event detected by a hotword detector in streaming audio captured by a user device. The method also processes the audio data to determine that the audio data corresponds to a query specifying an action to perform on the user device. Without triggering performance of the action on the user device or the other device, the method provides a notification for output from the user device where the notification prompts a user associated with the user device to provide an affirmative input indication in order to trigger performance of the action on the user device or the other device and, when the user fails to provide the affirmative input indication, instructs the user device or the other device to not perform the action specified by the query.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: A method for dynamically changing a graphical user interface element occurs in response to detecting that a temporal user interface element displayed on a user interface of user device. The method includes receiving, at the user device, a contextual signal characterizing a state of a user. The method further includes determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element. The methods also include, in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element, modifying a respective state of the temporal user interface element displayed on the user interface of the user device.

Abstract: Systems and methods are disclosed for displaying available or recommended electronic multimedia content to a user, including electronic media content on the Internet. According to one implementation, a method is provided that includes receiving a request from a user, the request specifying electronic media content desired by the user; analyzing an indexed web history of a plurality of other users, based on the request for desired content; and selecting and sorting a subset of available content groups, based on the request for desired content and the indexed web history. The method also includes selecting and sorting, for each selected and sorted content group, a subset of available content; providing instructions to display, to a user, the selected and sorted content groups along a first axis of a two-dimensional grid; and further providing instructions to display, to the user, the selected available content for each content group along a second axis of the two-dimensional grid.

Abstract: A method for dynamically changing a graphical user interface element occurs in response to detecting that a temporal user interface element displayed on a user interface of user device. The method includes receiving, at the user device, a contextual signal characterizing a state of a user. The method further includes determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element. The methods also include, in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element, modifying a respective state of the temporal user interface element displayed on the user interface of the user device.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: A method for a soft acceptance of a hotword receives audio data characterizing a soft hotword event detected by a hotword detector in streaming audio captured by a user device. The method also processes the audio data to determine that the audio data corresponds to a query specifying an action to perform on the user device. Without triggering performance of the action on the user device or the other device, the method provides a notification for output from the user device where the notification prompts a user associated with the user device to provide an affirmative input indication in order to trigger performance of the action on the user device or the other device and, when the user fails to provide the affirmative input indication, instructs the user device or the other device to not perform the action specified by the query.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: A method for dynamically changing a graphical user interface element occurs in response to detecting that a temporal user interface element displayed on a user interface of user device. The method includes receiving, at the user device, a contextual signal characterizing a state of a user. The method further includes determining, by the user device, that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element. The methods also include, in response to determining that the contextual signal characterizing the state of the user is indicative of the user intending to interact with a temporal user interface element, modifying a respective state of the temporal user interface element displayed on the user interface of the user device.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.

Abstract: Provided are methods, systems, and devices for generating semantic objects and an output based on the detection or recognition of the state of an environment that includes objects. State data, based in part on sensor output, can be received from one or more sensors that detect a state of an environment including objects. Based in part on the state data, semantic objects are generated. The semantic objects can correspond to the objects and include a set of attributes. Based in part on the set of attributes of the semantic objects, one or more operating modes, associated with the semantic objects can be determined. Based in part on the one or more operating modes, object outputs associated with the semantic objects can be generated. The object outputs can include one or more visual indications or one or more audio indications.

Abstract: A user can submit a visual query that includes one or more images. Various processing techniques such as optical character recognition (OCR) techniques can be used to recognize text (e.g. in the image, surrounding image(s), etc.) and/or various object detection techniques (e.g., machine-learned object detection models, etc.) may be used to detect objects (e.g., products, landmarks, animals, humans, etc.) within or related to the visual query. Content related to the detected text or object(s) can be identified and potentially provided to a user as search results or a proactive content feed. As such, aspects of the present disclosure enable the visual search system to more intelligently process a visual query to provide improved search results and content feeds, including those search results which are more personalized and/or consider contextual signals to account for implicit characteristics of the visual query and/or user's search intent.

Abstract: The present disclosure relates generally to the processing of machine-readable visual encodings in view of contextual information. One embodiment of aspects of the present disclosure comprises obtaining image data descriptive of a scene that includes a machine-readable visual encoding; processing the image data with a first recognition system configured to recognize the machine-readable visual encoding; processing the image data with a second, different recognition system configured to recognize a surrounding portion of the scene that surrounds the machine-readable visual encoding; identifying a stored reference associated with the machine-readable visual encoding based at least in part on one or more first outputs generated by the first recognition system based on the image data and based at least in part on one or more second outputs generated by the second recognition system based on the image data; and performing one or more actions responsive to identification of the stored reference.