Abstract: Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification.

Abstract: Efficient control and/or linking of smart network connected devices. Some implementations are directed to rendering, within an active application that enables control of multiple smart devices from multiple parties, both: (1) an interactive graphical interface element that, when interacted with, causes corresponding control of smart device(s) of a third-party (3P); and (2) a selectable element that, when selected, causes execution of a deeplink to cause a 3P application to open in a state that presents additional information for the smart device(s). Some implementations are additionally or alternatively directed to, in response to determining that a 3P application is installed at a client device, but a smart device topology lacks any entries for 3P smart devices of the 3P, efficiently adding one or more particular smart devices of the 3P to the smart device topology.

Abstract: In one example, a method includes receiving audio data generated by one or more microphones of a computing device, the audio data representing a spoken utterance; identifying, based on the audio data, a user that provided the spoken utterance; identifying, based on the audio data, an automation action associated with one or more automation devices, the automation action corresponding to the spoken utterance; determining whether the identified user is authorized to cause performance of the identified automation action; and responsive to determining that the identified user is authorized to cause performance of the identified automation action, causing the one or more automation devices to perform the identified automation action.

Abstract: Various implementations relate to techniques, for controlling smart devices, that are low latency and/or that provide computational efficiencies (client and/or server) and/or network efficiencies. Those implementations relate to generating and/or utilizing cache entries, of a cache that is stored locally at an assistant client device, in control of various smart devices (e.g., smart lights, smart thermostats, smart plugs, smart appliances, smart routers, etc.). Each of the cache entries includes a mapping of text to one or more corresponding semantic representations.

Abstract: In one example, a method includes receiving audio data generated by one or more microphones of a computing device, the audio data representing a spoken utterance; identifying, based on the audio data, a user that provided the spoken utterance; identifying, based on the audio data, an automation action associated with one or more automation devices, the automation action corresponding to the spoken utterance; determining whether the identified user is authorized to cause performance of the identified automation action; and responsive to determining that the identified user is authorized to cause performance of the identified automation action, causing the one or more automation devices to perform the identified automation action.

Abstract: Efficient control and/or linking of smart network connected devices. Some implementations are directed to rendering, within an active application that enables control of multiple smart devices from multiple parties, both: (1) an interactive graphical interface element that, when interacted with, causes corresponding control of smart device(s) of a third-party (3P); and (2) a selectable element that, when selected, causes execution of a deeplink to cause a 3P application to open in a state that presents additional information for the smart device(s). Some implementations are additionally or alternatively directed to, in response to determining that a 3P application is installed at a client device, but a smart device topology lacks any entries for 3P smart devices of the 3P, efficiently adding one or more particular smart devices of the 3P to the smart device topology.

Abstract: Various implementations relate to generating, locally at an assistant client device, specific control commands that, when transmitted to a corresponding smart device, are directly interpretable by the corresponding smart device to effectuate a state change at the corresponding smart device, or at a corresponding additional smart device directly controlled by the corresponding smart device. Various implementations additionally or alternatively relate to utilizing local assistant client devices in discovering, provisioning, and/or registering smart devices for an account of a user.

Abstract: The present disclosure provides systems and methods that perform structure-based access control. In particular, rather than relying upon a user-specific credential scheme, which can require manual sharing of user-specific credentials and/or switching between the multiple accounts to access the particular devices, applications, or services associated with such accounts, the systems and methods of the present disclosure facilitate user credentials to be inherited by or otherwise assigned to a structure identifier associated with a structure (e.g., a home in which the user resides), thereby generating a set of structure credentials. This enables other users in the structure, who may be part of a collaborative user group, to access devices, applications, and/or services using the structure credentials.

Abstract: Various implementations relate to techniques, for controlling smart devices, that are low latency and/or that provide computational efficiencies (client and/or server) and/or network efficiencies. Those implementations relate to generating and/or utilizing cache entries, of a cache that is stored locally at an assistant client device, in control of various smart devices (e.g., smart lights, smart thermostats, smart plugs, smart appliances, smart routers, etc.). Each of the cache entries includes a mapping of text to one or more corresponding semantic representations.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other connected devices, and other functions capable of being performed by the other connected devices. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with different connected devices. Each graphical control element can be arranged to reflect how each connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

Abstract: The present disclosure provides systems and methods that perform structure-based access control. In particular, rather than relying upon a user-specific credential scheme, which can require manual sharing of user-specific credentials and/or switching between the multiple accounts to access the particular devices, applications, or services associated with such accounts, the systems and methods of the present disclosure facilitate user credentials to be inherited by or otherwise assigned to a structure identifier associated with a structure (e.g., a home in which the user resides), thereby generating a set of structure credentials. This enables other users in the structure, who may be part of a collaborative user group, to access devices, applications, and/or services using the structure credentials.

Abstract: Remote automated assistant component(s) generate client device notification(s) based on a received IoT state change notification that indicates a change in at least one state associated with at least one IoT device. The generated client device notification(s) can each indicate the change in state associated with the at least one IoT device, and can optionally indicate the at least one IoT device. Further, the remote automated assistant component(s) can identify candidate assistant client devices that are associated with the at least one IoT device, and determine whether each of the one or more of the candidate assistant client device(s) should render a corresponding client device notification.

Abstract: The present disclosure is generally related to a data processing system to validate vehicular functions in a voice activated computer network environment. The data processing system can improve the efficiency of the network by discarding action data structures and requests that invalid prior to their transmission across the network. The system can invalidate requests by comparing attributes of a vehicular state to attributes of a request state.

Abstract: Various implementations relate to generating, locally at an assistant client device, specific control commands that, when transmitted to a corresponding smart device, are directly interpretable by the corresponding smart device to effectuate a state change at the corresponding smart device, or at a corresponding additional smart device directly controlled by the corresponding smart device. Various implementations additionally or alternatively relate to utilizing local assistant client devices in discovering, provisioning, and/or registering smart devices for an account of a user.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other connected devices, and other functions capable of being performed by the other connected devices. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with different connected devices. Each graphical control element can be arranged to reflect how each connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

Abstract: Example aspects of the present disclosure are directed to processing voice commands or utterances. For instance, data indicative of a voice utterance can be received. A device topology representation can be accessed. The device topology representation can define a plurality of smart devices associated with one or more structures. The device topology representation can further define a location of each of the plurality of devices within the associated structures. A transcription of the voice utterance can be determined based at least in part on the device topology representation. One or more selected devices and one or more actions to be performed by the one or more selected devices can be determined based at least in part on the determined transcription and the device topology representation.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated proactively and/or in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other connected devices, and other functions capable of being performed by the other connected devices. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with different connected devices. Each graphical control element can be arranged to reflect how each connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

Abstract: The present disclosure is generally related to a data processing system to validate vehicular functions in a voice activated computer network environment. The data processing system can improve the efficiency of the network by discarding action data structures and requests that invalid prior to their transmission across the network. The system can invalidate requests by comparing attributes of a vehicular state to attributes of a request state.

Abstract: Methods, apparatus, systems, and computer-readable media are provided for tailoring composite graphical assistant interfaces for interacting with multiple different connected devices. The composite graphical assistant interfaces can be generated proactively and/or in response to a user providing a request for an automated assistant to cause a connected device to perform a particular function. In response to the automated assistant receiving the request, the automated assistant can identify other connected devices, and other functions capable of being performed by the other connected devices. The other functions can then be mapped to various graphical control elements in order to provide a composite graphical assistant interface from which the user can interact with different connected devices. Each graphical control element can be arranged to reflect how each connected device is operating simultaneous to the presentation of the composite graphical assistant interface.

Abstract: The present disclosure provides systems and methods that perform structure-based access control. In particular, rather than relying upon a user-specific credential scheme, which can require manual sharing of user-specific credentials and/or switching between the multiple accounts to access the particular devices, applications, or services associated with such accounts, the systems and methods of the present disclosure facilitate user credentials to be inherited by or otherwise assigned to a structure identifier associated with a structure (e.g., a home in which the user resides), thereby generating a set of structure credentials. This enables other users in the structure, who may be part of a collaborative user group, to access devices, applications, and/or services using the structure credentials.