Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling one or more computing devices to perform a task using a hierarchical agent. One of the methods includes receiving an observation characterizing a state of the one or more computing devices at the time step; selecting a gesture class for the time step using a high-level agent; processing a mid-level input using a mid-level agent neural network conditioned on the selected gesture class to generate a mid-level output that comprises parameters that define a gesture from the selected gesture class; processing a low-level input using a low-level agent neural network to generate a policy output that defines a sequence of one or more actions for interacting with the one or more computing devices; and performing the sequence of one or more actions to interact with the one or more computing devices.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enabling a user to conduct a dialogue. Implementations of the system learn when to rely on supporting evidence, obtained from an external search system via a search system interface, and are also able to generate replies for the user that align with the preferences of a previously trained response selection neural network. Implementations of the system can also use a previously trained rule violation detection neural network to generate replies that take account of previously learnt rules.