Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent interacting with an environment. In one aspect, a method comprises: receiving a natural language text sequence that characterizes a task to be performed by the agent in the environment; generating an encoded representation of the natural language text sequence; and at each of a plurality of time steps: obtaining an observation image characterizing a state of the environment at the time step; processing the observation image to generate an encoded representation of the observation image; generating a sequence of input tokens; processing the sequence of input tokens to generate a policy output that defines an action to be performed by the agent in response to the observation image; selecting an action to be performed by the agent using the policy output; and causing the agent to perform the selected action.

Abstract: Asynchronous robotic control utilizing a trained critic network. During performance of a robotic task based on a sequence of robotic actions determined utilizing the critic network, a corresponding next robotic action of the sequence is determined while a corresponding previous robotic action of the sequence is still being implemented. Optionally, the next robotic action can be fully determined and/or can begin to be implemented before implementation of the previous robotic action is completed. In determining the next robotic action, most recently selected robotic action data is processed using the critic network, where such data conveys information about the previous robotic action that is still being implemented. Some implementations additionally or alternatively relate to determining when to implement a robotic action that is determined in an asynchronous manner.

Abstract: Using large-scale reinforcement learning to train a policy model that can be utilized by a robot in performing a robotic task in which the robot interacts with one or more environmental objects. In various implementations, off-policy deep reinforcement learning is used to train the policy model, and the off-policy deep reinforcement learning is based on self-supervised data collection. The policy model can be a neural network model. Implementations of the reinforcement learning utilized in training the neural network model utilize a continuous-action variant of Q-learning. Through techniques disclosed herein, implementations can learn policies that generalize effectively to previously unseen objects, previously unseen environments, etc.

Abstract: Various embodiments may provide an optical system for determining a refractive index of a sample. The optical system may include a laser source configured to emit a laser beam, and a non-linear crystal configured to generate, based on the laser beam, an infrared light beam which passes through the sample, and a visible signal light beam. The optical system may further include a first mirror configured to reflect the visible signal light beam, a second mirror configured to reflect the infrared light beam so that the reflected infrared light beam interacts with the reflected visible signal light beam in the non-linear crystal, and a photodetector configured to determine a property of the reflected visible signal light beam which has interacted with the reflected infrared light beam for determining the refractive index of the sample.

Abstract: Various embodiments may provide an optical system for determining a refractive index of a sample. The optical system may include a laser source configured to emit a laser beam, and a non-linear crystal configured to generate, based on the laser beam, an infrared light beam which passes through the sample, and a visible signal light beam. The optical system may further include a first mirror configured to reflect the visible signal light beam, a second mirror configured to reflect the infrared light beam so that the reflected infrared light beam interacts with the reflected visible signal light beam in the non-linear crystal, and a photodetector configured to determine a property of the reflected visible signal light beam which has interacted with the reflected infrared light beam for determining the refractive index of the sample.