Abstract: Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function.

Abstract: Deep machine learning methods and apparatus related to semantic robotic grasping are provided. Some implementations relate to training a training a grasp neural network, a semantic neural network, and a joint neural network of a semantic grasping model. In some of those implementations, the joint network is a deep neural network and can be trained based on both: grasp losses generated based on grasp predictions generated over a grasp neural network, and semantic losses generated based on semantic predictions generated over the semantic neural network. Some implementations are directed to utilization of the trained semantic grasping model to servo, or control, a grasping end effector of a robot to achieve a successful grasp of an object having desired semantic feature(s).

Abstract: Asynchronous robotic control utilizing a trained critic network. During performance of a robotic task based on a sequence of robotic actions determined utilizing the critic network, a corresponding next robotic action of the sequence is determined while a corresponding previous robotic action of the sequence is still being implemented. Optionally, the next robotic action can be fully determined and/or can begin to be implemented before implementation of the previous robotic action is completed. In determining the next robotic action, most recently selected robotic action data is processed using the critic network, where such data conveys information about the previous robotic action that is still being implemented. Some implementations additionally or alternatively relate to determining when to implement a robotic action that is determined in an asynchronous manner.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-policy evaluation of a control policy. One of the methods includes obtaining policy data specifying a control policy for controlling a source agent interacting with a source environment to perform a particular task; obtaining a validation data set generated from interactions of a target agent in a target environment; determining a performance estimate that represents an estimate of a performance of the control policy in controlling the target agent to perform the particular task in the target environment; and determining, based on the performance estimate, whether to deploy the control policy for controlling the target agent to perform the particular task in the target environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.

Abstract: Using large-scale reinforcement learning to train a policy model that can be utilized by a robot in performing a robotic task in which the robot interacts with one or more environmental objects. In various implementations, off-policy deep reinforcement learning is used to train the policy model, and the off-policy deep reinforcement learning is based on self-supervised data collection. The policy model can be a neural network model. Implementations of the reinforcement learning utilized in training the neural network model utilize a continuous-action variant of Q-learning. Through techniques disclosed herein, implementations can learn policies that generalize effectively to previously unseen objects, previously unseen environments, etc.

Abstract: Deep machine learning methods and apparatus related to semantic robotic grasping are provided. Some implementations relate to training a training a grasp neural network, a semantic neural network, and a joint neural network of a semantic grasping model. In some of those implementations, the joint network is a deep neural network and can be trained based on both: grasp losses generated based on grasp predictions generated over a grasp neural network, and semantic losses generated based on semantic predictions generated over the semantic neural network. Some implementations are directed to utilization of the trained semantic grasping model to servo, or control, a grasping end effector of a robot to achieve a successful grasp of an object having desired semantic feature(s).

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-policy evaluation of a control policy. One of the methods includes obtaining policy data specifying a control policy for controlling a source agent interacting with a source environment to perform a particular task; obtaining a validation data set generated from interactions of a target agent in a target environment; determining a performance estimate that represents an estimate of a performance of the control policy in controlling the target agent to perform the particular task in the target environment; and determining, based on the performance estimate, whether to deploy the control policy for controlling the target agent to perform the particular task in the target environment.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.

Abstract: A method for reconstructing a three-dimension image includes receiving a plurality of two-dimensional images and projection information of the two-dimensional images, projecting a plurality of rays onto the plurality of two-dimensional images, determining correspondence information between pixels of different ones of the plurality of two-dimensional images, determining a value of each of the pixels, and reconstructing a three-dimension image by integrating the plurality of rays, wherein a position on each ray can be associated to one pixel of the plurality of two-dimensional images.

Abstract: Systems and methods for sequence transcription with neural networks are provided. More particularly, a neural network can be implemented to map a plurality of training images received by the neural network into a probabilistic model of sequences comprising P(S|X) by maximizing log P(S|X) on the plurality of training images. X represents an input image and S represents an output sequence of characters for the input image. The trained neural network can process a received image containing characters associated with building numbers. The trained neural network can generate a predicted sequence of characters by processing the received image.

Abstract: Systems and methods for sequence transcription with neural networks are provided. More particularly, a neural network can be implemented to map a plurality of training images received by the neural network into a probabilistic model of sequences comprising P(S|X) by maximizing log P(S|X) on the plurality of training images. X represents an input image and S represents an output sequence of characters for the input image. The trained neural network can process a received image containing characters associated with building numbers. The trained neural network can generate a predicted sequence of characters by processing the received image.

Abstract: Systems and methods for updating geographic data based on a transaction are provided. In some aspects, one or more transaction records associated with a business are accessed from a memory. Each transaction record identifies a transaction time, geographic location data, and transaction information. A geocoded record of the business is selected to update, based on the geographic location data of the one or more transaction records. The selected geocoded record is updated based on at least one of the transaction time or the transaction information identified in the transaction records.

Abstract: A method for three-dimensional esophageal reconstruction includes acquiring a first X-ray image from a first angle with respect to a subject using a first X-ray imager. At least a second X-ray image is acquired from a second angle, different than the first angle, with respect to the subject using a second X-ray imager. Additional X-ray images may be acquired from additional angle. A three-dimensional model of the esophagus is generated from the at least two X-ray images acquired at different angles. A set of fluoroscopic X-ray images is acquired using either the first X-ray imager or the second X-ray imager. The three-dimensional model of the esophagus is registered to the acquired set of fluoroscopic X-ray images. The three-dimensional model of the esophagus is displayed overlaying the set of fluoroscopic X-ray images.

Abstract: A method for reconstructing a three-dimension image includes receiving a plurality of two-dimensional images and projection information of the two-dimensional images, projecting a plurality of rays onto the plurality of two-dimensional images, determining correspondence information between pixels of different ones of the plurality of two-dimensional images, determining a value of each of the pixels, and reconstructing a three-dimension image by integrating the plurality of rays, wherein a position on each ray can be associated to one pixel of the plurality of two-dimensional images.

Abstract: A method for three-dimensional esophageal reconstruction includes acquiring a first X-ray image from a first angle with respect to a subject using a first X-ray imager. At least a second X-ray image is acquired from a second angle, different than the first angle, with respect to the subject using a second X-ray imager. Additional X-ray images may be acquired from additional angle. A three-dimensional model of the esophagus is generated from the at least two X-ray images acquired at different angles. A set of fluoroscopic X-ray images is acquired using either the first X-ray imager or the second X-ray imager. The three-dimensional model of the esophagus is registered to the acquired set of fluoroscopic X-ray images. The three-dimensional model of the esophagus is displayed overlaying the set of fluoroscopic X-ray images.

Abstract: A method for imaging a myocardial surface includes receiving an image volume. A myocardial surface is segmented within the received image volume. A polygon mesh of the segmented myocardial surface is extracted. A surface texture is calculated from voxel information taken along a path normal to the surface of the myocardium. A view of the myocardial surface is rendered using the calculated surface texture.

Abstract: A method for simultaneous visualization of the outside and the inside of a surface model at a selected view orientation includes receiving a digitized representation of a surface of a segmented object, where the surface representation comprises a plurality of points, receiving a selection of a viewing direction for rendering the object, calculating an inner product image be calculating an inner product {right arrow over (n)}p·{right arrow over (d)} at each point on the surface mesh, where {right arrow over (n)}p is a normalized vector representing the normal direction of the surface mesh at a point p towards the exterior of the object and {right arrow over (d)} is a normalized vector representing the view direction, and rendering the object using an opacity that is a function of the denoised inner product image to yield a rendered object, where an interior of the object is rendered.