Abstract: A computer-implemented method of determining a pose of each of a plurality of objects includes, for each given object: using image data and associated depth information to estimate a pose of the given object. The method includes iteratively updating the estimated poses by: sampling, for each given object, a plurality of points from a predetermined model of the given object transformed in accordance with the estimated pose of the given object; determining first occupancy data for each given object dependent on positions of the points sampled from the predetermined model, relative to a voxel grid containing the given object; determining second occupancy data for each given object dependent on positions of the points sampled from the predetermined models of the other objects, relative to the voxel grid containing the given object; and updating the estimated poses of the plurality of objects to reduce an occupancy penalty.

Abstract: A computer-implemented method of estimating a pose of a target object in a three-dimensional scene includes: obtaining image data and associated depth information representing a view of the three-dimensional scene; processing the image data and the associated depth information to generate a volumetric reconstruction for each of a plurality of objects in the three-dimensional scene, including the target object; determining a volumetric grid containing the target object; generating, using the generated volumetric reconstructions, occupancy data indicating portions of the volumetric grid occupied by free space and portions of the volumetric grid occupied by objects other than the target object; and estimating the pose of the target object using the generated occupancy data and pointwise feature data for a plurality of points on a surface of the target object.

Abstract: A control system for a robotic device comprising a task embedding network to receive one or more demonstrations of a task and to generate a task embedding. The task embedding comprises a representation of the task, and each demonstration comprises one or more observations of a performance of the task. The control system includes a control network to receive the task embedding from the task embedding network and to apply a policy to map a plurality of successive observations of the robotic device to respective control instructions for the robotic device. The policy applied by the control network is modulated across the plurality of successive observations of the robotic device using the task embedding from the task embedding network.