Abstract: Systems and methods for visuotactile object pose estimation and shape completion are provided. In one embodiment, a method includes transforming at least one point cloud representation of an object into an input voxel grid of the visualized area of the object. The input voxel grid is a volumetric representation. The method further includes encoding the input voxel grid into a partial latent vector that lies on a partial latent space. The method yet further includes determining a mapping between the partial latent space and a complete latent space based on the sensor data. The method includes predicting a complete latent vector based on the complete latent space. The method also includes estimating a complete shape of an object based on the complete latent space. The method further includes estimating a 6D pose of the object based on the complete latent vector.

Abstract: Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a computer implemented method includes receiving image data, depth data, and tactile data about the object in the environment. The computer implemented method also includes generating a visual estimate of the object that includes an object point cloud. The computer implemented method further includes generating a tactile estimate of the object that includes a surface point cloud based on the tactile data. The computer implemented method yet further includes estimating a pose of the object based on the visual estimate and the tactile estimate by fusing the object point cloud and the surface point cloud in a 3D space. The pose is a six-dimensional pose.

Abstract: Pose fusion estimation may be achieved via a first and second set of sensors receiving a first and second set of data, passing the first and second set of data through a graph-based neural network to generate a set of geometric features to be passed through a pose fusion network to generate a first and second pose estimate. A second portion of the pose fusion network may receive the set of geometric features and generate a second set of geometric features and the second pose estimate based on the set of geometric features. A first portion of the pose fusion network may receive the first set of data and the second set of geometric features and generate the first pose estimate based on a fusion of the first set of data and the second set of geometric features.

Abstract: Systems and methods for visuo-tactile object pose estimation are provided. In one embodiment, a computer implemented method includes receiving image data, depth data, and tactile data about the object in the environment. The computer implemented method also includes generating a visual estimate of the object that includes an object point cloud. The computer implemented method further includes generating a tactile estimate of the object that includes a surface point cloud based on the tactile data. The computer implemented method yet further includes estimating a pose of the object based on the visual estimate and the tactile estimate by fusing the object point cloud and the surface point cloud in a 3D space. The pose is a six-dimensional pose.