Abstract: A method for estimating a camera pose includes recognizing a three-dimensional (3D) map representing a physical environment, the 3D map including 3D map features defined as 3D points. An obfuscated image representation is received, the representation derived from an original unobfuscated image of the physical environment captured by a camera. The representation includes a plurality of obfuscated features, each including (i) a two-dimensional (2D) line that passes through a 2D point in the original unobfuscated image at which an image feature was detected, and (ii) a feature descriptor that describes the image feature associated with the 2D point that the 2D line of the obfuscated feature passes through. Correspondences are determined between the obfuscated features and the 3D map features of the 3D map of the physical environment. Based on the determined correspondences, a six degree of freedom pose of the camera in the physical environment is estimated.

Abstract: Examples are disclosed that relate to representing recorded hand motion. One example provides a computing device comprising instructions executable by a logic subsystem to receive video data capturing hand motion relative to an object, determine a first pose of the object, and associate a first coordinate system with the object based on the first pose. The instructions are further executable to determine a representation of the hand motion in the first coordinate system, the representation having a time-varying pose relative to the first pose of the object, and configure the representation for display relative to a second instance of the object having a second pose in a second coordinate system, with a time-varying pose relative to the second pose that is spatially consistent with the time-varying pose relative to the first pose.

Abstract: To obtain a relative localization between a plurality of mobile devices, a first mobile device observes a second mobile device within a field of view of the first mobile device's camera at time t1, determines a first position of the first mobile device at t1, and receives from the second mobile device a second position of the second mobile device at t1. The first mobile device determines information about the first mobile device's orientation with respect to the second mobile device at t1 based at least in part on the first position and the observation of the second mobile device. The first mobile device identifies two constraints that relate the mobile devices' coordinate systems based at least in part on the second position and the orientation information. The first mobile device's pose relative to the second mobile device may be calculated once at least six constraints are accumulated.

Abstract: A data processing system is provided that includes a processor having associated memory, the processor being configured to execute instructions using portions of the memory to cause the processor to, at classification time, receive an input image frame from an image source. The input image frame includes an articulated object and a target object. The processor is further caused to process the input image frame using a trained neural network configured to, for each input cell of a plurality of input cells in the input image frame predict a three-dimensional articulated object pose of the articulated object and a three-dimensional target object pose of the target object relative to the input cell. The processor is further caused to output the three-dimensional articulated object pose and the three-dimensional target object pose from the neural network.

Abstract: Techniques for localizing a device based on images captured by the device. The techniques include receiving, from a device via a data communication network, image frame data for frames captured by an imaging camera included in the device, the frames including a first frame, automatically detecting real-world objects captured in the image frame data, automatically classifying the detected real-world objects as being associated with respective object classes, automatically identifying object class and instance dependent keypoints for the real-world objects based on the object classes associated with the real-world objects, and estimating a pose of the device for the first frame based on at least the identified object class and instance dependent keypoints.

Abstract: Techniques are provided to reduce power consumption when performing surface reconstruction. A surface mesh of an environment is generated, where the surface mesh is generated from multiple depth maps that are obtained of the environment. After the surface mesh is generated, a change detection image of that environment is captured while refraining from obtaining a new depth map of the environment. The change detection image is compared to the surface mesh. If a difference between the change detection image and the surface mesh is detected and if that difference satisfies a pre-determined difference threshold, then a new depth map of the environment is obtained. The surface mesh is then updated using the new depth map.

Abstract: A method for estimating a camera pose includes recognizing a three-dimensional (3D) map representing a physical environment, the 3D map including 3D map features defined as 3D points. An obfuscated image representation is received, the representation derived from an original unobfuscated image of the physical environment captured by a camera. The representation includes a plurality of obfuscated features, each including (i) a two-dimensional (2D) line that passes through a 2D point in the original unobfuscated image at which an image feature was detected, and (ii) a feature descriptor that describes the image feature associated with the 2D point that the 2D line of the obfuscated feature passes through. Correspondences are determined between the obfuscated features and the 3D map features of the 3D map of the physical environment. Based on the determined correspondences, a six degree of freedom pose of the camera in the physical environment is estimated.

Abstract: To obtain a relative localization between a plurality of mobile devices, a first mobile device observes a second mobile device within a field of view of the first mobile device's camera at time t1, determines a first position of the first mobile device at t1, and receives from the second mobile device a second position of the second mobile device at t1. The first mobile device determines information about the first mobile device's orientation with respect to the second mobile device at t1 based at least in part on the first position and the observation of the second mobile device. The first mobile device identifies two constraints that relate the mobile devices' coordinate systems based at least in part on the second position and the orientation information. The first mobile device's pose relative to the second mobile device may be calculated once at least six constraints are accumulated.

Abstract: Computing devices and methods for estimating a pose of a user computing device are provided. In one example a 3D map comprising a plurality of 3D points representing a physical environment is obtained. Each 3D point is transformed into a 3D line that passes through the point to generate a 3D line cloud. A query image of the environment captured by a user computing device is received, the query image comprising query features that correspond to the environment. Using the 3D line cloud and the query features, a pose of the user computing device with respect to the environment is estimated.

Abstract: Techniques for localizing a device based on images captured by the device. The techniques include receiving, from a device via a data communication network, image frame data for frames captured by an imaging camera included in the device, the frames including a first frame, automatically detecting real-world objects captured in the image frame data, automatically classifying the detected real-world objects as being associated with respective object classes, automatically identifying object class and instance dependent keypoints for the real-world objects based on the object classes associated with the real-world objects, and estimating a pose of the device for the first frame based on at least the identified object class and instance dependent keypoints.

Abstract: Stereo image reconstruction can be achieved by fusing a plurality of proposal cost volumes computed from a pair of stereo images, using a predictive model operating on pixelwise feature vectors that include disparity and cost values sparsely sampled form the proposal cost volumes to compute disparity estimates for the pixels within the image.

Abstract: Methods of generating a three dimensional representation of an object in a reference plane from a depth map including distances from a reference point to pixels in an image of the object taken from a reference point. Weights are assigned to respective voxels in a three dimensional grid along rays extending from the reference point through the pixels in the image based on the distances in the depth map from the reference point to the respective pixels, and a height map including an array of height values in the reference plane is formed based on the assigned weights. An n-layer height map may be constructed by generating a probabilistic occupancy grid for the voxels and forming an n-dimensional height map comprising an array of layer height values in the reference plane based on the probabilistic occupancy grid.

Abstract: Methods of generating a three dimensional representation of an object in a reference plane from a depth map including distances from a reference point to pixels in an image of the object taken from a reference point. Weights are assigned to respective voxels in a three dimensional grid along rays extending from the reference point through the pixels in the image based on the distances in the depth map from the reference point to the respective pixels, and a height map including an array of height values in the reference plane is formed based on the assigned weights. An n-layer height map may be constructed by generating a probabilistic occupancy grid for the voxels and forming an n-dimensional height map comprising an array of layer height values in the reference plane based on the probabilistic occupancy grid.