Abstract: A control system comprises a memory storing a sequence of sensor data received from one or more sensors. The control system has a processor which processes the sensor data to compute a sequence of derived sensor data values. An autoencoder receives the sequence of derived sensor data values and computes a forward prediction of the sequence of derived sensor data values, the autoencoder having been trained imposing a relationship on positions of the derived sensor data values encoded in a latent space of the autoencoder. A processor initiates control of an apparatus using the forward prediction.

Abstract: A method for predicting the pose of an articulated object includes receiving spatial information for n joints of the articulated object. The spatial information for the n joints is passed to a machine learning model previously trained to receive spatial information for n+m joints as input, wherein m>=1. From the machine learning model, a pose prediction for the articulated object is received as output based at least on the spatial information for the n joints, and without spatial information for the m joints.

Abstract: Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Abstract: A control system comprises a memory storing a sequence of sensor data received from one or more sensors. The control system has a processor which processes the sensor data to compute a sequence of derived sensor data values. An autoencoder receives the sequence of derived sensor data values and computes a forward prediction of the sequence of derived sensor data values, the autoencoder having been trained imposing a relationship on positions of the derived sensor data values encoded in a latent space of the autoencoder. A processor initiates control of an apparatus using the forward prediction.

Abstract: A control system comprises a memory storing a sequence of sensor data received from one or more sensors. The control system has a processor which processes the sensor data to compute a sequence of derived sensor data values. An autoencoder receives the sequence of derived sensor data values and computes a forward prediction of the sequence of derived sensor data values, the autoencoder having been trained imposing a relationship on positions of the derived sensor data values encoded in a latent space of the autoencoder. A processor initiates control of an apparatus using the forward prediction.

Abstract: Examples are disclosed that relate to a camera model for a machine vision application. One example provides instructions executable to receive image data obtained by an image sensor of a camera, the image data capturing a calibration pattern comprising a plurality of calibration features, for each of one or more imaged calibration features in the image data, determine an object space location of the imaged calibration feature, and determine a distance between the object space location and a corresponding ray of a camera model, the camera model defining a plurality of rays that each represent a relationship of an image space location on the image sensor to object space. The instructions are further executable to determine a value of a cost function based on the distances, adjust the camera model until the cost function meets a target condition, and use the camera model in a machine vision application.

Abstract: There is an apparatus for detecting pose of an object. The apparatus comprises a processor configured to receive captured sensor data depicting the object. It also has a memory storing a parameterized model of a class of 3D shape of which the object is a member, where an instance of the model is given as a mapping from a point in a 2D rectangular grid to a 3D position. The processor is configured to compute values of the parameters of the model by calculating an optimization to fit the model to the captured sensor data, using the parametrized mapping. The processor is configured to output the computed values of the parameters comprising at least global position and global orientation of the object.

Abstract: In various examples there is an apparatus for detecting position and orientation of an object. The apparatus comprises a memory storing at least one frame of captured sensor data depicting the object. The apparatus also comprises a trained machine learning system configured to receive the frame of the sensor data and to compute a plurality of two dimensional positions in the frame. Each predicted two dimensional position is a position of sensor data in the frame depicting a keypoint, where a keypoint is a pre-specified 3D position relative to the object. At least one of the keypoints is a floating keypoint depicting a pre-specified position relative to the object, lying inside or outside the object's surface. The apparatus comprises a pose detector which computes the three dimensional position and orientation of the object using the predicted two dimensional positions and outputs the computed three dimensional position and orientation.

Abstract: An apparatus for detecting pose of an object is described. The apparatus has a processor configured to receive captured sensor data depicting the object. The apparatus has a memory storing a model of a class of object of which the depicted object is a member, the model comprising a plurality of parameters specifying the pose, comprising global position and global orientation, of the model. The processor is configured to compute values of the parameters of the model by calculating an optimization to fit the model to the captured sensor data, wherein the optimization comprises iterated computation of updates to the values of the parameters and updates to values of variables representing correspondences between the captured sensor data and the model, the updates being interdependent in computation. The processor is configured to discard updates to values of the variables representing correspondences without applying the updates.

Abstract: A control system comprises a memory storing a sequence of sensor data received from one or more sensors. The control system has a processor which processes the sensor data to compute a sequence of derived sensor data values. An autoencoder receives the sequence of derived sensor data values and computes a forward prediction of the sequence of derived sensor data values, the autoencoder having been trained imposing a relationship on positions of the derived sensor data values encoded in a latent space of the autoencoder. A processor initiates control of an apparatus using the forward prediction.

Abstract: There is an apparatus for detecting pose of an object. The apparatus comprises a processor configured to receive captured sensor data depicting the object. It also has a memory storing a parameterized model of a class of 3D shape of which the object is a member, where an instance of the model is given as a mapping from a point in a 2D rectangular grid to a 3D position. The processor is configured to compute values of the parameters of the model by calculating an optimization to fit the model to the captured sensor data, using the parametrized mapping. The processor is configured to output the computed values of the parameters comprising at least global position and global orientation of the object.

Abstract: An apparatus for detecting pose of an object is described. The apparatus has a processor configured to receive captured sensor data depicting the object. The apparatus has a memory storing a model of a class of object of which the depicted object is a member, the model comprising a plurality of parameters specifying the pose, comprising global position and global orientation, of the model. The processor is configured to compute values of the parameters of the model by calculating an optimization to fit the model to the captured sensor data, wherein the optimization comprises iterated computation of updates to the values of the parameters and updates to values of variables representing correspondences between the captured sensor data and the model, the updates being interdependent in computation. The processor is configured to discard updates to values of the variables representing correspondences without applying the updates.

Abstract: In various examples there is an apparatus for detecting position and orientation of an object. The apparatus comprises a memory storing at least one frame of captured sensor data depicting the object. The apparatus also comprises a trained machine learning system configured to receive the frame of the sensor data and to compute a plurality of two dimensional positions in the frame. Each predicted two dimensional position is a position of sensor data in the frame depicting a keypoint, where a keypoint is a pre-specified 3D position relative to the object. At least one of the keypoints is a floating keypoint depicting a pre-specified position relative to the object, lying inside or outside the object's surface. The apparatus comprises a pose detector which computes the three dimensional position and orientation of the object using the predicted two dimensional positions and outputs the computed three dimensional position and orientation.

Abstract: Examples are disclosed that relate to a camera model for a machine vision application. One example provides instructions executable to receive image data obtained by an image sensor of a camera, the image data capturing a calibration pattern comprising a plurality of calibration features, for each of one or more imaged calibration features in the image data, determine an object space location of the imaged calibration feature, and determine a distance between the object space location and a corresponding ray of a camera model, the camera model defining a plurality of rays that each represent a relationship of an image space location on the image sensor to object space. The instructions are further executable to determine a value of a cost function based on the distances, adjust the camera model until the cost function meets a target condition, and use the camera model in a machine vision application.

Abstract: Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.

Abstract: A ground truth engine is described which has a memory holding a plurality of captured images depicting an articulated item. A processor of the engine is configured to access a parameterized, three dimensional (3D) model of the item. An optimizer of the ground truth engine is configured to compute ground truth values of the parameters of the 3D model for individual ones of the captured images, such that the articulated item depicted in the captured image fits the 3D model, the optimizer configured to take into account feedback data from one or more humans, about accuracy of a plurality of the computed values of the parameters.

Abstract: A computing device has an input configured to receive data captured by at least one capture device where the data depicts at least part of an object moving in an environment. The computing device has a tracker configured to track a real-world position and orientation of the object using the captured data. A processor at the computing device is configured to compute and output feedback about performance of the tracker, where the feedback encourages a user to adjust movement of the object for improved tracking of the object by the tracker.

Abstract: Techniques for human body pose estimation are disclosed herein. Depth map images from a depth camera may be processed to calculate a probability that each pixel of the depth map is associated with one or more segments or body parts of a body. Body parts may then be constructed of the pixels and processed to define joints or nodes of those body parts. The nodes or joints may be provided to a system which may construct a model of the body from the various nodes or joints.

Abstract: A ground truth engine is described which has a memory holding a plurality of captured images depicting an articulated item. A processor of the engine is configured to access a parameterized, three dimensional (3D) model of the item. An optimizer of the ground truth engine is configured to compute ground truth values of the parameters of the 3D model for individual ones of the captured images, such that the articulated item depicted in the captured image fits the 3D model, the optimizer configured to take into account feedback data from one or more humans, about accuracy of a plurality of the computed values of the parameters.

Abstract: A tracker is described which comprises an input configured to receive captured sensor data depicting an object. The tracker has a processor configured to access a rigged, smooth-surface model of the object and to compute values of pose parameters of the model by calculating an optimization to fit the model to data related to the captured sensor data. Variables representing correspondences between the data and the model are included in the optimization jointly with the pose parameters.