Abstract: A method of the present disclosure includes (a) extracting distinctive features used for respectively distinguishing a plurality of similar attitudes from which images similar to one another are obtained using a simulation model of an object, (b) capturing an object image of the object using a camera, (c) estimating a position and an attitude of the object using the object image, and (d) when the estimated attitude corresponds to one of the plurality of similar attitudes, determining the one of the plurality of similar attitudes as the attitude of the object using the distinctive features.

Abstract: A method of using both object features and scene features to track an object in a scene is provided. In one embodiment, the scene motion is compared with the object motion and if the motions differ greater than a threshold, then the pose from object tracker is used; otherwise, the pose from scene tracker is used. In another embodiment, the pose of an object is tracked by both scene tracker and object tracker and these poses are compared. If these comparison results in a difference greater than a threshold, the pose from object tracker is used; otherwise, the pose from scene tracker is used.

Abstract: Systems and method for predicting performance of pipelines for object detection, pose estimation, and object tracking are provided. In one embodiment, a plurality of regressors are stored where each regressor may be trained for a specific object detection (OD), pose estimation (PE), and/or tracking pipeline, so that each regressor outputs a score equal to or greater than that of another regressor in response to a first feature vector as input, the first feature vector corresponding to a first object that is more suitable for the specific pipeline than other pipelines. A second feature vector is stored corresponding to a second object. The regressors are run with the second feature vector as input to derive respective scores, and a recommendation of at least one pipeline is indicated based on the derived scores with respect to the second object. OD/PE/tracking may then be performed using the at least one indicated pipeline.

Abstract: A method of using both object features and scene features to track an object in a scene is provided. In one embodiment, the scene motion is compared with the object motion and if the motions differ greater than a threshold, then the pose from object tracker is used; otherwise, the pose from scene tracker is used. In another embodiment, the pose of an object is tracked by both scene tracker and object tracker and these poses are compared. If these comparison results in a difference greater than a threshold, the pose from object tracker is used; otherwise, the pose from scene tracker is used.

Abstract: Systems and method for predicting performance of pipelines for object detection, pose estimation, and object tracking are provided. In one embodiment, a plurality of regressors are stored where each regressor may be trained for a specific object detection (OD), pose estimation (PE), and/or tracking pipeline, so that each regressor outputs a score equal to or greater than that of another regressor in response to a first feature vector as input, the first feature vector corresponding to a first object that is more suitable for the specific pipeline than other pipelines. A second feature vector is stored corresponding to a second object. The regressors are run with the second feature vector as input to derive respective scores, and a recommendation of at least one pipeline is indicated based on the derived scores with respect to the second object. OD/PE/tracking may then be performed using the at least one indicated pipeline.

Abstract: A method includes acquiring, from a camera, an image frame including a representation of an object, and retrieving from a memory, data containing a template of a first pose of the object. A processor compares the first template to the image frame. A plurality of candidate locations in the image frame having a correlation with the template exceeding a predetermined threshold is determined. Edge registration on at least one candidate location of the plurality of candidate locations is performed to derive a refined pose of the object. Based at least in part on the performed edge registration, an initial pose of the object is determined, and a display image is output for display on a display device. The position at which the display image is displayed and/or the content of the display image is based at least in part on the determined initial pose of the object.

Abstract: A head-mounted display includes a camera that obtains an image of an object within a field of view. The head-mounted display further includes a processor configured to determine a plurality of feature points from the image and calculate a feature strength for each of the plurality of feature points. The processor is further configured to divide the image into a plurality of cells and select feature points having the highest feature strength from each cell and which have not yet been selected. The processor being further configured to detect and track the object within the field of view using the selected feature points.

Abstract: Multiple Holocam Orbs observe a real-life environment and generate an artificial reality representation of the real-life environment. Depth image data is cleansed of error due to LED shadow by identifying the edge of a foreground object in an (near infrared light) intensity image, identifying an edge in a depth image, and taking the difference between the start of both edges. Depth data error due to parallax is identified noting when associated text data in a given pixel row that is progressing in a given row direction (left-to-right or right-to-left) reverses order. Sound sources are identified by comparing results of a blind audio source localization algorithm, with the spatial 3D model provided by the Holocam Orb. Sound sources that corresponding to identifying 3D objects are associated together. Additionally, types of data supported by a standard movie data container, such as an MPEG container, is expanding to incorporate free viewpoint data (FVD) model data.

Abstract: Multiple Holocam Orbs observe a real-life environment and generate an artificial reality representation of the real-life environment. Depth image data is cleansed of error due to LED shadow by identifying the edge of a foreground object in an (near infrared light) intensity image, identifying an edge in a depth image, and taking the difference between the start of both edges. Depth data error due to parallax is identified noting when associated text data in a given pixel row that is progressing in a given row direction (left-to-right or right-to-left) reverses order. Sound sources are identified by comparing results of a blind audio source localization algorithm, with the spatial 3D model provided by the Holocam Orb. Sound sources that corresponding to identifying 3D objects are associated together. Additionally, types of data supported by a standard movie data container, such as an MPEG container, is expanding to incorporate free viewpoint data (FVD) model data.

Abstract: A head-mounted display includes a camera that obtains an image of an object within a field of view. The head-mounted display further includes a processor configured to determine a plurality of feature points from the image and calculate a feature strength for each of the plurality of feature points. The processor is further configured to divide the image into a plurality of cells and select feature points having the highest feature strength from each cell and which have not yet been selected. The processor being further configured to detect and track the object within the field of view using the selected feature points.

Abstract: A method includes acquiring, from a camera, an image frame including a representation of an object, and retrieving from a memory, data containing a template of a first pose of the object. A processor compares the first template to the image frame. A plurality of candidate locations in the image frame having a correlation with the template exceeding a predetermined threshold is determined. Edge registration on at least one candidate location of the plurality of candidate locations is performed to derive a refined pose of the object. Based at least in part on the performed edge registration, an initial pose of the object is determined, and a display image is output for display on a display device. The position at which the display image is displayed and/or the content of the display image is based at least in part on the determined initial pose of the object.