Abstract: In one embodiment, a method includes determining correspondence data between a sequence of images based on identified features in the sequence of images and predicted pose based on motion data, and determining current state information based on the correspondence data and the motion data. The current state information comprises at least a current pose of the wearable device relative to the environment capture by the one or more cameras. Furthermore, the method comprises receiving map points in a three-dimensional map and their associated descriptors based on the identified features in the sequence of images and identifying one or more of the map points in the sequence of images based on the associated descriptors associated with the map points. The current state information is further determined based on the identified one or more of the map points.

Abstract: In one embodiment, a computing system accesses a set of 3D locations associated with features in an environment previously captured by a camera from a previous camera pose. The computing system determines a predicted camera pose using the previous camera pose and motion measurements generated using a motion sensor associated with the camera. The computing system projects the set of 3D locations toward the predicted camera pose and onto a 2D image plane associated with the camera. The computing system generates, based on the projected set of 3D locations on the 2D image plane, an activation map specifying a subset of the pixel sensors of the camera that are to be activated. The computing system instructs, using the activation map, the camera to activate the subset of pixel sensors to capture a new image of the environment. The computing system reads pixel values of the new image.

Abstract: In one embodiment, a method includes determining correspondence data between a sequence of images based on identified features in the sequence of images and predicted pose based on motion data, and determining current state information based on the correspondence data and the motion data. The current state information comprises at least a current pose of the wearable device relative to the environment capture by the one or more cameras. Furthermore, the method comprises receiving map points in a three-dimensional map and their associated descriptors based on the identified features in the sequence of images and identifying one or more of the map points in the sequence of images based on the associated descriptors associated with the map points. The current state information is further determined based on the identified one or more of the map points.

Abstract: In one embodiment, a method includes determining correspondence data between a sequence of images based on identified features in the sequence of images and predicted pose based on motion data, and determining current state information based on the correspondence data and the motion data. The current state information comprises at least a current pose of the wearable device relative to the environment capture by the one or more cameras. Furthermore, the method comprises receiving map points in a three-dimensional map and their associated descriptors based on the identified features in the sequence of images and identifying one or more of the map points in the sequence of images based on the associated descriptors associated with the map points. The current state information is further determined based on the identified one or more of the map points.

Abstract: A system and method for a 3-D motion estimation and online temporal calibration having one or more than one processor, one or more than one camera, one or more than one inertial sensor, a storage for storing data and instructions. The stored instructions comprise a filter estimation module, high-precision navigation instructions and consistent state estimates in scenarios involving features, with both constant and time-varying offsets from the filter estimation module.

Abstract: A mobile platform having a camera for outputting image data of features with unknown locations, an inertial measurement unit for outputting inertial measurements of the features, where the coordinates of the features are unknown, a processor, storage and an extended-Kalman filter-based estimator executable on the processor for processing the inertial measurement and features of the image data, where a state vector of the estimator contains a sliding window of states for determining the position and orientation of the mobile platform.

Abstract: A motion estimation and correction system and methods are shown comprising: an image acquisition device configured to acquire an image via scanning an image frame over a period of time, an inertial measurement unit configured to measure at least one of a position, an orientation, and a movement of the image acquisition device during the period of time and output an indication of the movement as detected; and a state estimation module, operatively coupled to the inertial measurement unit and the image acquisition device, configured to estimate a state related to at least one of position and orientation of the image acquisition device based on the at least one of the position, the orientation, and the movement. In one example, systems and methods include using an inertial measurement unit along with an image acquisition device. In one example, the image acquisition device includes a rolling-shutter camera.

Abstract: A hybrid estimator system using visual and inertial sensors for real-time pose tracking on devices with limited processing power using at least one processor, a memory, a storage and communications through a protocol and one or more than one software module for a hybrid estimator, real-time algorithm selection to process different measurements, statistical learning for these characteristics to compute the expected device computing cost of any strategy for allocating measurements to algorithms, and algorithm selection based on the statistical learning module.

Abstract: A hybrid estimator system using visual and inertial sensors for real-time pose tracking on devices with limited processing power using at least one processor, a memory, a storage and communications through a protocol and one or more than one software module for a hybrid estimator, real-time algorithm selection to process different measurements, statistical learning for these characteristics to compute the expected device computing cost of any strategy for allocating measurements to algorithms, and algorithm selection based on the statistical learning module.

Abstract: A motion estimation and correction system and methods are shown comprising: an image acquisition device configured to acquire an image via scanning an image frame over a period of time, an inertial measurement unit configured to measure at least one of a position, an orientation, and a movement of the image acquisition device during the period of time and output an indication of the movement as detected; and a state estimation module, operatively coupled to the inertial measurement unit and the image acquisition device, configured to estimate a state related to at least one of position and orientation of the image acquisition device based on the at least one of the position, the orientation, and the movement. In one example, systems and methods include using an inertial measurement unit along with an image acquisition device. In one example, the image acquisition device includes a rolling-shutter camera.

Abstract: A mobile platform having a camera for outputting image data of features with unknown locations, an inertial measurement unit for outputting inertial measurements of the features, where the coordinates of the features are unknown, a processor, storage and an extended-Kalman filter-based estimator executable on the processor for processing the inertial measurement and features of the image data, where a state vector of the estimator contains a sliding window of states for determining the position and orientation of the mobile platform.

Abstract: A system and method for a 3-D motion estimation and online temporal calibration having one or more than one processor, one or more than one camera, one or more than one inertial sensor, a storage for storing data and instructions. The stored instructions comprise a filter estimation module, high-precision navigation instructions and consistent state estimates in scenarios involving features, with both constant and time-varying offsets from the filter estimation module.