Abstract: A computer-implemented method, apparatus, computer readable medium and mobile device for determining a 6DOF pose from an input image. The process of determining 6DOF pose may include processing an input image to create one or more static representations of the input image, creating a dynamic representation of the input image from an estimated 6DOF pose and a 2.5D reference map, and measuring correlation between the dynamic representation and the one or more static representations of the input image. The estimated 6DOF may be iteratively adjusted according to the measured correlation error until a final adjusted dynamic representation meets an output threshold.

Abstract: A computer-implemented method, apparatus, computer readable medium and mobile device for determining a 6DOF pose from an input image. The process of determining 6DOF pose may include processing an input image to create one or more static representations of the input image, creating a dynamic representation of the input image from an estimated 6DOF pose and a 2.5D reference map, and measuring correlation between the dynamic representation and the one or more static representations of the input image. The estimated 6DOF may be iteratively adjusted according to the measured correlation error until a final adjusted dynamic representation meets an output threshold.

Abstract: For an image processing concept, a database is provided in a device, the database comprising data derived from a plurality of frames of a video sequence. A live video feed is obtained from a camera of the device. Information is extracted from a selected image of the live video feed. A search is performed in the database based on the extracted information to retrieve a list of potential frames out of the plurality of frames. An initial pose of the selected image is estimated with respect to one frame of the list of potential frames as a function of the extracted information and the data derived from the one frame. Respective subsequent poses for subsequent images from the live video feed are iteratively estimated, wherein for each of the subsequent images the associated subsequent pose is estimated based on said subsequent image and a respective previously estimated pose. The live video feed is augmented on the device with virtual information based on the estimated initial pose and the subsequent poses.

Abstract: Techniques are presented for monocular visual simultaneous localization and mapping (SLAM) based on detecting a translational motion in the movement of the camera using at least one motion sensor, while the camera is performing panoramic SLAM, and initializing a three dimensional map for tracking of finite features. Motion sensors may include one or more sensors, including inertial (gyroscope, accelerometer), magnetic (compass), vision (camera) or any other sensors built into mobile devices.

Abstract: Disclosed are a system, apparatus, and method for multiple client simultaneous localization and mapping. Tracking and mapping may be performed locally and independently by each of a plurality of clients. At configurable points in time map data may be sent to a server for stitching and fusion. In response to successful stitching and fusion to one or more maps known to the server, updated position and orientation information relative to the server's maps may be sent back to the clients. Clients may update their local map data with the received server location data. Clients may receive additional map data from the server, which can be used for extending their maps. Clients may send queries to the server for 3D maps, and the queries may include metadata.

Abstract: A computer-implemented method, apparatus, computer readable medium and mobile device for initializing a 3-Dimensional (3D) map may include obtaining, from a camera, a single image of an urban outdoor scene and estimating an initial pose of the camera. An untextured model of a geographic region may be obtained. Line features from the single image may be extracted and the orientation may be determined with respect to the untextured model and using the extracted line features, the orientation of the camera in 3 Degrees of Freedom (3DOF). In response to determining the orientation of the camera, a translation in 3DOF with respect to the untextured model may be determined using the extracted line features. The 3D map may be initialized based on the determined orientation and translation.

Abstract: Techniques are presented for monocular visual simultaneous localization and mapping (SLAM) based on detecting a translational motion in the movement of the camera using at least one motion sensor, while the camera is performing panoramic SLAM, and initializing a three dimensional map for tracking of finite features. Motion sensors may include one or more sensors, including inertial (gyroscope, accelerometer), magnetic (compass), vision (camera) or any other sensors built into mobile devices.

Abstract: A mobile device uses vision and orientation sensor data jointly for six degree of freedom localization, e.g., in wide-area environments. An image or video stream is captured while receiving geographic orientation data and may be used to generate a panoramic cylindrical map of an environment. A bin of model features stored in a database is accessed based on the geographic orientation data. The model features are from a pre-generated reconstruction of the environment produced from extracted features from a plurality of images of the environment. The reconstruction is registered to a global orientation and the model features are stored in bins based on similar geographic orientations. Features from the panoramic cylindrical map are matched to model features in the bin to produce a set of corresponding features, which are used to determine a position and an orientation of the camera.

Abstract: Pose estimation is performed using a scene structure captured in a query image and reference images from a database. Each of the reference images has an associated position estimate. Direction vectors are generated that describe directions between an unknown position of a camera center for the query image and a reference camera center for each reference image based on the query image and the plurality of reference images. The direction vectors may be generated using, e.g., homographies, essential matrices, or fundamental matrices. A pose estimation with six degrees of freedom is determined using the direction vectors and the associated position estimate for each reference image. The pose estimation, for example, may be determined by solving a three-point pose problem using the direction vectors and the associated position estimate for each reference image.

Abstract: Exemplary methods, apparatuses, and systems for performing wide area localization from simultaneous localization and mapping (SLAM) maps are disclosed. A mobile device can select a first keyframe based SLAM map of the local environment with one or more received images. A respective localization of the mobile device within the local environment can be determined, and the respective localization may be based on the keyframe based SLAM map. The mobile device can send the first keyframe to a server and receive a first global localization response representing a correction to a local map on the mobile device. The first global localization response can include rotation, translation, and scale information. A server can receive keyframes from a mobile device, and localize the keyframes within a server map by matching keyframe features received from the mobile device to server map features.

Abstract: A real-time panoramic mapping process is presented for generating a panoramic image from a plurality of image frames that are being captured by one or more cameras of a device. The proposed mapping process may be used to clear-out an unwanted portion from the panoramic image and replace it with correct information from other images of the same scene. Moreover, brightness seams may be blended while constructing the panoramic image. The proposed real-time panoramic mapping process may be performed on a parallel processor.

Abstract: A mobile device uses vision and orientation sensor data jointly for six degree of freedom localization, e.g., in wide-area environments. An image or video stream is captured while receiving geographic orientation data and may be used to generate a panoramic cylindrical map of an environment. A bin of model features stored in a database is accessed based on the geographic orientation data. The model features are from a pre-generated reconstruction of the environment produced from extracted features from a plurality of images of the environment. The reconstruction is registered to a global orientation and the model features are stored in bins based on similar geographic orientations. Features from the panoramic cylindrical map are matched to model features in the bin to produce a set of corresponding features, which are used to determine a position and an orientation of the camera.

Abstract: Pose estimation is performed using a scene structure captured in a query image and reference images from a database. Each of the reference images has an associated position estimate. Direction vectors are generated that describe directions between an unknown position of a camera center for the query image and a reference camera center for each reference image based on the query image and the plurality of reference images. The direction vectors may be generated using, e.g., homographies, essential matrices, or fundamental matrices. A pose estimation with six degrees of freedom is determined using the direction vectors and the associated position estimate for each reference image. The pose estimation, for example, may be determined by solving a three-point pose problem using the direction vectors and the associated position estimate for each reference image.

Abstract: Real-time localization is performed using at least a portion of a panoramic image captured by a camera on a mobile device. A panoramic cylindrical map is generated using images captured by the camera, e.g., as the camera rotates. Extracted features from the panoramic cylindrical map are compared to features from a pre-generated three-dimensional model of the environment. The resulting set of corresponding features may be used to determine the pose of the camera. For example, the set of corresponding features may be converted into rays between the panoramic cylindrical map and the three-dimensional model, where the intersection of the rays is used to determine the pose. The relative orientation of the camera may also be tracked by comparing features from each new image to the panoramic cylindrical map, and the tracked orientation may be fused with the pose.