Abstract: A mobile platform uses orientation sensors and vision-based tracking to track with absolute orientation. The mobile platform generates a panoramic map by rotating a camera, which is compared to an image frame produced by the camera, to determine the orientation of the camera with respect to the panoramic map. The mobile platform also estimates an orientation of the panoramic map with respect to a world reference frame, e.g., magnetic north, using orientation sensors, including at least one accelerometer and a magnetic sensor. The orientation of the camera with respect to the world reference frame is then determined using the orientation of the camera with respect to the panoramic map and the orientation of the panoramic map with respect to the world reference frame. A filter, such as a Kalman filter, provides an accurate and stable estimate of the orientation of the panoramic map with respect to the world reference frame.

Abstract: A mobile platform maps and tracks the orientation of the mobile platform in real-time as the mobile platform rotates. The mobile platform captures a plurality of camera images as the camera rotates and projects the plurality of camera images consecutively to a panoramic cylindrical map. The map is extended by projecting areas of any camera image that correspond to unmapped portions of the panoramic cylindrical map. Keypoints are extracted from mapped portions of the map and using the keypoints the orientation of the camera is determined. Accumulated error is removed when closing the panoramic cylindrical map by extending the map horizontally beyond 360 degrees to produce overlapping regions and extracting and matching keypoints from the overlapping regions. Additionally, an incorrect estimate of the initial orientation of the map may be corrected by determining the correct orientation and re-projecting the map onto a map with the correct orientation.

Abstract: A mobile platform generates a panoramic map by rotating a camera and stores the panoramic map. The mobile platform performs a computer vision operation on the panoramic map while continuing to generate the panoramic map. The mobile platform may determine the location of the camera when the panoramic map is generated, produce an annotation on the panoramic map and transmit the determined location, annotation, and portion to a storage device. In another example, the mobile platform may receive an annotation, a portion of a source panoramic map associated with the annotation, and a location where the source panoramic map was generated. The mobile platform may compare the portion of the source panoramic map to the panoramic map to identify a position on the panoramic map that corresponds to the portion of the source panoramic map, and display the annotation based on the determined position on the panoramic map.