Abstract: Real-time localization is performed using at least a portion of a panoramic image captured by a camera on a mobile device. A panoramic cylindrical map is generated using images captured by the camera, e.g., as the camera rotates. Extracted features from the panoramic cylindrical map are compared to features from a pre-generated three-dimensional model of the environment. The resulting set of corresponding features may be used to determine the pose of the camera. For example, the set of corresponding features may be converted into rays between the panoramic cylindrical map and the three-dimensional model, where the intersection of the rays is used to determine the pose. The relative orientation of the camera may also be tracked by comparing features from each new image to the panoramic cylindrical map, and the tracked orientation may be fused with the pose.