Abstract: A visualization system using mixed reality provides an “extended view” to the pilot or other crew of an aircraft. The extended view enables the user to visualize obstructed objects projected onto the real world scene viewed by the pilot using an optical the see-through display (OSTD) thus providing the user with greater situational awareness. A LIDAR point cloud model of the environment around the aircraft is generated from two or more laser scanning devices providing a 360 degree panoramic view. The combined point cloud model is aligned with the pilot's real world view based on the pilot's “pose” and projected onto the OSTD as an overlay so that the pilot sees the combined point cloud model in relation to the real world as seen by the pilot. One aspect of the invention is that the raw sensor data from the LIDAR devices is displayed on the actual real-world scene as a point cloud without any further image processing to enhance the user's perception.

Abstract: A visualization system using mixed reality provides an “extended view” to the pilot or other crew of an aircraft. The extended view enables the user to visualize obstructed objects projected onto the real world scene viewed by the pilot using an optical the see-through display (OSTD) thus providing the user with greater situational awareness. A LIDAR point cloud model of the environment around the aircraft is generated from two or more laser scanning devices providing a 360 degree panoramic view. The combined point cloud model is aligned with the pilot's real world view based on the pilot's “pose” and projected onto the OSTD as an overlay so that the pilot sees the combined point cloud model in relation to the real world as seen by the pilot. One aspect of the invention is that the raw sensor data from the LIDAR devices is displayed on the actual real-world scene as a point cloud without any further image processing to enhance the user's perception.