Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.