Abstract: A head-mounted device may include one or more sensors for capturing images of a real-world environment and for determining where the head-mounted device is facing in the real-world environment. The sensors can be used to capture images for creating a lighting map that indicates where one or more light sources are located within the environment and for creating a physical map representing a geometry of the environment. The sensors can be used to predict a direction in which the head-mounted device will be facing in the real-world environment. One of the sensors can include an image sensor for capturing images that are displayed to a user of the head-mounted device. One or more settings of the image sensor can be adjusted based on the lighting map and based on the predicted direction of where the head-mounted device will be facing when capturing a future frame.

Abstract: The present disclosure relates to techniques for transitioning between imagery and sounds of two different environments, such as a virtual environment and a real environment. A view of a first environment and audio associated with the first environment are provided. In response to detecting a transition event, a view of the first environment combined with a second environment is provided. The combined view includes imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value. In addition, in response to detecting a transition event, the first environment audio is mixed with audio associated with the second environment.

Abstract: The present disclosure relates to techniques for presenting a combined view of a virtual environment and a real environment in response to detecting a transition event associated with an object in the real environment. While presenting the combined view, if an input of a first type is detected, the combined view is adjusted by increasing the visibility of imagery of the virtual environment and decreasing the visibility of imagery of the real environment. If an input of a second type is detected, the combined view is adjusted by decreasing the visibility of the imagery of the virtual environment and increasing the visibility of the imagery of the real environment.

Abstract: The present disclosure relates to techniques for inserting imagery from a real environment into a virtual environment. While presenting (e.g., displaying) the virtual environment at an electronic device, a proximity of the electronic device to a physical object located in a real environment is detected. In response to detecting that the proximity of the electronic device to the physical object is less than a first threshold distance, imagery of the physical object is isolated from other imagery of the real environment. The isolated imagery of the physical object is inserted into the virtual environment at a location corresponding to the location of the physical object in the real environment. The imagery of the physical object has a first visibility value associated with the proximity of the electronic device to the physical object.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: An optical lens arrangement comprises a first Fresnel lens element and a second lens element. The first Fresnel lens element defines a flat surface side and an opposite faceted surface side defining wedge and draft faces. The flat surface side faces towards the eye of a user and the opposite faceted surface side faces away from the eye of the user. The second lens element interfaces with the faceted surface side of first Fresnel lens. The second lens element is selected from the group consisting of: a second Fresnel lens element, a singlet lens element, a doublet lens element and any combination thereof. The first Fresnel lens is proximal relative to the eye of the user and the second lens element is distal relative to the eye of the user. Head mounted devices (HMD) including these optical lens arrangements are provided. Methods of making such optical lens arrangements and HMDs are also provided.

Abstract: Some implementations provide improved user experiences on head mounted devices (HMDs) that provide near eye viewing, e.g., HMDs that display distorted images and provide lenses that undistort the images for the user. The images are produced using distortion that is corrected dynamically based on context to conserve device resources. To do so, a context associated with a state of the user, the HMD, or content being viewed on the HMD is tracked during the user experience. For example, the device may predict pupil position, eye state, eye gaze direction, or eye fixation, content type, connection mode, and other context. The device uses the tracked context to determine how to correct distortion for the images at different points during the user experience. For example, new distortion corrections may be computed and used while the user's gaze is moving and previously-determined distortion corrections may be used while the user's gaze is fixed.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: The present disclosure relates to techniques for transitioning between imagery and sounds of two different environments, such as a virtual environment and a real environment. A view of a first environment and audio associated with the first environment are provided. In response to detecting a transition event, a view of the first environment combined with a second environment is provided. The combined view includes imagery of the first environment at a first visibility value and imagery of the second environment at a second visibility value. In addition, in response to detecting a transition event, the first environment audio is mixed with audio associated with the second environment.

Abstract: The present disclosure relates to techniques for inserting imagery from a real environment into a virtual environment. While presenting (e.g., displaying) the virtual environment at an electronic device, a proximity of the electronic device to a physical object located in a real environment is detected. In response to detecting that the proximity of the electronic device to the physical object is less than a first threshold distance, imagery of the physical object is isolated from other imagery of the real environment. The isolated imagery of the physical object is inserted into the virtual environment at a location corresponding to the location of the physical object in the real environment. The imagery of the physical object has a first visibility value associated with the proximity of the electronic device to the physical object.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: An optical lens arrangement comprises a first Fresnel lens element and a second lens element. The first Fresnel lens element defines a flat surface side and an opposite faceted surface side defining wedge and draft faces. The flat surface side faces towards the eye of a user and the opposite faceted surface side faces away from the eye of the user. The second lens element interfaces with the faceted surface side of first Fresnel lens. The second lens element is selected from the group consisting of: a second Fresnel lens element, a singlet lens element, a doublet lens element and any combination thereof. The first Fresnel lens is proximal relative to the eye of the user and the second lens element is distal relative to the eye of the user. Head mounted devices (HMD) including these optical lens arrangements are provided. Methods of making such optical lens arrangements and HMDs are also provided.

Abstract: An optical lens arrangement comprises a first Fresnel lens element and a second lens element, The first Fresnel lens element defines a flat surface side and an opposite faceted surface side defining wedge and draft faces. The flat surface side faces towards the eye of a user and the opposite faceted surface side faces away from the eye of the user. The second lens element interfaces with the faceted surface side of first Fresnel lens. The second lens element is selected from the group consisting of: a second Fresnel lens element, a singlet lens element, a doublet lens element and any combination thereof. The first Fresnel lens is proximal relative to the eye of the user and the second lens element is distal relative to the eye of the user. Head mounted devices (HMD) including these optical lens arrangements are provided. Methods of making such optical lens arrangements and HMDs are also provided.

Abstract: A Head-Mounted Display with camera sensors to perform chroma keying in a mixed reality context is presented. Low latency is achieved by embedding the processing in the HMD itself, specifically, format camera images, detect the selected color range and make a composite with the virtual content.

Abstract: A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Abstract: An optical lens arrangement comprises a first Fresnel lens element and a second lens element, The first Fresnel lens element defines a flat surface side and an opposite faceted surface side defining wedge and draft faces. The flat surface side faces towards the eye of a user and the opposite faceted surface side faces away from the eye of the user. The second lens element interfaces with the faceted surface side of first Fresnel lens. The second lens element is selected from the group consisting of: a second Fresnel lens element, a singlet lens element, a doublet lens element and any combination thereof. The first Fresnel lens is proximal relative to the eye of the user and the second lens element is distal relative to the eye of the user. Head mounted devices (HMD) including these optical lens arrangements are provided. Methods of making such optical lens arrangements and HMDs are also provided.