Abstract: Some implementations provide a multi-focus display system that renders images at multiple focus distances for display in conjunction with the use of appropriately powered lenses. For example, an HMD may include a fast switching lens element that allows quickly alternating between two or more focus distances. The displayed images are configured to correspond to the alternating focus distances by adjusting a high-frequency part of the images. This can provide a more natural user experience that will include near objects that require the user's eye to focus on a close focal depth plane and far objects that require the user's eye to focus on a far focal depth plane. Moreover, the user experience can be provided with little or no loss of brightness and without requiring processor and resource intensive computations.

Abstract: Various implementations disclosed herein include devices, systems, and methods that enable improved display of virtual content in computer generated reality (CGR) environments. In some implementations, the virtual content is viewed by an observer and includes multiple content elements, each of which may be rendered at an independent rate based on a change of view of the observer.

Abstract: In one implementation, a method of frame rate extrapolation is performed by a device including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes displaying, on the display, the image of the scene at a first time. The method includes generating an extrapolated image by transforming, using the one or more processors, the image of the scene based on movement of the device, wherein the extrapolated image includes a first area including a first plurality of pixels having respective first pixel values based on a single depth and a second area including a second plurality of pixels having respective second pixel values based on a plurality of depths. The method includes displaying, on the display, the extrapolated image at a second time after the first time.

Abstract: Various implementations provided herein provide a computer-generated reality (CGR) experience on a head mounted device (HMD) that includes intentionally blurring virtual content to provide a more natural user experience. For example, while the user is focused on an object 10 feet away, a virtual object 3 feet away that is not focused upon but still visible may be intentionally blurred so that it does not appear crisp. The blurring depends on the distance of what the user is focusing on and the distance of the virtual object that the user is not focusing on. The CGR environment may involve virtual reality, mixed reality, optical see through, or video pass through.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Abstract: Some implementations disclosed herein include devices, systems, and methods that display images at multiple focus distances on HMDs. The systems can avoid or mitigate vergence-accommodation conflict (VAC) and depth-blur conflict (DBC) experienced by users on traditional HMDs. Some implementations use a geometric phase lens and a circular polarization switch. The geometric phase lens passively changes its power depending on the light's circular polarization. The circular polarization switch changes the polarization of the light at appropriate times. With the circular polarization switch and geometric phase lens positioned over an HMD screen, the system can quickly change the apparent distance to the screen, which in turn changes the accommodative state of the eye needed to focus on the rendered image.

Abstract: Various implementations provided herein provide content on an optical see-through display using enhanced local contrast. In some implementations, the enhanced local contrast may be used to provide an apparent reduction in brightness (e.g., a shadow) or other visual effect. For example, the appearance of a virtual shadow of a virtual cup on a real table of a physical environment can be provided even though the brightness of the table cannot be reduced. The appearance of the shadow may be provided by selectively enhancing contrast that the user cognitively interprets to see a relatively “darker” area where the shadow should be (e.g., via an optical illusion/effect).

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

Abstract: In one implementation, a method of compensating for motion blur while presenting content on a display of an electronic device through display actuation is disclosed. The method involves detecting movement of the electronic device using a sensor of the electronic device while presenting content on a display of the electronic device. An inter-frame movement of the electronic device is determined based on the movement of the electronic device. The display is moved using an actuator of the electronic device that is coupled to the display such that movement of the display opposes the inter-frame movement of the electronic device.

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

Abstract: In one implementation, a method includes receiving a warped image representing simulated reality (SR) content (e.g., to be displayed in a display space), the warped image having a plurality of pixels at respective locations uniformly spaced in a grid pattern in a warped space, wherein the plurality of pixels are respectively associated with a plurality of respective pixel values and a plurality of respective scaling factors indicating a plurality of respective resolutions at a plurality of respective locations of the SR content (e.g., in the display space). The method includes processing the warped image in the warped space based on the plurality of respective scaling factors to generate a processed warped image and transmitting the processed warped image.

Abstract: In one implementation, a method of scene camera retargeting is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes determining, using the one or more processors, a depth of the scene. The method further includes transforming, using the one or more processors, the image of the scene based on the depth and a difference between a point-of-view of the scene camera and a point-of-view of a user. The method further includes displaying, on the display, the transformed image.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Abstract: In one implementation, a method of scene camera retargeting is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes determining, using the one or more processors, a depth of the scene. The method further includes transforming, using the one or more processors, the image of the scene based on the depth and a difference between a point-of-view of the scene camera and a point-of-view of a user. The method further includes displaying, on the display, the transformed image.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Abstract: Various implementations provided herein provide content on an optical see-through display using enhanced local contrast. In some implementations, the enhanced local contrast may be used to provide an apparent reduction in brightness (e.g., a shadow) or other visual effect. For example, the appearance of a virtual shadow of a virtual cup on a real table of a physical environment can be provided even though the brightness of the table cannot be reduced. The appearance of the shadow may be provided by selectively enhancing contrast that the user cognitively interprets to see a relatively “darker” area where the shadow should be (e.g., via an optical illusion/effect).

Abstract: In one implementation, a method of scene camera retargeting is performed by a head-mounted device (HMD) including one or more processors, non-transitory memory, a scene camera, and a display. The method includes capturing, using the scene camera, an image of a scene. The method includes determining, using the one or more processors, a depth of the scene. The method further includes transforming, using the one or more processors, the image of the scene based on the depth and a difference between a point-of-view of the scene camera and a point-of-view of a user. The method further includes displaying, on the display, the transformed image.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.

Abstract: A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user's environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.

Abstract: In various implementations, a method includes obtaining a first frame that is characterized by a first resolution associated with a first memory allocation. In some implementations, the method includes down-converting the first frame from the first resolution to a second resolution that is lower than the first resolution initially defining the first frame in order to produce a reference frame. In some implementations, the second resolution is associated with a second memory allocation that is less than a target memory allocation derived from the first memory allocation. In some implementations, the method includes storing the reference frame in a non-transitory memory. In some implementations, the method includes obtaining a second frame that is characterized by the first resolution. In some implementations, the method includes performing an error correction operation on the second frame based on the reference frame stored in the non-transitory memory.