Abstract: The present disclosure relates to techniques for providing an interactive computer-generated reality environment for creating a virtual drawing using one or more electronic devices. Specifically, the described techniques provide a user with a computer-generated reality environment, which can be based on different types of realities including virtual reality and mixed reality, for creating a virtual drawing on a drawing surface within the computer-generated reality environment. The computer-generated reality environment provides the user with a realistic and immersive experience while creating the virtual drawing.

Abstract: A head-mounted device includes one or more displays configured to present media content. The media content may be presented in a user interface window. The head-mounted device may include a display controller configured to adjust a frame rate of the one or more displays and may include a frame rate management circuit configured to determine whether to adjust the frame rate of the one or more displays based on the type of the media content being presented in the user interface window and based on additional information such as a preferred frame rate associated with the media content, a size of the user interface window relative to the total display area of the one or more displays, point of gaze data, hand gestures data, head pose data, data associated with other body parts, audio information and other data.

Abstract: A method is performed at an electronic device with one or more processors, a non-transitory memory, one or more environmental sensors, and a display. The method includes determining a first positional value associated with a physical agent based on environmental data from the one or more environmental sensors. The method includes determining that a portion of computer-generated content satisfies an occlusion criterion with respect to a corresponding portion of the physical agent, based on the first positional value. The method includes, in response to determining that the occlusion criterion is satisfied and determining that the physical agent satisfies a movement criterion or a pose criterion, generating a mesh associated with the physical agent based on the first positional value, and displaying the mesh on the display.

Abstract: In one implementation, a method is performed for tiered posture awareness. The method includes: while presenting a three-dimensional (3D) environment, via the display device, obtaining head pose information for a user associated with the computing system; determining an accumulated strain value for the user based on the head pose information; and in accordance with a determination that the accumulated strain value for the user exceeds a first posture awareness threshold: determining a location for virtual content based on a height value associated with the user and a depth value associated with the 3D environment; and presenting, via the display device, the virtual content at the determined location while continuing to present the 3D environment via the display device.

Abstract: The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, information related to an object to be simulated and rendered in the CGR environment is provided to the CGR platform, and a three-dimensional representation of the object is displayed in the CGR environment.

Abstract: Providing a visual treatment based on proximity to an obstruction includes collecting, by a device, a sensor data for an environment, determining a status for each of a plurality of regions of the environment, where at least one region of the environment is assigned an occupied status, and in accordance with a determination that the device satisfies a predetermined closeness threshold to the at least one region of the environment assigned an occupied status, causing a visual treatment to be rendered by the device, where the visual treatment indicates a location of the at least one region of the environment having an occupied status.

Abstract: An electronic device may include a stereoscopic display with a plurality of lenticular lenses that extend across the length of the display. The lenticular lenses may be configured to enable stereoscopic viewing of the display such that a viewer perceives three-dimensional images. The display may have different viewing zones that account for horizontal parallax as a viewer moves horizontally relative to the display. The display may be dimmed globally, blurred, and/or composited with a default image based on the detected vertical position of the viewer. The display may render content that compensates for the real-time vertical positions of multiple viewers.

Abstract: A method is performed at an electronic device with one or more processors, a non-transitory memory, one or more environmental sensors, and a display. The method includes displaying computer-generated content on the display. The method includes determining a first positional value associated with a physical object, based on environmental data from the one or more environmental sensors. The method includes identifying a portion of the computer-generated content that satisfies an occlusion criterion with respect to a corresponding portion of the physical object, based on the first positional value. The method includes, in response to identifying that the occlusion criterion is satisfied, generating a mesh associated with the physical object based on the first positional value, and displaying the mesh on the display. The mesh overlaps with the portion of the computer-generated content.

Abstract: In some implementations, a method is performed at an electronic device including one or more processors, a non-transitory memory, a rendering system, and a display. The method includes determining a first rendering characteristic based on a first viewing angle of a first individual with respect to the display. The method includes determining a second rendering characteristic based on a second viewing angle of a second individual with respect to the display. The first rendering characteristic is different from the second rendering characteristic. The method includes generating, via the rendering system, first display content data according to the first rendering characteristic, and generating, via the rendering system, second display content data according to the second rendering characteristic. The first display content data is associated with the first viewing angle. The second display content data is associated with the second viewing angle.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide a CGR environment in which virtual objects from one or more apps are included. User interactions with the virtual objects are detected and interpreted by a system that is separate from the apps that provide the virtual objects. The system detects user interactions received via one or more input modalities and interprets those user interactions as events. These events provide a higher-level, input modality-independent, abstractions of the lower-level input-modality dependent user interactions that are detected. The system uses UI capability data provided by the apps to interpret user interactions with respect to the virtual object provided by the apps. For example, the UI capability data can identify whether a virtual object is moveable, actionable, hover-able, etc. and the system interprets user interactions at or near the virtual object accordingly.

Abstract: Various implementations disclosed herein include devices, systems, and methods for adjusting display of an image based on device position. In some implementations, a device includes an image sensor, a display, a non-transitory memory and one or more processors coupled with the image sensor, the display and the non-transitory memory. In some implementations, a method includes obtaining a first image of a body part of a user of the device. In some implementations, the method includes capturing, via the image sensor, a second image of the body part. In some implementations, the method includes determining, based on the second image, a position of the body part relative to the device. In some implementations, the method includes generating an adjusted image by adjusting the first image based on the position of the body part relative to the device. In some implementations, the method includes displaying the adjusted image on the display.

Abstract: Various implementations disclosed herein include devices, systems, and methods for displaying image data based on ambient light. In some implementations, a device includes an image sensor, an environmental sensor, a display, a non-transitory memory and one or more processors. In some implementations, a method includes capturing, via the image sensor, first image data that corresponds to a body part of a user of the device. In some implementations, the method includes detecting, via the environmental sensor, environmental data that indicates a current ambient light condition of a physical environment surrounding the device. In some implementations, the method includes generating second image data by modifying the first image data based on a function of the current ambient light condition and a threshold ambient light condition detected during enrollment. In some implementations, the method includes displaying the second image data on the display.

Abstract: The present disclosure relates to techniques for providing an interactive computer-generated reality environment for creating a virtual drawing using one or more electronic devices. Specifically, the described techniques provide a user with a computer-generated reality environment, which can be based on different types of realities including virtual reality and mixed reality, for creating a virtual drawing on a drawing surface within the computer-generated reality environment. The computer-generated reality environment provides the user with a realistic and immersive experience while creating the virtual drawing.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide a CGR environment in which virtual objects from one or more apps are included. User interactions with the virtual objects are detected and interpreted by a system that is separate from the apps that provide the virtual objects. The system detects user interactions received via one or more input modalities and interprets those user interactions as events. These events provide a higher-level, input modality-independent, abstractions of the lower-level input-modality dependent user interactions that are detected. The system uses UI capability data provided by the apps to interpret user interactions with respect to the virtual object provided by the apps. For example, the UI capability data can identify whether a virtual object is moveable, actionable, hover-able, etc. and the system interprets user interactions at or near the virtual object accordingly.

Abstract: The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, information related to an object to be simulated and rendered in the CGR environment is provided to the CGR platform, and a three-dimensional representation of the object is displayed in the CGR environment.

Abstract: A method is performed at an electronic device with one or more processors, a non-transitory memory, one or more environmental sensors, and a display. The method includes displaying computer-generated content on the display. The method includes determining a first positional value associated with a physical object, based on environmental data from the one or more environmental sensors. The method includes identifying a portion of the computer-generated content that satisfies an occlusion criterion with respect to a corresponding portion of the physical object, based on the first positional value. The method includes, in response to identifying that the occlusion criterion is satisfied, generating a mesh associated with the physical object based on the first positional value, and displaying the mesh on the display. The mesh overlaps with the portion of the computer-generated content.

Abstract: The present disclosure relates to providing a computer-generated reality (CGR) platform for generating CGR environments including virtual and augmented reality environments. In some embodiments, the platform includes an operating-system-level (OS-level) process that simulates and renders content in the CGR environment, and one or more application-level processes that provide information related to the content to be simulated and rendered to the OS-level process.

Abstract: The present disclosure relates generally to for displaying a set of images based on captured image data. In some embodiments, a system displays a first set of images corresponding to a first perspective in a virtual reality (VR) (or mixed reality (MR)) environment. The system receives a request while displaying the first set of images. The system captures image data corresponding to a second perspective in the VR (or MR) environment in response to receiving the request. The system displays a second set of images based on the captured image data.

Abstract: A device implementing a system for managing multi-modal rendering of application content includes at least one processor configured to receive content, provided by an application running on a device, for display. The at least one processor is further configured to determine that the content corresponds to two-dimensional content. The at least one processor is further configured to identify a portion of the two-dimensional content for enhancement by a three-dimensional render. The at least one processor is further configured to enhance, in response to the determining, the portion of the two-dimensional content by the three-dimensional renderer. The at least one processor is further configured to provide for display of the enhanced portion of the two-dimensional content on a display of the device.

Abstract: A device implementing a system for managing multi-modal rendering of application content includes at least one processor configured to receive content, provided by an application running on a device, for displaying in a three-dimensional display mode. The at least one processor is further configured to determine that the content corresponds to two-dimensional content. The at least one processor is further configured to identify a portion of the two-dimensional content for enhancement by a three-dimensional render. The at least one processor is further configured to enhance, in response to the determining, the portion of the two-dimensional content by the three-dimensional renderer. The at least one processor is further configured to provide for display of the enhanced portion of the two-dimensional content on a display of the device in the three-dimensional display mode.