Abstract: Techniques are disclosed relating to kernel task scheduling. In various embodiments, a computing device receives, at a first scheduler, a compute graph defining interrelationships for a set of tasks to be performed by the computing device. In some embodiment, the set of tasks are performed to provide an extended reality (XR) experience to a user. The first scheduler determines a schedule for implementing the set of tasks based on the interrelationships defined in the compute graph and issues instructions to cause a second scheduler of the computing device to schedule performance of the set of tasks in accordance with the determined schedule.

Abstract: An electronic device is provided that includes at least one image sensor for acquiring a video feed and one or more displays for presenting a passthrough video feed to a user. The electronic device can include a hierarchical failure detection scheme for detecting critical failures on the device. The hierarchical failure detection scheme may include monitoring a condition of a first subsystem with a second subsystem, monitoring a condition of the second subsystem with a third subsystem, monitoring a condition of the third subsystem with a fourth subsystem, and so on. The displays can operate in a first video passthrough mode or a second video passthrough mode based on the condition of the first subsystem as monitored by the second subsystem, the condition of the second subsystem as monitored by the third subsystem, and/or the condition of the third subsystem as monitored by the fourth subsystem.

Abstract: Aspects of the subject technology may provide time-synchronized image frames from multiple cameras to various system and/or application processes running on an electronic device. In one or more implementations, a frame identifier may be determined for each image frame from each camera based on a system pulse associated with the capture of the image frame. By generating frame identifiers based for images from multiple cameras based on a centralized source such as the system pulses, subsequent processes can immediately identify images from multiple cameras having the same frame identifier for co-processing of those images.

Abstract: An electronic device may include software and hardware subsystems that are centrally controlled by a user experience manager. The user experience manager can identify a current user context and enforce a corresponding power and performance policy for the various subsystems that is optimized for the current user context. The user experience manager can provide a set of restrictions at the launch of a user experience, can allow the various subsystems to vary their dynamic behavior based on current operating conditions as long as the dynamic adjustments do not violate the restrictions, and can perform a series of thermal mitigation operations as the internal temperature of the electronic device varies. The centralized user experience manager can also be configured to predict a user context based on monitored states of the subsystems and monitored application usage on the electronic device.

Abstract: Generating synthesized data includes capturing one or more frames of a scene at a first frame rate by one or more cameras of a wearable device, determining body position parameters for the frames, and obtaining geometry data for the scene in accordance with the one or more frames. The frames, body position parameters, and geometry data are applied to a trained network which predicts one or more additional frames. With respect to virtual data, generating a synthesized frame includes determining current body position parameters in accordance with the one or more frames, predicting a future gaze position based on the current body position parameters, and rendering, at a first resolution, a gaze region of a frame in accordance with the future gaze position. A peripheral region is predicted for the frame at a second resolution, and the combined regions form a frame that is used to drive a display.

Abstract: A head-mounted device is provided that includes one or more cameras configured to acquire a raw video feed and one or more displays configured to present a passthrough video feed to a user. Generation of the passthrough video feed can involve processing the raw video feed using an image signal processor and auxiliary compute blocks. One or more of the auxiliary compute blocks can be bypassed in response to detecting one or more failures associated with the auxiliary compute blocks. Configured and operated in this way, the head-mounted device can fall back to a more reliable passthrough video feed without having to power cycle the head-mounted device when a failure occurs.

Abstract: A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: A method is provided that includes determining a gaze position of a user relative to mixed-reality content displayed in a first frame, setting a binning mode for a first camera based on the determined gaze position, and capturing, using the first camera, passthrough content for a second frame at a resolution determined by the binning mode.

Abstract: A request to transition a computing system from a first state to a second state is received, and a respective manifest is compiled for each of a plurality of processors of the computing system. Each manifest comprises a transition identifier representing a command to transition from the first state to the second state and an action time for executing one or more operations associated with the transition identifier. The respective manifests are dispatched to the plurality of processors, and status reports are received from the plurality of processors regarding the transition from the first state to the second state.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: In some implementations, a mobile device can be configured with virtual motion fences that delineate domains of motion detectable by the mobile device. In some implementations, the mobile device can be configured to invoke an application or function when the mobile device enters or exits a motion domain (by crossing a motion fence). In some implementations, entering or exiting a motion domain can cause components of the mobile device to power on or off (or awaken or sleep) in an incremental manner.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: In some implementations, a mobile device can be configured with virtual motion fences that delineate domains of motion detectable by the mobile device. In some implementations, the mobile device can be configured to invoke an application or function when the mobile device enters or exits a motion domain (by crossing a motion fence). In some implementations, entering or exiting a motion domain can cause components of the mobile device to power on or off (or awaken or sleep) in an incremental manner.