Abstract: Generating synthesized data includes capturing one or more frames of a scene at a first frame rate by one or more cameras of a wearable device, determining body position parameters for the frames, and obtaining geometry data for the scene in accordance with the one or more frames. The frames, body position parameters, and geometry data are applied to a trained network which predicts one or more additional frames. With respect to virtual data, generating a synthesized frame includes determining current body position parameters in accordance with the one or more frames, predicting a future gaze position based on the current body position parameters, and rendering, at a first resolution, a gaze region of a frame in accordance with the future gaze position. A peripheral region is predicted for the frame at a second resolution, and the combined regions form a frame that is used to drive a display.

Abstract: A head-mounted device is provided that includes one or more cameras configured to acquire a raw video feed and one or more displays configured to present a passthrough video feed to a user. Generation of the passthrough video feed can involve processing the raw video feed using an image signal processor and auxiliary compute blocks. One or more of the auxiliary compute blocks can be bypassed in response to detecting one or more failures associated with the auxiliary compute blocks. Configured and operated in this way, the head-mounted device can fall back to a more reliable passthrough video feed without having to power cycle the head-mounted device when a failure occurs.

Abstract: A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: A method is provided that includes determining a gaze position of a user relative to mixed-reality content displayed in a first frame, setting a binning mode for a first camera based on the determined gaze position, and capturing, using the first camera, passthrough content for a second frame at a resolution determined by the binning mode.

Abstract: A request to transition a computing system from a first state to a second state is received, and a respective manifest is compiled for each of a plurality of processors of the computing system. Each manifest comprises a transition identifier representing a command to transition from the first state to the second state and an action time for executing one or more operations associated with the transition identifier. The respective manifests are dispatched to the plurality of processors, and status reports are received from the plurality of processors regarding the transition from the first state to the second state.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: A mobile device includes a display, at least one sensor, and a wireless transceiver. The mobile device also includes control circuitry coupled to the display, the at least one sensor, and the wireless transceiver. The control circuitry is configured to obtain content primitives from the at least one sensor, to perform content provisioning operations to obtain content based at least in part on the content primitives, and to display the obtained content on the display, wherein at least some of the content is virtual content. In response to a bandwidth condition of the wireless communication channel being less than a threshold, the control circuitry is configured to perform adjusted content provisioning operations that involve increasing an amount of image processing operations performed by the mobile device to obtain the content.

Abstract: One implementation forms a composited stream of computer-generated reality (CGR) content using multiple data streams related to a CGR experience to facilitate recording or streaming. A media compositor obtains a first data stream of rendered frames and a second data stream of additional data. The rendered frame content (e.g., 3D models) represents real and virtual content rendered during a CGR experience at a plurality of instants in time. The additional data of the second data stream relates to the CGR experience, for example, relating to audio, audio sources, metadata identifying detected attributes of the CGR experience, image data, data from other devices involved in the CGR experience, etc. The media compositor forms a composited stream that aligns the rendered frame content with the additional data for the plurality of instants in time, for example, by forming time-stamped, n-dimensional datasets (e.g., images) corresponding to individual instants in time.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: In some implementations, a mobile device can be configured with virtual motion fences that delineate domains of motion detectable by the mobile device. In some implementations, the mobile device can be configured to invoke an application or function when the mobile device enters or exits a motion domain (by crossing a motion fence). In some implementations, entering or exiting a motion domain can cause components of the mobile device to power on or off (or awaken or sleep) in an incremental manner.

Abstract: An electronic display includes a display side and an ambient light sensor configured to measure received light received through the display side. The electronic display also includes multiple pixels located between the display side and the ambient light sensor. The multiple pixels are configured to emit display light through the display side.

Abstract: In some implementations, a mobile device can be configured with virtual motion fences that delineate domains of motion detectable by the mobile device. In some implementations, the mobile device can be configured to invoke an application or function when the mobile device enters or exits a motion domain (by crossing a motion fence). In some implementations, entering or exiting a motion domain can cause components of the mobile device to power on or off (or awaken or sleep) in an incremental manner.

Abstract: Automatic display brightness adjustments may be made by an electronic device based on ambient light sensor data. Proximity sensor data from a light-based proximity sensor, from nearby capacitive sensor electrodes in a touch screen, or from other proximity sensing components may be used to determine whether the ambient light sensor is being shadowed by a hand or other external object. Ambient light sensor data associated with blocked sensor conditions can be suppressed. A transient event filter may be used to remove spikes from ambient light sensor data. A display brightness baseline may be adaptively adjusted. Short changes in ambient light level may result in corresponding momentary adjustments to display brightness. Longer changes in ambient light level may be associated with persistent changes in the display brightness baseline.

Abstract: In some implementations, a mobile device can be configured with virtual motion fences that delineate domains of motion detectable by the mobile device. In some implementations, the mobile device can be configured to invoke an application or function when the mobile device enters or exits a motion domain (by crossing a motion fence). In some implementations, entering or exiting a motion domain can cause components of the mobile device to power on or off (or awaken or sleep) in an incremental manner.

Abstract: Automatic display brightness adjustments may be made by an electronic device based on ambient light sensor data. Proximity sensor data from a light-based proximity sensor, from nearby capacitive sensor electrodes in a touch screen, or from other proximity sensing components may be used to determine whether the ambient light sensor is being shadowed by a hand or other external object. Ambient light sensor data associated with blocked sensor conditions can be suppressed. A transient event filter may be used to remove spikes from ambient light sensor data. A display brightness baseline may be adaptively adjusted. Short changes in ambient light level may result in corresponding momentary adjustments to display brightness. Longer changes in ambient light level may be associated with persistent changes in the display brightness baseline.