Abstract: Various implementations disclosed herein include devices, systems, and methods that mitigate eye tracking loss. For example, a process may capture sensor data of a 3D region. The process may further generate eye tracking data comprising a 3D position of an eye based on tracking the 3D position of the eye. The 3D position may be updated over time based on the captured sensor data. The process may further detect a condition interfering with updating the 3D position of the eye based on the captured sensor data. The process may further update the eye tracking data using a smoothed 3D position of the eye. The smoothed 3D position may be determined based on previously-tracked 3D positions of the portion of the eye and a current 3D position of the portion of the eye determined based on current sensor data.

Abstract: An audio renderer can have a machine learning model that jointly processes audio and visual information of an audiovisual recording. The audio renderer can generate output audio channels. Sounds captured in the audiovisual recording and present in the output audio channels are spatially mapped based on the joint processing of the audio and visual information by the machine learning model. Other aspects are described.

Abstract: In one implementation, a method of generating an image is performed by a device including one or more processors and non-transitory memory. The method includes generating a first resolution function based on a formula with a set of variables having a first set of values. The method includes generating a first image based on first content and the first resolution function. The method includes detecting a resolution constraint. The method includes generating a second resolution function based on the formula with the set of variables having a second set of values, wherein the second resolution function has a summation value that satisfies the resolution constraint. The method includes generating a second image based on second content and the second resolution function.

Abstract: A method and device for modifying virtual content according to various simulation characteristics includes obtaining first virtual content; obtaining one or more simulation characteristics; and generating second virtual content by modifying the first virtual content according to the one or more simulation characteristics; and presenting the second virtual content.

Abstract: Various implementations disclosed herein include devices, systems, and methods for presenting content based on environmental data. In various implementations, a device includes a display, a non-transitory memory and one or more processors coupled with the display and the non-transitory memory. In some implementations, a method includes obtaining environmental data associated with a physical environment in which the device is located. In some implementations, the method includes selecting, from a plurality of presentation modes, a first presentation mode for content based on the environmental data. In some implementations, the method includes presenting the content in accordance with the first presentation mode.

Abstract: Eye tracking is performed by determining an initial pupil position of a user in relation to a lens situated in front of the user, detecting a change in pupil position in relation to the lens to an updated pupil position in relation to the lens, and determining that the updated pupil position is outside a bounding box associated with the lens. The updated pupil position is a replacement pupil position with a replacement pupil position within the bounding box associated with the lens, and the updated pupil position is utilized for eye-tracking functionality. Eye tracking is also performed by determining that a first pixel associated with a gaze direction is outside a visibility region, identifying a replacement pixel within the visibility region, determining an updated gaze angle based on the replacement pixel, and performing eye tracking using the updated gaze angle.

Abstract: In various implementations, a method of presenting video streams at a head-mountable device (HMD) includes generating a first video stream at a first frame rate for a first display portion. In some implementations, the first frame rate indicates a rate at which frames are presented by the first display portion. In various implementations, the method includes generating a second video stream at a second frame rate for a second display portion. In some implementations, the second frame rate indicates a rate at which frames are presented by the second display portion. In some implementations, the second frame rate is within a threshold relative to the first frame rate. In various implementations, the method includes temporally shifting the second video stream relative to the first video stream so that a majority of refresh times of the first display portion are different from refresh times of the second display portion.

Abstract: In one implementation, a method of displaying an animation is performed at a device including an optical see-through display, one or more processors, and a non-transitory memory. The method includes receiving a request to display a first animation of an object exhibiting a response characteristic. The method includes determining a metric characterizing an amount of processing power for the device to display the first animation on the optical see-through display. The method includes, in response to a determination that the metric exceeds a threshold associated with the device, selecting a second animation of the object exhibiting the response characteristic. The method includes displaying the second animation.

Abstract: An audio renderer can have a machine learning model that jointly processes audio and visual information of an audiovisual recording. The audio renderer can generate output audio channels. Sounds captured in the audiovisual recording and present in the output audio channels are spatially mapped based on the joint processing of the audio and visual information by the machine learning model. Other aspects are described.