Abstract: A method of operating an electronic device such as a head-mounted device to mitigate judder, double images, ghosting, and other display artifacts is provided. The method can include acquiring images of a scene with one or more cameras, outputting the acquired images of the scene with one or more displays, sensing light in the scene with a flicker sensor, obtaining a frequency and a phase of the sensed light, and setting or locking a frame rate of the cameras based on the frequency of the sensed light. The method can also include shifting or locking exposure periods of the cameras based on the phase of the sensed light. Determination of whether judder is present can involve computing a display artifact severity parameter and comparing that parameter to a threshold.

Abstract: A method of operating an electronic device to mask noise at low scene brightness levels is provided. The method can include acquiring images of a scene and determining a scene brightness level of the acquired images. In response to determining that the scene brightness level is in a first range, the acquired images can be displayed as passthrough content in accordance with a 1:1 nits-to-nits mapping where the perceived brightness of the passthrough content matches the scene brightness level. In response to determining that the scene brightness is less than the first range, the perceived brightness of the passthrough content can be dimmed to be less than the scene brightness level. The method can also include generating virtual content. The passthrough content can be adjusted in accordance with a first scene-brightness-to-display-brightness profile, whereas the virtual content can be adjusted in accordance with a second scene-brightness-to-display-brightness profile.

Abstract: Various implementations provide passthrough video based on adjusting camera parameters based on environment modeling. An environment characteristic may be determined based on modeling the physical environment based on sensor data captured via one or more sensors. For example, this may involve determining environment light source optical characteristics, environment surfaces optical characteristics, a 3D mapping of the environment, user behavior, a prediction of optical characteristics of light coming in the camera, and the like. The method may involve, based on the environment characteristic, determining a camera parameter for an image captured via the image sensor. For example, the method may determine exposure, gain, tone mapping, color balance, noise reduction, sharpness enhancement. The method may determine the camera parameter based on user information, e.g., user preferences, user activity, etc.

Abstract: Fiducial patterns that produce diffraction patterns at a camera sensor are etched or otherwise provided on the surface of a cover glass (CG) in front of a camera. An active light source injects light into the cover glass or into a diffractive optical element to strengthen the signal from the fiducial pattern, or alternatively an active light source is used to reflect light off a reflective fiducial pattern on the cover glass, thus requiring fewer frames to capture and process the diffraction pattern caused by the fiducial pattern.

Abstract: In one implementation, a method of pipelined blending an image with virtual content is performed by a device including an image sensor, a display, one or more processors, and non-transitory memory. The method includes capturing, with the image sensor, a first portion of an image of a physical environment. The method includes warping the first portion of the image of the physical environment to generate a warped first portion. The method includes blending the warped first portion with a first portion of virtual content to generate a blended first portion. The method includes displaying, on the display, the blended first portion. The method includes capturing, with the image sensor, a second portion of the image of the physical environment. The method includes warping the second portion of the image of the physical environment to generate a warped second portion. The method includes blending the warped second portion with a second portion of the virtual content to generate a blended second portion.

Abstract: A head-mounted device is provided that includes one or more image sensors configured to capture a video feed, one or more motion sensors configured to detect motion, and control circuitry configured to analyze a lighting condition of the captured video feed and to perform auto white balancing operations on the captured video feed. An update frequency or a color adaptation speed of the auto white balancing operations can be determined based on the lighting condition and the detected motion. The color adaptation speed of the auto white balancing operations can be adjusted only in response to detecting, using the one or more motion sensors, an amount of motion exceeding a threshold.

Abstract: Various implementations disclosed herein improve the appearance of captured video by accounting for motion-induced blur, noise, lighting, and/or other factors related to the appearance of video captured by rotating or otherwise moving wearable electronic device. Some implementations reduce or eliminate camera motion blur during video capture by dynamically adjusting the exposure used to capture the video based on user motion. For example, exposure may be reduced to reduce the amount of blur during motion. Since such a reduced exposure captures a lesser amount of light, gain may be increased such that the captured video maintains a relatively constant image brightness in spite of the changing exposure over time.

Abstract: Circuits, methods, and apparatus that can transfer data and power between an optical-communication device and an electronic device without the need for electrical contacts on the electronic device. An example can include infrared transmitters and receivers in both the optical-communication device and the electronic device for optical communication. This can further include a coil in the optical-communication device that can be powered to inductively induce currents in a corresponding coil in the electronic device to charge a battery. One or more attachment components, such as magnets or magnetic elements, can be used to secure the optical-communication device to the electronic device. The optical-communication device to the electronic device can each include an alignment magnet or magnetic element.