Abstract: Frame capture techniques may include obtaining, from at least one sensor, sensor data indicating a motion rate of the system in the environment, receiving a request for the image data of the environment, and selecting, by the controller, a frame rate based on one or more requesting client applications and the sensor data. In addition, the frame tracking techniques may include detecting, based on a motion parameter in the sensor data and by the controller, that a motion of the system falls below a predetermined threshold; and triggering frame capture by the camera in accordance with the sensor data and the selected frame rate, the frame capture being triggered in accordance with the request and the detection.

Abstract: Methods and apparatus for camera focusing for video passthrough devices. Gaze information from a gaze tracking subsystem, either alone or along with depth information from a depth tracking system, may be leveraged to determine depths at which to focus. Gaze information, or a combination of depth and gaze information, may be used. As an alternative, the user can manually control the focus distance. For example, a manual bifocal method may provide two focus distances (near focus and far focus.

Abstract: The present disclosure generally relates to updating user interfaces based on user presence and/or outputting information about detected real-world object.

Abstract: A spatial light system that may, for example, be used to project image content onto surfaces of a room. The system may include two or more projection units for emitting light onto surfaces within a room. The system may include a controller and two or more projection units, each module including an LED array, an array of light pipes, and a condenser lens. Two or more projection units may be connected to a flexible strip that provides power and data (e.g., serial data) to the projection units. Two or more flex strips may be connected together. The flex strips provide a serially-connected, flexible modular architecture for spatial light systems that allow the projection units to be conformed to a variety of configurations and shapes.

Abstract: A spatial light system that may, for example, be used to project image content onto surfaces of a room. The system may include two or more projection units for emitting light onto surfaces within a room. The system may include a controller and two or more projection units, each module including an LED array, an array of light pipes, and a condenser lens. Two or more projection units may be connected to a flexible strip that provides power and data (e.g., serial data) to the projection units. Two or more flex strips may be connected together. The flex strips provide a serially-connected, flexible modular architecture for spatial light systems that allow the projection units to be conformed to a variety of configurations and shapes.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.

Abstract: In one implementation, an apparatus includes: a display to emit light in a first wavelength range; one or more light sources to emit light in a second wavelength range; a camera to detect the light in the second wavelength range; and an eyepiece to reflect and refract the light in the first wavelength range while passing, without substantial distortion, the light in the second wavelength range, wherein the eyepiece includes two lens halves separated by a retarder that changes light in the first wavelength range from a first polarization to a second polarization different from the first polarization.

Abstract: Methods and apparatus for generating images to be displayed on lenticular displays. In these methods, a fixed mesh is generated offline, and in real-time texture information is mapped to the fixed mesh. In an offline process, texture and 3D mesh information for an object is used to render UV map views for multiple viewpoints of the object, view maps are generated from display calibration data, and a lenticular to UV map is generated from the UV map views and view maps. In real-time, texture information is captured, and a composite process is performed that generates a lenticular image for multiple viewpoints by sampling pixels from the texture based on the lenticular to UV map. The lenticular image is then displayed on the lenticular display. Detected positions of persons in the environment may be used to limit the number of viewpoints that are generated during the real-time composite process.

Abstract: A portable electronic device is provided. The portable electronic device includes a camera module having an optical axis. The portable electronic device also includes a display panel. The portable electronic device further includes a display printed circuit board (PCB) including a cutout centered around the optical axis. In addition, the portable electronic device includes a mounting bracket fixedly coupling the camera module to the display PCB. The portable electronic device includes a housing configured to retain the camera module, the display panel, the display PCB, and the mounting bracket therein. In some aspects, the mounting bracket may be fixedly attached to an interior surface within the portable electronic device and configured to maintain a position of the camera module.

Abstract: Embodiments disclosed herein are directed to devices and methods for performing spatially-varying brightness correction of a flash image. The spatially-varying brightness correction utilizes a set of images that includes one or more images captured under flash illumination and one or more images captured without flash illumination, which are used to decompose one of the flash images or an image generated therefrom into a flash contribution image and an ambient contribution image. The flash contribution image and the ambient contribution image are recombined to generate a corrected flash image, and information about the flash module and/or scene is used to adjust the relative contributions of these images during recombination.

Abstract: An electronic device is provided. The electronic device includes a camera module having an optical axis and a set of one or more lenses centered along the optical axis. The electronic device also includes a display panel including an enhanced light transmission region. The enhanced light transmission region permits a greater amount of light to pass therethrough compared to at least one other region of the display panel. The electronic device further includes a display printed circuit board (PCB). In addition, the electronic device includes an aperture stop positioned between the set of one or more lenses and the display panel. The electronic device includes a transparent cover positioned over the display panel on an opposite side of the display panel from the display PCB. The electronic device also includes a housing configured to retain the camera module, the display panel, the display PCB, and the aperture stop therein.

Abstract: An image sensing device includes a detector assembly, which includes a matrix of optical sensing elements having a predefined pitch. Each optical sensing element includes an active area having a width that is less than 90% of the pitch. An array of optical apertures are respectively aligned with the optical sensing elements such that each optical aperture is positioned at a distance from a respective optical sensing element that is no less than twice the width of the active area. Objective optics are configured to focus light from a scene onto the detector assembly.

Abstract: This disclosure describes a novel lens system having an entrance pupil at a location matching, or nearly matching, the eye's optical system entrance pupil. The described lens system is significantly thinner than a single-aperture lens with matched entrance pupil and is achieved by using an array of camera elements. Each camera element contains camera optics called “lenslets” and a sensor (e.g., an image or depth sensor), where a lenslet can include one or more lens elements (e.g., a compound lens). Individual camera elements can be arranged such that THEIR optical axes intersect at, or near, the eye's entrance pupil location. In addition, each camera element's field-of-view (FOV) corresponds to a small sector of a large FOV. The FOVs of adjacent camera elements can be non- or slightly-overlapping so that a wide-angle image can be formed by concatenation of the individual camera element images.

Abstract: An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system includes at least one eye tracking camera, an illumination source that emits infrared light towards the user's eyes, and diffraction gratings located at the eyepieces. The diffraction gratings redirect or reflect at least a portion of infrared light reflected off the user's eyes, while allowing visible light to pass. The cameras capture images of the user's eyes from the infrared light that is redirected or reflected by the diffraction gratings.

Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user's eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

Abstract: An eye tracking system for detecting position and movements of a user's eyes in a head-mounted display (HMD). The eye tracking system includes at least one eye tracking camera, an illumination source that emits infrared light towards the user's eyes, and diffraction gratings located at the eyepieces. The diffraction gratings redirect or reflect at least a portion of infrared light reflected off the user's eyes, while allowing visible light to pass. The cameras capture images of the user's eyes from the infrared light that is redirected or reflected by the diffraction gratings.

Abstract: A device for MR/VR systems that includes a two-dimensional array of cameras that capture images of respective portions of a scene. The cameras are positioned along a spherical surface so that the cameras have adjacent fields of view. The entrance pupils of the cameras are positioned at or near the user’s eye while the cameras also form optimized images at the sensor. Methods for reducing the number of cameras in an array, as well as methods for reducing the number of pixels read from the array and processed by the pipeline, are also described.

Abstract: A light source module includes an array of illumination elements and an optional projecting lens. The light source module is configured to receive or generate a control signal for adjusting different ones of the illumination elements to control a light field emitted from the light source module. In some embodiments, the light source module is also configured to adjust the projecting lens responsive to objects in an illuminated scene and a field of view of an imaging device. A controller for a light source module may determine a light field pattern based on various parameters including a field of view of an imaging device, an illumination sensitivity model of the imaging device, depth, ambient illumination and reflectivity of objects, configured illumination priorities including ambient preservation, background illumination and direct/indirect lighting balance, and so forth.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.

Abstract: A device such as a stylus may have a color sensor. The color sensor may have a color sensing light detector having a plurality of photodetectors each of which measures light for a different respective color channel. The color sensor may also have a light emitter. The light emitter may have an adjustable light spectrum. The light spectrum may be adjusted during color sensing measurements using information such as ambient light color measurements made with a color ambient light sensor that has a plurality of photodetectors each of which measures light for a different respective color channel. An inertial measurement unit may be used to measure the angular orientation between the stylus and an external object during color measurements. Arrangements in which the light emitter is modulated during color sensing may also be used. Measurements from the stylus may be transmitted wirelessly to external equipment.