Abstract: A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.

Abstract: Systems and methods are provided for computing gaze. The systems and methods access an uncorrected gaze vector computed based on a center point of a pupil of a user. The systems and methods process an image of the pupil by a machine learning model to predict an estimated error in a gaze vector, the machine learning model trained to establish a relationship between a plurality of ground truth gaze vectors and uncorrected gaze vectors for a plurality of pupil parameters, the pupil parameters including diameters, gaze angles, or gaze eccentricities. The systems and methods generate a corrected gaze vector by applying the estimated error in the gaze vector predicted by the machine learning model to the uncorrected gaze vector that has been computed based on the center point of the pupil of the user.

Abstract: A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.

Abstract: An extended Reality (XR) display system includes a Light Emitting Diode (LED) display controller, and a Light Emitting Diode (LED) near-eye display element operatively coupled to the LED display driver. The LED near-eye display element includes one or more motors and an LED array operably connected to the one or more motors. During operation, the LED display driver receives video data including a rendered virtual object of an XR experience and generates LED array control signals based on the video data, the LED array control signals causing one or more LEDs of the LED array to be energized in a sequence. The LED display driver also generates synchronized motor control signals and simultaneously communicates the LED array control signals to the LED array and the synchronized motor control signals to the one or more motors causing the LED near-eye display element to display the rendered virtual object.

Abstract: A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.

Abstract: An eXtended Reality (XR) display system includes a Light Emitting Diode (LED) display controller, and a Light Emitting Diode (LED) near-eye display element operatively coupled to the LED display driver. The LED near-eye display element includes one or more motors and an LED array operably connected to the one or more motors. During operation, the LED display driver receives video data including a rendered virtual object of an XR experience and generates LED array control signals based on the video data, the LED array control signals causing one or more LEDs of the LED array to be energized in a sequence. The LED display driver also generates synchronized motor control signals and simultaneously communicates the LED array control signals to the LED array and the synchronized motor control signals to the one or more motors causing the LED near-eye display element to display the rendered virtual object.

Abstract: An energy-efficient adaptive 3D sensing system. The adaptive 3D sensing system includes one or more cameras and one or more projectors. The adaptive 3D sensing system captures images of a real-world scene using the one or more cameras and computes depth estimates and depth estimate confidence values for pixels of the images. The adaptive 3D sensing system computes an attention mask based on the one or more depth estimate confidence values and commands the one or more projectors to send a distributed laser beam into one or more areas of the real-world scene based on the attention mask. The adaptive 3D sensing system captures 3D sensing image data of the one or more areas of the real-world scene and generates 3D sensing data for the real-world scene based on the 3D sensing image data.

Abstract: A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.

Abstract: An energy-efficient adaptive 3D sensing system. The adaptive 3D sensing system includes one or more cameras and one or more projectors. The adaptive 3D sensing system captures images of a real-world scene using the one or more cameras and computes depth estimates and depth estimate confidence values for pixels of the images. The adaptive 3D sensing system computes an attention mask based on the one or more depth estimate confidence values and commands the one or more projectors to send a distributed laser beam into one or more areas of the real-world scene based on the attention mask. The adaptive 3D sensing system captures 3D sensing image data of the one or more areas of the real-world scene and generates 3D sensing data for the real-world scene based on the 3D sensing image data.

Abstract: An extended Reality (XR) display system includes a Light Emitting Diode (LED) display controller, and a Light Emitting Diode (LED) near-eye display element operatively coupled to the LED display driver. The LED near-eye display element includes one or more motors and an LED array operably connected to the one or more motors. During operation, the LED display driver receives video data including a rendered virtual object of an XR experience and generates LED array control signals based on the video data, the LED array control signals causing one or more LEDs of the LED array to be energized in a sequence. The LED display driver also generates synchronized motor control signals and simultaneously communicates the LED array control signals to the LED array and the synchronized motor control signals to the one or more motors causing the LED near-eye display element to display the rendered virtual object.

Abstract: A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.

Abstract: An energy-efficient adaptive 3D sensing system. The adaptive 3D sensing system includes one or more cameras and one or more projectors. The adaptive 3D sensing system captures images of a real-world scene using the one or more cameras and computes depth estimates and depth estimate confidence values for pixels of the images. The adaptive 3D sensing system computes an attention mask based on the one or more depth estimate confidence values and commands the one or more projectors to send a distributed laser beam into one or more areas of the real-world scene based on the attention mask. The adaptive 3D sensing system captures 3D sensing image data of the one or more areas of the real-world scene and generates 3D sensing data for the real-world scene based on the 3D sensing image data.

Abstract: A system for deformation or bending correction in an Augmented Reality (AR) system. Sensors are positioned in a frame of a head-worn AR system to sense forces or pressure acting on the frame by temple pieces attached to the frame. The sensed forces or pressure are used in conjunction with a model of the frame to determine a corrected model of the frame. The corrected model is used to correct video data captured by the AR system and to correct a video virtual overlay that is provided to a user wearing the head-worn AR system.

Abstract: A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.

Abstract: A system for hand tracking for an Augmented Reality (AR) system. The AR system uses a camera of the AR system to capture tracking video frame data of a hand of a user of the AR system. The AR system generates a skeletal model based on the tracking video frame data and determines a location of the hand of the user based on the skeletal model. The AR system causes a steerable camera of the AR system to focus on the hand of the user.

Abstract: A miniaturized active dual pixel stereo system and method for close range depth extraction includes a projector adapted to project a locally distinct projected pattern onto an image of a scene and a dual pixel sensor including a dual pixel sensor array that generates respective displaced images of the scene. A three-dimensional image is generated from the displaced images of the scene by projecting the locally distinct projected pattern onto the image of the scene, capturing the respective displaced images of the scene using the dual pixel sensor, generating disparity images from the respective displaced images of the scene, determining depth to each pixel of the disparity images, and generating the three-dimensional image from the determined depth to each pixel. A three-dimensional image of a user's hands generated by the active dual pixel stereo system may be processed by gesture recognition software to provide an input to an electronic eyewear device.