Abstract: In particular embodiments, a computing system may receive a color image captured by a color camera and a monochrome image captured by a monochrome camera. The color camera and the monochrome camera are associated with an artificial reality system. The computing system may compute, for each of the color and monochrome images, histogram statistics and perform, based on the histogram statistics, tone map matching to normalize the monochrome image with respect to the color image. The computing system may perform local motion estimation to calculate motion vectors indicating pixel correspondence between the normalized monochrome image and the color image. The computing system may generate a mono-color merged image for display on the artificial reality system by adding, for each pixel in the normalized monochrome image, color information extracted from corresponding pixel in the color image using the motion vectors.

Abstract: The disclosure describes artificial reality systems and techniques for providing artificial reality content to a user. For example, an artificial reality system includes a head-mounted display (HMD) configured to output artificial reality content, the HMD including a set of second image capture devices configured to capture image data indicative of a focal point of a gaze of the user and a varifocal display having a focal length that is modifiable based on the focal point of the user. Additionally, the system includes a depth engine configured to generate, based on the real-world image data and depth data associated with the real-world image data, a three-dimensional (3D) scene of the physical environment of the user and generate artificial reality content as an overlay to the 3D scene of the physical environment for display on the varifocal display of the HMD based on the focal point of the user.

Abstract: A method for managing a user interface in a smart glass is provided. The method includes identifying an eye gesture of the user of the smart glass, based on an eye-tracking signal indicative of a pupil location of the user, assessing a user intention for an application displayed in a user interface in the headset based on the eye gesture, and activating the application based on the user intention. A smart glass for use with the above method is also provided.

Abstract: Eye data is captured with one or more sensors of the head mounted device. The one or more sensors are configured to sense an eyebox region. User notifications are controlled based on the eye data.

Abstract: In one aspect, the systems and methods described herein may include a first device comprising a first ultra-wideband (UWB) antenna. The first device may establish a connection with a second device having a second UWB antenna. The first device may determine an orientation of the first device relative to the second device, according to one or more UWB measurements between the first UWB antenna of the first device and the second UWB antenna of the second device. The first device may receive audio signals corresponding to the second device, and can render the audio signals into audio output to a user of the first device according to the determined orientation.

Abstract: Methods and systems for adjusting device functions based on ambient conditions or battery status are disclosed. When environmental or device conditions reach a threshold level, device function such as display brightness or display refresh rate may be adjusted. One such example, may include a method and system for image fusion with a wide camera and a second image from a narrow camera to create a composite image with unnoticeable blending between the images.

Abstract: A method of operating a head mounted device includes capturing eye data with one or more sensors of the head mounted device. The one or more sensors are configured to sense an eyebox region. Operations of the head mounted device are adjusted in response to the eye data.

Abstract: A method of operating a head mounted device includes capturing eye data with one or more sensors of the head mounted device. The one or more sensors are configured to sense an eyebox region. The eye data may include images. Operations of the head mounted device are adjusted in response to the eye data.

Abstract: A method for managing a user interface in a smart glass is provided. The method includes identifying an eye gesture of the user of the smart glass, based on an eye-tracking signal indicative of a pupil location of the user, assessing a user intention for an application displayed in a user interface in the headset based on the eye gesture, and activating the application based on the user intention. A smart glass for use with the above method is also provided.

Abstract: A device is provided, including a frame that supports two eyepieces, a capacitive sensor mounted on the frame, an inertial measurement unit mounted on the frame, and a circuit component inside the frame, wherein the circuit component electrically couples the capacitive sensor and the inertial measurement unit with a processor and a memory inside the frame. A method for using the above device is also provided.

Abstract: A gaze direction of a user is determined by an eye-tracking system of a head mounted device. Audio data generated by at least one microphone is captured. Gaze-guided audio is generated from the audio data based on the gaze direction of the user.

Abstract: In one aspect, the systems and methods described herein may include a first device comprising a first ultra-wideband (UWB) antenna. The first device may establish a connection with a second device having a second UWB antenna. The first device may determine an orientation of the first device relative to the second device, according to one or more UWB measurements between the first UWB antenna of the first device and the second UWB antenna of the second device. The first device may receive audio signals corresponding to the second device, and can render the audio signals into audio output to a user of the first device according to the determined orientation.

Abstract: A disclosed computer-implemented method may include directing a display device included in a head-mounted display worn by a user to illuminate, via a projection of a line at an illumination time, a portion of a cornea of the user. The method may further include detecting, via an image sensor at a detection time, a portion of the projection of the line reflected by the portion of the cornea of the user and identifying a distortion of the projection of the line reflected by the portion of the cornea of the user. The method may also include determining a shape of the cornea of the user based on the illumination time, the detection time, and the distortion of the projection of the line reflected by the portion of the cornea. Various other methods, systems, apparatuses, and computer-readable media are also disclosed.

Abstract: A device is provided, including a frame, providing mechanical support to at least two eyepieces, a capacitive sensor mounted on the frame, an inertial measurement sensor mounted on the frame, and a flexible circuit component inside the frame and electrically coupling the capacitive sensor and the inertial measurement sensor with a processor circuit and a memory circuit inside the frame. A method for using the above device is also provided.

Abstract: A client device of a user (e.g., a headset) provides a software-based user interface, not relying solely on physical buttons built into the client device itself. The client device's user interface system can render the user interface using graphical, virtual user interface elements that take the place of physical buttons, reducing device manufacturing costs, improving user interface reliability, and allowing greater user interface flexibility. Information on user movements can then be obtained using a separate wearable device, and the client device can use such information to determine whether the user's movements indicate interactions with the virtual user interface elements, taking appropriate actions if so. The client device's user interface system can also render the user interface using sounds made to appear to emanate from particular locations. Subsequent user movements can then be analyzed to determine whether they indicate the particular locations corresponding to the sound-based elements.

Abstract: A smart glass including photoplethysmography and electrocardiogram sensors to determine a health condition of the user is provided. The smart glass includes a frame for holding two eyepieces, the frame having two nose pads to rest on a user's nose, and two arms to rest on two user's ears, a sensor mounted on at least one of the nose pads or the arms, and configured to collect an optical signal from a user's blood vessel, and a processor configured to obtain a waveform from the optical signal or the electrical signal, and to determine a cardiovascular parameter based on the waveform.

Abstract: Various aspects of the subject technology relate to prediction of eye movements of a user of a head-mountable display device. Predictive foveated display systems and methods, using the predicted eye movements are also disclosed. Predictive variable focus display systems and methods using the predicted eye movements are also disclosed. Predicting eye movements may include predicting a future gaze location and/or predicting a future vergence plane for the user's eyes, based on the current motion of one or both of the user's eyes. The predicted gaze location may be used to pre-render a foveated display image frame with a high-resolution region at the predicted gaze location. The predicted vergence plane may be used to modify an image plane of a display assembly to mitigate or avoid a vergence/accommodation conflict for the user.

Abstract: The disclosed eye-tracking systems may include a visible light display having visible light pixels, at least one infrared pixel positioned within bounds of the visible light display, and an infrared sensor positioned and configured to detect infrared light originating from the at least one infrared pixel and reflected from an eye of the user. Related head-mounted display systems and methods for eye tracking are also disclosed.

Abstract: A headset for use with immersive reality applications is provided. The headset includes a left eyepiece and a right eyepiece mounted on a frame, a display in at least one of the left eyepiece or the right eyepiece, the display comprising an array of multiple light emitting pixels, an ambient light sensor to measure an amount of ambient light, and a processor configured to control a light intensity of the light emitting pixels based on the amount of ambient light. A method for using the above headset is also provided.

Abstract: A method for managing a user interface in a smart glass is provided. The method includes identifying an eye gesture of the user of the smart glass, based on an eye-tracking signal indicative of a pupil location of the user, assessing a user intention for an application displayed in a user interface in the headset based on the eye gesture, and activating the application based on the user intention. A smart glass for use with the above method is also provided.