Abstract: At least one sensor signal may be received from at least one motion sensor of a head-mounted device (HMD), the HMD being associated with a user interface (UI) displaying a UI selection element. A first value of the at least one sensor signal corresponding to a first position of the HMD and a second value of the at least one sensor signal corresponding to a second position of the HMD may be determined, and an initialization point within the UI may also be determined. Then, a relative displacement of the UI selection element may be determined, based on the initialization point, the first value, and the second value, and the UI selection element may be moved within the UI, based on the relative displacement.

Abstract: Smart glasses including a first audio device, a second audio device, a frame including a first portion, a second portion, and a third portion, the second portion and the third portion are moveable in relation to the first portion, the second portion including the first audio device and the third portion including the second audio device, and a processor configured to cause the first audio device to generate a signal, receive the signal via the second audio device, estimate a distance based on the received signal, and determine a configuration of the frame.

Abstract: Implementations relate to generation of segmentation masks for images in a zero-shot, unsupervised manner. Implementations also relate to generation of labels for the segmentation layers of the segmentation mask. Implementations use self-attention maps from a pass of the image through a generative image model to determine the segmentation mask and may use cross-attention maps generated when a prompt describing the image is provided with the image to the generative image model. Implementations aggregate maps from different resolutions to determine the mask and labels. The disclosed techniques enable accurate segmentation for any image without apriori training, facilitating applications in image processing, computer vision, extended reality applications, and robotics.

Abstract: Techniques of operating an AR system include determining hand gestures formed by a user based on a sequence of two-dimensional images through skin of the user's wrist acquired from a near-infrared camera. Specifically, an image capture device disposed on a band worn around a user's wrist includes a source of electromagnetic radiation, e.g., light-emitting diodes in the infrared (IR) wavelength band that emit the radiation into the user's wrist and an IR detector which produces the sequence of two-dimensional images of a region within a dermal layer in the user's wrist. From this sequence, gesture detection circuitry determines values of a biological flow metric, e.g., a change in perfusion index (PI) between frames of the sequence, based on a trained model that generates the metric from the sequence. Finally, the gesture detection circuitry maps the values of the biological flow metric to specific hand/finger movements that determine a gesture.

Abstract: Techniques include determining hand gestures formed by a user based on an electrical impedance tomograph of the wrist. For example, a user may be outfitted with a flexible wristband that fits snugly around the wrist and contains a plurality of electrodes, e.g., 32 electrodes. When a current is applied to a first subset of the electrodes, e.g., two of 32 electrodes, the electric field induced through at least one cross-section of the wrist will in turn induce a voltage across adjacent pairs of a second subset of the electrodes (e.g., the other 30 of 32 electrodes). From this current and induced voltage, one may use techniques of electrical impedance tomography (EIT) to determine the electrical impedance throughout the at least one cross-section of the wrist, e.g., in an electrical impedance tomograph. One may use a neural network to map the electrical impedance tomograph to a hand gesture.

Abstract: Techniques of operating an AR system include determining hand gestures formed by a user based on a sequence of two-dimensional images through skin of the user's wrist acquired from a near-infrared camera. Specifically, an image capture device disposed on a band worn around a user's wrist includes a source of electromagnetic radiation, e.g., light-emitting diodes in the infrared (IR) wavelength band that emit the radiation into the user's wrist and an IR detector which produces the sequence of two-dimensional images of a region within a dermal layer in the user's wrist. From this sequence, gesture detection circuitry determines values of a biological flow metric, e.g., a change in perfusion index (PI) between frames of the sequence, based on a trained model that generates the metric from the sequence. Finally, the gesture detection circuitry maps the values of the biological flow metric to specific hand/finger movements that determine a gesture.

Abstract: Techniques of operating an AR system include determining hand gestures formed by a user based on a sequence of two-dimensional images through skin of the user's wrist acquired from a near-infrared camera. Specifically, an image capture device disposed on a band worn around a user's wrist includes a source of electromagnetic radiation, e.g., light-emitting diodes in the infrared (IR) wavelength band that emit the radiation into the user's wrist and an IR detector which produces the sequence of two-dimensional images of a region within a dermal layer in the user's wrist. From this sequence, gesture detection circuitry determines values of a biological flow metric, e.g., a change in perfusion index (PI) between frames of the sequence, based on a trained model that generates the metric from the sequence. Finally, the gesture detection circuitry maps the values of the biological flow metric to specific hand/finger movements that determine a gesture.

Abstract: Smart glasses including a first audio device, a second audio device, a frame including a first portion, a second portion, and a third portion, the second portion and the third portion are moveable in relation to the first portion, the second portion including the first audio device and the third portion including the second audio device, and a processor configured to cause the first audio device to generate a signal, receive the signal via the second audio device, estimate a distance based on the received signal, and determine a configuration of the frame.

Abstract: In one embodiment, a computing system may receive, from a second computing system, video streams of a scene, the video streams including at least a first image and a second image that are simultaneously captured by a first camera and a second camera of the second computing system, respectively. The system may determine, using a sensor system, a viewpoint of a viewer with respect to a display region of a monoscopic display associated with the first computing system. The system may generate an output image of the scene by blending, according to blending proportions computed using the viewpoint of the viewer, corresponding portions of the first image and the second image. The system may display the output image in the display region of the monoscopic display.

Abstract: In one embodiment, a computing system may receive, from a second computing system, video streams of a scene, the video streams including at least a first image and a second image that are simultaneously captured by a first camera and a second camera of the second computing system, respectively. The system may determine, using a sensor system, a viewpoint of a viewer with respect to a display region of a monoscopic display associated with the first computing system. The system may generate an output image of the scene by blending, according to blending proportions computed using the viewpoint of the viewer, corresponding portions of the first image and the second image. The system may display the output image in the display region of the monoscopic display.

Abstract: The disclosed camera system may include a primary camera and a plurality of secondary cameras that each have a maximum horizontal FOV that is less than a maximum horizontal FOV of the primary camera. Two of the plurality of secondary cameras may be positioned such that their maximum horizontal FOVs overlap in an overlapped horizontal FOV and the overlapped horizontal FOV may be at least as large as a minimum horizontal FOV of the primary camera. The camera system may also include an image controller that simultaneously activates two or more of the primary camera and the plurality of secondary cameras when capturing images from a portion of an environment included within the overlapped horizontal FOV. Various other systems, devices, assemblies, and methods are also disclosed.

Abstract: A process executes at an electronic system. The process identifies device characteristics of an imaging device that includes signal emitters and signal detectors. The process illuminates a field of view by signals from the signal emitters according to a modulation signal generated by the imaging device. At each of the signal detectors, the process obtains a response signal, and samples the response signals to form a response vector. The process obtains a lookup table corresponding to the modulation signal and the device characteristics. The field of view is partitioned into a 3-dimensional plurality of voxels, and the lookup table specifies, for each voxel, expected signals received by the signal detectors when the voxel is filled and the signal emitters illuminate the field of view according to the modulation signal. The process compares the response vector to the lookup table to determine which voxels are filled.

Abstract: A method generates depth maps at a camera having illuminators, a lens assembly, an image sensing element, a processor, and memory. The illuminators operate in a first mode to provide illumination, the lens assembly focuses incident light on the image sensing element, the memory stores programs for execution by the processor, and the processor executes the programs to control operation of the camera. The method reconfigures the illuminators to operate in a second mode, where each of a plurality of subsets of the illuminators provides illumination of a scene separately. For each subset, the process activates the illuminators in the subset without activating illuminators not in the subset and receives reflected illumination from the scene incident on the lens assembly and focused onto the image sensing element. The measured light intensity values of the received reflected illumination at the image sensing element are transmitted to a remote server for processing.

Abstract: Systems and methods of using active infrared (AIR) sensors to map a room of a home or building and determine whether an external portal (e.g., window and/or door) of the room is open or closed are provided. In particular, the systems and methods include outputting infrared (IR) light from an IR light source of an active infrared (AIR) sensor, receiving reflected IR light with a light sensor, and determining, with a processor coupled to the light sensor, whether a window of a room is open according to the received reflected IR light.

Abstract: In an implementation of the disclosed subject matter, a device may emit a first emission sequence of infrared radiation at a subject, and capture a first reflected sequence of infrared radiation reflected from the subject. The first emission sequence may be compared to the first reflected sequence, and, based on the comparison, a sequence of variations may be determined. The sequence of variations may be compared to signal pattern stored in a sleep profile for the subject. The subject may be determined to have exhibited sleep behavior based on the comparison of the sequence of variations to the signal pattern stored in the sleep profile. In response to determining the subject has exhibited sleep behavior, the device may capture a second reflected sequence of radiation reflected from the subject. A breathing rate of the subject and/or a heart rate of the subject may be determined based on the second reflected sequence.

Abstract: A device may emit a first emission sequence of infrared radiation at a subject, and capture a first reflected sequence of infrared radiation reflected from the subject. The first emission sequence may be compared to the first reflected sequence, and, based on the comparison, a sequence of variations may be determined. The sequence of variations may be compared to a signal pattern stored in a sleep profile for the subject. The subject may be determined to have exhibited a sleep behavior based on the comparison of the sequence of variations to the signal pattern stored in the sleep profile. In response to determining that the subject has exhibited the sleep behavior, the device may capture a second reflected sequence of radiation reflected from the subject. A breathing rate of the subject and/or a heart rate of the subject may be determined based on the second reflected sequence.

Abstract: A process executes at an electronic system. The process identifies device characteristics of an imaging device that includes signal emitters and signal detectors. The process illuminates a field of view by signals from the signal emitters according to a modulation signal generated by the imaging device. At each of the signal detectors, the process obtains a response signal, and samples the response signals to form a response vector. The process obtains a lookup table corresponding to the modulation signal and the device characteristics. The field of view is partitioned into a 3-dimensional plurality of voxels, and the lookup table specifies, for each voxel, expected signals received by the signal detectors when the voxel is filled and the signal emitters illuminate the field of view according to the modulation signal. The process compares the response vector to the lookup table to determine which voxels are filled.

Abstract: A method generates depth maps at a camera having illuminators, a lens assembly, an image sensing element, a processor, and memory. The illuminators operate in a first mode to provide illumination, the lens assembly focuses incident light on the image sensing element, the memory stores programs for execution by the processor, and the processor executes the programs to control operation of the camera. The method reconfigures the illuminators to operate in a second mode, where each of a plurality of subsets of the illuminators provides illumination of a scene separately. For each subset, the process activates the illuminators in the subset without activating illuminators not in the subset and receives reflected illumination from the scene incident on the lens assembly and focused onto the image sensing element. The measured light intensity values of the received reflected illumination at the image sensing element are transmitted to a remote server for processing.

Abstract: Techniques of tracking a hand controller in a VR system involves detecting, by photodiodes in the hand controller, patterns of diffuse radiation generated by diffuse LEDs in the HMD. Such techniques can also include comparing the detected patterns to those simulated offline previously and represented in a lookup table (LUT). By looking up the detected patterns in the LUT, the VR system may determine the position and/or orientation of the hand controller with sub-millimeter accuracy. Some advantages of the improved techniques can be in the simplicity and/or low cost of the components without sacrificing accuracy in deriving the position and orientation of the hand controller.

Abstract: A method generates depth maps at a camera having illuminators, a lens assembly, an image sensing element, a processor, and memory. The illuminators operate in a first mode to provide illumination, the lens assembly focuses incident light on the image sensing element, the memory stores image data from the image sensing element, and the processor executes programs to control operation of the camera. The method reconfigures the illuminators to operate in a second mode, where each of a plurality of subsets of the illuminators provides illumination of a scene separately. For each subset, the process activates the illuminators in the subset without activating illuminators not in the subset and receives reflected illumination from the scene incident on the lens assembly and focused onto the image sensing element. The measured light intensity of the received reflected illumination at the image sensing element is stored in association with activation of the subset.