Abstract: Various implementations disclosed herein include devices, systems, and methods that match image portions of a first set of image portions and image portions of another set of image portions. This may be performed in a such a way that the sum of descriptor distances of the matched image portions is optimized in a globally optimal way. In some implementations, the matching is subject to one or more constraints, e.g., ensuring that each image portion of the first data set is matched with at least one image portion of the second data set and ensuring that each image portion of the second data set is matched with only one image portion of the first data set.

Abstract: Various implementations disclosed herein include methods, electronic devices, and systems for performing perceptual-based color correction based on chromaticity values. To that end, in some implementations, a method is performed at an electronic device with one or more processors, a non-transitory memory, and a see-through display. The method includes determining a chromaticity value associated with ambient light from a physical environment. The chromaticity value quantifies the ambient light. The method includes determining a set of color correction values based on a function of the chromaticity value and image data. The set of color correction values and the chromaticity value together satisfy one or more perceptual criteria. The method includes modifying the image data in order to generate display data based on a function of the set of color correction values. The method includes displaying the display data on the see-through display.

Abstract: Various implementations disclosed herein include devices, systems, and methods for verifying that an image includes a complete code of a time-varying visual marker that displays codes sequentially on a display. In some implementations, the verification determines that the image include a complete code rather than combinations of sequentially-displayed codes that may be included in an image based on use of a rolling shutter (e.g., in a camera of a detecting electronic device). In some implementations, the verification involves comparing a first verification portion of an image to a second opposing verification portion of an image. Various implementations disclosed herein include devices, systems, and methods for modifying image capture parameters (e.g., frame rate) to ensure capture of all codes of a time-varying visual marker.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide color visual markers that include colored markings that encode data, where the colors of the colored markings are determined by scanning (e.g., detecting the visual marker using a sensor of an electronic device) the visual marker itself. In some implementations, a visual marker is detected in an image of a physical environment. In some implementations, the visual marker is detected in the image by detecting a predefined shape of a first portion of the visual marker in the image. Then, a color-interpretation scheme is determined for interpreting colored markings of the visual marker that encode data by identifying a set of colors at a corresponding set of predetermined locations on the visual marker. Then, the data of the visual marker is decoded using the colored markings and the set of colors of the color-interpretation scheme.

Abstract: Various implementations disclosed herein include devices, systems, and methods that select colors for visual markers that include colored markings encoding data. In some implementations, an input is received selecting a source image and the devices, systems, and methods determine colors based on the source image and based on distances between the colors exceeding a spatial distance threshold in a 3D color space. In some implementations, the devices, systems, and methods generate, based on the determined colors, an appearance of a visual marker comprising graphical elements encoding data using the determined colors.

Abstract: Performing a corrective operation for environmental conditions related to a predetermined eye condition includes obtaining environment sensor data from a one or more sensors of the device, determining a current context for the device based on the environment sensor data, and determining, based on the current context, that an eye state criterion is satisfied. In response to determining that the eye state criterion is satisfied, a corrective operation is determined in accordance with the eye state criterion, and the corrective operation is performed. When performed, the corrective operation is configured to resolve an environmental condition associated with the eye state criterion.

Abstract: Various implementations disclosed herein include devices, systems, and methods that select colors for visual markers that include colored markings encoding data. In some implementations, the colors are automatically or semi-automatically selected. In some implementations, the colors are selected to remain sufficiently detectable despite changes in lighting conditions or printing/display conditions. In some implementations, a set of colors selectable for use in a visual marker is obtained. Then, measures of distance between a plurality of colors of the set of colors is determined, and a subset of the set of colors for the visual marker is selected based on the measure of distance between colors of the subset of colors. In some implementations, the visual marker appearance includes graphical elements encoding data using the subset of colors. In some implementations, input is received using a GUI on a display to determine multiple colors based on a source image.

Abstract: Various implementations disclosed herein include devices, systems, and methods that estimate a location of a light source based on ambient light data. For example, an example process may include acquiring ambient light data from an ambient light sensor (ALS) during movement of a device in a physical environment, acquiring motion data from a motion sensor during the movement of the device, determining, based on the ambient light data and the motion data, estimates of three-dimensional (3D) locations of a light source with respect to the device during the movement of the device, and tracking a location of the device in a 3D coordinate system during the movement of the device based on the estimates of the 3D locations of the light source with respect to the device during the movement of the device.

Abstract: Implementations of the subject technology provide visualizations of non-visible features of a physical environment, at the location of the non-visible features in the physical environment. The non-visible features may include wireless communications signals, sounds, airflow, gases, subsonic and/or ultrasonic waves, hidden objects, or the like. A device may store visual contexts for visualizations of particular non-visible features. The device may obtain a depth map that allows the device to determine the location of the non-visible feature in the physical environment and to overlay the visualization on a user's view of that location. In this way, the non-visible feature can be visualized its correct location, orientation, direction and/or strength in the physical environment.

Abstract: Various implementations disclosed herein include devices, systems, and methods that provide a visual marker including a plurality of markings arranged in a corresponding plurality of shapes. In some implementations, each marking is formed of a set of sub-markings separated by gaps and arranged according to a respective shape, and the gaps of the plurality of markings are configured to encode data and indicate orientation of the visual marker. In some implementations, the plurality of markings are arranged in a plurality of concentric rings of increasing size. In some implementations, the orientation is encoded in a first set of gaps and data in a second set of gaps of the gaps in the plurality of markings.

Abstract: Various implementations disclosed herein include methods, electronic devices, and systems for performing perceptual-based color correction based on chromaticity values. To that end, in some implementations, a method is performed at an electronic device with one or more processors, a non-transitory memory, and a see-through display. The method includes determining a chromaticity value associated with ambient light from a physical environment. The chromaticity value quantifies the ambient light. The method includes determining a set of color correction values based on a function of the chromaticity value and image data. The set of color correction values and the chromaticity value together satisfy one or more perceptual criteria. The method includes modifying the image data in order to generate display data based on a function of the set of color correction values. The method includes displaying the display data on the see-through display.

Abstract: Various implementations disclosed herein include devices, systems, and methods that match image portions of a first set of image portions and image portions of another set of image portions. This may be performed in a such a way that the sum of descriptor distances of the matched image portions is optimized in a globally optimal way. In some implementations, the matching is subject to one or more constraints, e.g., ensuring that each image portion of the first data set is matched with at least one image portion of the second data set and ensuring that each image portion of the second data set is matched with only one image portion of the first data set.

Abstract: Implementations of the subject technology provide visualizations of non-visible features of a physical environment, at the location of the non-visible features in the physical environment. The non-visible features may include wireless communications signals, sounds, airflow, gases, subsonic and/or ultrasonic waves, hidden objects, or the like. A device may store visual contexts for visualizations of particular non-visible features. The device may obtain a depth map that allows the device to determine the location of the non-visible feature in the physical environment and to overlay the visualization on a user's view of that location. In this way, the non-visible feature can be visualized its correct location, orientation, direction and/or strength in the physical environment.

Abstract: Various implementations disclosed herein include devices, systems, and methods for verifying that an image includes a complete code of a time-varying visual marker that displays codes sequentially on a display. In some implementations, the verification determines that the image include a complete code rather than combinations of sequentially-displayed codes that may be included in an image based on use of a rolling shutter (e.g., in a camera of a detecting electronic device). In some implementations, the verification involves comparing a first verification portion of an image to a second opposing verification portion of an image. Various implementations disclosed herein include devices, systems, and methods for modifying image capture parameters (e.g., frame rate) to ensure capture of all codes of a time-varying visual marker.

Abstract: Implementations of the subject technology provide visualizations of non-visible features of a physical environment, at the location of the non-visible features in the physical environment. The non-visible features may include wireless communications signals, sounds, airflow, gases, subsonic and/or ultrasonic waves, hidden objects, or the like. A device may store visual contexts for visualizations of particular non-visible features. The device may obtain a depth map that allows the device to determine the location of the non-visible feature in the physical environment and to overlay the visualization on a user's view of that location. In this way, the non-visible feature can be visualized its correct location, orientation, direction and/or strength in the physical environment.

Abstract: Implementations of the subject technology provide visualizations of non-visible features of a physical environment, at the location of the non-visible features in the physical environment. The non-visible features may include wireless communications signals, sounds, airflow, gases, subsonic and/or ultrasonic waves, hidden objects, or the like. A device may store visual contexts for visualizations of particular non-visible features. The device may obtain a depth map that allows the device to determine the location of the non-visible feature in the physical environment and to overlay the visualization on a user's view of that location. In this way, the non-visible feature can be visualized its correct location, orientation, direction and/or strength in the physical environment.

Abstract: Implementations of the subject technology provide visualizations of non-visible features of a physical environment, at the location of the non-visible features in the physical environment. The non-visible features may include wireless communications signals, sounds, airflow, gases, subsonic and/or ultrasonic waves, hidden objects, or the like. A device may store visual contexts for visualizations of particular non-visible features. The device may obtain a depth map that allows the device to determine the location of the non-visible feature in the physical environment and to overlay the visualization on a user's view of that location. In this way, the non-visible feature can be visualized its correct location, orientation, direction and/or strength in the physical environment.

Abstract: Various implementations disclosed herein include devices, systems, and methods that select colors for visual markers that include colored markings encoding data. In some implementations, the colors are automatically or semi-automatically selected. In some implementations, the colors are selected to remain sufficiently detectable despite changes in lighting conditions or printing/display conditions. In some implementations, a set of colors selectable for use in a visual marker is obtained. Then, measures of distance between a plurality of colors of the set of colors is determined, and a subset of the set of colors for the visual marker is selected based on the measure of distance between colors of the subset of colors. In some implementations, the visual marker appearance includes graphical elements encoding data using the subset of colors. In some implementations, input is received using a GUI on a display to determine multiple colors based on a source image.