Abstract: An electronic device may include one or more sensors that capture sensor data for a physical environment around the electronic device. The sensor data may be used to determine a scene understanding data set for an extended reality environment including the electronic device. The scene understanding data set may include information such as spatial information, information regarding physical objects in the extended reality environment, and information regarding virtual objects in the extended reality environment. When providing scene understanding data to one or more applications running on the electronic device, spatial and/or temporal restrictions may be applied to the scene understanding data set. Scene understanding data that is associated with locations within a boundary and that is associated with times after a cutoff time may be provided to an application.

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a particular object instance in CGR environments. In some implementations, an object type of an object depicted in an image of a physical environment is identified. Then, a particular instance is determined based on the object type and the image. In some implementations, objects of the particular instance have a set of characteristics that differs from sets of characteristics associated with other instances of the object type. Then, the set of characteristics of the particular instance of the object depicted in the physical environment is obtained.

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a particular object instance in CGR environments. In some implementations, an object type of an object depicted in an image of a physical environment is identified. Then, a particular instance is determined based on the object type and the image. In some implementations, objects of the particular instance have a set of characteristics that differs from sets of characteristics associated with other instances of the object type. Then, the set of characteristics of the particular instance of the object depicted in the physical environment is obtained.

Abstract: An electronic device: obtains a plurality of sets of images; synthesizes a three-dimensional point cloud for each of the plurality of sets of images; constructs planes using the respective three-dimensional point clouds; and generates a merged set of quadtrees characterizing a merged set of planes across the three-dimensional point clouds.

Abstract: In one implementation, a method of generating a horizontal plane hypothesis includes obtaining a point cloud of a scene including a plurality of points in a gravity-aligned coordinate system. The method includes generating, based on the plurality of points, a height histogram indicative of a plurality of non-overlapping height ranges in the gravity-aligned coordinate system and a respective plurality of weights. The method includes generating one or more horizontal plane hypotheses based on the height histogram.

Abstract: In accordance with some embodiments, a method is performed at a device with one or more processors, non-transitory memory, and an image sensor. The method includes synthesizing, from image data that corresponds to a physical perspective of an object relative to the device, a surface mask characterizing a portion of a surface of the object based on a grid arranged on the object, wherein the grid is represented in the image data. The method includes generating a virtual perspective of a model associated with the object having a surface oriented to match the surface mask within a predefined confidence threshold.

Abstract: Implementations disclosed herein more accurately and efficiently identify planar region boundaries corresponding to surfaces in a physical environment based on identifying intersections. In some implementations a detected planar region is altered, e.g., trimmed or extended, based on detecting an intersection of the planar region with an expansion of another planar region. In other implementations, a detected planar region is altered, e.g., trimmed or extended, based on detecting an intersection of the planar region with a detected edge line.

Abstract: In one implementation, a method of generating a horizontal plane hypothesis includes obtaining a point cloud of a scene including a plurality of points in a gravity-aligned coordinate system. The method includes generating, based on the plurality of points, a height histogram indicative of a plurality of non-overlapping height ranges in the gravity-aligned coordinate system and a respective plurality of weights. The method includes generating one or more horizontal plane hypotheses based on the height histogram.

Abstract: Various implementations disclosed herein include devices, systems, and methods that determine a particular object instance in CGR environments. In some implementations, an object type of an object depicted in an image of a physical environment is identified. Then, a particular instance is determined based on the object type and the image. In some implementations, objects of the particular instance have a set of characteristics that differs from sets of characteristics associated with other instances of the object type. Then, the set of characteristics of the particular instance of the object depicted in the physical environment is obtained.

Abstract: In one implementation, a method of generating a horizontal plane hypothesis includes obtaining a point cloud of a scene including a plurality of points in a gravity-aligned coordinate system. The method includes generating, based on the plurality of points, a height histogram indicative of a plurality of non-overlapping height ranges in the gravity-aligned coordinate system and a respective plurality of weights. The method includes generating one or more horizontal plane hypotheses based on the height histogram.

Abstract: An electronic device: obtains a plurality of sets of images; synthesizes a three-dimensional point cloud for each of the plurality of sets of images; constructs planes using the respective three-dimensional point clouds; and generates a merged set of quadtrees characterizing a merged set of planes across the three-dimensional point clouds.

Abstract: An electronic device: displays, on a display of the electronic device, a reticle element in a first appearance state overlaid on a media capture preview of objects in a field of view of an image sensor of the electronic device, where the media capture preview changes as the objects in the field of view of the image sensor change; detects a plane in the media capture preview; and, in response to detecting the plane, displays, on the display, the reticle element in a second appearance state overlaid on the media capture preview, where the reticle element corresponds to an indication of a portion of the extent of the plane while displayed in the second appearance state.

Abstract: An electronic device: displays, on a display of the electronic device, a reticle element in a first appearance state overlaid on a media capture preview of objects in a field of view of an image sensor of the electronic device, where the media capture preview changes as the objects in the field of view of the image sensor change; detects a plane in the media capture preview; and, in response to detecting the plane, displays, on the display, the reticle element in a second appearance state overlaid on the media capture preview, where the reticle element corresponds to an indication of a portion of the extent of the plane while displayed in the second appearance state.