Abstract: Improved techniques for performing object segmentation are disclosed. Surface reconstruction (SR) data corresponding to an environment is accessed. This SR data is used to generate a detailed three-dimensional (3D) representation of the environment. The SR data is also used to infer a high-level 3D structural representation of the environment. The high-level 3D structural representation is inferred using machine learning that is performed on the surface reconstruction data to identify a structure of the environment. The high-level 3D structural representation is then cut from the detailed 3D representation. This cutting process generates a clutter mesh comprising objects that remain after the cut and that are distinct from the structure. Object segmentation is then performed on the remaining objects to identify those objects.

Abstract: Improved techniques for performing object segmentation are disclosed. Surface reconstruction (SR) data corresponding to an environment is accessed. This SR data is used to generate a detailed three-dimensional (3D) representation of the environment. The SR data is also used to infer a high-level 3D structural representation of the environment. The high-level 3D structural representation is inferred using machine learning that is performed on the surface reconstruction data to identify a structure of the environment. The high-level 3D structural representation is then cut from the detailed 3D representation. This cutting process generates a clutter mesh comprising objects that remain after the cut and that are distinct from the structure. Object segmentation is then performed on the remaining objects to identify those objects.

Abstract: A method is disclosed for generating a digital twin of a physical environment. Depth data for the physical environment is received from a depth sensing device. A three-dimensional map of the physical environment is then generated based at least on the received depth data, and a digital twin of the physical environment is then generated based on the generated three-dimensional map. Information is received regarding the location of one or more networked devices within the generated three-dimensional map. Each of the one or more networked devices is associated with a digital twin of the networked device. Coordinate locations are established in the generated three-dimensional map for each networked device. Each established coordinate location is associated with a device identity.

Abstract: A computer-implemented technique is described herein for discriminatively apprising the user of the existence of some, but not necessarily all, physical objects in a physical environment in the course of the user's interaction with a virtual environment. In operation, the technique allows the user to selectively identify one or more objects-of-interest, such as people, walls, computing devices, etc. The technique then uses a scene analysis component to automatically detect the presence of the selected objects-of-interest in the physical environment, while the user interacts with the virtual environment. The technique provides alert information which notifies the user of the existence of any objects-of-interest that it detects. By virtue of the above-summarized strategy, the technique can apprise the user of objects-of-interest in the user's vicinity without cluttering the user's virtual experience with extraneous information pertaining to the physical environment.

Abstract: A computer-implemented technique is described herein for discriminatively apprising the user of the existence of some, but not necessarily all, physical objects in a physical environment in the course of the user's interaction with a virtual environment. In operation, the technique allows the user to selectively identify one or more objects-of-interest, such as people, walls, computing devices, etc. The technique then uses a scene analysis component to automatically detect the presence of the selected objects-of-interest in the physical environment, while the user interacts with the virtual environment. The technique provides alert information which notifies the user of the existence of any objects-of-interest that it detects. By virtue of the above-summarized strategy, the technique can apprise the user of objects-of-interest in the user's vicinity without cluttering the user's virtual experience with extraneous information pertaining to the physical environment.