Abstract: In one embodiment, a method includes accessing a digital image captured by a camera that is connected to a machine-detectable object, detecting a reflection of the machine-detectable object in the digital image, computing, in response to the detection, a plane that is coincident with a reflective surface associated with the reflection, determining a boundary of the reflective surface in the plane based on at least one of a plurality of cues, and storing information associated with the reflective surface, where the information includes a pose of the reflective surface and the boundary of the reflective surface in a 3D model of a physical environment, and where the information associated with the reflective surface and the 3D model are configured to be used to render a reconstruction of the physical environment.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment and determining, from the real-time data, current object data for the environment. The current object data may include both state data and relationship data for objects in the environment. The method may also include determining object deltas between the current object data and prior object data from an event graph. The prior object data may include prior state data and prior relationship data for the objects. The method may include detecting an unknown state for one of the objects, inferring a state for the object based on the event graph, and updating the event graph based on the object deltas and the inferred state. The method may further include sending updated event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method may include determining a local position and a local orientation of a local device in an environment and receiving, by the local device and from a mapping system, object data for objects within the environment. The object data may include position data and orientation data for the objects and relationship data between the objects. The method may also include deriving, based on the object data received from the mapping system, and the local position and orientation of the local device, a contextual rendering of the objects that provides contextual data that modifies a user's view of the environment. The method may include displaying, using the local device, the contextual rendering of at least one of the plurality of objects to modify the user's view of the environment. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes accessing a digital image captured by a camera that is connected to a machine-detectable object, detecting a reflection of the machine-detectable object in the digital image, computing, in response to the detection, a plane that is coincident with a reflective surface associated with the reflection, determining a boundary of the reflective surface in the plane based on at least one of a plurality of cues, and storing information associated with the reflective surface, where the information includes a pose of the reflective surface and the boundary of the reflective surface in a 3D model of a physical environment, and where the information associated with the reflective surface and the 3D model are configured to be used to render a reconstruction of the physical environment.

Abstract: The disclosed computer-implemented method may include comprising identifying, within a real-world environment, a position of a user relative to a safety boundary. The position of the user is identified by a head-mounted display system comprising a display device. The display device is configured to at least partially obscure visibility of the real-world environment to the user. The method may further include selecting, based on the position of the user, at least a portion of a model of the real-world environment, rendering the portion of the model of the real-world environment, and displaying the rendered portion of the model of the real-world environment via the display device as a notification of the position of the user relative to the safety boundary. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes accessing a digital image captured by a camera that is connected to a machine-detectable object, detecting a reflection of the machine-detectable object in the digital image, computing, in response to the detection, a plane that is coincident with a reflective surface associated with the reflection, determining a boundary of the reflective surface in the plane based on at least one of a plurality of cues, and storing information associated with the reflective surface, where the information includes a pose of the reflective surface and the boundary of the reflective surface in a 3D model of a physical environment, and where the information associated with the reflective surface and the 3D model are configured to be used to render a reconstruction of the physical environment.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment and determining, from the real-time data, current object data for the environment. The current object data may include both state data and relationship data for objects in the environment. The method may also include determining object deltas between the current object data and prior object data from an event graph. The prior object data may include prior state data and prior relationship data for the objects. The method may include detecting an unknown state for one of the objects, inferring a state for the object based on the event graph, and updating the event graph based on the object deltas and the inferred state. The method may further include sending updated event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment. The method may also include determining, from the real-time data, current mapping and object data. The current mapping data may include coordinate data for the environment and the current object data may include both state data and relationship data for objects in the environment. The method may also include determining mapping deltas between the current mapping data and baseline map data and determining object deltas between the current object data and an event graph. The event graph may include prior state data and prior relationship data for objects. The method may also include updating the baseline map data and the event graph based on the deltas and sending updated baseline map data and event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment. The method may also include determining, from the real-time data, current mapping and object data. The current mapping data may include coordinate data for the environment and the current object data may include both state data and relationship data for objects in the environment. The method may also include determining mapping deltas between the current mapping data and baseline map data and determining object deltas between the current object data and an event graph. The event graph may include prior state data and prior relationship data for objects. The method may also include updating the baseline map data and the event graph based on the deltas and sending updated baseline map data and event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A head mounted display (HMD) dynamically generates a model of an area. The HMD includes a depth camera assembly (DCA), a color camera, and a processing circuitry. The processing circuitry receives, from the DCA, a frame of depth image data, generates a depth map of a portion of the area based on the frame of the depth image data, receives a frame of color image data from the camera, determines a location in a model of the area that corresponds with the portion of the area of the depth map based on the frame of the color image data, and update the model of the area by combining the depth map of the portion of area with one or more other depth maps of one or more other portions of the area based on the location in the model.

Abstract: A method for mapping an environment comprises moving a sensor along a path from a start location (P0) through the environment, the sensor generating a sequence of images, each image associated with a respective estimated sensor location and comprising a point cloud having a plurality of vertices, each vertex comprising an (x,y,z)-tuple and image information for the tuple. The sequence of estimated sensor locations is sampled to provide a pose graph (P) comprising a linked sequence of nodes, each corresponding to a respective estimated sensor location. For each node of the pose graph (P), a respective cloud slice (C) comprising at least of portion of the point cloud for the sampled sensor location is acquired. A drift between an actual sensor location (Pi+1) and an estimated sensor location (Pi) on the path is determined.

Abstract: A method for mapping an environment comprises moving a sensor along a path from a start location (P0) through the environment, the sensor generating a sequence of images, each image associated with a respective estimated sensor location and comprising a point cloud having a plurality of vertices, each vertex comprising an (x,y,z)-tuple and image information for the tuple. The sequence of estimated sensor locations is sampled to provide a pose graph (P) comprising a linked sequence of nodes, each corresponding to a respective estimated sensor location. For each node of the pose graph (P), a respective cloud slice (C) comprising at least of portion of the point cloud for the sampled sensor location is acquired. A drift between an actual sensor location (Pi+1) and an estimated sensor location (Pi) on the path is determined.