Abstract: In one embodiment, a method includes accessing a digital image captured by a camera that is connected to a machine-detectable object, detecting a reflection of the machine-detectable object in the digital image, computing, in response to the detection, a plane that is coincident with a reflective surface associated with the reflection, determining a boundary of the reflective surface in the plane based on at least one of a plurality of cues, and storing information associated with the reflective surface, where the information includes a pose of the reflective surface and the boundary of the reflective surface in a 3D model of a physical environment, and where the information associated with the reflective surface and the 3D model are configured to be used to render a reconstruction of the physical environment.

Abstract: In one embodiment, a method includes accessing a calibration model for a camera rig. The method includes accessing multiple observations of an environment captured by the camera rig from multiple poses in the environment. The method includes generating an environmental model including geometry of the environment based on at least the observations, the poses, and the calibration model. The method includes determining, for one or more of the poses, one or more predicted observations of the environment based on the environmental model and the poses. The method includes comparing the predicted observations to the observations corresponding to the poses from which the predicted observations were determined. The method includes revising the calibration model based on the comparison. The method includes revising the environmental model based on at least a set of observations of the environment and the revised calibration model.

Abstract: The disclosed computer-implemented method may include comprising identifying, within a real-world environment, a position of a user relative to a safety boundary. The position of the user is identified by a head-mounted display system comprising a display device. The display device is configured to at least partially obscure visibility of the real-world environment to the user. The method may further include selecting, based on the position of the user, at least a portion of a model of the real-world environment, rendering the portion of the model of the real-world environment, and displaying the rendered portion of the model of the real-world environment via the display device as a notification of the position of the user relative to the safety boundary. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: In one embodiment, a method includes accessing a digital image captured by a camera that is connected to a machine-detectable object, detecting a reflection of the machine-detectable object in the digital image, computing, in response to the detection, a plane that is coincident with a reflective surface associated with the reflection, determining a boundary of the reflective surface in the plane based on at least one of a plurality of cues, and storing information associated with the reflective surface, where the information includes a pose of the reflective surface and the boundary of the reflective surface in a 3D model of a physical environment, and where the information associated with the reflective surface and the 3D model are configured to be used to render a reconstruction of the physical environment.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment and determining, from the real-time data, current object data for the environment. The current object data may include both state data and relationship data for objects in the environment. The method may also include determining object deltas between the current object data and prior object data from an event graph. The prior object data may include prior state data and prior relationship data for the objects. The method may include detecting an unknown state for one of the objects, inferring a state for the object based on the event graph, and updating the event graph based on the object deltas and the inferred state. The method may further include sending updated event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A depth measurement assembly (DMA) includes a structured light emitter, an augmented camera, and a controller. The structured light emitter projects structured light into a local area under instructions from the controller. The augmented camera generates image data of an object illuminated with the structured light pattern projected by the structured light emitter in accordance with camera instructions generated by the controller. The augmented camera includes a high speed computation tracking sensor that comprises a plurality of augmented photodetectors. Each augmented photodetector converts light to data and stores the data in its own memory unit. The controller receives the image data and determines depth information of the object in the local area based in part on the image data. The depth measurement unit can be incorporated into a head-mounted display (HMD).

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment. The method may also include determining, from the real-time data, current mapping and object data. The current mapping data may include coordinate data for the environment and the current object data may include both state data and relationship data for objects in the environment. The method may also include determining mapping deltas between the current mapping data and baseline map data and determining object deltas between the current object data and an event graph. The event graph may include prior state data and prior relationship data for objects. The method may also include updating the baseline map data and the event graph based on the deltas and sending updated baseline map data and event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: The disclosed computer-implemented method may include receiving, from devices in an environment, real-time data associated with the environment. The method may also include determining, from the real-time data, current mapping and object data. The current mapping data may include coordinate data for the environment and the current object data may include both state data and relationship data for objects in the environment. The method may also include determining mapping deltas between the current mapping data and baseline map data and determining object deltas between the current object data and an event graph. The event graph may include prior state data and prior relationship data for objects. The method may also include updating the baseline map data and the event graph based on the deltas and sending updated baseline map data and event graph data to the devices. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: A head mounted display (HMD) dynamically generates a model of an area. The HMD includes a depth camera assembly (DCA), a color camera, and a processing circuitry. The processing circuitry receives, from the DCA, a frame of depth image data, generates a depth map of a portion of the area based on the frame of the depth image data, receives a frame of color image data from the camera, determines a location in a model of the area that corresponds with the portion of the area of the depth map based on the frame of the color image data, and update the model of the area by combining the depth map of the portion of area with one or more other depth maps of one or more other portions of the area based on the location in the model.

Abstract: An imaging device operates as an event camera. The device includes an event sensor and a controller. The sensor comprises a plurality of photodiodes that asynchronously output data values corresponding to relative intensity changes within a local area. The controller populates an event matrix based in part on data values asynchronously received from the sensor and positions of photodiodes associated with the received data values over a first time period. The controller populates a change matrix based in part on a threshold intensity value and the photodiodes associated with the received data values over the first time period, and generates an image for the first time period using the event matrix and the change matrix.

Abstract: An imaging device operates as an event camera. The device includes an event sensor and a controller. The sensor comprises a plurality of photodiodes that asynchronously output data values corresponding to relative intensity changes within a local area. The controller populates an event matrix based in part on data values asynchronously received from the sensor and positions of photodiodes associated with the received data values over a first time period. The controller populates a change matrix based in part on a threshold intensity value and the photodiodes associated with the received data values over the first time period, and generates an image for the first time period using the event matrix and the change matrix.