Abstract: An improved human-computer interface (“HCI”) is disclosed herein for viewing a three-dimensional (“3D”) representation of a real-world environment from different, changing, and/or multiple perspectives. An AR device may capture, in real-time, a 3D representation of a scene using a surface reconstruction (“SR”) camera and a traditional Red Green & Blue (“RGB”) camera. The 3D representation may be transmitted to and viewed on a user's computing device, enabling the user to navigate the 3D representation. The user may view the 3D representation in a free-third-person mode, enabling the user to virtually walk or fly through the representation captured by the AR device. The user may also select a floor plan mode for a top-down or isomorphic perspective. Enabling a user to view a scene from different perspectives enhances understanding, speeds trouble-shooting, and fundamentally improves the capability of the computing device, the AR device, and the combination thereof.

Abstract: An improved human-computer interface (“HCI”) is disclosed herein for viewing a three-dimensional (“3D”) representation of a real-world environment from different, changing, and/or multiple perspectives. An AR device may capture, in real-time, a 3D representation of a scene using a surface reconstruction (“SR”) camera and a traditional Red Green & Blue (“RGB”) camera. The 3D representation may be transmitted to and viewed on a user's computing device, enabling the user to navigate the 3D representation. The user may view the 3D representation in a free-third-person mode, enabling the user to virtually walk or fly through the representation captured by the AR device. The user may also select a floor plan mode for a top-down or isomorphic perspective. Enabling a user to view a scene from different perspectives enhances understanding, speeds trouble-shooting, and fundamentally improves the capability of the computing device, the AR device, and the combination thereof.

Abstract: The techniques introduced here enable a display system, such as an HMD device, to generate and display to a user a holographic structure matching a real-world structure. In some embodiments vertices, edges and planes of the holographic schematics are generated via the use of a peripheral tool that is positioned by a user. In other embodiments, other user input indicates the bounds of the holographic schematic. In response to user action, a holographic schematic is made to appear including corresponding real-world size measurements. The corresponding measurements are used to develop a holographic structure that integrates with the holographic schematic.

Abstract: A method, medium, and virtual object for providing a virtual representation with an attribute are described. The virtual representation is generated based on a digitization of a real-world object. Properties of the virtual representation, such as colors, shape similarities, volume, surface area, and the like are identified and an amount or degree of exhibition of those properties by the virtual representation is determined. The properties are employed to identify attributes associated with the virtual representation, such as temperature, weight, or sharpness of an edge, among other attributes of the virtual object. A degree of exhibition of the attributes is also determined based on the properties and their degrees of exhibition. Thereby, the virtual representation is provided with one or more attributes that instruct presentation and interactions of the virtual representation in a virtual world.

Abstract: The techniques introduced here enable a display system, such as an HMD device, to generate and display to a user a holographic structure matching a real-world structure. In some embodiments vertices, edges and planes of the holographic schematics are generated via the use of a peripheral tool that is positioned by a user. In other embodiments, other user input indicates the bounds of the holographic schematic. In response to user action, a holographic schematic is made to appear including corresponding real-world size measurements. The corresponding measurements are used to develop a holographic structure that integrates with the holographic schematic.

Abstract: An improved human-computer interface (“HCI”) is disclosed herein for viewing a three-dimensional (“3D”) representation of a real-world environment from different, changing, and/or multiple perspectives. An AR device may capture, in real-time, a 3D representation of a scene using a surface reconstruction (“SR”) camera and a traditional Red Green & Blue (“RGB”) camera. The 3D representation may be transmitted to and viewed on a user's computing device, enabling the user to navigate the 3D representation. The user may view the 3D representation in a free-third-person mode, enabling the user to virtually walk or fly through the representation captured by the AR device. The user may also select a floor plan mode for a top-down or isomorphic perspective. Enabling a user to view a scene from different perspectives enhances understanding, speeds trouble-shooting, and fundamentally improves the capability of the computing device, the AR device, and the combination thereof.

Abstract: To digitize an object, a camera captures images of different sides of the object with color and depth data. At least two different sides of the object are identified from the images, and constructions are created of the sides of the object from the images. Points of the constructions to connect to one another are determined and used to align the constructions. The construction are merged to generate a rendition of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. The rendition of the object can be provided for display as a digital representation of the object and potentially used in different applications (e.g., games, Web, etc.).

Abstract: The techniques introduced here enable a display system, such as an HMD device, to generate and display to a user a holographic structure matching a real-world structure. In some embodiments vertices, edges and planes of the holographic schematics are generated via the use of a peripheral tool that is positioned by a user. In other embodiments, other user input indicates the bounds of the holographic schematic. In response to user action, a holographic schematic is made to appear including corresponding real-world size measurements. The corresponding measurements are used to develop a holographic structure that integrates with the holographic schematic.

Abstract: The techniques introduced here enable a display system, such as an HMD device, to generate and display to a user a holographic structure matching a real-world structure. In some embodiments vertices, edges and planes of the holographic schematics are generated via the use of a peripheral tool that is positioned by a user. In other embodiments, other user input indicates the bounds of the holographic schematic. In response to user action, a holographic schematic is made to appear including corresponding real-world size measurements. The corresponding measurements are used to develop a holographic structure that integrates with the holographic schematic.

Abstract: To digitize an object, a camera captures images of different sides of the object with color and depth data. At least two different sides of the object are identified from the images, and constructions are created of the sides of the object from the images. Points of the constructions to connect to one another are determined and used to align the constructions. The construction are merged to generate a rendition of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. The rendition of the object can be provided for display as a digital representation of the object and potentially used in different applications (e.g., games, Web, etc.).

Abstract: Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. For both front and back images, the closest point to the camera is determined by analyzing the depth data. From the closest points, edges of the object are found by noting large differences in depth data. The depth data is also used to construct point cloud constructions of the front and back of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. Eventually, a digital representation is presented to the user and potentially used in different applications (e.g., games, Web, etc.).

Abstract: Methods, systems, and computer-storage media having computer-usable instructions embodied thereon, for controlling objects in a virtual environment are provided. Real-world objects may be received into a virtual environment. The real-world objects may be any non-human object. An object skeleton may be identified and mapped to the object. A user skeleton of the real-world user may also be identified and mapped to the object skeleton. By mapping the user skeleton to the object skeleton, movements of the user control the movements of the object in the virtual environment.

Abstract: Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. The object is recognized and digitized using color and depth data of the image. The user's client queries a server managing images uploaded by other users for virtual renditions of the object, as recognized in the other images. The virtual renditions from the other images are merged with the digitized version of the object in the image captured by the user to create a composite rendition of the object.

Abstract: Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. For both front and back images, the closest point to the camera is determined by analyzing the depth data. From the closest points, edges of the object are found by noting large differences in depth data. The depth data is also used to construct point cloud constructions of the front and back of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. Eventually, a digital representation is presented to the user and potentially used in different applications (e.g., games, Web, etc.).

Abstract: Systems, methods, and computer media for generating an avatar reflecting a player's current appearance. Data describing the player's current appearance is received. The data includes a visible spectrum image of the player, a depth image including both the player and a current background, and skeletal data for the player. The skeletal data indicates an outline of the player's skeleton. Based at least in part on the received data, one or more of the following are captured: a facial appearance of the player; a hair appearance of the player; a clothing appearance of the player; and a skin color of the player. A 3D avatar resembling the player is generated by combining the captured facial appearance, hair appearance, clothing appearance, and/or skin color with predetermined avatar features.

Abstract: Systems, methods, and computer media for generating an avatar reflecting a player's current appearance. Data describing the player's current appearance is received. The data includes a visible spectrum image of the player, a depth image including both the player and a current background, and skeletal data for the player. The skeletal data indicates an outline of the player's skeleton. Based at least in part on the received data, one or more of the following are captured: a facial appearance of the player; a hair appearance of the player; a clothing appearance of the player; and a skin color of the player. A 3D avatar resembling the player is generated by combining the captured facial appearance, hair appearance, clothing appearance, and/or skin color with predetermined avatar features.

Abstract: Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. The object is recognized and digitized using color and depth data of the image. The user's client queries a server managing images uploaded by other users for virtual renditions of the object, as recognized in the other images. The virtual renditions from the other images are merged with the digitized version of the object in the image captured by the user to create a composite rendition of the object.

Abstract: Methods, systems, and computer-storage media having computer-usable instructions embodied thereon, for controlling objects in a virtual environment are provided. Real-world objects may be received into a virtual environment. The real-world objects may be any non-human object. An object skeleton may be identified and mapped to the object. A user skeleton of the real-world user may also be identified and mapped to the object skeleton. By mapping the user skeleton to the object skeleton, movements of the user control the movements of the object in the virtual environment.

Abstract: Systems, methods, and computer media for generating an avatar reflecting a player's current appearance. Data describing the player's current appearance is received. The data includes a visible spectrum image of the player, a depth image including both the player and a current background, and skeletal data for the player. The skeletal data indicates an outline of the player's skeleton. Based at least in part on the received data, one or more of the following are captured: a facial appearance of the player; a hair appearance of the player; a clothing appearance of the player; and a skin color of the player. A 3D avatar resembling the player is generated by combining the captured facial appearance, hair appearance, clothing appearance, and/or skin color with predetermined avatar features.

Abstract: Digitizing objects in a picture is discussed herein. A user presents the object to a camera, which captures the image comprising color and depth data for the front and back of the object. For both front and back images, the closest point to the camera is determined by analyzing the depth data. From the closest points, edges of the object are found by noting large differences in depth data. The depth data is also used to construct point cloud constructions of the front and back of the object. Various techniques are applied to extrapolate edges, remove seams, extend color intelligently, filter noise, apply skeletal structure to the object, and optimize the digitization further. Eventually, a digital representation is presented to the user and potentially used in different applications (e.g., games, Web, etc.).