Abstract: This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Abstract: This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Abstract: This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Abstract: This disclosure describes an artificial reality system that presents artificial reality content in the context of a physical environment that includes a mirror or other reflective surface. In one example, this disclosure describes a method that includes capturing capture data representative of a physical environment, wherein the physical environment includes a reflective surface and a plurality of objects, determining a pose of the HMD, determining a map of the physical environment, wherein the map includes position information about the reflective surface and position information about each of the plurality of physical objects in the physical environment, identifying a visible object from among the plurality of physical objects, and generating artificial reality content associated with the visible object.

Abstract: In general, the disclosure describes artificial reality (AR) systems and techniques for generating and presenting virtual surfaces within an artificial reality environment and for facilitating user interaction with the virtual surfaces using a physical peripheral device. For example, AR systems are described that generate and render virtual surfaces, such as a virtual pinboard or a virtual drawing surface (e.g., a virtual canvas) in an artificial reality environment, for display to a user. The AR systems enable the user to interact with the virtual surfaces using a physical peripheral device, which may be manipulated and otherwise interacted with by the user to provide input to an AR system through pose tracking of the peripheral device and/or via one or more input devices of the peripheral device, such as a presence-sensitive surface.

Abstract: In some examples, a virtual surface presented by an artificial reality (AR) system may be a virtual drawing surface in an artificial reality environment, with which users may interact using a physical peripheral device. For example, the AR system may enable a user to draw or write on a surface of the peripheral device and may simultaneously, i.e., along with the user inputs, render virtual markings corresponding to the user inputs on a virtual drawing surface locked to the surface of the peripheral device. In some cases, the AR system enables the user to “transfer” the virtual markings rendered on the surface of the peripheral device (e.g., a virtual drawing) to another virtual drawing surface (e.g., to a planar or other surface in the artificial reality environment).

Abstract: An artificial reality system includes a head mounted display (HMD) and a compass generator that generates a proximity compass used to locate virtual or physical elements within an artificial reality environment. The artificial reality system can render a proximity compass around or in proximity to a virtual device, a virtual or physical object, or at a designated location. The proximity compass includes graphical elements that can represent virtual experiences (applications, games, utilities etc.), physical or virtual objects, and physical or virtual locations within the artificial reality environment. The positioning of a graphical element within the proximity compass represents a direction of the associated experience, object or location with respect to the HMD. The graphical elements within the proximity compass may rotate around a virtual hand-held device when the user changes the orientation of the hand-held device.

Abstract: In some examples, a method comprises: obtaining, by an artificial reality system including a head mounted display (HMD), image data via an image capture device, the HMD configured to output artificial reality content; detecting, by the artificial reality system, a physical peripheral device from the image data; detecting, by the artificial reality system, a pose of the peripheral device; generating, by the artificial reality system, based on the pose of the peripheral device, a virtual pointer along at least a portion of a line between the peripheral device and a virtual surface, the virtual pointer pointing at a location of the virtual surface; performing, by the artificial reality system, one or more actions based on the location of the virtual surface; and rendering, by the artificial reality system, the virtual pointer and the virtual surface for display at the HMD.

Abstract: In general, the disclosure describes artificial reality systems and techniques for generating and presenting a virtual user interface with which users may interact using a physical peripheral device. In some examples, an artificial reality system includes an image capture device configured to capture image data; a head-mounted display (HMD) configured to output artificial reality content; a user interface engine configured to detect a peripheral device from the image data, wherein the user interface engine is configured to generate a virtual user interface comprising one or more virtual user interface elements; and a rendering engine configured to render the artificial reality content and to render, at a user interface position locked relative to a position of the peripheral device in an artificial reality environment, the virtual user interface for display at the HMD.