Abstract: Techniques for providing multi-screen interaction in an interactive application. A primary device establishes an application session. The application session includes a virtual environment relating to a physical environment in which the primary user device is located. The primary user device is configured to display a graphical image relating to a primary virtual camera view of the virtual environment. A secondary virtual camera view of the virtual environment, relating to the primary virtual camera view, is determined. The primary user device transmits image data relating to the secondary virtual camera view from the primary user device to a secondary user device, which is configured to display a graphical image based on the image data. Responsive to receiving an interaction request from the secondary user device, the primary user device controls the application session based on the interaction request.

Abstract: To prevent virtual content from occluding physical objects in an augmented reality (AR) system, the embodiments herein describe generating a pass-through texture using a pre-generated model of a physical object. In one embodiment, the AR system determines the location of the physical object as well as its orientation. With this information, the system rotates and re-sizes the pre-generated model of the object to match its location and orientation in the real-world. The system then generates the pass-through texture from the pre-generated model and inserts the texture into the virtual content. When the virtual content is displayed, the pass-through texture permits light from the real-world view to pass through substantially unaffected by the virtual content. In this manner, the physical object (which aligns with the location of the pass-through texture in the virtual content) can be seen by the user—i.e., is not occluded by the virtual content.

Abstract: Embodiments provide for reconciling a first map of an environment created by a first device with a second map of the environment created by a second device by determining a first confidence score for a first location of a given object in the first map; determining a second confidence score for a second location of the given object in the second map; inserting the given object into a reconciled map at a position based on the first confidence score and the second confidence score; anchoring an Augmented Reality object in the reconciled map at coordinates based on the position of the given object; and outputting the reconciled map to an Augmented Reality device.

Abstract: Systems, methods and articles of manufacture relating to calibration of a magnetometer in an electronic device suitable for use in an augmented reality (AR) and/or virtual reality (VR) video game. An electronic video game device includes a magnetometer. A prompt is displayed in the video game requesting a user-performed first gameplay motion using the electronic video game device where the first gameplay motion is a pre-defined motion used for calibrating the magnetometer. A first plurality of measurements is received from the magnetometer relating to a magnetic field in a physical gameplay environment, where the first plurality of measurements is taken by the magnetometer during the first gameplay motion. The first plurality of measurements is analyzed to detect a magnetic bias in the magnetometer. The magnetometer is calibrated to offset the magnetic bias.

Abstract: To capture augmented reality (AR) on a head mounted display the embodiments described herein include the optical display of virtual components of an AR on an AR device. A real world view within a field of view of the display of the AR device is captured, creating a composite image that combines the virtual components and real world physical components within the field of view of the AR device.

Abstract: Embodiments provide for tracking location and resolving drift in Augmented Reality (AR) devices. The AR devices includes computing devices having screens on a first face and cameras on a second, opposite face to project an image onto optical arrangements for viewing by wearers of the AR devices. The AR devices map locations for real objects in the environment to a virtual environment; anchor virtual objects at anchor locations within the virtual environment; capture station keeping images of the environment from a first Field of View via the camera; determine a second, different Field of View in the environment for the wearer of the AR device based on the relative locations of real objects present in the station keeping images; and output images depicting the virtual objects at positions on the screen to depict the virtual objects in the physical environment at the anchor locations.

Abstract: Embodiments provide for reconciling a first map of an environment created by a first device with a second map of the environment created by a second device by determining a first confidence score for a first location of a given object in the first map; determining a second confidence score for a second location of the given object in the second map; inserting the given object into a reconciled map at a position based on the first confidence score and the second confidence score; anchoring an Augmented Reality object in the reconciled map at coordinates based on the position of the given object; and outputting the reconciled map to an Augmented Reality device.

Abstract: Embodiments provide for tracking location and resolving drift in Augmented Reality (AR) devices. The AR devices includes computing devices having screens on a first face and cameras on a second, opposite face to project an image onto optical arrangements for viewing by wearers of the AR devices. The AR devices map locations for real objects in the environment to a virtual environment; anchor virtual objects at anchor locations within the virtual environment; capture station keeping images of the environment from a first Field of View via the camera; determine a second, different Field of View in the environment for the wearer of the AR device based on the relative locations of real objects present in the station keeping images; and output images depicting the virtual objects at positions on the screen to depict the virtual objects in the physical environment at the anchor locations.

Abstract: Techniques for providing multi-screen interaction in an interactive application. A primary device establishes an application session. The application session includes a virtual environment relating to a physical environment in which the primary user device is located. The primary user device is configured to display a graphical image relating to a primary virtual camera view of the virtual environment. A secondary virtual camera view of the virtual environment, relating to the primary virtual camera view, is determined. The primary user device transmits image data relating to the secondary virtual camera view from the primary user device to a secondary user device, which is configured to display a graphical image based on the image data. Responsive to receiving an interaction request from the secondary user device, the primary user device controls the application session based on the interaction request.

Abstract: Embodiments herein provide a method for determining, utilizing an output from a beacon device detected using one or more sensors, a physical location of an interactive device in a physical space. Additionally, the method includes receiving an indication that an augmented reality scene is being displayed, where the augmented reality scene includes the physical space and a first virtual element. The method also includes identifying a predefined dynamic based on characteristics of the interactive device and the first virtual element. Finally, the method includes determining a physical movement to perform based on the determined physical location of the interactive device and the predefined dynamic, and activating the one or more actuators to cause the determined physical movement.

Abstract: Embodiments provide for autonomous drone play and directional alignment by in response to receiving a command for a remotely controlled device to perform a behavior, monitoring a first series of actions performed by the remotely controlled device that comprise the behavior; receiving feedback related to how the remotely controlled device performs the behavior, wherein the feedback is received from at least one of a user, a second device, and environmental sensors; updating, according to the feedback, a machine learning model used by the remotely controlled device to produce a second, different series of actions to perform the behavior; and in response to receiving a subsequent command to perform the behavior, instructing the remotely controlled device to perform the second series of actions.

Abstract: To prevent virtual content from occluding physical objects in an augmented reality (AR) system, the embodiments herein describe generating a pass-through texture using a pre-generated model of a physical object. In one embodiment, the AR system determines the location of the physical object as well as its orientation. With this information, the system rotates and re-sizes the pre-generated model of the object to match its location and orientation in the real-world. The system then generates the pass-through texture from the pre-generated model and inserts the texture into the virtual content. When the virtual content is displayed, the pass-through texture permits light from the real-world view to pass through substantially unaffected by the virtual content. In this manner, the physical object (which aligns with the location of the pass-through texture in the virtual content) can be seen by the user—i.e., is not occluded by the virtual content.

Abstract: Embodiments herein provide a method for determining, utilizing an output from a beacon device detected using one or more sensors, a physical location of an interactive device in a physical space. Additionally, the method includes receiving an indication that an augmented reality scene is being displayed, where the augmented reality scene includes the physical space and a first virtual element. The method also includes identifying a predefined dynamic based on characteristics of the interactive device and the first virtual element. Finally, the method includes determining a physical movement to perform based on the determined physical location of the interactive device and the predefined dynamic, and activating the one or more actuators to cause the determined physical movement.

Abstract: Systems, methods and articles of manufacture relating to calibration of a magnetometer in an electronic device suitable for use in an augmented reality (AR) and/or virtual reality (VR) video game. An electronic video game device includes a magnetometer. A prompt is displayed in the video game requesting a user-performed first gameplay motion using the electronic video game device where the first gameplay motion is a pre-defined motion used for calibrating the magnetometer. A first plurality of measurements is received from the magnetometer relating to a magnetic field in a physical gameplay environment, where the first plurality of measurements is taken by the magnetometer during the first gameplay motion. The first plurality of measurements is analyzed to detect a magnetic bias in the magnetometer. The magnetometer is calibrated to offset the magnetic bias.