Abstract: Optical combiners are provided. The optical combiner may have a see through optically transparent substrate and a patterned region included in the optically transparent substrate and disposed along a wave propagation axis of the substrate. The patterned region may be partially optically reflective and partially optically transparent. The patterned region may comprise a plurality of optically transparent regions of the optically transparent substrate and a plurality of optically reflective regions inclined relative to the optical transparent substrate wave propagation axis. Augmented reality optical apparatus, such a head up display, may include the optical combiner.

Abstract: An augmented-reality (AR) device comprises a physical sensor and a display. The AR device identifies, using the physical sensor, a location and orientation of the AR device within a physical environment. A selection of a type of a virtual sensor and a placement location of the virtual sensor within the physical environment is received at the AR device. The placement location of the virtual sensor is determined relative to the location and orientation of the AR device within the physical environment. The AR device provides the type of the virtual sensor and the placement location of the virtual sensor to a server that generates virtual content for the virtual sensor. The AR device displays the virtual content at the placement location of the virtual sensor in the display.

Abstract: Optical combiners are provided. The optical combiner may have a see through optically transparent substrate and a patterned region included in the optically transparent substrate and disposed along a wave propagation axis of the substrate. The patterned region may be partially optically reflective and partially optically transparent. The patterned region may comprise a plurality of optically transparent regions of the optically transparent substrate and a plurality of optically reflective regions inclined relative to the optical transparent substrate wave propagation axis. Augmented reality optical apparatus, such a head up display, may include the optical combiner.

Abstract: Methods based on growth pattern models are utilized to determine patterns of reflective dots in optical combiners or other components for augmented reality (AR), head mounted displays (HMD) and/or head up display (HUD) applications. Optical combiners including the reflective dots arranged in the grown patterns are provided.

Abstract: Disclosed are systems, methods, and non-transitory computer-readable media for reducing latency in augmented reality displays. A display controller receives, from a GPU, a stream of image pixels of a frame of virtual content to be presented on a display of a display device. The stream of image pixels is received via a high-speed bulk interface that transfers data at least as fast as can be consumed by the display. As the stream of image pixel is received, the display controller converts each respective image pixel from a data format used to transmit the stream of image pixels via the high-speed bulk interface to a data format that is compatible for display by the display. Each converted image pixel is stored in a pixel cell of the display, after which the frame is presented on the display.

Abstract: Systems and methods for providing and/or presenting, to a user, a user interface for an environment that includes virtual objects are disclosed. Exemplary implementations may: obtain, from electronic storage, information regarding virtual objects in a virtual three-dimensional space that has a virtual three-dimensional volume; determine a subset of voxels from the set of voxels such that the subset of voxels encompasses a three-dimensional volume that includes at least part of a first external surface of the first virtual object; determine proximity information for the first virtual object; determine a manipulation granularity; adjust the manipulation granularity based on the proximity information; receive particular user input from the user having a particular input magnitude; manipulate the first virtual object within the virtual three-dimensional space in accordance with the received particular user input; and effectuate presentation of the user interface to the user through a client computing platform.

Abstract: Systems and methods for generating composite depth images are disclosed. Exemplary implementations may: capture, by a depth sensor, a set of depth images over a capture period of time; generate, by an inertial sensor, inertial signals that convey values of one or more inertial parameters characterizing motion of the depth sensor during the capture period of time; select a target capture position based on one or more of the capture positions of the set of depth images; generate, using the values of the one or more inertial parameters during the capture period of time, re-projected depth images; and generate a composite depth image by combining multiple depth images, such multiple depth images including a first re-projected depth image and a second re-projected depth image.

Abstract: A device has an optical sensor, an inertial sensor, and a hardware processor. The optical sensor generates image data. The inertial sensor generates inertia data. The hardware processor receives an augmented reality (AR) authoring template authored at a client device, generates media content using the image data, and receives a selection of spatial coordinates within a three-dimensional region using the inertia data and the image data. The three-dimensional region is identified in the AR authoring template. The hardware processor further identifies an entry in the AR authoring template corresponding to the media content, places the media content at the selected spatial coordinates in the entry in the AR authoring template, and forms AR content using the media content at the selected spatial coordinates placed in the entry in the AR authoring template.

Abstract: Optical adaptive viewport display systems and methods are provided. One such optical adaptive viewport display system has an adaptive pupil device which is optical coupled to an optical combiner. The adaptive pupil device is optically couplable to an image projector and is configured to select a sub-pupil from the pupil of the projector. The selected sub-pupil is optically relayed by relay optics from the adaptive pupil device to an eyebox. The relay optics includes an optical combiner.

Abstract: Optical hyperfocal reflective systems and methods are provided. One such optical hyperfocal reflective system has an optical substrate; an optical input coupling portion configured to input couple a collimated display image to the optical substrate; and an optical hyperfocal output coupling portion integrated with said optical substrate. The optical output coupling portion includes at least one hyperfocal reflective view port formed from a discrete optical hyperfocal reflector spot integrated with the optical substrate.

Abstract: A mobile device identifies a user task provided by an augmented reality application at a mobile device. The mobile device identifies a first physical tool valid for performing the user task from a tool compliance library based on the user task. The mobile device detects and identifies a second physical tool present at the mobile device. The mobile device determines whether the second physical tool matches the first physical tool. The mobile device display augmented reality content that identifies at least one of a missing physical tool, an unmatched physical tool, or a matched physical tool based on whether the second physical tool matches the first physical tool.

Abstract: A head mounted display (HMD) system includes an HMD device worn on a head of a user. The HMD device incorporates electroencephalography (EEG) interfaces for monitoring the brain of a human subject during interaction with the HMD device. Fluctuations in electrical potential that are observed via the EEG interfaces may be used to detect event-related potentials (ERPs). The HMD system may programmatically perform one or more operations in response to detecting ERPs. The HMD system may further include off-board devices that communicate with the HMD device over a wireless communications network.

Abstract: A head-mounted display (HMD) system includes an HMD device wearable upon a head of a user, and a peripheral device that is dockable with the HMD device. The peripheral device comprises a plurality of light source elements disposed along a perimeter of a face of the peripheral device and a peripheral electronic control system. The peripheral electronic control system is configured to form an optically detectable light pattern with the plurality of light source elements by controlling each of the light source elements. The HMD device is configured to detect the optically detectable light pattern with a camera and to display an augmented reality (AR) content based on the identified optically detectable light pattern.

Abstract: A head-mounted display (HMD) system includes an HMD device wearable upon a head of a user, and a peripheral device that is dockable with the HMD device via a pair of electronic connectors. The HMD device and the peripheral device may communicate with each other via wireless communications in a docked configuration, and may communicate with each other via wired communications over the pair of electronic connectors in the docked configuration. The HMD system supports a variety of different modes of operation that may be varied responsive to whether the peripheral device is in the docked configuration or the undocked configuration with the HMD device.

Abstract: Techniques of tracking a user's location are disclosed. In some embodiments, a mobile device captures first sensor data using at least one sensor, determines that a predetermined hazard criteria is not satisfied by an environment of a user of the mobile device, suppresses transmission of a representation of the captured first sensor data to a remote computing device based on the determination that the predetermined hazard criteria is not satisfied, captures second sensor data using the sensor(s), determines that the predetermined hazard criteria is satisfied by the environment of the user, and transmits a representation of the captured second sensor data to the remote computing device based on the determination that the predetermined hazard criteria is satisfied by the environment of the user.

Abstract: Optical combiners are provided. The optical combiner may have a see through optically transparent substrate and a patterned region included in the optically transparent substrate and disposed along a wave propagation axis of the substrate. The patterned region may be partially optically reflective and partially optically transparent. The patterned region may comprise a plurality of optically transparent regions of the optically transparent substrate and a plurality of optically reflective regions inclined relative to the optical transparent substrate wave propagation axis. Augmented reality optical apparatus, such a head up display, may include the optical combiner.

Abstract: An augmented-reality (AR) device detects a physical device located within a first predefined distance of the augmented-reality device. The AR device detects, using an optical sensor, a physical object located within a second predefined distance of the physical device, the physical object being electronically unconnected to the physical device. The AR device determines that the physical object is associated with the physical device and identifies a command based on an identification of both the physical object and the physical device. The physical device is configured to operate the command and display, in the display, virtual content as an overlay to the physical object.

Abstract: An image localization system is described. A server stores a localization model that associates location and orientation data with each picture among a plurality of pictures. The server receives a query for a location and an orientation of a device. The query includes a first picture. A second picture that matches the first picture is identified. The server determines the location and the orientation of the device based on the location and orientation data corresponding to the second picture. The location and orientation data indicates position, orientation, three-dimensional geometry, gyroscope measurement, and accelerometer measurement.

Abstract: A computing device provides augmented reality images of an environment in which the computing device is used. The computing device is further configured to display a graphical user interface for interacting with the computing device. The graphical user interface may be displayed according to one or more configured graphical user interface layouts. The computing device further includes an inertial measurement unit, which provides input for interacting with one or more portions of the graphical user interface. As a user of the computing device moves the computing device, corresponding graphical changes are made to the displayed graphical user interface. In this way, by moving the computing device, the user is able to interact with, and provide input to, the computing device.

Abstract: A system and method for offloading object detection are described. A server receives first sensor data from a first sensor of an augmented reality (AR) display device. The first sensor data indicates a pose of the AR display device relative to a first reference coordinate system. The server detects a physical object using second sensor data received from a second sensor of the AR display device. The server determines, based on the second sensor data, a pose of the physical object relative to the AR display device. The server then determines the pose of the physical object relative to the first reference coordinate system based on the pose of the physical object relative to the AR display device and the pose of the AR display device relative to the first reference coordinate system.