Abstract: Techniques for tracking eye movement in an augmented reality system identify a plurality of base images of an object or a portion thereof. A search image may be generated based at least in part upon at least some of the plurality of base images. A deep learning result may be generated at least by performing a deep learning process on a base image using a neural network in a deep learning mode. A captured image may be localized at least by performing an image registration process on the captured image and the search image using a Kalman filter model and the deep learning result.

Abstract: A method of fabricating a blazed diffraction grating comprises providing a master template substrate and imprinting periodically repeating lines on the master template substrate in a plurality of master template regions. The periodically repeating lines in different ones of the master template regions extend in different directions. The method additionally comprises using at least one of the master template regions as a master template to imprint at least one blazed diffraction grating pattern on a grating substrate.

Abstract: A wearable device can include an inward-facing imaging system configured to acquire images of a user's periocular region. The wearable device can determine a relative position between the wearable device and the user's face based on the images acquired by the inward-facing imaging system. The relative position may be used to determine whether the user is wearing the wearable device, whether the wearable device fits the user, or whether an adjustment to a rendering location of virtual object should be made to compensate for a deviation of the wearable device from its normal resting position.

Abstract: Described herein are techniques and technologies to identify an encrypted content within a field of view of a user of a VR/AR system and process the encrypted content appropriately. The user of the VR/AR technology may have protected content in a field of view of the user. Encrypted content is mapped to one or more protected surfaces on a display device. Contents mapped to a protected surface may be rendered on the display device but prevented from being replicated from the display device.

Abstract: Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Abstract: A cross reality system enables any of multiple devices to efficiently render shared location-based content. The cross reality system may include a cloud-based service that responds to requests from devices to localize with respect to a stored map. The service may return to the device information that localizes the device with respect to the stored map. In conjunction with localization information, the service may provide information about locations in the physical world proximate the device for which virtual content has been provided. Based on information received from the service, the device may render, or stop rendering, virtual content to each of multiple users based on the user's location and specified locations for the virtual content.

Abstract: An example a head-mounted display device includes a light projector and an eyepiece. The eyepiece is arranged to receive light from the light projector and direct the light to a user during use of the wearable display system. The eyepiece includes a waveguide having an edge positioned to receive light from the display light source module and couple the light into the waveguide. The waveguide includes a first surface and a second surface opposite the first surface. The waveguide includes several different regions, each having different grating structures configured to diffract light according to different sets of grating vectors.

Abstract: An optical scanner is described and includes an optical beam assembly and a transducer. The optical beam assembly includes a pivoting portion having a cantilevered beam; a stationary portion; a first torsional flexure coupling a first side of the pivoting portion to the stationary portion; and a second torsional flexure coupling a second side of the pivoting portion to the stationary portion. The transducer includes a first magnetic element disposed on the pivoting portion and a second magnetic element disposed on the stationary portion. The first and second magnetic elements are configured to generate magnetic fields that interact to rotate the pivoting portion relative to the stationary portion about an axis defined by the first and second torsional flexures.

Abstract: A cross reality system enables portable devices to access stored maps and efficiently and accurately render virtual content specified in relation to those maps. The system may process images acquired with a portable device to quickly and accurately localize the portable device to the persisted maps by constraining the result of localization based on the estimated direction of gravity of a persisted map and the coordinate frame in which data in a localization request is posed. The system may actively align the data in the localization request with an estimated direction of gravity during the localization processing, and/or a portable device may establish a coordinate frame in which the data in the localization request is posed aligned with an estimated direction of gravity such that the subsequently acquired data for inclusion in a localization request, when posed in that coordinate frame, is passively aligned with the estimated direction of gravity.

Abstract: A multiple-input, multiple-output (MIMO) transceiver comprises a plurality of RF chains, a plurality of antennas, a plurality of switching components, and control circuitry operatively coupled to the plurality of switching components. In some examples, a total quantity of RF chains included in the plurality of RF chains is equal to a first value, and a total quantity of antennas included in the plurality of antennas is equal to a second value that is less than the first value.

Abstract: Disclosed herein are systems and methods for presenting an audio signal associated with presentation of a virtual object colliding with a surface. The virtual object and the surface may be associated with a mixed reality environment. Generation of the audio signal may be based on at least one of an audio stream from a microphone and a video stream form a sensor. In some embodiments, the collision between the virtual object and the surface is associated with a footstep on the surface.

Abstract: An image display system includes an optical subsystem configured to emit a modulated light beam, and a scanning mirror for generating a reflected light beam that is scanned according to randomly selected or pseudo-randomly selected scan patterns to generate multiple image fields of a multiple interlaced scan image. A plurality of different scan patterns can be cycled through, randomly or pseudo-randomly selected, for the different image fields to reduce artifacts that may be observed while viewing a projected image.

Abstract: Examples of a light field metrology system for use with a display are disclosed. The light field metrology may capture images of a projected light field, and determine focus depths (or lateral focus positions) for various regions of the light field using the captured images. The determined focus depths (or lateral positions) may then be compared with intended focus depths (or lateral positions), to quantify the imperfections of the display. Based on the measured imperfections, an appropriate error correction may be performed on the light field to correct for the measured imperfections. The display can be an optical display element in a head mounted display, for example, an optical display element capable of generating multiple depth planes or a light field display.

Abstract: An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.

Abstract: Disclosed are methods and systems for a scripting framework and implementations therefor for mixed reality software applications of heterogeneous systems. These methods or systems create a mixed-reality software application that executes across heterogeneous platforms on a server-side instance of a scripting framework and manage a change in the mixed-reality software application using the server-side instance of the scripting framework. Moreover, the change in the mixed-reality software application using a client-side instance of the scripting framework; and the mixed-reality software application may be interactively executed on a mixed-reality device.

Abstract: An apparatus configured to be head-worn by a user, includes: a transparent screen configured to allow the user to see therethrough; a sensor system configured to sense a characteristic of a physical object in an environment in which the user is located; and a processing unit coupled to the sensor system, the processing unit configured to: cause the screen to display a user-controllable object, and cause the screen to display an image of a feature that is resulted from a virtual interaction between the user-controllable object and the physical object, so that the feature will appear to be a part of the physical object in the environment or appear to be emanating from the physical object.

Abstract: A virtual, augmented, or mixed reality display system includes a display configured to display virtual, augmented, or mixed reality image data, the display including one or more optical components which introduce optical distortions or aberrations to the image data. The system also includes a display controller configured to provide the image data to the display. The display controller includes memory for storing optical distortion correction information, and one or more processing elements to at least partially correct the image data for the optical distortions or aberrations using the optical distortion correction information.

Abstract: Systems and methods are disclosed for operating a head-mounted display system based on user perceptibility. The display system may be an augmented reality display system configured to provide virtual content on a plurality of depth planes by presenting the content with different amounts of wavefront divergence. Some embodiments include obtaining an image captured by an imaging device of the display system. Whether a threshold measure or more of motion blur is determined to be exhibited in one or more regions of the image. Based on a determination that the threshold measure or more of motion blur is exhibited in one or more regions of the image, one or more operating parameters of the wearable display are adjusted. Example operating parameter adjustments comprise adjusting the depth plane on which content is presented (e.g., by switching from a first depth plane to a second depth plane), adjusting a rendering quality, and adjusting power characteristics of the system.

Abstract: Expandable band systems for wearable devices, such as spatial computing headsets, are provided which have flexible and conformable form factors that enable users to reliably to secure such wearable devices in place. Further, in the context of spatial computing headsets with an optics assembly supported by opposing temple arms, the expandable band systems provide protection against over-extension of the temple arms or extreme deflections that may otherwise arise from undesirable torsional loading of the temple arms.