Abstract: A high-resolution image sensor suitable for use in an augmented reality (AR) system to provide low latency image analysis with low power consumption. The AR system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object, selectively output imaging information for that region, and synchronously output high-resolution image frames. The region may be updated dynamically as the image sensor and/or the object moves. The image sensor may output the high-resolution image frames less frequently than the region being updated when the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an AR scene can be developed.

Abstract: An eye tracking system includes a pair of glasses including two frames, a projector coupled to the pair of glasses and operable to project a beam of light, and an eyepiece mounted in one of the two frames and optically coupled to the projector. The eyepiece is operable to direct at least a portion of the beam of light towards an eye of a user. The eye tracking system also includes one or more optical sensors coupled to at least one of the two frames of the pair of glasses and operable to detect a set of reflected signals off the eye of the user, and a processor coupled to the projector and the one or more optical sensors, wherein the processor is operable to determine an eye orientation based on the set of reflected signals.

Abstract: Disclosed is an improved systems and method for navigation and manipulation of browser windows in a 3D mixed reality environment. An improved approach is provided to view a user's windows, regardless of the current location for the user relative to one or more previously-opened windows. A method for displaying windows in a computing environment includes receiving an instruction to select multiple open windows. The method also includes retrieving information for the multiple open windows, where the multiple open windows are associated with different physical locations. The method further includes displaying a representation of the multiple open windows in a single user interface. Moreover, the method includes upon receiving a selection of a selected window of the multiple open windows, loading the selected window into a foreground of a field of view for a user.

Abstract: An optical device includes variable optical material that alters at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light through the optical device in response to a stimulus provided by the device. The device can sense intensity and/or spectral characteristics of ambient light and provide appropriate stimulus to various portions of the optical device to activate the variable optical material and alter at least one of: incident ambient light, spectral content of incident ambient light or direction of incident ambient light.

Abstract: An annular axial flux motor includes a rotor mounted on an annular subsection of a rotatable cam ring and a stator mounted on an annular subsection of a carrier frame. The rotor includes two Halbach arrays of permanent magnets spaced from each other on the cam ring along an axial direction. The stator includes multiple phase electrical windings printed on multiple layers of a printed circuit board (PCB) that are stacked along the axial direction. The multiple layers are positioned between the Halbach arrays, with active side of the Halbach arrays facing to opposite sides of the multiple layers. The Halbach arrays are configured to generate a symmetrical magnetic field and the multiple phase electrical windings are configured to have a same rotor-dependent torque constant, such that the stator can generate a constant torque to rotate the rotor and the cam ring within a finite travel range.

Abstract: Very high refractive index (n>2.2) lightguide substrates enable the production of 70° field of view eyepieces with all three color primaries in a single eyepiece layer. Disclosed herein are viewing optics assembly architectures that make use of such eyepieces to reduce size and cost, simplifying manufacturing and assembly, and better-accommodating novel microdisplay designs.

Abstract: Wearable and optical display systems and methods for operation thereof incorporating monovision display techniques are disclosed. A wearable device may include left and right optical stacks configured to switch between displaying virtual content at a first focal plane or a second focal plane. The wearable device may determine whether or not an activation condition is satisfied. In response to determining that the activation condition is satisfied, a monovision display mode associated with the wearable device may be activated, which may include causing the left optical stack to display the virtual content at the first focal plane and causing the right optical stack to display the virtual content at the second focal plane.

Abstract: A wearable display system including multiple cameras and a processor is disclosed. A greyscale camera and a color camera can be arranged to provide a central view field associated with both cameras and a peripheral view field associated with one of the two cameras. One or more of the two cameras may be a plenoptic camera. The wearable display system may acquire light field information using the at least one plenoptic camera and create a world model using the first light field information and first depth information stereoscopically determined from images acquired by the greyscale camera and the color camera. The wearable display system can track head pose using the at least one plenoptic camera and the world model. The wearable display system can track objects in the central view field and the peripheral view fields using the one or two plenoptic cameras, when the objects satisfy a depth criterion.

Abstract: An image sensor suitable for use in an augmented reality system to provide low latency image analysis with low power consumption. The augmented reality system can be compact, and may be small enough to be packaged within a wearable device such as a set of goggles or mounted on a frame resembling ordinary eyeglasses. The image sensor may receive information about a region of an imaging array associated with a movable object and selectively output imaging information for that region. The region may be updated dynamically as the image sensor and/or the object moves. Such an image sensor provides a small amount of data from which object information used in rendering an augmented reality scene can be developed. The amount of data may be further reduced by configuring the image sensor to output indications of pixels for which the measured intensity of incident light changes.

Abstract: A cross reality system enables portable devices to access stored maps and efficiently and accurately render virtual content specified in relation to those maps. The system may process images acquired with a portable device to quickly and accurately localize the portable device to the persisted maps by constraining the result of localization based on the estimated direction of gravity of a persisted map and the coordinate frame in which data in a localization request is posed. The system may actively align the data in the localization request with an estimated direction of gravity during the localization processing, and/or a portable device may establish a coordinate frame in which the data in the localization request is posed aligned with an estimated direction of gravity such that the subsequently acquired data for inclusion in a localization request, when posed in that coordinate frame, is passively aligned with the estimated direction of gravity.

Abstract: An image display system includes an optical subsystem configured to emit a modulated light beam, and a scanning mirror for generating a reflected light beam that is scanned according to randomly selected or pseudo-randomly selected scan patterns to generate multiple image fields of a multiple interlaced scan image. A plurality of different scan patterns can be cycled through, randomly or pseudo-randomly selected, for the different image fields to reduce artifacts that may be observed while viewing a projected image.

Abstract: A wearable device includes a left optical stack having a left eyepiece configured to receive left virtual image light, a left accommodating lens, and a left compensating lens. The wearable device also includes a right optical stack having a right eyepiece configured to receive right virtual image light, a right accommodating lens, and a right compensating lens. An optical power of the left accommodating lens is equal in magnitude to an optical power of the left compensating lens, an optical power of the right accommodating lens is equal in magnitude to an optical power of the right compensating lens, and the optical power of the left accommodating lens and the optical power of the right accommodating lens differ by an offset amount.

Abstract: Disclosed are methods and systems for a scripting framework and implementations therefor for mixed reality software applications of heterogeneous systems. These methods or systems create a mixed-reality software application that executes across heterogeneous platforms on a server-side instance of a scripting framework and manage a change in the mixed-reality software application using the server-side instance of the scripting framework. Moreover, the change in the mixed-reality software application using a client-side instance of the scripting framework; and the mixed-reality software application may be interactively executed on a mixed-reality device.

Abstract: A head mounted display system for displaying image content to a user comprises at least one display configured to be worn by a user to present virtual content to first and second eyes of a user, one or more inwardly facing sensors or camera configured to monitor one or both of the users eye and processing electronics. This head mounted display system is configured such that virtual content activity can be initiated and/or driven from eye inputs such as gaze direction, eyelid motions (e.g., blinking), and/or other eye gestures.

Abstract: Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object. upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.

Abstract: A method for computing a total weight array includes receiving an image frame captured by a content capture device and identifying a plurality of objects in the image frame. Each object of the plurality of objects corresponds to one of a plurality of pixel groups. The method also includes providing a plurality of neural networks and calculating, for each object of the plurality of objects, an object weight using a corresponding neural network of the plurality of neural networks. The method further includes computing the total weight array by summing the object weight for each of the plurality of objects.

Abstract: A distributed, cross reality system efficiently and accurately compares location information that includes image frames. Each of the frames may be represented as a numeric descriptor that enables identification of frames with similar content. The resolution of the descriptors may vary for different computing devices in the distributed system based on degree of ambiguity in image comparisons and/or computing resources for the device. A descriptor computed for a cloud-based component operating on maps of large areas that can result in ambiguous identification of multiple image frames may use high resolution descriptors. High resolution descriptors reduce computationally intensive disambiguation processing. A portable device, which is more likely to operate on smaller maps and less likely to have the computational resources to compute a high resolution descriptor, may use a lower resolution descriptor.

Abstract: Methods and systems are disclosed for presenting virtual objects on a limited number of depth planes using, e.g., an augmented reality display system. A farthest one of the depth planes is within a mismatch tolerance of optical infinity. The display system may switch the depth plane on which content is actively displayed, so that the content is displayed on the depth plane on which a user is fixating. The impact of errors in fixation tracking is addressed using partially overlapping depth planes. A fixation depth at which a user is fixating is determined and the display system determines whether to adjust selection of a selected depth plane at which a virtual object is presented. The determination may be based on whether the fixation depth falls within a depth overlap region of adjacent depth planes. The display system may switch the active depth plane depending upon whether the fixation depth falls outside the overlap region.

Abstract: Techniques are described for operating an optical system. In some embodiments, light associated with a world object is received at the optical system. Virtual image light is projected onto an eyepiece of the optical system. A portion of a system field of view of the optical system to be at least partially dimmed is determined based on information detected by the optical system. A plurality of spatially-resolved dimming values for the portion of the system field of view may be determined based on the detected information. The detected information may include light information, gaze information, and/or image information. A dimmer of the optical system may be adjusted to reduce an intensity of light associated with the world object in the portion of the system field of view according to the plurality of dimming values.

Abstract: A display device comprises a waveguide configured to guide light in a lateral direction parallel to an output surface of the waveguide. The waveguide is further configured to outcouple the guided light through the output surface. The display device additionally comprises a broadband adaptive lens assembly configured to incouple and to diffract therethrough the outcoupled light from the waveguide. The broadband adaptive lens assembly comprises a first waveplate lens comprising a liquid crystal (LC) layer arranged such that the waveplate lens has birefringence (?n) that varies in a radially outward direction from a central region of the first waveplate lens and configured to diffract the outcoupled light at a diffraction efficiency greater than 90% within a wavelength range including at least 450 nm to 630 nm. The broadband adaptive lens assembly is configured to be selectively switched between a plurality of states having different optical powers.