Abstract: A system that efficiently estimates an object's orientation using magnetic, angular rate, and gravity sensors. The object may be for example a virtual reality headset or a user in a virtual reality environment. Magnetic and gravity data are used to correct errors that accumulate from integrating angular velocity. Unlike systems that use Kalman filter approaches, embodiments of the system apply a simple, highly efficient technique to generate magnetic and gravity error vectors; these error vectors are added directly to the angular velocity prior to integration. Error calculations are performed in the sensor reference frame rather than in the Earth reference frame. Magnetic error correction uses only the horizontal component of the magnetic field, which is efficiently calculated by subtracting off the projection of the magnetic field onto the measured gravity vector. Sensors and processors for calculating orientation may be integrated into a low-latency virtual reality display system.

Abstract: A virtual reality system that uses gestures to obtain commands from a user. Embodiments may use sensors mounted on a virtual reality headset to detect head movements, and may recognize selected head motions as gestures associated with commands. Commands associated with gestures may modify the user's virtual reality experience, for example by selecting or modifying a virtual world or by altering the user's viewpoint within the virtual world. Embodiments may define specific gestures to place the system into command mode or user input mode, for example to temporarily disable normal head tracking within the virtual environment. Embodiments may also recognize gestures of other body parts, such as wrist movements measured by a smart watch.

Abstract: A virtual reality headset that tracks motion of a user relative to a moving vehicle in which the user is located. The system updates the headset's display based on relative motion, rather than on absolute motion due to vehicle movement. Data from sensors on the headset, the vehicle, or both is analyzed to determine the pose (orientation, position, or both) of the headset relative to the vehicle. Sensors may include headset-mounted cameras that observe the vehicle's interior from the user's viewpoint. Analysis of camera images may use optical flow or it may compare images to reference images captured from known poses. The system may also combine data from headset sensors and vehicle sensors to calculate the relative pose. Headset-mounted cameras may function both as relative pose sensors for virtual reality, and as image capture devices for display of the real environment on the headset.

Abstract: A display system for a head mounted device that provides a wide field-of-view image to the user. The system may use a pair of displays angled relative to one another, and a lens or lenses between the displays and the user's eyes to generate a wide field-of-view image. Lenses may be for example gradient index lenses, Fresnel lenses, or holographic optical elements in order to provide significant and complex bending of light across the field of view with relatively thin lenses. The system may provide lower resolution images at the periphery of the user's field of view, using for example light emitting elements at the periphery that are directed by the lens or lenses towards the outside edges of the field of view.

Abstract: A virtual reality virtual theater system that generates or otherwise displays a virtual theater, for example in which to view videos, such as movies or television. Videos may be for example 2D or 3D movies or television programs. Embodiments create a virtual theater environment with elements that provide a theater-like viewing experience to the user. Virtual theaters may be for example indoor movie theaters, drive-in movie theaters, or home movie theaters. Virtual theater elements may include for example theater fixtures, décor, and audience members; these elements may be selectable or customizable by the user. One or more embodiments also render sound by placing virtual speakers in a virtual theater and projecting sounds from these virtual speakers onto virtual microphones corresponding to the position and orientation of the user's ears. Embodiments may also employ minimal latency processing to improve the theater experience.

Abstract: A low-latency virtual reality display system that provides rapid updates to virtual reality displays in response to a user's movements. Display images generated by rendering a 3D virtual world are modified using an image warping approximation, which allows updates to be calculated and displayed quickly. Image warping is performed using various rerendering approximations that use simplified 3D models and simplified 3D to 2D projections. An example of such a rerendering approximation is a shifting of all pixels in the display by a pixel translation vector that is calculated based on the user's movements; pixel translation may be done very quickly, possibly using hardware acceleration, and may be sufficiently realistic particularly for small changes in a user's position and orientation. Additional features include techniques to fill holes in displays generated by image warping, synchronizing full rendering with approximate rerendering, and using prediction of a user's future movements to reduce apparent latency.