Abstract: A computer game can be viewed on a TV through an augmented reality (AR) head-mounted display (HMD). The wearer can make a gesture such as a first to cause the HMD to present 2D rectangles and 3D objects representing locations in the room that the game can be “presented” in. The wearer may then open his hand and “cast” toward one of the room locations, and the game is transferred to the HMD for presentation in the associated rectangle (in 2D) or 3D object (in 3D).

Abstract: A mat has an embedded array of light sensors under a micro-faceted lens array to provide an array of light intensity values for various incident angles of incident real world light. Virtual objects may be projected onto the mat and, using the information from the light sensors, virtually illuminated to match the real-world lighting environment. A touch sensor array may be embedded in the mat to allow users to interact with the virtual objects and provide other input. The surface of the micro-faceted lens array may be coated with a patterned infrared (IR) reflective coating, such that the mat passively reflects a specific band of IR wavelengths in a pattern to act as a fiducial marker.

Abstract: A spectator of a computer simulation who is not playing the computer simulation and is associated with a spectator device for viewing the simulation without controlling it is identified. An indication can be received from the spectator that a portion of the computer simulation contains an error. The indication is transmitted to a computer network, from whence is received a corrected version of the computer simulation that does not contain the error. The corrected version of the computer simulation can then be presented. The spectator may be awarded for reporting the error.

Abstract: A mobile device such as a mobile phone is used to image a nearby user for eye tracking at a closer range than would be provided by cameras on a display device such as a TV and send the eye tracking data to a server streaming video being presented on the display. The eye tracking data is used by the server to identify a region of interest (ROI) in the video, which is more highly compressed than regions outside the ROI to facilitate efficient compression of the video game image while preserving the quality of the ROI in the image where the user is looking.

Abstract: A projection system projects images onto a projection surface in, for example, a computer game head-mounted display (HMD). The light is projected through a spatial light modulator that contains a phase-only image of a Freeform Fourier Lens that is a combination of a Fresnel lens, an X-phase grating, a Y-phase grating, and a radial grating. The freeform lens is a condensing freeform lens that causes the gradual shrinking of portions of the laser-projected image, decreasing the perceived pixel pitch in at least one foveal area on the projection surface compared to a non-modulated laser image. The center positions of the Fresnel lens and radial grating can be changed in the X and Y axes, moving the condensed foveal areas in accordance with eye tracking of the user. In effect, the system projects a Foveated image that contains variable pixel pitch such that a user perceives a higher visual acuity in his gaze direction to the projected surface.

Abstract: Stereoscopic viewing systems may be adjusted for a user's inter-pupillary distance (IPD). An inter-pupillary distance (IPD) value is determined from a signal from a position sensitive device coupled to two moveable sighting fixtures mounted to a head mounted display (HMD) having two fixed optics configured for a specific inter-pupillary distance. The position sensitive device is configured to produce a signal that corresponds to a distance between the two sighting fixtures. The images are warped with a processor to optimize display of the images with the two fixed optics for the determined IPD value.

Abstract: Alpha values from an image frame buffer are used to control one or more of a transparent display and an occlusion screen adjacent to the transparent display to render images with coloring perceptible to a person viewing the transparent display.

Abstract: To improve the social interaction and interaction with external objects for a user wearing an AR head-mounted display (HMD), cameras on the HMD detect that other users (“social targets”) are looking at and/or speaking to the user. The region around the social target in the AR user's view is determined and in response an optical occlusion system in the HMD is changed to allow more outside light to reveal the HMD user's eyes to the other users, The HMD optionally decreases the brightness of the virtual images presented in the HMD to allow the HMD user to improve his real world social and scene interaction.

Abstract: To facilitate control of an AR HMD, a camera unit in a camera sensor system generates RGB/IR images and the system also extrapolates images for times in the future based on light intensity change signals from an event detection sensor (EDS) for HMD pose tracking, hand tracking, and eye tracking. The times in the future are provided by an HMD application which defines the future times, and the RGB/IR images and extrapolated images are sent back to the application. In this way, the camera sensor system enables improved performance tracking (equivalent to using very high-speed camera) at lower bandwidth and power consumption.

Abstract: A spectator of a computer simulation who is not playing the computer simulation and is associated with a spectator device for viewing the simulation without controlling it is identified. An indication can be received from the spectator that a portion of the computer simulation contains an error. The indication is transmitted to a computer network, from whence is received a corrected version of the computer simulation that does not contain the error. The corrected version of the computer simulation can then be presented. The spectator may be awarded for reporting the error.

Abstract: Methods and systems for using a position of a mobile device with an integrated display as an input to a video game or other presentation are presented. Embodiments include rendering an avatar on a mobile device such that it appears to overlay a competing user in the real world. Using the mobile device's position, view direction, and the other user's mobile device position, an avatar (or vehicle, etc.) is depicted at an apparently inertially stabilized location of the other user's mobile device or body. Some embodiments may estimate the other user's head and body positions and angles and reflect them in the avatar's gestures.

Abstract: A method for assisting game play. The method includes monitoring game play of the user playing a gaming application, wherein the user has a defined task to accomplish, wherein the task is associated with a task type. The method includes determining a task type proficiency rule for the task type based on results of a plurality of players taking on a plurality of tasks having the task type. The method includes determining a player proficiency score for accomplishing the task based on the task type proficiency rule. The method includes determining a user predictive rate of success in accomplishing the task based on the player proficiency score, the task type proficiency rule, and the task. The method includes determining a recommendation for the user based on the user predictive rate of success.

Abstract: A method for gaming. The method includes assigning a default game play profile to a user, wherein the default game play profile includes a default game play style that simulates human game play, and wherein the default game play profile is configured to control game play for the user based on the default game play style. The method includes monitoring a plurality of game plays of the user playing a plurality of gaming applications. The method includes generating a user game play profile of the user by adjusting the default game play style based on the plurality of game plays, wherein the user game play profile incudes a user game play style customized to the user. The method includes controlling an instance of a first gaming application based on the user game play style of the user game play profile.

Abstract: A volume holographic film (such as a photopolymer) that is pre-recorded with patterns subsequently is used to encode LED or low-power laser light reflections from an eye into a binary pattern that can be read at very high speeds by a relatively simple complementary metal-oxide-semiconductor (CMOS) sensor that may be similar to a high framerate, low resolution mouse sensor. The low-resolution mono images from the film are translated into eye poses using, for instance, a look up table that correlates binary patterns to X, Y positions or using a pre-trained convolutional neural network to robustly interpret many variations of the binary patterns for conversion to X, Y positions.

Abstract: A mobile device such as a mobile phone is used to image a nearby user for eye tracking at a closer range than would be provided by cameras on a display device such as a TV and send the eye tracking data to a server streaming video being presented on the display. The eye tracking data is used by the server to identify a region of interest (ROI) in the video, which is more highly compressed than regions outside the ROI to facilitate efficient compression of the video game image while preserving the quality of the ROI in the image where the user is looking.

Abstract: A mat has an embedded array of light sensors under a micro-faceted lens array to provide an array of light intensity values for various incident angles of incident real world light. Virtual objects may be projected onto the mat and, using the information from the light sensors, virtually illuminated to match the real-world lighting environment. A touch sensor array may be embedded in the mat to allow users to interact with the virtual objects and provide other input. The surface of the micro-faceted lens array may be coated with a patterned infrared (IR) reflective coating, such that the mat passively reflects a specific band of IR wavelengths in a pattern to act as a fiducial marker.

Abstract: A method for gaming. The method includes assigning a default game play profile to a user, wherein the default game play profile includes a default game play style that simulates human game play, and wherein the default game play profile is configured to control game play for the user based on the default game play style. The method includes monitoring a plurality of game plays of the user playing a plurality of gaming applications. The method includes generating a user game play profile of the user by adjusting the default game play style based on the plurality of game plays, wherein the user game play profile includes a user game play style customized to the user. The method includes controlling an instance of a first gaming application based on the user game play style of the user game play profile.

Abstract: A method for assisting game play. The method includes monitoring game play of the user playing a gaming application, wherein the user has a defined task to accomplish, wherein the task is associated with a task type. The method includes determining a task type proficiency rule for the task type based on results of a plurality of players taking on a plurality of tasks having the task type. The method includes determining a player proficiency score for accomplishing the task based on the task type proficiency rule. The method includes determining a user predictive rate of success in accomplishing the task based on the player proficiency score, the task type proficiency rule, and the task. The method includes determining a recommendation for the user based on the user predictive rate of success.

Abstract: A stereoscopic VR HMD has left and right display portions in which one of the display portions is arranged with a transformation to improve the perceived virtual image resolution and therefore the perceived virtual image quality. The transformation can include offsetting sub-pixels of one of the display portions in the horizontal and if desired vertical dimensions relative to the other display portion. By applying very small (sub-millimeter) offsets along with appropriately adjusted image data, two slightly different virtual images form a stereoscopic image with a perceived increased of resolution in the binocular overlap zone established by the display portions.

Abstract: A projection system projects images onto a projection surface in, for example, a computer game head-mounted display (HMD). The light is projected through a spatial light modulator that contains a phase-only image of a Freeform Fourier Lens that is a combination of a Fresnel lens, an X-phase grating, a Y-phase grating, and a radial grating. The freeform lens is a condensing freeform lens that causes the gradual shrinking of portions of the laser-projected image, decreasing the perceived pixel pitch in at least one foveal area on the projection surface compared to a non-modulated laser image. The center positions of the Fresnel lens and radial grating can be changed in the X and Y axes, moving the condensed foveal areas in accordance with eye tracking of the user. In effect, the system projects a Foveated image that contains variable pixel pitch such that a user perceives a higher visual acuity in his gaze direction to the projected surface.