Abstract: A device for interfacing with an interactive program, including: a housing, including, an IR projector for projecting an IR pattern into an interactive environment; a camera for capturing images of the projected IR pattern; an image processor configured to process the captured images for identification of the projected IR pattern; a transceiver configured to communicate with a computing device that executes the interactive program, the transceiver configured to transmit data generated by the image processor to the computing device, such that a position of the device in the interactive environment is determined based on the captured images of the IR pattern, the determined position of the device being utilized to update a state of the executing interactive program at the computing device.

Abstract: A controller (110) for controlling an execution of a game program by a processor for enabling an interactive game to be played by a user includes a body (111) having a section to be oriented towards a screen when a progress of a game provided via execution of the game apparatus is displayed upon the screen, and at least one photonically detectable (“PD”) element (e.g. 122, 124, 126, and/or 128) assembled with the body, a position of the photonically detectable element within an image being recordable by an image capture device (112) when the section is oriented at least partly towards the screen, wherein positions of the PD element at different points in time are quantifiable to quantify movement of the body in space.

Abstract: A system is provided, including the following: an input device, the input device including, a light emitting diode (LED) array; an image capture device configured to capture images of an interactive environment in which the input device is disposed, the image capture device configured to generate captured image data that is processed to determine a movement of the input device in the interactive environment; wherein a gearing is established that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display, the gearing changing to different settings during the movement of the image as the movement of the image is rendered to the display.

Abstract: A method for executing a game presented on a screen of a head mounted display (HMD) is provided. Execution renders interactive scenes of the game on the screen. The method includes receiving data of the HMD worn by a user to identify an HMD spatial position. The method also includes receiving data of a controller held by the user to identify a controller spatial position. During game play, the method includes rendering a point of view into the interactive scenes of the game on the screen, where the point of view is a volume of space rendered in three-dimensions. The method includes adjusting the point of view as the HMD spatial position is detected to change. The controller can drive interactivity in the game and changes in position of the controller and the HMD are correlated to provide an immersive three-dimensional game play experience.

Abstract: A device for enabling user control for interfacing with a video game is provided, including the following: a projector for projecting a radiation pattern; a camera for capturing images of the projected radiation pattern; an image processor configured to process the captured images for identification of the projected radiation pattern; a transceiver configured to communicate with a game console that executes the video game, the transceiver configured to transmit data generated by the image processor to the game console, such that a position of the device is determined based on the captured images of the radiation pattern, the determined position of the device being utilized to update a state of the executing video game at the game console; a harness for attaching the device to the user.

Abstract: Systems and methods for executing content to be rendered on a screen of a head mounted display (HMD) are provided. One method includes executing the content to render interactive scenes on the screen and tracking an orientation direction of the HMD when worn on a head of a user and the interactive scenes are being rendered on the screen. The method includes changing view directions into the interactive scenes based on changes in the orientation direction of the HMD, such that movements of the head of the user causes the changes in the view directions into the interactive scenes. The method further includes receiving images of a real world space using a camera of the HMD. The camera of the HMD is configured to capture a location of real world objects in the real world space relative to the user of the HMD.

Abstract: A method is provided, including: receiving inertial data from an input device, the inertial data being generated from one or more inertial sensors of the input device; receiving captured image data from an image capture device configured to capture images of an interactive environment in which the input device is disposed, the input device having a light emitting diode (LED) array that generates infrared light; processing the inertial data and the captured image data to determine a movement of the input device in the interactive environment; establishing a gearing that adjusts an amount by which the movement of the input device is mapped to movement of an image that is rendered to a display; changing the gearing to different settings during the movement of the image as the movement of the image is rendered to the display.

Abstract: Methods, systems, and computer programs are presented for managing the sensory stimulus generated by a head mounted display (HMD). One method includes an operation for identifying the game state of a game being executed for display on a head mounted display (HMD). Further, the method includes an operation for determining the game intensity value for the game state based on user intensity ratings obtained from user play of the game. The user intensity ratings represent the level of sensory stimulus presented to users during the user play of the game via the HMDs of the respective users. Further, the method includes an operation for presenting the game intensity value to the HMD for rendering during execution of the game.

Abstract: A method and system for generating view adaptive spatial audio is disclosed. The method includes facilitating receipt of a spatial audio. The spatial audio comprises a plurality of audio adaptation sets, each audio adaptation set associated with a region among a plurality of regions, each audio adaptation set comprising one or more audio signals encoded at one or more bit rates, each of the one or more audio signals segmented into a plurality of audio segments. The method includes detecting a change in region from a source region to a destination region associated with a change in a head orientation of a user. The source region and the destination region are from among the plurality of regions. Further, the method includes facilitating a playback of the spatial audio by at least in part performing crossfading between at least one audio segment each of the source region and the destination region.

Abstract: A tracking device for use in obtaining information for controlling an execution of a game program by a processor for enabling an interactive game to be played by a user and related apparatus are disclosed.

Abstract: A system for interfacing with an interactive program is provided, including: a computing device for executing the interactive program, the interactive program being stored on a memory, the computing device enabling user control and input; a display device for rendering image content associated with an interactive program, the display device being configured to be attached to the user; wherein the computing device is configured to receive data from an image capture device to determine and track a position of the display device; wherein the computing device is configured to define at least two interactive zones, each interactive zone being defined by a spatial region configured to change a function of the display device when the display device is moved between the at least two interactive zones; and wherein the computing device is configured to set the function of the display device.

Abstract: A method and system for view optimization of a 360 degrees video is provided. The method includes generating two-dimensional video frame from the 360 degrees video. The macroblocks are generated for the two-dimensional video frame. A foveated region of interest for the two-dimensional video frame is defined based on a given view orientation. DCT (Discrete Cosine Transform) coefficients are generated for the macroblocks. View adaptive DCT domain filtering is then performed on the DCT coefficients using the foveated region of interest. Quantization offset is calculated for the DCT coefficients using the foveated region of interest. The DCT coefficients are quantized using the quantization offset to generate encoded two-dimensional video frame for the view orientation. A new view orientation is then set as the given view orientation and steps of generating, performing, calculating, and quantizing are performed for each view orientation and each video frame to generate view optimized video.

Abstract: A method and system for constructing view from multiple video streams is provided. The method includes receiving a view independent stream. The method further includes selecting a first view dependent stream, wherein the view independent stream and the first view dependent stream has at least one different geometry. The method also includes generating end user views corresponding to the view independent stream and the first view dependent stream. Further, the method includes blending the end user views to generate a view for display.

Abstract: Methods and system for reconstructing 360-degree video is disclosed. A video sequence V1 including a plurality of frames associated with spherical content at a first frame rate and a video sequence V2 including a plurality of frames associated with a predefined viewport at a second frame rate is received by a processor. The first frame rate is lower than the second frame rate. An interpolated video sequence V1? of the video sequence V1 is generated by creating a plurality of intermediate frames between a set of consecutive frames of the plurality of frames of the sequence V1 corresponding to the second frame rate of the video sequence V2. A pixel based blending of each intermediate frame of the plurality of the intermediate frames of sequence V1? with a corresponding frame of the plurality of frames the sequence V2 is performed to generate a fused video sequence Vm for displaying.

Abstract: A method and system for layer based encoding of a 360 degrees video is provided. The method includes receiving, by a server, an input video. The input video includes multiple groups of pictures (GOPs). Each GOP starts from a major anchor frame of the input video and includes frames till next major anchor frame. The method also includes generating a first layer. The first layer includes one encoded frame per GOP. The method further includes generating a first sub-layer. The first sub-layer includes encoded frames of multiple mini-GOPs and reconstructed frames of encoded frames of the first layer. Each mini-GOP includes frames between two major anchor frames. Furthermore, the method includes outputting encoded video including the first layer and the first sub-layer.

Abstract: Systems and methods for executing content to be rendered on a screen of a head mounted display (HMD) are provided. One method includes executing the content to render interactive scenes on the screen and tracking an orientation direction of the HMD when worn on a head of a user and the interactive scenes are being rendered on the screen. The method includes changing view directions into the interactive scenes based on changes in the orientation direction of the HMD, such that movements of the head of the user causes the changes in the view directions into the interactive scenes. The method further includes receiving images of a real world space using a camera of the HMD. The camera of the HMD is configured to capture a location of real world objects in the real world space relative to the user of the HMD.

Abstract: Methods, systems, and computer programs are presented for managing the sensory stimulus generated by a head mounted display (HMD). One method includes an operation for identifying the game state of a game being executed for display on a head mounted display (HMD). Further, the method includes an operation for determining the game intensity value for the game state based on user intensity ratings obtained from user play of the game. The user intensity ratings represent the level of sensory stimulus presented to users during the user play of the game via the HMDs of the respective users. Further, the method includes an operation for presenting the game intensity value to the HMD for rendering during execution of the game.

Abstract: Systems and methods for executing a game presented on a screen of a head mounted display include executing a game. The execution of the game renders interactive scenes of the game on the screen of the HMD. Images identifying a shift in gaze direction of the user wearing the HMD, are received. The gaze shift is detected during viewing of the interactive scenes presented on the HMD screen. Real-world images that are in line with the gaze direction of the user, are captured from a forward-facing camera of the HMD. A portion of the screen is transitioned from a non-transparent mode to a semi-transparent mode in response to the shift in the gaze direction such that at least part of the real world images are presented in the portion of the screen rendering the interactive scenes of the game. The transparent mode is discontinued after a period of time.

Abstract: A device for enabling user control for interfacing with a video game is provided, including the following: a projector for projecting a radiation pattern; a camera for capturing images of the projected radiation pattern; an image processor configured to process the captured images for identification of the projected radiation pattern; a transceiver configured to communicate with a game console that executes the video game, the transceiver configured to transmit data generated by the image processor to the game console, such that a position of the device is determined based on the captured images of the radiation pattern, the determined position of the device being utilized to update a state of the executing video game at the game console; a harness for attaching the device to the user.

Abstract: A method for sharing content with other HMDs includes rendering content of a virtual environment scene on a display screen of a head-mounted display associated with a first user. The display screen rendering the virtual environment scene represents a virtual reality space of the first user. A request to share the virtual reality space of the first user is detected. The request targets a second user. In response to detecting acceptance of the request to share, the virtual reality space of the first user is shared with the second user. The sharing allows synchronizing the virtual environment scene rendered on the head mounted display of the first and the second users.