Abstract: A game controller includes an image capture unit; a body; at least one input device assembled with the body, the input device manipulable by a user to register an input from the user; an inertial sensor operable to produce information for quantifying a movement of said body through space; and at least one light source assembled with the body; and a processor coupled to the image capture unit and the inertial sensor. The processor is configured to track the body by analyzing a signal from the inertial sensor and analyzing an image of the light source from the image capture unit. The processor is configured to establish a gearing between movement of the body and actions to be applied by a computer program.

Abstract: A game controller is disclosed.

Abstract: One or more images can be captured with a depth camera having a capture location in a coordinate space. First and second objects in the one or more images can be identified and assigned corresponding first and second object locations in the coordinate space. A relative position can be identified in the coordinate space between the first object location and the second object location when viewed from the capture location by computing an azimuth angle and an altitude angle between the first object location and the object location in relation to the capture location. The relative position includes a dimension of depth with respect to the coordinate space. The dimension of depth is determined from analysis of the one or more images. A state of a computer program is changed based on the relative position.

Abstract: A method and apparatus for capturing image and sound during interactivity with a computer program is provided. The apparatus includes an image capture unit that is configured to capture one or more image frames. Also provided is a sound capture unit. The sound capture unit is configured to identify one or more sound sources. The sound capture unit generates data capable of being analyzed to determine a zone of focus at which to process sound to the substantial exclusion of sounds outside of the zone of focus. In this manner, sound that is captured and processed for the zone of focus is used for interactivity with the computer program.

Abstract: Methods for tracking a controller of a video game and for providing inputs to a system are disclosed.

Abstract: Systems, methods and for analyzing game control input data are disclosed. A machine-readable medium having embodied thereon instructions for analyzing game control input data is also disclosed.

Abstract: A method of moving objects in a graphical user interface, includes obtaining a video image of a user of the interface; displaying the video image on a display such that the video image is superposed with one or more objects displayed on the display; and moving one or more objects displayed on the display based on recognition of motions of the video image of the user. Recognition of motions of the video image may include recognition of motions of an image of the user's hand.

Abstract: A method for interfacing with a video game is provided, including: rendering display data on a main display, the display data defining a scene rendered by the video game, the display data being configured to include a visual cue; capturing the display data by an image capture device incorporated into a portable device; analyzing the captured display data to identify the visual cue; in response to identification of the visual cue, determining additional information that is in addition to the scene of the video game that is displayed on the main display, the additional information defining graphics or text to be added to the scene of the video game; presenting the additional information on a personal display incorporated into the portable device, the presentation of the additional information being synchronized to the rendering of the display data on the main display.

Abstract: A method for detecting direction conveyed by a user when interfacing with a computer program executed on a computing device. The computing device being interfaced with an image capture device. The method includes detecting a human head of a person in the image taken with the image capture device and assigning the human head a head location. And, detecting an object held by the person in the image and assigning the object an object location. The method determines a relative position in space between the head location and the object location from the capture location, such that the relative position is used to identify a pointing direction of the object.

Abstract: A method of moving objects in a graphical user interface, includes obtaining a video image of a user of the interface; displaying the video image on a display such that the video image is superposed with one or more objects displayed on the display; and moving one or more objects displayed on the display based on recognition of motions of the video image of the user. Recognition of motions of the video image may include recognition of motions of an image of the user's hand.

Abstract: A portable device configured to provide an augmented reality experience is provided. The portable device has a display screen configured to display a real world scene. The device includes an image capture device associated with the display screen. The image capture device is configured to capture image data representing the real world scene. The device includes image recognition logic configured to analyze the image data representing the real world scene. Image generation logic responsive to the image recognition logic is included. The image generation logic is configured to incorporate an additional image into the real world scene. A computer readable medium and a system providing an augmented reality environment are also provided.

Abstract: A controller provides inputs to control execution of a program by combining controller input information and supplementary information utilizing a value of a first one of the controller input information or the supplementary input information as a modifying input to the game program for modifying a mapping or gearing of an input controlling a still active function activated in accordance with at least one of a second one of the controller input information or the supplementary input information or vice versa. It is emphasized that this abstract is provided to comply with the rules requiring an abstract that will allow a searcher or other reader to quickly ascertain the subject matter of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims.

Abstract: A method for controlling a view of a scene is provided. The method initiates with detecting an initial location of a control object. An initial view of the scene is displayed on a virtual window, the initial view defined by a view-frustum based on a projection of the initial location of the control object through outer edges of the virtual window. Movement of the control object to a new location is detected. An updated view of the scene is displayed on the virtual window, the updated view defined by an updated view-frustum based on a projection of the new location of the control object through the outer edges of the virtual window.

Abstract: One or more images can be captured with a depth camera having a capture location in a coordinate space. First and second objects in the one or more images can be identified and assigned corresponding first and second object locations in the coordinate space. A relative position can be identified in the coordinate space between the first object location and the second object location when viewed from the capture location by computing an azimuth angle and an altitude angle between the first object location and the object location in relation to the capture location. The relative position includes a dimension of depth with respect to the coordinate space. The dimension of depth is determined from analysis of the one or more images. A state of a computer program is changed based on the relative position.

Abstract: A controller provides inputs to control execution of a program by combining controller input information and supplementary information indicating a three-dimensional motion of the controller. The combined input can be obtained by assigning the value of the controller input information as coarse control information and assigning the value of the supplementary input information as fine control information or vice versa.

Abstract: Detecting direction pointing direction when interfacing with a computer program is described. Two or more stereo images presented in front of two or more corresponding image capture devices can be captured. Each image capture device having a capture location in a coordinate space. The image capture devices can be synchronized with a strobe signal that is visible to each image capture device. When a person is captured in the image, first and second body parts of the person in the image can be identified and assigned first and second locations in the coordinate space. A relative position that includes a dimension of depth can be identified in coordinate space between the first location and the second location when viewed from the capture location.

Abstract: An electronic device may be controlled using nerve analysis by measuring a nerve activity level for one or more body parts of a user of the device using one or more nerve sensors associated with the electronic device. A relationship can be determined between the user's one or more body parts and an intended interaction by the user with one or more components of the electronic device using each nerve activity level determined. A control input or reduced set of likely actions can be established for the electronic device based on the relationship determined.

Abstract: A peripheral device for communicating with a computer gaming system having an image capture device associated therewith is provided. The image capture device is configured to capture image data of the peripheral device and the computer system is configured to exchange wireless communication data with the peripheral device. The peripheral device includes a body having a first location and a second location, where the first location is defined for a first light emitting object and the second location is defined for a second light emitting object. The first and second illuminating objects having a size that is identifiable in captured image data. The first location on the body is at a fixed predetermined distance from the second location. The peripheral device includes one or more buttons, and circuitry interfaced with the first and second light emitting objects.

Abstract: One or more images can be captured with a depth camera having a capture location in a coordinate space. First and second objects in the one or more images can be identified and assigned corresponding first and second object locations in the coordinate space. A relative position can be identified in the coordinate space between the first object location and the second object location when viewed from the capture location by computing an azimuth angle and an altitude angle between the first object location and the object location in relation to the capture location. The relative position includes a dimension of depth with respect to the coordinate space. The dimension of depth is determined from analysis of the one or more images. A state of a computer program is changed based on the relative position.

Abstract: An input device for interfacing with a computing device is provided. The input device includes a body configured to be held within a human hand. The input device includes a light emitting diode (LED) affixed to the body and a power supply for the LED. A mode change activator is integrated into the body, where the mode change activator is configured to cause a change of a color of a light originating from the LED. The color change is capable of being detected to cause a mode change at the computing device. Methods for detecting input commands from an input source within a field of sight of an image capture device, and a computing system which includes the input device are provided.