Abstract: A multi-channel image capture system includes: a multi-channel image sensor including a plurality of first pixels configured to detect light in a first band and a plurality of second pixels configured to detect light in a second band different from the first band; an image signal processor coupled to the multi-channel image sensor, the image signal processor being configured to: store a first plurality of capture parameters and a second plurality of capture parameters; control the multi-channel image sensor to capture a first image frame according to the first plurality of capture parameters; control the multi-channel image sensor to capture a second image frame according to the second plurality of capture parameters; and transmit the first image frame and the second image frame to a host processor.

Abstract: Embodiments in accordance with this invention disclose systems and methods for implementing head tracking based graphical user interfaces that incorporate gesture reactive interface objects. The disclosed embodiments perform a method in which a GUI includes interface objects is rendered and displayed. Image data of an interaction zone is captured. A targeting gestured targeting a targeted interface object is detected in the captured image data and a set of 3D head interaction gestures are enabled. Additional image data is captured. Motion of at least a portion of a human head is detected and one of the 3D head interactions is identified. The rendering of the interface is modified in response to the detection of one of the 3D head interactions and the modified interface is displayed.

Abstract: Natural three-dimensional (xw, yw, zw, tw) gesture player interaction with a two-dimensional game application rendered on a two or three dimensional display includes mapping acquired (xw, yw, zw, tw) gesture data to virtual game-world (xv, yv, zv, tv) coordinates or vice versa, and scaling if needed. The game application is caused to render at least one image on the display responsive to the mapped and scaled (xw, yw, zw) data, where the display and game interaction is rendered from the player's perception viewpoint. The (xw, yw, zw) data preferably is acquired using spaced-apart two-dimensional cameras coupled to software to reduce the acquired images to a relatively small number of landmark points, from which player gestures may be recognized. The invention may be implemented in a handheld device such as a smart phone or tablet, which device may include a gyroscope and/or accelerometer.

Abstract: A projection system configured to emit patterned light along a projection optical axis includes: a diffractive optical element configured to perform a collimation function on the light emitted by the light emitter and to perform a pattern generation function to replicate the collimated light in a pattern, the pattern having substantially no collimated zero-order; and a light emitter configured to emit light toward the diffractive optical element, wherein the collimation function is configured to collimate the light emitted from the light emitter, and wherein the pattern generation function is configured to replicate the collimated light to produce the patterned light.

Abstract: A sponsor of ads included in media content broadcast to devices by a media broadcast system for viewing by users can receive realtime feedback from users indicative of user evaluation of the presently broadcast and viewed ad. User devices anonymously acquire, process, analyze and broadcast user responses to broadcast ads viewed on the device, the responses preferably made with natural user gestures. User responses broadcast from the device are received by the media broadcast system. Ad sponsors may the customized the ad and/or future ads for the user based upon feedback and, if present, a user history profile. Broadcast ads can allow user to preselect desired ads by interacting with sponsor logos or icons presented on the device. Gesture data can be acquired, processed and broadcast to the media broadcast system for latent, incomplete, user responses, and for responses made during non-ad portions of the broadcast media.

Abstract: A method for three-dimensional reconstruction of a scene includes: controlling a projection source to emit invisible light toward the scene; while the projection source is emitting light: controlling a first multi-channel image sensor to capture a first image, the first multi-channel image sensor including visible light detecting pixels and invisible light detecting pixels, the first image including a first invisible light channel and a first visible light channel; controlling a second multi-channel image sensor to capture a second image substantially simultaneously as the capture of the first image, the second multi-channel image sensor including visible light detecting pixels and invisible light detecting pixels, the second image including a second invisible light channel and a second visible light channel; performing stereo analysis of the first image and the second image in accordance with the invisible light channels and the visible light channels to generate a depth map.

Abstract: A video transmitting system includes: a display configured to display an image in a first direction; cameras including: a first camera adjacent a first edge of the display; and a second camera adjacent a second edge of the display, at least a portion of the display being proximal a convex hull that includes the first camera and the second camera, the first camera and the second camera having substantially overlapping fields of view encompassing the first direction; and an image processor to: receive a position of a virtual camera relative to the cameras substantially within the convex hull and substantially on the display, the virtual camera having a field of view encompassing the first direction; receive raw images captured by the cameras at substantially the same time; and generate processed image data from the raw images for synthesizing a view in accordance with the position of the virtual camera.

Abstract: A projection system configured to emit patterned light along a projection optical axis includes: a diffractive optical element configured to perform a collimation function on the light emitted by the light emitter and to perform a pattern generation function to replicate the collimated light in a pattern, the pattern having substantially no collimated zero-order; and a light emitter configured to emit light toward the diffractive optical element, wherein the collimation function is configured to collimate the light emitted from the light emitter, and wherein the pattern generation function is configured to replicate the collimated light to produce the patterned light.

Abstract: Dynamic projection of at least first and second patterns contributes detectable disparity onto a scene that includes a target object. The scene is imaged with two-dimensional cameras whose acquired imagery includes disparity contributions whose presence enable a three-dimensional reconstruction depth map to be rapidly and accurately generated. In one embodiment coherent light is input to a first DOE within whose near range output is disposed a second DOE, whose far range output projects an image. Electronically varying effective optical distance between the two DOEs varies the pattern projected from the second DOE. A processor system and algorithms enable dynamic intelligent selection of projected patterns to more readily discern target object characteristics: shape, size, velocity. Patterns can implement spatio-temporal depth reconstruction, spatio-temporal depth reconstruction, and even single-camera spatio-temporal light coding reconstruction.

Abstract: User wearable eye glasses include a pair of two-dimensional cameras that optically acquire information for user gestures made with an unadorned user object in an interaction zone responsive to viewing displayed imagery, with which the user can interact. Glasses systems intelligently signal process and map acquired optical information to rapidly ascertain a sparse (x,y,z) set of locations adequate to identify user gestures. The displayed imagery can be created by glasses systems and presented with a virtual on-glasses display, or can be created and/or viewed off-glasses. In some embodiments the user can see local views directly, but augmented with imagery showing internet provided tags identifying and/or providing information as to viewed objects. On-glasses systems can communicate wirelessly with cloud servers and with off-glasses systems that the user can carry in a pocket or purse.

Abstract: A method and system analyzes data acquired by image systems to more rapidly identify objects of interest in the data. In one embodiment, z-depth data are segmented such that neighboring image pixels having similar z-depths are given a common label. Blobs, or groups of pixels with a same label, may be defined to correspond to different objects. Blobs preferably are modeled as primitives to more rapidly identify objects in the acquired image. In some embodiments, a modified connected component analysis is carried out where image pixels are pre-grouped into regions of different depth values preferably using a depth value histogram. The histogram is divided into regions and image cluster centers are determined. A depth group value image containing blobs is obtained, with each pixel being assigned to one of the depth groups.

Abstract: A multi-channel image capture system includes: a multi-channel image sensor including a plurality of first pixels configured to detect light in a first band and a plurality of second pixels configured to detect light in a second band different from the first band; an image signal processor coupled to the multi-channel image sensor, the image signal processor being configured to: store a first plurality of capture parameters and a second plurality of capture parameters; control the multi-channel image sensor to capture a first image frame according to the first plurality of capture parameters; control the multi-channel image sensor to capture a second image frame according to the second plurality of capture parameters; and transmit the first image frame and the second image frame to a host processor.

Abstract: User interaction with a display is detected substantially simultaneously using at least two cameras whose intersecting FOVs define a three-dimensional hover zone within which user interactions can be imaged. Separately and collectively image data is analyzed to identify a relatively few user landmarks. A substantially unambiguous correspondence is established between the same landmark on each acquired image, and a three-dimensional reconstruction is made in a common coordinate system. Preferably cameras are modeled to have characteristics of pinhole cameras, enabling rectified epipolar geometric analysis to facilitate more rapid disambiguation among potential landmark points. Consequently processing overhead is substantially reduced, as are latency times. Landmark identification and position information is convertible into a command causing the display to respond appropriately to a user gesture.

Abstract: A method and system analyzes data acquired by image systems to more rapidly identify objects of interest in the data. In one embodiment, z-depth data are segmented such that neighboring image pixels having similar z-depths are given a common label. Blobs, or groups of pixels with a same label, may be defined to correspond to different objects. Blobs preferably are modeled as primitives to more rapidly identify objects in the acquired image. In some embodiments, a modified connected component analysis is carried out where image pixels are pre-grouped into regions of different depth values preferably using a depth value histogram. The histogram is divided into regions and image cluster centers are determined. A depth group value image containing blobs is obtained, with each pixel being assigned to one of the depth groups.