Abstract: A method and apparatus for segmenting an image are provided. The method may include the steps of clustering pixels from one of a plurality of images into one or more segments, determining one or more unstable segments changing by more than a predetermined threshold from a prior of the plurality of images, determining one or more segments transitioning from an unstable to a stable segment, determining depth for one or more of the one or more segments that have changed by more than the predetermined threshold, determining depth for one or more of the one or more transitioning segments, and combining the determined depth for the one or more unstable segments and the one or more transitioning segments with a predetermined depth of all segments changing less than the predetermined threshold from the prior of the plurality of images.

Abstract: A near-touch interface is provided that utilizes stereo cameras and a series of targeted structured light tessellations, emanating from the screen as a light source and incident on objects in the field-of-view. After radial distortion from a series of wide-angle lenses is mitigated, a surface-based spatio-temporal stereo algorithm is utilized to estimate initial depth values. Once these values are calculated, a subsequent refinement step may be applied in which light source tessellations are used to flash a structure onto targeted components of the scene, where initial near-interaction disparity values have been calculated. The combination of a spherical stereo algorithm, and smoothing with structured light source tessellations, provides for a very reliable and fast near-field depth engine, and resolves issues that are associated with depth estimates for embedded solutions of this approach.

Abstract: A gesture control system includes a processor, the processor in communication with a plurality of sensors. The processor is configured to perform the steps of detecting, using the plurality of sensors, a gesture in a volume occupied by a plurality of occupants, analyzing a prior knowledge to associate the gesture with one of the plurality of occupants, and generating an output, the output being determined by the gesture and the one of the plurality of occupants.

Abstract: A system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level comprises a low-lux sensor equipped to capture image data in an environment with an illumination level below an illumination threshold, a high-lux sensor equipped to capture image data in the environment with the illumination level above the illumination threshold, and an object recognition module for activating the sensors. The object recognition module determines the illumination level of the environment and activates the low-lux sensor if the illumination level is below the illumination threshold. If the illumination level is above the threshold, the object recognition module activates the high-lux sensor.

Abstract: Method, computer program and system for tracking movement of a subject. The method includes receiving data from a distributed network of camera sensors employing one or more emitted light sources associated with one or more of the one or more camera sensors to generate a volumetric three-dimensional representation of the subject, identifying a plurality of clusters within the volumetric three-dimensional representation that correspond to motion features indicative of movement of the motion features of the subject, presenting one or more objects on one or more three dimensional display screens, and using the plurality of fixed position sensors to track motion of the motion features of the subject and track manipulation of the motion features of the volumetric three-dimensional representation to determine interaction of one or more of the motion features of the subject and one or more of the one or more objects on the three dimensional display.

Abstract: A system and method for combining two separate types of human machine interfaces, e.g., a voice signal and a gesture signal, performing voice recognition to a voice signal and gesture recognition to the gesture signal. Based on a confidence determination using the voice recognition result and the gesture recognition result the system can, for example, immediately perform the command/request, request confirmation of the command/request or determine that the command/request was not identified.

Abstract: A method and apparatus for creating an adaptive mosaic pixel-wise virtual Bayer pattern. The method may include receiving a plurality of monochromatic images from an array of imaging elements, creating a reference ordered set at infinity from the plurality of monochromatic images, running a demosaicing process on the reference ordered set, and creating a color image from the demosaiced ordered set. One or more offset artifacts resulting from the demosaicing process may be computed at a distance other than infinity, the ordered set may be modified in accordance with the computed offsets, and a demosaiced image may be regenerated in accordance with modified ordered set. The regenerating process may be repeated for one or more of the one or more offsets.

Abstract: A method and system for generating a disparity map. The method comprises the steps of generating a first disparity map based upon a first image and a second image acquired at a first time, acquiring at least a third image and a fourth image at a second time, and determining one or more portions comprising a difference between one of the first and second images and a corresponding one of the third and fourth images. A disparity map update is generated for the one or more determined portions, and a disparity map is generated based upon the third image and the fourth image by combining the disparity map update and the first disparity map.

Abstract: A near-touch interface is provided that utilizes stereo cameras and a series of targeted structured light tessellations, emanating from the screen as a light source and incident on objects in the field-of-view. After radial distortion from a series of wide-angle lenses is mitigated, a surface-based spatio-temporal stereo algorithm is utilized to estimate initial depth values. Once these values are calculated, a subsequent refinement step may be applied in which light source tessellations are used to flash a structure onto targeted components of the scene, where initial near-interaction disparity values have been calculated. The combination of a spherical stereo algorithm, and smoothing with structured light source tessellations, provides for a very reliable and fast near-field depth engine, and resolves issues that are associated with depth estimates for embedded solutions of this approach.

Abstract: A gesture-enabled keyboard and method are defined. The gesture-enabled keyboard includes a keyboard housing including one or more keyboard keys for typing and a pair of stereo camera sensors mounted within the keyboard housing, a field of view of the pair of stereo camera sensors projecting substantially perpendicularly to the plane of the keyboard housing. A background of the field of view is updated when one or more alternative input devices are in use. A gesture region including a plurality of interaction zones and a virtual membrane defining a region of transition from one of the plurality of interaction zones to another of the plurality of interaction zones is defined within the field of view of the pair of stereo camera sensors. Gesture interaction is enabled when one or more gesture objects are positioned within the gesture region, and when one or more alternative input devices are not in use.

Abstract: A system and method for combining two separate types of human machine interfaces, e.g., a voice signal and a gesture signal, performing voice recognition to a voice signal and gesture recognition to the gesture signal. Based on a confidence determination using the voice recognition result and the gesture recognition result the system can, for example, immediately perform the command/request, request confirmation of the command/request or determine that the command/request was not identified.

Abstract: A system and method for combining two separate types of human machine interfaces, e.g., a voice signal and a gesture signal, performing voice recognition to a voice signal and gesture recognition to the gesture signal. Based on a confidence determination using the voice recognition result and the gesture recognition result the system can, for example, immediately perform the command/request, request confirmation of the command/request or determine that the command/request was not identified.

Abstract: A method and apparatus for processing image data is provided. The method includes the steps of employing a main processing network for classifying one or more features of the image data, employing a monitor processing network for determining one or more confusing classifications of the image data, and spawning a specialist processing network to process image data associated with the one or more confusing classifications.

Abstract: An in-vehicle computing system allows a user to control components of the vehicle by performing gestures. The user provides a selecting input to indicate that he wishes to control one of the components. After the component is identified, the user performs a gesture to control the component. The gesture and the component that was previously selected are analyzed to generate a command for the component. Since the command is based on both the gesture and the identified component, the user can perform the same gesture in the same position within the vehicle to control different components.

Abstract: A system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level comprises a low-lux sensor equipped to capture image data in an environment with an illumination level below an illumination threshold, a high-lux sensor equipped to capture image data in the environment with the illumination level above the illumination threshold, and an object recognition module for activating the sensors. The object recognition module determines the illumination level of the environment and activates the low-lux sensor if the illumination level is below the illumination threshold. If the illumination level is above the threshold, the object recognition module activates the high-lux sensor.

Abstract: A user, such as the driver of a vehicle, to retrieve information related to a point of interest (POI) near the vehicle by pointing at the POI or performing some other gesture to identify the POI. Gesture recognition is performed on the gesture to generate a target region that includes the POI that the user identified. After generating the target region, information about the POI can be retrieved by querying a server-based POI service with the target region or by searching in a micromap that is stored locally. The retrieved POI information can then be provided to the user via a display and/or speaker in the vehicle. This process beneficially allows a user to rapidly identify and retrieve information about a POI near the vehicle without having to navigate a user interface by manipulating a touchscreen or physical buttons.

Abstract: A three-dimensional virtual-touch human-machine interface system (20) and a method (100) of operating the system (20) are presented. The system (20) incorporates a three-dimensional time-of-flight sensor (22), a three-dimensional autostereoscopic display (24), and a computer (26) coupled to the sensor (22) and the display (24). The sensor (22) detects a user object (40) within a three-dimensional sensor space (28). The display (24) displays an image (42) within a three-dimensional display space (32). The computer (26) maps a position of the user object (40) within an interactive volumetric field (36) mutually within the sensor space (28) and the display space (32), and determines when the positions of the user object (40) and the image (42) are substantially coincident. Upon detection of coincidence, the computer (26) executes a function programmed for the image (42).

Abstract: A method and system for generating a disparity map. The method comprises the steps of generating a first disparity map based upon a first image and a second image acquired at a first time, acquiring at least a third image and a fourth image at a second time, and determining one or more portions comprising a difference between one of the first and second images and a corresponding one of the third and fourth images. A disparity map update is generated for the one or more determined portions, and a disparity map is generated based upon the third image and the fourth image by combining the disparity map update and the first disparity map.

Abstract: A method and apparatus for segmenting an image are provided. The method may include the steps of clustering pixels from one of a plurality of images into one or more segments, determining one or more unstable segments changing by more than a predetermined threshold from a prior of the plurality of images, determining one or more segments transitioning from an unstable to a stable segment, determining depth for one or more of the one or more segments that have changed by more than the predetermined threshold, determining depth for one or more of the one or more transitioning segments, and combining the determined depth for the one or more unstable segments and the one or more transitioning segments with a predetermined depth of all segments changing less than the predetermined threshold from the prior of the plurality of images.

Abstract: A system for capturing image data for gestures from a passenger or a driver in a vehicle with a dynamic illumination level comprises a low-lux sensor equipped to capture image data in an environment with an illumination level below an illumination threshold, a high-lux sensor equipped to capture image data in the environment with the illumination level above the illumination threshold, and an object recognition module for activating the sensors. The object recognition module determines the illumination level of the environment and activates the low-lux sensor if the illumination level is below the illumination threshold. If the illumination level is above the threshold, the object recognition module activates the high-lux sensor.