Abstract: Apparatus and methods for determining whether a live, human eye is presented for iris-based recognition. An image is presented on a computer screen and reflected back from the eye. The reflected image is captured and processed to determine whether the reflection was consistent with a human eye.

Abstract: A method and apparatus for performing video surveillance of a field of view is disclosed. In one embodiment, a method for performing surveillance of the field of view includes monitoring the field of view and detecting a moving object in the field of view, where the motion is detected based on a spatio-temporal signature (e.g., a set of descriptive feature vectors) of the moving object.

Abstract: A sentient system combines detection, tracking, and immersive visualization of a cluttered and crowded environment, such as an office building, terminal, or other enclosed site using a network of stereo cameras. A guard monitors the site using a live 3D model, which is updated from different directions using the multiple video streams. As a person moves within the view of a camera, the system detects its motion and tracks the person's path, it hands off the track to the next camera when the person goes out of that camera's view. Multiple people can be tracked simultaneously both within and across cameras, with each track shown on a map display. The track system includes a track map browser that displays the tracks of all moving objects as well as a history of recent tracks and a video flashlight viewer that displays live immersive video of any person that is being tracked.

Abstract: An image-based tele-presence system forward warps video images selected from a plurality fixed imagers using local depth maps and merges the warped images to form high quality images that appear as seen from a virtual position. At least two images, from the images produced by the imagers, are selected for creating a virtual image. Depth maps are generated corresponding to each of the selected images. Selected images are warped to the virtual viewpoint using warp parameters calculated using corresponding depth maps. Finally the warped images are merged to create the high quality virtual image as seen from the selected viewpoint. The system employs a video blanket of imagers, which helps both optimize the number of imagers and attain higher resolution. In an exemplary video blanket, cameras are deployed in a geometric pattern on a surface.

Abstract: A method and system for performing automated training environment monitoring and evaluation. The training environment may include a mixed reality elements to enhance a training experience.

Abstract: A method and apparatus for indexing digital video and audio signals using a digital database. A user may index the digital images by content within the images, through annotation, and the like. The database can contain high resolution and low resolution versions of the audio-video content. The indexed video can be used to create web pages that enable a viewer to access the video clips. The indexed video may also be used to author digital video disks (DVDs). The video may be enhanced to achieve DVD quality or be accessed to enhancing the digital signals and indexing the digital signals. The user may choose to enhance the digital signals by combining frames into a panorama, enhancing the resolution of the frames, filtering the images, and the like.

Abstract: A method and apparatus for video surveillance is disclosed. In one embodiment, a sequence of scene imagery representing a field of view is received. One or more moving objects are identified within the sequence of scene imagery and then classified in accordance with one or more extracted spatio-temporal features. This classification may then be applied to determine whether the moving object and/or its behavior fits one or more known events or behaviors that are causes for alarm.

Abstract: A unified approach, a fusion technique, a space-time constraint, a methodology, and system architecture are provided. The unified approach is to fuse the outputs of monocular and stereo video trackers, RFID and localization systems and biometric identification systems. The fusion technique is provided that is based on the transformation of the sensory information from heterogeneous sources into a common coordinate system with rigorous uncertainties analysis to account for various sensor noises and ambiguities. The space-time constraint is used to fuse different sensor using the location and velocity information. Advantages include the ability to continuously track multiple humans with their identities in a large area. The methodology is general so that other sensors can be incorporated into the system. The system architecture is provided for the underlying real-time processing of the sensors.

Abstract: A method and apparatus for processing a video sequence is disclosed. The apparatus may include a video processor for detecting and tracking at least one identifiable face in a video sequence. The method may include performing face detection of at least one identifiable face, selecting a face template including face features used to represent the at least one identifiable face, processing the video sequence to detect faces similar to the at least one identifiable face, and tracking the at least one identifiable face in the video sequence.

Abstract: A method and apparatus for detecting objects (e.g., bags, vehicles, etc.) left in a field of view are disclosed. A long-term representation and a short-term representation of the field of view are constructed, and a difference between the long-term representation and the short-term representation is calculated. One or more criteria may optionally be applied to this difference to determine whether the difference represents an object that was left in the field of view.

Abstract: A method and apparatus for high-resolution 3D imaging ladar system which can penetrate foliage and camouflage to sample fragments of concealed surfaces of interest is disclosed. Samples collected while the ladar moves can be integrated into a coherent object shape. In one embodiment, a system and method for automatic data-driven registration of ladar frames, comprises a coarse search stage, a pairwise fine registration stage using an iterated closest points algorithm, and a multi-view registration strategy. After alignment and aggregation, it is often difficult for human observers to find, assess and recognize objects from a point cloud display. Basic display manipulations, surface fitting techniques, and clutter suppression to enhance visual exploitation of 3D imaging ladar data may be utilized.

Abstract: A scaleable video encoder has one or more encoding modes in which at least some, and possibly all, of the motion information used during motion-based predictive encoding of a video stream is excluded from the resulting encoded video bitstream, where a corresponding video decoder is capable of performing its own motion computation to generate its own version of the motion information used to perform motion-based predictive decoding in order to decode the bitstream to generate a decoded video stream. All motion computation, whether at the encoder or the decoder, is preferably performed on decoded data. For example, frames may be encoded as either H, L, or B frames, where H frames are intra-coded at full resolution and L frames are intra-coded at low resolution. The motion information is generated by applying motion computation to decoded L and H frames and used to generate synthesized L frames.

Abstract: A method and apparatus for performing iris recognition from at least one image is disclosed. A plurality of cameras is used to capture a plurality of images where at least one of the images contains a region having at least a portion of an iris. At least one of the plurality of images is then processed to perform iris recognition.

Abstract: A scalable architecture for providing real-time multi-camera distributed video processing and visualization. An exemplary system comprises at least one video capture and storage system for capturing and storing a plurality of input videos, at least one vision based alarm system for detecting and reporting alarm situations or events, and at least one video rendering system (e.g., a video flashlight system) for displaying an alarm situation in a context that speeds up comprehension and response. One advantage of the present architecture is that these systems are all scalable, such that additional sensors (e.g., cameras, motion sensors, infrared sensors, chemical sensors, biological sensors, temperature sensors and like) can be added in large numbers without overwhelming the ability of security forces to comprehend the alarm situation.

Abstract: Method and apparatus for dynamically placing sensors in a 3D model is provided. Specifically, in one embodiment, the method selects a 3D model and a sensor for placement into the 3D model. The method renders the sensor and the 3D model in accordance with sensor parameters associated with the sensor and parameters desired by a user. In addition, the method determines whether an occlusion to the sensor is present.

Abstract: A method and apparatus are disclosed that estimate the brightness or other feature values of unchanging or slowly changing regions of an image in a sequence of video images even when the regions is obscured by objects over large portions of the video sequence. The apparatus and method generate a histogram for each image region position over a plurality of image frames in the sequence. The mode, or most frequently occurring value, of the image region as indicated by the histogram is selected as representing the unchanging portion of the image. The mode values of all of the regions are then assembled to form a composite image of the unchanging or slowly changing feature values. According to one method, the histogram is generated using a recursive filter. In order to process images that exhibit some motion from frame to frame, the images in the video sequence may be aligned before generating the histogram.

Abstract: A method and apparatus for accurately computing parallax information as captured by imagery of a scene. The method computes the parallax information of each point in an image by computing the parallax within windows that are offset with respect to the point for which the parallax is being computed. Additionally, parallax computations are performed over multiple frames of imagery to ensure accuracy of the parallax computation and to facilitate correction of occluded imagery.

Abstract: A system and method for accurately mapping between image coordinates and geo-coordinates, called geo-spatial registration. The system utilizes the imagery and terrain information contained in the geo-spatial database to precisely align geodetically calibrated reference imagery with an input image, e.g., dynamically generated video images, and thus achieve a high accuracy identification of locations within the scene. When a sensor, such as a video camera, images a scene contained in the geo-spatial database, the system recalls a reference image pertaining to the imaged scene. This reference image is aligned very accurately with the sensor's images using a parametric transformation. Thereafter, other information that is associated with the reference image can easily be overlaid upon or otherwise associated with the sensor imagery.

Abstract: A method and apparatus for providing immersive surveillance wherein a remote security guard may monitor a scene using a variety of imagery sources that are rendered upon a model to provide a three-dimensional conceptual view of the scene. Using a view selector, the security guard may dynamically select a camera view to be displayed on his conceptual model, perform a walk through of the scene, identify moving objects and select the best view of those moving objects and so on.

Abstract: An image-based tele-presence system forward warps video images selected from a plurality fixed imagers using local depth maps and merges the warped images to form high quality images that appear as seen from a virtual position. At least two images, from the images produced by the imagers, are selected for creating a virtual image. Depth maps are generated corresponding to each of the selected images. Selected images are warped to the virtual viewpoint using warp parameters calculated using corresponding depth maps. Finally the warped images are merged to create the high quality virtual image as seen from the selected viewpoint. The system employs a video blanket of imagers, which helps both optimize the number of imagers and attain higher resolution. In an exemplary video blanket, cameras are deployed in a geometric pattern on a surface.