Abstract: Described is system and method for robust ground-plane homography estimation using adaptive feature selection. The system determines feature correspondences of an image that correspond with at least one moving object in each image in a set of images. Additionally, feature correspondences of the image that correspond with at least one above-ground object are determined in each image. Feature correspondences that correspond with each moving object in each image are excluded, and feature correspondences that correspond with each above-ground object in each image are excluded. Each image is divided into a plurality of sub-regions comprising features correspondences. The number of feature correspondences in each sub-region is limited to a predetermined threshold to ensure that feature correspondences are evenly distributed over each image. Finally, a ground-plane homography estimation between the set of images is generated.

Abstract: Described is a system for rapid object detection combining structural information with bio-inspired attentional mechanisms. The system oversegments an input image into a set of superpixels, where each superpixel comprises a plurality of pixels. For each superpixel, a bounding box defining a region of the input image representing a detection hypothesis is determined. An average residual saliency (ARS) is calculated for the plurality of pixels belonging to each superpixel. Each detection hypothesis that is out of a range of a predetermined threshold value for object size is eliminated. Next, each remaining detection hypothesis having an ARS below a predetermined threshold value is eliminated. Then, color contrast is calculated for the region defined by the bounding box for each remaining detection hypothesis. Each detection hypothesis having a color contrast below a predetermined threshold is eliminated. Finally, the remaining detection hypotheses are output to a classifier for object recognition.

Abstract: Described is a system for selective color processing for vision systems. The system receives a multi-band image as input. As an optional step, the multi-band image is preprocessed, and a transformation is performed to transform the multi-band image into a color space. A metric function is applied to the transformed image to generate a distance map comprising intensities which vary based on a similarity between an intensity of a pixel color and an intensity of a target color. A contrast enhancement process is applied to the distance map to normalize the distance map to a range of values. The range of values is expanded near the intensity of the target color. Finally, an output response map for the target color of interest is generated, such that the output response map has high responses in regions which are similar to the target color to aid in detection and recognition processes.

Abstract: Described is method for object cueing in motion imagery. Key points and features are extracted from motion imagery, and features between consecutive image frames of the motion imagery are compared to identify similar image frames. A candidate set of matching keypoints is generated by matching keypoints between the similar image frames. A ground plane homography model that fits the candidate set of matching keypoints is determined to generate a set of correct matching keypoints. Each image frame of a set of image frames within a selected time window is registered into a reference frame's coordinate system using the homography transformation. A difference image is obtained between the reference frame and each registered image frame, resulting in multiple difference images. The difference images are then accumulated to calculate a detection image which is used for detection of salient regions. Object cues for surveillance use are produced based on the detected salient regions.

Abstract: Described is a system for object detection using classification-based learning. A fusion method is selected, then a video sequence is processed to generate detections for each frame, wherein a detection is a representation of an object candidate. The detections are fused to generate a set of fused detections for each frame. The classification module generates a classification score labeling each fused detection based on a predetermined classification threshold. Otherwise, a token indicating that the classification module has abstained from generating a classification score is generated. The scoring module produces a confidence score for each fused detection based on a set of learned parameters from the learning module and the set of fused detections. The set of fused detections are filtered by the accept-reject module based on one of the classification score or the confidence score. Finally, a set of final detections representing an object is output.

Abstract: Described is a system for stabilizing, detecting, and recognizing objects in video captured from a mobile platform. The system first receives a video (with a plurality of image frames) captured from a mobile platform. The video is stabilized by registering the image frames to a global coordinate system to generate stabilized image frames. A bio-inspired attention algorithm is applied to the stabilized image frames to produce a set of locations in the stabilized image frames that are salient points representative of an object of interest. An image chip is generated that surrounds each salient point. High-dimensional feature vectors are extracted from the image chip. The feature vectors are then classified as an object class. Thus, through classifying the feature vectors, an object of interest can be identified in the video as captured from the mobile platform.

Abstract: Described is a system for multiple-object recognition in visual images. The system is configured to receive an input test image comprising at least one object. Keypoints representing the object are extracted using a local feature algorithm. The keypoints from the input test image are matched with keypoints from at least one training image stored in a training database, resulting in a set of matching keypoints. A clustering algorithm is applied to the set of matching keypoints to detect inliers among the set of matching keypoints. The inliers and neighboring keypoints in a vicinity of the inliers are removed from the input test image. An object label and an object boundary for the object are generated, and the object in the input test image is identified and segmented. Also described is a method and computer program product for multiple-object recognition in visual images.

Abstract: The present invention relates to a system for identifying regions of interest in visual imagery. The system is configured to receive a series of consecutive frames representing a scene as captured from N sensors. The frames include at least a current frame and a previous frame. A surprise map can be generated based on features found in the current frame and the previous frame. The surprise map having a plurality of values corresponding to spatial locations within the scene. Based on the values, a surprise in the scene can be identified if a value in the surprise map exceeds a predetermined threshold.

Abstract: Described is a system for multi-class classifier threshold-offset estimation for visual object recognition. The system receives an input image with input features for classifying. A pair-wise classifier is trained for each pair of a plurality of object classes. A set of classification responses is generated, and a multi-class receiver-operating-characteristics (ROC) curve is computed for a set of threshold-offsets. An objective function of classification performance is computed from the ROC curve and optimized using particle swarm optimization (PSO) to generate a set of optimized threshold-offsets. The optimized threshold-offsets are then applied to the classification responses. The resulting classification responses are compared to a predetermined value to classify each input feature as belonging to one object class or another. The tuning of the threshold-offsets with (PSO) improves classification performance in a visual object recognition system.

Abstract: Described is a system for optimizing rapid serial visual presentation (RSVP). A similarity metric is computed for RSVP images, and the images are sequenced according to the similarity metrics. The sequenced images are presented to a user, and neural signals are received to detect a P300 signal. A neural score for each image is computed, and the system is optimized to model the neural scores. The images are resequenced according a predictive model to output a sequence prediction which does not cause a false P300 signal. Additionally, the present invention describes computing a set of motion surprise maps from image chips. The image chips are labeled as static or moving and prepared into RSVP datasets. Neural signals are recorded in response to the RSVP datasets, and an EEG score is computed from the neural signals. Each image chip is then classified as containing or not containing an item of interest.

Abstract: The present invention creates and stores target representations in several coordinate representations based on biologically inspired models of the human vision system. By using biologically inspired target representations a computer can be programmed for robot control without using kinematics to relate a target position in camera eyes to a target position in body or head coordinates. The robot sensors and appendages are open loop controlled to focus on the target. In addition, the invention herein teaches a scenario and method to learn the mappings between coordinate representations using existing machine learning techniques such as Locally Weighted Projection Regression.

Abstract: To improve the scheduling and tasking of sensors, the present disclosure describes an improved planning system and method for the allocation and management of sensors. In one embodiment, the planning system uses a branch and bound approach of tasking sensors using a heuristic to expedite arrival at a deterministic solution. In another embodiment, a progressive lower bound is applied to the branch and bound approach. Also, in another embodiment, a hybrid branch and bound approach is used where both local and global planning are employed in a tiered fashion.

Abstract: A bio-inspired actionable intelligence method and system is disclosed. The actionable intelligence method comprises recognizing entities in an imagery signal, detecting and classifying anomalous entities, and learning new hierarchal relationships between different classes of entities. A knowledge database is updated after each new learning experience to aid in future searches and classification. The method can accommodate incremental learning via Adaptive Resonance Theory (ART).

Abstract: Described is a Distributed Resource Allocation System (DRAS) for sensor control and planning. The DRAS comprises an information framework module that is configured to specify performance goals, assess current performance state, and includes sensor models to achieve the performance goals. The DRAS is configured to further allocate the sensors to achieve the performance goals. Once allocated, the DRAS then reassesses the current performance state and continues reallocating the sensors until the current performance state is most similar to the performance goals.

Abstract: A method and apparatus for controlling robots based on prioritized targets extracted from fused visual and auditory saliency maps. The fused visual and auditory saliency map may extend beyond the immediate visual range of the robot yet the methods herein allow the robot to maintain an awareness of targets outside the immediate visual range. The fused saliency map may be derived in a bottom-up or top-down approach and may be feature based or object based.

Abstract: To improve the scheduling and tasking of resources, the present disclosure describes an improved planning system and method for the allocation and management of resources. The planning system uses a branch and bound approach of tasking resources using a heuristic to expedite arrival at a deterministic solution. For each possible functional mode of the resources, an upper bound is determined. The upper bounds are employed in an objective function for the branch and bound process to determine the functional mode in which to place the resources and to determine movement paths for the resources, all in an environment where a hostile force may attempt to destroy the resources.

Abstract: The present disclosure describes a fused saliency map from visual and auditory saliency maps. The saliency maps are in azimuth and elevation coordinates. The auditory saliency map is based on intensity, frequency and temporal conspicuity maps. Once the auditory saliency map is determined, the map is converted into azimuth and elevation coordinates by processing selected snippets of sound from each of four microphones arranged on a robot head to detect the location of the sound source generating the saliencies.

Abstract: Described is a system for finding salient regions in imagery. The system improves upon the prior art by receiving an input image of a scene and dividing the image into a plurality of image sub-regions. Each sub-region is assigned a coordinate position within the image such that the sub-regions collectively form the input image. A plurality of local saliency maps are generated, where each local saliency map is based on a corresponding sub-region and a coordinate position representative of the corresponding sub-region. Finally, the plurality of local saliency maps is combined according to their coordinate positions to generate a single global saliency map of the input image of the scene.

Abstract: Described is a bio-inspired vision system for attention and object segmentation capable of computing attention for a natural scene, attending to regions in a scene in their rank of saliency, and extracting the boundary of an attended proto-object based on feature contours to segment the attended object. The attention module can work in both a bottom-up and a top-down mode, the latter allowing for directed searches for specific targets. The region growing module allows for object segmentation that has been shown to work under a variety of natural scenes that would be problematic for traditional object segmentation algorithms. The system can perform at multiple scales of object extraction and possesses a high degree of automation. Lastly, the system can be used by itself for stand-alone searching for salient objects in a scene, or as the front-end of an object recognition and online labeling system.

Abstract: A system for intelligent goal-directed search in large volume visual imagery using a cognitive-neural subsystem is disclosed. The system comprises an imager, a display, a display processor, a cognitive-neural subsystem, a system controller, and operator controls. The cognitive-neural subsystem comprises a cognitive module, a neural module, and an adaptation module. The cognitive module is configured to extract a set of regions of interest from the image using a cognitive algorithm. The neural module is configured to refine the set of regions of interest using a neural processing algorithm. The adaptation module is configured to bias the cognitive algorithm with information gained from the neural module to improve future searches. The system functions in a plurality of operating modes, including: batch mode, semi-interactive mode, real-time mode, and roaming mode.