Abstract: A system and process for tracking an object state over time using particle filter sensor fusion and a plurality of logical sensor modules is presented. This new fusion framework combines both the bottom-up and top-down approaches to sensor fusion to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers can be designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Different from the traditional fusion algorithms, the present framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle non-stationary situations, the present framework evaluates the performance of the individual trackers and dynamically updates their object states.

Abstract: A system and process for tracking an object state over time using particle filter sensor fusion and a plurality of logical sensor modules is presented. This new fusion framework combines both the bottom-up and top-down approaches to sensor fusion to probabilistically fuse multiple sensing modalities. At the lower level, individual vision and audio trackers can be designed to generate effective proposals for the fuser. At the higher level, the fuser performs reliable tracking by verifying hypotheses over multiple likelihood models from multiple cues. Different from the traditional fusion algorithms, the present framework is a closed-loop system where the fuser and trackers coordinate their tracking information. Furthermore, to handle non-stationary situations, the present framework evaluates the performance of the individual trackers and dynamically updates their object states.

Abstract: A system and method for object tracking using probabilistic mode-based multi-hypothesis tracking (MHT) provides for robust and computationally efficient tracking of moving objects such as heads and faces in complex environments. A mode-based multi-hypothesis tracker uses modes that are local maximums which are refined from initial samples in a parametric state space. Because the modes are highly representative, the mode-based multi-hypothesis tracker effectively models non-linear probabilistic distributions using a small number of hypotheses. Real-time tracking performance is achieved by using a parametric causal contour model to refine initial contours to nearby modes. In addition, one common drawback of conventional MHT schemes, i.e., producing only maximum likelihood estimates instead of a desired posterior probability distribution, is addressed by introducing an importance sampling framework into MHT, and estimating the posterior probability distribution from the importance function.

Abstract: Automatic detection and tracking of multiple individuals includes receiving a frame of video and/or audio content and identifying a candidate area for a new face region in the frame. One or more hierarchical verification levels are used to verify whether a human face is in the candidate area, and an indication made that the candidate area includes a face if the one or more hierarchical verification levels verify that a human face is in the candidate area. A plurality of audio and/or video cues are used to track each verified face in the video content from frame to frame.