Abstract: An action recognition system and method are provided. The system includes an image capture device configured to capture a video sequence formed from image frames and depicting a set of objects. The system includes a processor configured to detect the objects to form object detections. The processor is configured to track the object detections over the frames to form tracked detections. The processor is configured to generate for a current frame, responsive to conditions, sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous frame; and (ii) detection proposals for the object derived from the current frame. The processor is configured to control a hardware device to perform a response action in response to an identification of an action type of an action performed by the object, the identification being based on the sparse object proposals.

Abstract: A system and method are provided for driving assistance. The system includes an image capture device configured to capture a video sequence, relative to an outward view from a vehicle, which includes a set of objects and is formed from a set of image frames. The system includes a processor configured to detect the objects to form a set of object detections, and track the set of object detections over the frames to form tracked detections. The processor is configured to generate for a current frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous frame; and (ii) detection proposals for the object derived from the current frame. The processor is configured to perform an action to mitigate a likelihood of potential harmful due to a current object location.

Abstract: A system and method are provided. The system includes an image capture device configured to capture a video sequence formed from a set of input image frames and including a set of objects. The system further includes a processor configured to detect the objects to form object detections and track the object detections over the input frames to form tracked detections. The processor is also configured to generate for a current input frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous input frame; and (ii) detection proposals for the object derived from the current frame. The processor is additionally configured to provide a user perceptible indication of the current location of the object, based on the set of sparse object proposals.

Abstract: A surveillance system and method are provided. The surveillance system includes an image capture device configured to capture a video sequence of a target area that includes a set of objects and is formed from a set of image frames. The surveillance system also includes a processor. The processor is configured to detect the objects to form object detections, and track the object detections over the frames to form tracked detections. The processor is further configured to generate for a current input frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous frame; and (ii) detection proposals for the object derived from the current frame. The processor is additionally configured to provide a user perceptible indication of the current location of the object, based on the sparse object proposals.

Abstract: A surveillance system and method are provided. The surveillance system includes at least one camera configured to capture a set of images of a given target area that includes a set of objects to be tracked. The surveillance system includes a memory storing a learning model configured to perform multi-object tracking by jointly learning arbitrarily parameterized and differentiable cost functions for all variables in a linear program that associates object detections with bounding boxes to form trajectories. The surveillance system includes a processor configured to perform surveillance of the target area to (i) detect the objects and track locations of the objects by applying the learning model to the images in a surveillance task that uses the multi-object tracking, and (ii), provide a listing of the objects and their locations for surveillance task. A bi-level optimization is used to minimize a loss defined on a solution of the linear program.

Abstract: A multi-object tracking system and method are provided. The multi-object tracking system includes at least one camera configured to capture a set of input images of a set of objects to be tracked. The multi-object tracking system further includes a memory storing a learning model configured to perform multi-object tracking by jointly learning arbitrarily parameterized and differentiable cost functions for all variables in a linear program that associates object detections with bounding boxes to form trajectories. The multi-object tracking system also includes a processor configured to (i) detect the objects and track locations of the objects by applying the learning model to the set of input images in a multi-object tracking task, and (ii), provide a listing of the objects and the locations of the objects for the multi-object tracking task. A bi-level optimization is used to minimize a loss defined on a solution of the linear program.

Abstract: A computer-implemented method for detecting objects by using subcategory-aware convolutional neural networks (CNNs) is presented. The method includes generating object region proposals from an image by a region proposal network (RPN) which utilizes subcategory information, and classifying and refining the object region proposals by an object detection network (ODN) that simultaneously performs object category classification, subcategory classification, and bounding box regression. The image is an image pyramid used as input to the RPN and the ODN. The RPN and the ODN each include a feature extrapolating layer to detect object categories with scale variations among the objects.

Abstract: Methods and systems for predicting a trajectory include determining prediction samples for agents in a scene based on a past trajectory. The prediction samples are ranked according to a likelihood score that incorporates interactions between agents and semantic scene context. The prediction samples are iteratively refined using a regression function that accumulates scene context and agent interactions across iterations. A response activity is triggered when the prediction samples satisfy a predetermined condition.

Abstract: A computer-implemented method is provided for video annotation of a video sequence that includes a plurality of frames. The method includes storing, by a computer, information representative of a bounding box around a given target object in the video sequence in both a first frame and a last frame of the video sequence that include the target object. The method further includes generating, by the computer based on the information, estimated box annotations of the given target object in suggested frames from among the plurality of frames. The suggested frames are determined based on an annotation uncertainty measure calculated using a set of already provided annotations for at least some of the plurality of frames together with optical flow information for the video sequence. The method also includes displaying, by the computer, various ones of the estimated box annotations of the given target object in the suggested frames.

Abstract: Systems and methods are disclosed to provide an Advanced Warning System (AWS) for a driver of a vehicle, by capturing traffic scene types from a single camera video; generating real-time monocular SFM and 2D object detection from the single camera video; detecting a ground plane from the real-time monocular SFM and the 2D object detection; performing dense 3D estimation from the real-time monocular SFM and the 2D object detection; generating a joint 3D object localization from the ground plane and dense 3D estimation; and communicating a situation that requires caution to the driver.

Abstract: A baby detection system and corresponding method are provided. The baby detection system includes a camera configured to capture an input image of a subject purported to be a baby and presented at an electronic-gate system. The baby detection system further includes a memory storing a deep learning model configured to perform a baby detection task for an electronic-gate application corresponding to the electronic-gate system. The baby detection system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A mass transit surveillance system and corresponding method are provided. The mass transit surveillance system includes a camera configured to capture an input image of a subject purported to be a baby and presented at a mass transit environment. The mass transit surveillance system further includes a memory storing a deep learning model configured to perform a baby detection task for the mass transit environment. The mass transit surveillance system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A baby detection system and corresponding method are provided. The baby detection system includes a camera configured to capture an input image of a subject purported to be a baby and presented at an electronic-gate system. The baby detection system further includes a memory storing a deep learning model configured to perform a baby detection task for an electronic-gate application corresponding to the electronic-gate system. The baby detection system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A smuggling detection system and corresponding method are provided. The smuggling detection system includes a camera configured to capture an input image of a subject purported to be a baby. The smuggling detection system further includes a memory storing a deep learning model configured to perform a baby detection task for a smuggling detection application. The smuggling detection system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A computer-implemented method is provided for video annotation of a video sequence that includes a plurality of frames. The method includes storing, by a computer, information representative of a bounding box around a given target object in the video sequence in both a first frame and a last frame of the video sequence that include the target object. The method further includes generating, by the computer based on the information, estimated box annotations of the given target object in suggested frames from among the plurality of frames. The suggested frames are determined based on an annotation uncertainty measure calculated using a set of already provided annotations for at least some of the plurality of frames together with optical flow information for the video sequence. The method also includes displaying, by the computer, various ones of the estimated box annotations of the given target object in the suggested frames.

Abstract: A computer-implemented method for training a convolutional neural network (CNN) is presented. The method includes receiving regions of interest from an image, generating one or more convolutional layers from the image, each of the one or more convolutional layers having at least one convolutional feature within a region of interest, applying at least one cascaded rejection classifier to the regions of interest to generate a subset of the regions of interest, and applying scale dependent pooling to convolutional features within the subset to determine a likelihood of an object category.

Abstract: A computer-implemented method for detecting objects by using subcategory-aware convolutional neural networks (CNNs) is presented. The method includes generating object region proposals from an image by a region proposal network (RPN) which utilizes subcategory information, and classifying and refining the object region proposals by an object detection network (ODN) that simultaneously performs object category classification, subcategory classification, and bounding box regression. The image is an image pyramid used as input to the RPN and the ODN. The RPN and the ODN each include a feature extrapolating layer to detect object categories with scale variations among the objects.

Abstract: Systems and methods are disclosed to provide an Advanced Warning System (AWS) for a driver of a vehicle, by capturing traffic scene types from a single camera video; generating real-time monocular SFM and 2D object detection from the single camera video; detecting a ground plane from the real-time monocular SFM and the 2D object detection; performing dense 3D estimation from the real-time monocular SFM and the 2D object detection; generating a joint 3D object localization from the ground plane and dense 3D estimation; and communicating a situation that requires caution to the driver.

Abstract: Systems and methods are disclosed to track targets in a video by capturing a video sequence, detecting data association between detections and targets, where detections are generated using one or more image based detectors (tracking-by-detections); identifying one or more target of interests and estimating a motion of each individual; and applying an Aggregated Local Flow Descriptor to accurately measure an affinity between a pair of detections and a Near Online Multi-target Tracking to perform multiple target tracking given a video sequence.