Abstract: A traffic enforcement system and corresponding method are provided. The traffic enforcement system includes a camera configured to capture an input image of one or more subjects in a motor vehicle. The traffic enforcement system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image. The traffic enforcement system also includes a processor configured to apply the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.

Abstract: A mass transit surveillance system and corresponding method are provided. The mass transit surveillance system includes a camera configured to capture an input image of a subject purported to be a baby and presented at a mass transit environment. The mass transit surveillance system further includes a memory storing a deep learning model configured to perform a baby detection task for the mass transit environment. The mass transit surveillance system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A computer-implemented method, system, and computer program product are provided for object detection utilizing an online flow guided memory network. The method includes receiving a plurality of videos, each of the plurality of videos including a plurality of frames. The method also includes generating, with a feature extraction network, a frame feature map for a current frame of the plurality of frames. The method additionally includes aggregating a memory feature map from the frame feature map and previous memory feature maps from previous frames on a plurality of time axes, with the plurality of time axes including a first time axis at a first frame increment and a second time axis at a second frame increment. The method further includes predicting, with a task network, an object from the memory feature map. The method also includes controlling an operation of a processor-based machine to react in accordance with the object.

Abstract: A computer-implemented method, system, and computer program product are provided for activity recognition. The method includes receiving, by a processor, a plurality of videos, the plurality of videos including labeled videos and unlabeled videos. The method also includes extracting, by the processor with a feature extraction convolutional neural network (CNN), frame features for frames from each of the plurality of videos. The method additionally includes estimating, by the processor with a feature aggregation system, a vector representation for one of the plurality of videos responsive to the frame features. The method further includes classifying, by the processor, an activity from the vector representation. The method also includes controlling an operation of a processor-based machine to react in accordance with the activity.

Abstract: A computer-implemented method, system, and computer program product are provided for activity recognition in a mobile device. The method includes receiving a plurality of unlabeled videos from one or more cameras. The method also includes generating a classified video for each of the plurality of unlabeled videos by classifying an activity in each of the plurality of unlabeled videos. The method additionally includes storing the classified video in a location in a memory designated for videos of the activity in each of the classified videos.

Abstract: A computer-implemented method, system, and computer program product are provided for activity recognition in a surveillance system. The method includes receiving a plurality of unlabeled videos from one or more cameras. The method also includes classifying an activity in each of the plurality of unlabeled videos. The method additionally includes controlling an operation of a processor-based machine to react in accordance with the activity.

Abstract: A computer-implemented method, system, and computer program product are provided for object detection utilizing an online flow guided memory network. The method includes receiving, by a processor, a plurality of videos, each of the plurality of videos including a plurality of frames. The method also includes generating, by the processor with a feature extraction network, a frame feature map for a current frame of the plurality of frames. The method additionally includes determining, by the processor, a memory feature map from the frame feature map and a previous memory feature map from a previous frame by warping the previous memory feature map. The method further includes predicting, by the processor with a task network, an object from the memory feature map. The method also includes controlling an operation of a processor-based machine to react in accordance with the object.

Abstract: Systems and methods for generating an occlusion-aware bird's eye view map of a road scene include identifying foreground objects and background objects in an input image to extract foreground features and background features corresponding to the foreground objects and the background objects, respectively. The foreground objects are masked from the input image with a mask. Occluded objects and depths of the occluded objects are inferred by predicting semantic features and depths in masked areas of the masked image according to contextual information related to the background features visible in the masked image. The foreground objects and the background objects are mapped to a three-dimensional space according to locations of each of the foreground objects, the background objects and occluded objects using the inferred depths. A bird's eye view is generated from the three-dimensional space and displayed with a display device.

Abstract: Systems and methods for generating an occlusion-aware bird's eye view map of a road scene include identifying foreground objects and background objects in an input image to extract foreground features and background features corresponding to the foreground objects and the background objects, respectively. The foreground objects are masked from the input image with a mask. Occluded objects and depths of the occluded objects are inferred by predicting semantic features and depths in masked areas of the masked image according to contextual information related to the background features visible in the masked image. The foreground objects and the background objects are mapped to a three-dimensional space according to locations of each of the foreground objects, the background objects and occluded objects using the inferred depths. A bird's eye view is generated from the three-dimensional space and displayed with a display device.

Abstract: A computer-implemented method, system, and computer program product are provided for facial recognition. The method includes receiving, by a processor device, a plurality of images. The method also includes extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images. The method additionally includes generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors. The method further includes classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector. The method also includes control an operation of a processor-based machine to react in accordance with the identity.

Abstract: A computer-implemented method, system, and computer program product are provided for facial recognition. The method includes receiving, by a processor device, a plurality of images. The method also includes extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images. The method additionally includes generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors. The method further includes classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector. The method also includes control an operation of a processor-based machine to react in accordance with the identity.

Abstract: A computer-implemented method, system, and computer program product are provided for facial recognition. The method includes receiving, by a processor device, a plurality of images. The method also includes extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images. The method additionally includes generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors. The method further includes classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector. The method also includes control an operation of a processor-based machine to react in accordance with the identity.

Abstract: A computer-implemented method, system, and computer program product are provided for facial recognition. The method includes receiving, by a processor device, a plurality of images. The method also includes extracting, by the processor device with a feature extractor utilizing a convolutional neural network (CNN) with an enlarged intra-class variance of long-tail classes, feature vectors for each of the plurality of images. The method additionally includes generating, by the processor device with a feature generator, discriminative feature vectors for each of the feature vectors. The method further includes classifying, by the processor device utilizing a fully connected classifier, an identity from the discriminative feature vector. The method also includes control an operation of a processor-based machine to react in accordance with the identity.

Abstract: Systems and methods for vehicle surveillance include a camera for capturing target images of vehicles. An object recognition system is in communication with the camera, the object recognition system including a processor for executing a synthesizer module for generating a plurality of viewpoints of a vehicle depicted in a source image, and a domain adaptation module for performing domain adaptation between the viewpoints of the vehicle and the target images to classifying vehicles of the target images regardless of the viewpoint represented in the target images. A display is in communication with the object recognition system for displaying each of the target images with labels corresponding to the vehicles of the target images.

Abstract: Systems and methods for performing domain adaptation include collecting a labeled source image having a view of an object. Viewpoints of the object in the source image are synthesized to generate view augmented source images. Photometrics of each of the viewpoints of the object are adjusted to generate lighting and view augmented source images. Features are extracted from each of the lighting and view augmented source images with a first feature extractor and from captured images captured by an image capture device with a second feature extractor. The extracted features are classified using domain adaptation with domain adversarial learning between extracted features of the captured images and extracted features of the lighting and view augmented source images. Labeled target images are displayed corresponding to each of the captured images including labels corresponding to classifications of the extracted features of the captured images.

Abstract: Systems and methods for correspondence estimation and flexible ground modeling include communicating two-dimensional (2D) images of an environment to a correspondence estimation module, including a first image and a second image captured by an image capturing device. First features, including geometric features and semantic features, are hierarchically extract from the first image with a first convolutional neural network (CNN) according to activation map weights, and second features, including geometric features and semantic features, are hierarchically extracted from the second image with a second CNN according to the activation map weights. Correspondences between the first features and the second features are estimated, including hierarchical fusing of geometric correspondences and semantic correspondences. A 3-dimensional (3D) model of a terrain is estimated using the estimated correspondences belonging to the terrain surface.

Abstract: A computer-implemented method, system, and computer program product are provided for a stabilization system utilizing pose estimation in an aerial drone. The method includes receiving, by a pose estimation system, a plurality of images from one or more cameras. The method also includes predicting, by the pose estimation system, a pose from the score map and a combined feature map, the combined feature map correlated from a pair of the plurality of images. The method additionally includes moving, by a propulsion system, the aerial drone responsive to the pose.

Abstract: A computer-implemented method, system, and computer program product are provided for a guidance control system utilizing pose estimation in an autonomous vehicle. The method includes receiving, by a pose estimation system, a plurality of images from one or more cameras. The method also includes predicting, by the pose estimation system, a pose from the score map and a combined feature map, the combined feature map correlated from a pair of the plurality of images. The method additionally includes moving, by a propulsion system, the autonomous vehicle responsive to the pose.

Abstract: A computer-implemented method, system, and computer program product are provided for pose estimation. The method includes receiving, by a processor, a plurality of images from one or more cameras. The method also includes generating, by the processor with a feature extraction convolutional neural network (CNN), a feature map for each of the plurality of images. The method additionally includes estimating, by the processor with a feature weighting network, a score map from a pair of the feature maps. The method further includes predicting, by the processor with a pose estimation CNN, a pose from the score map and a combined feature map. The method also includes controlling an operation of a processor-based machine to change a state of the processor-based machine, responsive to the pose.

Abstract: Systems and methods for detecting traffic scenarios include an image capturing device which captures two or more images of an area of a traffic environment with each image having a different view of vehicles and a road in the traffic environment. A hierarchical feature extractor concurrently extracts features at multiple neural network layers from each of the images, with the features including geometric features and semantic features, and for estimating correspondences between semantic features for each of the images and refining the estimated correspondences with correspondences between the geometric features of each of the images to generate refined correspondence estimates. A traffic localization module uses the refined correspondence estimates to determine locations of vehicles in the environment in three dimensions to automatically determine a traffic scenario according to the locations of vehicles. A notification device generates a notification of the traffic scenario.