Abstract: A face recognition system is provided. The system includes a device configured to capture an input image of a subject. The system further includes a processor. The processor estimates, using a 3D Morphable Model (3DMM) conditioned Generative Adversarial Network, 3DMM coefficients for the subject of the input image. The subject varies from an ideal front pose. The processor produces, using an image generator, a synthetic frontal face image of the subject of the input image based on the input image and the 3DMM coefficients. An area spanning the frontal face of the subject is made larger in the synthetic image than in the input image. The processor provides, using a discriminator, a decision indicative of whether the subject of the synthetic image is an actual person. The processor provides, using a face recognition engine, an identity of the subject in the input image based on the synthetic and input images.

Abstract: A system is provided for unsupervised cross-domain image generation relative to a first and second image domain that each include real images. A first generator generates synthetic images similar to real images in the second domain while including a semantic content of real images in the first domain. A second generator generates synthetic images similar to real images in the first domain while including a semantic content of real images in the second domain. A first discriminator discriminates real images in the first domain against synthetic images generated by the second generator. A second discriminator discriminates real images in the second domain against synthetic images generated by the first generator. The discriminators and generators are deep neural networks and respectively form a generative network and a discriminative network in a cyclic GAN framework configured to increase an error rate of the discriminative network to improve synthetic image quality.

Abstract: A video retrieval system is provided that includes a server for retrieving video sequences from a remote database responsive to a text specifying a face recognition result as an identity of a subject of an input image. The face recognition result is determined by a processor of the server, which estimates, using a 3DMM conditioned Generative Adversarial Network, 3DMM coefficients for the subject of the input image. The subject varies from an ideal front pose. The processor produces a synthetic frontal face image of the subject of the input image based on the input image and coefficients. An area spanning the frontal face of the subject is made larger in the synthetic than in the input image. The processor provides a decision of whether the synthetic image subject is an actual person and provides the identity of the subject in the input image based on the synthetic and input images.

Abstract: A computer-implemented method, system, and computer program product is provided for pose-invariant facial recognition. The method includes generating, by a processor using a recognition neural network, a rich feature embedding for identity information and non-identity information for each of one or more images. The method also includes generating, by the processor using a Siamese reconstruction network, one or more pose-invariant features by employing the rich feature embedding for identity information and non-identity information. The method additionally includes identifying, by the processor, a user by employing the one or more pose-invariant features. The method further includes controlling an operation of a processor-based machine to change a state of the processor-based machine, responsive to the identified user in the one or more images.

Abstract: A video surveillance system is provided. The system includes a device configured to capture an input image of a subject located in an area. The system further includes a processor. The processor estimates, using a three-dimensional Morphable Model (3DMM) conditioned Generative Adversarial Network, 3DMM coefficients for the subject of the input image. The subject varies from an ideal front pose. The processor produces, using an image generator, a synthetic frontal face image of the subject of the input image based on the input image and coefficients. An area spanning the frontal face of the subject is made larger in the synthetic than in the input image. The processor provides, using a discriminator, a decision of whether the subject of the synthetic image is an actual person. The processor provides, using a face recognition engine, an identity of the subject in the input image based on the synthetic and input images.

Abstract: A face recognition system is provided that includes a device configured to capture a video sequence formed from a set of unlabeled testing video frames. The system includes a processor configured to pre-train a face recognition engine formed from reference CNNs on a still image domain that includes labeled training still image frames of faces. The processor adapts the face recognition engine to a video domain to form an adapted engine, by applying non-reference CNNs to domains including the still image and video domains and a degraded image domain. The degraded image domain includes labeled synthetically degraded versions of the frames included in the still image domain. The video domain includes random unlabeled training video frames. The processor recognizes, using the adapted engine, identities of persons corresponding to at least one face in the video sequence to obtain a set of identities. A display device displays the set of identities.

Abstract: Systems and methods for training semantic segmentation. Embodiments of the present invention include predicting semantic labeling of each pixel in each of at least one training image using a semantic segmentation model. Further included is predicting semantic boundaries at boundary pixels of objects in the at least one training image using a semantic boundary model concurrently with predicting the semantic labeling. Also included is propagating sparse labels to every pixel in the at least one training image using the predicted semantic boundaries. Additionally, the embodiments include optimizing a loss function according the predicted semantic labeling and the propagated sparse labels to concurrently train the semantic segmentation model and the semantic boundary model to accurately and efficiently generate a learned semantic segmentation model from sparsely annotated training images.

Abstract: A surveillance system and method are provided. The surveillance system includes at least one camera configured to capture a set of images of a given target area that includes a set of objects to be tracked. The surveillance system includes a memory storing a learning model configured to perform multi-object tracking by jointly learning arbitrarily parameterized and differentiable cost functions for all variables in a linear program that associates object detections with bounding boxes to form trajectories. The surveillance system includes a processor configured to perform surveillance of the target area to (i) detect the objects and track locations of the objects by applying the learning model to the images in a surveillance task that uses the multi-object tracking, and (ii), provide a listing of the objects and their locations for surveillance task. A bi-level optimization is used to minimize a loss defined on a solution of the linear program.

Abstract: A system and method are provided. The system includes an image capture device configured to capture a video sequence formed from a set of input image frames and including a set of objects. The system further includes a processor configured to detect the objects to form object detections and track the object detections over the input frames to form tracked detections. The processor is also configured to generate for a current input frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous input frame; and (ii) detection proposals for the object derived from the current frame. The processor is additionally configured to provide a user perceptible indication of the current location of the object, based on the set of sparse object proposals.

Abstract: An action recognition system and method are provided. The system includes an image capture device configured to capture a video sequence formed from image frames and depicting a set of objects. The system includes a processor configured to detect the objects to form object detections. The processor is configured to track the object detections over the frames to form tracked detections. The processor is configured to generate for a current frame, responsive to conditions, sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous frame; and (ii) detection proposals for the object derived from the current frame. The processor is configured to control a hardware device to perform a response action in response to an identification of an action type of an action performed by the object, the identification being based on the sparse object proposals.

Abstract: A surveillance system and method are provided. The surveillance system includes an image capture device configured to capture a video sequence of a target area that includes a set of objects and is formed from a set of image frames. The surveillance system also includes a processor. The processor is configured to detect the objects to form object detections, and track the object detections over the frames to form tracked detections. The processor is further configured to generate for a current input frame, responsive to conditions, a set of sparse object proposals for a current location of an object based on: (i) the tracked detections of the object from an immediately previous frame; and (ii) detection proposals for the object derived from the current frame. The processor is additionally configured to provide a user perceptible indication of the current location of the object, based on the sparse object proposals.

Abstract: A multi-object tracking system and method are provided. The multi-object tracking system includes at least one camera configured to capture a set of input images of a set of objects to be tracked. The multi-object tracking system further includes a memory storing a learning model configured to perform multi-object tracking by jointly learning arbitrarily parameterized and differentiable cost functions for all variables in a linear program that associates object detections with bounding boxes to form trajectories. The multi-object tracking system also includes a processor configured to (i) detect the objects and track locations of the objects by applying the learning model to the set of input images in a multi-object tracking task, and (ii), provide a listing of the objects and the locations of the objects for the multi-object tracking task. A bi-level optimization is used to minimize a loss defined on a solution of the linear program.

Abstract: An action recognition system and method are provided. The action recognition system includes an image capture device configured to capture an actual image depicting an object. The action recognition system includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN thereto to generate an image pair including a 2D and 3D geometric structure of the object. The processor is configured to control a device to perform a response action in response to an identification of an action performed by the object, wherein the identification of the action is based on the image pair.

Abstract: A system and method are provided. The system includes an image capture device configured to capture an actual image depicting an object. The system also includes a processor. The processor is configured to render, based on a set of 3D Computer Aided Design (CAD) models, a set of synthetic images with corresponding intermediate shape concept labels. The processor is also configured to form a multi-layer Convolutional Neural Network (CNN) which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is further configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to output an image pair including a 2D geometric structure and a 3D geometric structure of the object depicted in the actual image.

Abstract: A system and method are provided for driving assistance. The system includes an image capture device configured to capture an actual image relative to an outward view from a motor vehicle and depicting an object. The system further includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is further configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is also configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to output an image pair including a 2D and 3D geometric structure of the object. The processor is additionally configured to perform an action to mitigate a likelihood of harm involving the motor vehicle, based on the image pair.

Abstract: A facility access control system and corresponding method are provided. The facility access control system includes a camera configured to capture an input image of a subject attempting to enter or exit a restricted facility. The facility access control system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task. The facility access control system also includes a processor configured to apply the deep learning model to the input image to recognize an identity of the subject in the input image regarding being authorized for access to the facility and a liveness of the subject. The liveness detection task is configured to evaluate a plurality of different distracter modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.

Abstract: A surveillance system and method are provided. The surveillance system includes an image capture device configured to capture an actual image of a target area depicting an object. The surveillance system further includes a processor. The processor is configured to render, based on a set of 3D Computer Aided Design (CAD) models, synthetic images with intermediate shape corresponding concept labels. The processor is further configured to form a multi-layer Convolutional Neural Network (CNN) which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is also configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to generate an image pair including a 2D and 3D geometric structure of the object depicted in the actual image. The surveillance system further includes a display device configured to display the image pair.

Abstract: A smuggling detection system and corresponding method are provided. The smuggling detection system includes a camera configured to capture an input image of a subject purported to be a baby. The smuggling detection system further includes a memory storing a deep learning model configured to perform a baby detection task for a smuggling detection application. The smuggling detection system also includes a processor configured to apply the deep learning model to the input image to provide a baby detection result of either a presence or an absence of an actual baby in relation to the subject purported to be the baby. The baby detection task is configured to evaluate one or more different distractor modalities corresponding to one or more different physical spoofing materials to prevent baby spoofing for the baby detection task.

Abstract: A face recognition system and corresponding method are provided. The face recognition system includes a camera configured to capture an input image of a subject purported to be a person. The face recognition system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task. The face recognition system also includes a processor configured to apply the deep learning model to the input image to recognize an identity of the subject in the input image and a liveness of the subject. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.

Abstract: A login access control system is provided. The login access control system includes a camera configured to capture an input image of a subject purported to be a person and attempting to login to a system to access secure data. The login access control system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task. The login access control system also includes a processor configured to apply the deep learning model to the input image to recognize an identity of the subject in the input image regarding being authorized for access to the secure data and a liveness of the subject. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.