Abstract: A method for acquiring privacy-enhancing encodings in an optical domain before image capture is presented. The method includes feeding a differentiable sensing model with a plurality of images to obtain encoded images, the differentiable sensing model including parameters for sensor optics, integrating the differentiable sensing model into an adversarial learning framework where parameters of attack networks, parameters of utility networks, and the parameters of the sensor optics are concurrently updated, and, once adversarial training is complete, validating efficacy of a learned sensor design by fixing the parameters of the sensor optics and training the attack networks and the utility networks to learn to estimate private and public attributes, respectively, from a set of the encoded images.

Abstract: A method provided for 3D object localization predicts pairs of 2D bounding boxes. Each pair corresponds to a detected object in each of the two consecutive input monocular images. The method generates, for each detected object, a relative motion estimation specifying a relative motion between the two images. The method constructs an object cost volume by aggregating temporal features from the two images using the pairs of 2D bounding boxes and the relative motion estimation to predict a range of object depth candidates and a confidence score for each object depth candidate and an object depth from the object depth candidates. The method updates the relative motion estimation based on the object cost volume and the object depth to provide a refined object motion and a refined object depth. The method reconstructs a 3D bounding box for each detected object based on the refined object motion and refined object depth.

Abstract: A computer-implemented method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework includes receiving a set of data including face recognition data, liveness data and material data associated with at least one face image, obtaining a shared feature from the set of data using a backbone neural network structure, performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction, and aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance.

Abstract: A method is provided for drone-video-based action recognition. The method learns a transformation for each of target video clips taken from a set of target videos, responsive to original features extracted from the target video clips. The transformation corrects differences between a target drone domain corresponding to the target video clips and a source non-drone domain corresponding to source video clips taken from a set of source videos. The method adapts the target to the source domain by applying the transformation to the original features to obtain transformed features for the target video clips. The method converts the original and transformed features of same ones of the target video clips into a single classification feature for each of the target videos. The method classifies a human action in a new target video relative to the set of source videos using the single classification feature for each of the target videos.

Abstract: A method for correcting blur effects is presented. The method includes generating a plurality of images from a camera, synthesizing blurred images from sharp image counterparts to generate training data to train a structure-and-motion-aware convolutional neural network (CNN), and predicting a camera motion and a depth map from a single blurred image by employing the structure-and-motion-aware CNN to remove blurring from the single blurred image.

Abstract: A computer-implemented method is provided for domain adaptation between a source domain and a target domain. The method includes applying, by a hardware processor, an attention network to features extracted from images included in the source and target domains to provide attended features relating to a given task to be domain adapted between the source and target domains. The method further includes applying, by the hardware processor, a deformation network to at least some of the attended features to align the attended features between the source and target domains using warping to provide attended and warped features. The method also includes training, by the hardware processor, a target domain classifier using the images from the source domain. The method additionally includes classifying, by the hardware processor using the trained target domain classifier, at least one image from the target domain.

Abstract: Methods and systems for object detection include training dataset-specific object detectors using respective annotated datasets, each of the annotated datasets including annotations for a respective set of one or more object classes. The annotated datasets are cross-annotated using the dataset-specific object detectors. A unified object detector is trained, using the cross-annotated datasets, to detect all of the object classes of the annotated datasets. Objects are detected in an input image using the unified object detector.

Abstract: Methods and systems for training a neural network include generate an image of a mask. A copy of an image is generated from an original set of training data. The copy is altered to add the image of a mask to a face detected within the copy. An augmented set of training data is generated that includes the original set of training data and the altered copy. A neural network model is trained to recognize masked faces using the augmented set of training data.

Abstract: A method for correcting rolling shutter (RS) effects is presented. The method includes generating a plurality of images from a camera, synthesizing RS images from global shutter (GS) counterparts to generate training data to train the structure-and-motion-aware convolutional neural network (CNN), and predicting an RS camera motion and an RS depth map from a single RS image by employing a structure-and-motion-aware CNN to remove RS distortions from the single RS image.

Abstract: Methods and systems for training a trajectory prediction model and performing a vehicle maneuver include encoding a set of training data to generate encoded training vectors, where the training data includes trajectory information for agents over time. Trajectory scenarios are simulated based on the encoded training vectors, with each simulated trajectory scenario representing one or more agents with respective agent trajectories, to generate simulated training data. A predictive neural network model is trained using the simulated training data to generate predicted trajectory scenarios based on a detected scene.

Abstract: Systems and methods for recognizing fine-grained objects are provided. The system divides unlabeled training data from a target domain into two or more target subdomains using an attribute annotation. The system ranks the target subdomains based on a similarity to the source domain. The system applies multiple domain discriminators between each of the target subdomains and a mixture of the source domain and preceding target domains. The system recognizes, using the multiple domain discriminators for the target domain, fine-grained objects.

Abstract: Systems and methods for performing domain adaptation include collecting a labeled source image having a view of an object. Viewpoints of the object in the source image are synthesized to generate view augmented source images. Photometrics of each of the viewpoints of the object are adjusted to generate lighting and view augmented source images. Features are extracted from each of the lighting and view augmented source images with a first feature extractor and from captured images captured by an image capture device with a second feature extractor. The extracted features are classified using domain adaptation with domain adversarial learning between extracted features of the captured images and extracted features of the lighting and view augmented source images. Labeled target images are displayed corresponding to each of the captured images including labels corresponding to classifications of the extracted features of the captured images.

Abstract: A method for jointly removing rolling shutter (RS) distortions and blur artifacts in a single input RS and blurred image is presented. The method includes generating a plurality of RS blurred images from a camera, synthesizing RS blurred images from a set of GS sharp images, corresponding GS sharp depth maps, and synthesized RS camera motions by employing a structure-and-motion-aware RS distortion and blur rendering module to generate training data to train a single-view joint RS correction and deblurring convolutional neural network (CNN), and predicting an RS rectified and deblurred image from the single input RS and blurred image by employing the single-view joint RS correction and deblurring CNN.

Abstract: Systems and methods are provided for producing a road layout model. The method includes capturing digital images having a perspective view, converting each of the digital images into top-down images, and conveying a top-down image of time t to a neural network that performs a feature transform to form a feature map of time t. The method also includes transferring the feature map of the top-down image of time t to a feature transform module to warp the feature map to a time t+1, and conveying a top-down image of time t+1 to form a feature map of time t+1. The method also includes combining the warped feature map of time t with the feature map of time t+1 to form a combined feature map, transferring the combined feature map to a long short-term memory (LSTM) module to generate the road layout model, and displaying the road layout model.

Abstract: Methods and systems for occlusion detection include detecting a set of foreground object masks in an image, including a mask of a visible portion of a foreground object and a mask of the foreground object that includes at least one occluded portion, using a machine learning model. A set of background object masks is detected in the image, including a mask of a visible portion of a background object and a mask of the background object that includes at least one occluded portion, using the machine learning model. The set of foreground object masks and the set of background object masks are merged using semantic merging. A computer vision task is performed that accounts for the at least one occluded portion of at least one object of the merged set.

Abstract: A computer-implemented method for implementing face spoofing detection using a physical-cue-guided multi-source multi-channel framework includes receiving a set of data including face recognition data, liveness data and material data associated with at least one face image, obtaining a shared feature from the set of data using a backbone neural network structure, performing, based on the shared feature, a pretext task corresponding to face recognition, a first proxy task corresponding to depth estimation, a liveness detection task, and a second proxy task corresponding to material prediction, and aggregating outputs of the pretext task, the first proxy task, the liveness detection task and the second proxy task using an attention mechanism to boost face spoofing detection performance.

Abstract: Methods and systems for object detection include training dataset-specific object detectors using respective annotated datasets, each of the annotated datasets including annotations for a respective set of one or more object classes. The annotated datasets are cross-annotated using the dataset-specific object detectors. A unified object detector is trained, using the cross-annotated datasets, to detect all of the object classes of the annotated datasets. Objects are detected in an input image using the unified object detector.

Abstract: A method for simultaneous multi-agent recurrent trajectory prediction is presented. The method includes reconstructing a topological layout of a scene from a dataset including real-world data, generating a road graph of the scene, the road graph capturing a hierarchical structure of interconnected lanes, incorporating vehicles from the scene on the generated road graph by utilizing tracklet information available in the dataset, assigning the vehicles to their closest lane identifications, and identifying diverse plausible behaviors for every vehicle in the scene. The method further includes sampling one behavior from the diverse plausible behaviors to select an associated velocity profile sampled from the real-world data of the dataset that resembles the sampled one behavior and feeding the road graph and the sampled velocity profile with a desired destination to a dynamics simulator to generate a plurality of simulated diverse trajectories output on a visualization device.

Abstract: Systems and methods for adapting semantic segmentation across domains is provided. The method includes inputting a source image into a segmentation network, and inputting a target image into the segmentation network. The method further includes identifying category wise features for the source image and the target image using category wise pooling, and discriminating between the category wise features for the source image and the target image. The method further includes training the segmentation network with a pixel-wise cross-entropy loss on the source image, and a weak image classification loss and an adversarial loss on the target image, and outputting a semantically segmented target image.

Abstract: A computer-implemented method for implementing face recognition includes receiving training data including a plurality of augmented images each corresponding to a respective one of a plurality of input images augmented by one of a plurality of variations, splitting a feature embedding generated from the training data into a plurality of sub-embeddings each associated with one of the plurality of variations, associating each of the plurality of sub-embeddings with respective ones of a plurality of confidence values, and applying a plurality of losses including a confidence-aware identification loss and a variation-decorrelation loss to the plurality of sub-embeddings and the plurality of confidence values to improve face recognition performance by learning the plurality of sub-embeddings.