Abstract: A method includes obtaining a time-series of training samples that include one or more states, a ground truth value, an output value produced in the presence of the one or more states, and an actual error value that is defined as a difference between the ground truth value and the output value. The method also includes training a machine learning model using the time-series of training samples such that the machine learning model is configured to determine a condition-dependent error distribution for a current time step based on simulated states for the current time step.

Abstract: A method includes determining a detection output that represents an object in a two-dimensional image using a detection model, wherein the detection output includes a shape definition that describes a shape and size of the object; defining a three-dimensional representation based on the shape definition, wherein the three-dimensional representation includes a three-dimensional model that represents the object that is placed in three-dimensional space according to a position and a rotation; determining a three-dimensional detection loss that describes a difference between the three-dimensional representation and three-dimensional sensor information; and updating the detection model based on the three-dimensional detection loss.

Abstract: A method includes obtaining surface samples that represent three-dimensional locations of surfaces of an environment; generating a voxelized representation of the surfaces of the environment in three-dimensional space using the surface samples; obtaining an image that shows the surfaces of the environment; associating each of the surface samples with image information that corresponds to a portion of the image that is spatially correlated with a respective one of the surface samples; determining voxel features for voxels from the voxelized representation based on the surface samples and the image information using a first trained machine learning model, wherein the voxel features each describe three-dimensional shapes present within a respective one of the voxels; and detecting objects based on the voxel features.

Abstract: A method includes obtaining training samples that include images that depict objects and annotations of annotated key point locations for the objects. The method also includes training a machine learning model to determine estimated key point locations for the objects and key point uncertainty values for the estimated key point locations by minimizing a loss function that is based in part on a key point localization loss value that represents a difference between the annotated key point locations and the estimated key point locations values and is weighted by the key point uncertainty values.

Abstract: A neural network is trained to defend against adversarial attacks, such as by preparing an input image for classification by a neural network where the input image includes a noise-based perturbation. The input image is divided into source patches. Replacement patches are selected for the source patches by searching a patch library for candidate patches available for replacing ones of those source patches, such as based on sizes of those source patches. A denoised image reconstructed from a number of replacement patches is then output to the neural network for classification. The denoised image may be produced based on reconstruction errors determined for individual candidate patches identified from the patch library. Alternatively, the denoised image may be selected from amongst a number of candidate denoised images. A set of training images is used to construct the patch library, such as based on salient data within patches of those training images.

Abstract: A method for detecting objects in an environment includes obtaining, from one or more sensors, distance measurements from the one or more sensors to portions of the environment, generating a representation of a three-dimensional space using the distance measurements, identifying object features in the representation of the three-dimensional space using a neural network, comparing the object features to pre-defined three-dimensional templates to generate scores that represent correspondence of the object features to the pre-defined three-dimensional templates, and determining a location and a rotational orientation for a three-dimensional object based on the scores.

Abstract: A method and system detects actions of an object in a scene by first acquiring a video of the scene as a sequence of images, wherein each image includes pixels, wherein the video is partitioned into chunks. The object in the video is tracked. For each object and each chunk of the video, trajectories of the pixels within a bounding box located over the object are tracked, and cropped trajectories and cropped images for one or more images in the chunk are produced using the bounding box. Then, the cropped trajectories and cropped images are passed to a recurrent neural network (RNN) that outputs a relative score for each action of interest.

Abstract: A method detects an object in an image. The method extracts a first feature vector from a first region of an image using a first subnetwork and determines a second region of the image by processing the first feature vector with a second subnetwork. The method also extracts a second feature vector from the second region of the image using the first subnetwork and detects the object using a third subnetwork on a basis of the first feature vector and the second feature vector to produce a bounding region surrounding the object and a class of the object. The first subnetwork, the second subnetwork, and the third subnetwork form a neural network. Also, a size of the first region differs from a size of the second region.

Abstract: A computer-implemented method generates a multimodal digital image by processing a vector with a first neural network to produce a first modality of the digital image and processing the vector with a second neural network to produce a second modality of the digital image. A structure and a number of layers of the first neural network are identical to a structure and a number of layers of the second neural network. Also, at least one layer in the first neural network has parameters identical to parameters of a corresponding layer in the second neural network, and at least one layer in the first neural network has parameters different from parameters of a corresponding layer in the second neural network.

Abstract: A method for detecting an object in an image includes extracting a first feature vector from a first region of an image using a first subnetwork, determining a second region of the image by resizing the first region into a fixed ratio using a second subnetwork, wherein a size of the first region is smaller than a size of the second region, extracting a second feature vector from the second region of the image using the second subnetwork, classifying a class of the object using a third subnetwork on a basis of the first feature vector and the second feature vector, and determining the class of object in the first region according to a result of the classification, wherein the first subnetwork, the second subnetwork, and the third subnetwork form a neural network, wherein steps of the method are performed by a processor.

Abstract: A method detects an object in an image. The method extracts a first feature vector from a first region of an image using a first subnetwork and determines a second region of the image by processing the first feature vector with a second subnetwork. The method also extracts a second feature vector from the second region of the image using the first subnetwork and detects the object using a third subnetwork on a basis of the first feature vector and the second feature vector to produce a bounding region surrounding the object and a class of the object. The first subnetwork, the second subnetwork, and the third subnetwork form a neural network. Also, a size of the first region differs from a size of the second region.

Abstract: A computer-implemented method generates a multimodal digital image by processing a vector with a first neural network to produce a first modality of the digital image and processing the vector with a second neural network to produce a second modality of the digital image. A structure and a number of layers of the first neural network are identical to a structure and a number of layers of the second neural network. Also, at least one layer in the first neural network has parameters identical to parameters of a corresponding layer in the second neural network, and at least one layer in the first neural network has parameters different from parameters of a corresponding layer in the second neural network.

Abstract: A method upsamples an image using a non-linear fully connected neural network to produce only global details of an upsampled image and interpolates the image to produce a smooth upsampled image. The method concatenates the global details and the smooth upsampled image into a tensor and applies a sequence of nonlinear convolutions to the tensor using a convolutional neural network to produce the upsampled image.

Abstract: A method labels an image of a street view by first extracting, for each pixel, an appearance feature for inferring a semantic label, a depth feature for inferring a depth label. Then, a column-wise labeling procedure is applied to the features to jointly determine the semantic label and the depth label for each pixel using the appearance feature and the depth feature, wherein the column-wise labeling procedure is according to a model of the street view, and wherein each column of pixels in the images includes at most four layers.

Abstract: A method and system detects actions of an object in a scene by first acquiring a video of the scene as a sequence of images, wherein each image includes pixels, wherein the video is partitioned into chunks. The object in the video is tracked. For each object and each chunk of the video, trajectories of the pixels within a bounding box located over the object are tracked, and cropped trajectories and cropped images for one or more images in the chunk are produced using the bounding box. Then, the cropped trajectories and cropped images are passed to a recurrent neural network (RNN) that outputs a relative score for each action of interest.

Abstract: A method upsamples an image using a non-linear fully connected neural network to produce only global details of an upsampled image and interpolates the image to produce a smooth upsampled image. The method concatenates the global details and the smooth upsampled image into a tensor and applies a sequence of nonlinear convolutions to the tensor using a convolutional neural network to produce the upsampled image.

Abstract: A computer-implemented method for semantic segmentation of an image determines unary energy of each pixel in an image using a first subnetwork, determines pairwise energy of at least some pairs of pixels of the image using a second subnetwork, and determines, using a third subnetwork, an inference on a Gaussian random field (GRF) minimizing an energy function including a combination of the unary energy and the pairwise energy. The GRF inference defining probabilities of semantic labels for each pixel in the image, and the method converts the image into a semantically segmented image by assigning to a pixel in the semantically segmented image a semantic label having the highest probability for a corresponding pixel in the image among the probabilities determined by the third subnetwork. The first subnetwork, the second subnetwork, and the third subnetwork are parts of a neural network.

Abstract: A method for face alignment operates on a face image and a set of intitial landmark locations by first aligning globally the initial locations to a set of landmark locations of a face with a prototype shape to obtain global alignment parameters, and then warping the initial locations and the image from a coordinate frame of the image to a coordinate frame of the prototype shape according to the global alignment parameters to obtain warped landmark locations and a warped face image. Features are extracted from the warped face image at the warped landmark locations, and a regression function is applied to the features to obtain updated landmark locations in the coordinate frame of the prototype shape. Finally, the updated landmark locations in the coordinate frame of the prototype shape are warped to the coordinate frame of the image, to obtain updated landmark locations.

Abstract: A sensor acquires an input image X of a scene. The image includes noise with a variance ?2. A deep Gaussian conditional random field (GCRF) network is applied to the input image to produce an output image Y, where the output image is denoised, and wherein the deep GCRF includes a prior generation (PgNet) network followed by an inference network (InfNet), wherein the PgNet produces patch covariance priors ?ij for patches centered on every pixel (i,j) in the input image, and wherein the InfNet is applied to the patch covariance priors and the input image to solve the GCRF.

Abstract: A vehicle includes a set of devises for determining a state of the vehicle at an instant of time and a processor operatively connected to a memory for processing parameters of the state of the vehicle collected over a plurality of instances of time for a period of time from an initial instance of time till a current instant of time to determine a set of driving conditions leading to a current state of the vehicle at the current instant of time. Each driving condition indicates a condition of an operation of the vehicle for the period of time. The vehicle also includes a communication module for outputting the current state of the vehicle determined by the set of devices at the current instant of time and for outputting the set of driving conditions leading to the current state of the vehicle.