Abstract: A training system includes: a model; and a training module configured to: construct a first pair of images of at least a first portion of a first human captured at different times; construct a second pair of images of at least a second portion of a second human captured at the same time from different points of view; input the first and second pairs of images to the model; the model configured to: generate first and second reconstructed images of the at least the first portion of the first human based on the first and second pairs, respectively, and the training module is configured to selectively adjust one or more parameters of the model based on: the first reconstructed image and the second reconstructed image.

Abstract: A method includes: performing unsupervised pre-training of a model, the model including and a decoder including: obtaining a first image and a second image under different conditions or from different viewpoints; encoding, by the encoder, the first image into a representation of the first image and the second image into a representation of the second image; transforming the representation of the first image into a transformed representation; decoding, by the decoder, the transformed representation into a reconstructed image, where the transforming of the representation of the first image and the decoding of the transformed representation is based on the representation of the first image and the representation of the second image; and adjusting one or more parameters of at least one of the encoder and the decoder based on minimizing a loss; and fine-tuning the model, initialized with a set of task specific encoder parameters, for a geometric vision task.

Abstract: A motion generation system includes: a model configured to generate latent indices for a sequence of images including an entity performing an action based on an action label and a duration of the sequence; and a decoder module configured to: decode the latent indices and to generate the sequence of images including the entity performing the action based on the latent indices; and output the sequence of images including the entity performing the action to a display control module configured to display the sequence of images including the entity performing the action sequentially on a display.

Abstract: A system includes: a hand module to, based on a demonstration of a human hand grasping an object, determine first and second vectors that are normal to and parallel to a palm of the human hand, respectively, and a position of the human hand; a gripper module to determine third and fourth vectors that are normal to and parallel to a palm of a gripper of a robot, respectively, and a present position of the gripper; and an actuation module to: move the gripper when open such that the present position of the gripper is at the position of the human hand, the third and first vectors are aligned, and the fourth and second vectors are aligned; close fingers of the gripper based on minimizing a first loss; and actuate the fingers of the gripper to minimize a second loss determined based on the first loss and a third loss.

Abstract: A computer-implemented method includes: obtaining a pair of images depicting a same scene, the pair of images including a first image with a first pixel grid and a second image with a second pixel grid, the first pixel grid different than the second pixel grid; by a neural network module having a first set of parameters: generating a first feature map based on the first image; and generating a second feature map based on the second image; determining a first correlation volume based on the first and second feature maps; iteratively determining a second correlation volume based on the first correlation volume; determining a loss for the first and second feature maps based on the second correlation volume; generating a second set of the parameters based on minimizing a loss function using the loss; and updating the neural network module to include the second set of parameters.

Abstract: A computer-implemented method of recognition of actions performed by individuals includes: by one or more processors, obtaining images including at least a portion of an individual by the one or more processors, based on the images, generating implicit representations of poses of the individual in the images; and by the one or more processors, determining an action performed by the individual and captured in the images by classifying the implicit representations of the poses of the individual.

Abstract: A system for generating whole body poses includes: a body regression module configured to generate a first pose of a body of an animal in an input image by regressing from a stored body anchor pose; a face regression module configured to generate a second pose of a face of the animal in the input image by regressing from a stored face anchor pose; an extremity regression module configured to generate a third pose of an extremity of the animal in the input image by regressing from a stored extremity anchor pose; and a pose module configured to generate a whole body pose of the animal in the input image based on the first pose, the second pose, and the third pose.

Abstract: A training system includes: a neural network model configured to determine three-dimensional coordinates of joints, respectively, representing poses of animals in images, where the neural network model is trained using a first training dataset including: images including animals; and coordinates of joints of the animals in the images, respectively; and a training module configured to, after the training of the neural network model using the first training dataset, train the neural network model using a second training dataset including motion capture data, where the motion capture data does not include images of animals and includes measured coordinates at points, respectively, on animals.

Abstract: A computer-implemented method of recognition of actions performed by individuals includes: by one or more processors, obtaining images including at least a portion of an individual; by the one or more processors, based on the images, generating implicit representations of poses of the individual in the images; and by the one or more processors, determining an action performed by the individual and captured in the images by classifying the implicit representations of the poses of the individual.

Abstract: Am image retrieval system includes: a neural network (NN) module configured to generate local features based on an input image; an iterative attention module configured to, via T iterations, generate an ordered set of super features in the input image based on the local features, where T is an integer greater than 1; and a selection module configured to select a second image from a plurality of images in an image database based on the second image having a second ordered set of super features that most closely match the ordered set of super features in the input image, where the super features in the set of super features do not include redundant local features of the input image.

Abstract: A method of semi-supervised learning includes inputting an image; generating a weak augmentation version and a strong augmentation version of the inputted image; predicting a class of the weak augmentation version of the inputted image; determining if the predicted class of the weak augmentation version of the inputted image is confident; using a pseudo-label to train a model using the strong augmentation version of the inputted image when the predicted class of the weak augmentation version of the selected image is confident; and using a self-supervised loss based on deep clustering to train a model using the strong augmentation version of the selected image when the predicted class of the weak augmentation version of the selected image is not confident.

Abstract: A system for generating whole body poses includes: a body regression module configured to generate a first pose of a body of an animal in an input image by regressing from a stored body anchor pose; a face regression module configured to generate a second pose of a face of the animal in the input image by regressing from a stored face anchor pose; an extremity regression module configured to generate a third pose of an extremity of the animal in the input image by regressing from a stored extremity anchor pose; and a pose module configured to generate a whole body pose of the animal in the input image based on the first pose, the second pose, and the third pose.

Abstract: A system for generating whole body poses includes: a body regression module configured to generate a first pose of a body of an animal in an input image by regressing from a stored body anchor pose; a face regression module configured to generate a second pose of a face of the animal in the input image by regressing from a stored face anchor pose; an extremity regression module configured to generate a third pose of an extremity of the animal in the input image by regressing from a stored extremity anchor pose; and a pose module configured to generate a whole body pose of the animal in the input image based on the first pose, the second pose, and the third pose.

Abstract: A computer-implemented method for training a neural network to perform a data processing task includes: for each data sample of a set of labeled data samples: by a first loss function for the data processing task, computing a first loss for that data sample; and by a second loss function, automatically computing a weight value for the data sample based on the first loss, the weight value indicative of a reliability of a label of the data sample predicted by the neural network for the data sample and dictating the extent to which that data sample impacts training of the neural network; and training the neural network with the set of labelled data samples according to their respective weight value.

Abstract: A training system includes: an encoder module configured to receive a query image and to generate a first vector representative of one or more features in the query image using an encoder; a mixing module configured to generate a second vector by mixing a third vector, representative of one or more features in a second image that is classified as a negative relative to the query image, with a fourth vector; and an adjustment module configured to train the encoder by selectively adjusting one or more parameters of the encoder based on the first vector and the second vector.

Abstract: A system for generating whole body poses includes: a body regression module configured to generate a first pose of a body of an animal in an input image by regressing from a stored body anchor pose; a face regression module configured to generate a second pose of a face of the animal in the input image by regressing from a stored face anchor pose; an extremity regression module configured to generate a third pose of an extremity of the animal in the input image by regressing from a stored extremity anchor pose; and a pose module configured to generate a whole body pose of the animal in the input image based on the first pose, the second pose, and the third pose.

Abstract: A method for training a convolutional recurrent neural network for semantic segmentation in videos, includes (a) training, using a set of semantically segmented training images, a first convolutional neural network; (b) training, using a set of semantically segmented training videos, a convolutional recurrent neural network, corresponding to the first convolutional neural network, wherein a convolutional layer has been replaced by a recurrent module having a hidden state. The training of the convolutional recurrent neural network, for each pair of successive frames (t?1, t?1; T2) of a video of the set of semantically segmented training videos includes warping an internal state of a recurrent layer according to an estimated optical flow between the frames of the pair of successive frames, so as to adapt the internal state to the motion of pixels between the frames of the pair and learning parameters of at least the recurrent module.

Abstract: A system for detecting and describing keypoints in images is described. A camera is configured to capture an image including a plurality of pixels. A fully convolutional network is configured to jointly and concurrently: generate descriptors for each of the pixels, respectively; generate reliability scores for each of the pixels, respectively; and generate repeatability scores for each of the pixels, respectively. A scoring module is configured to generate scores for the pixels, respectively, based on the reliability scores and the repeatability scores of the pixels, respectively. A keypoint list module is configured to: select X of the pixels having the X highest scores, where X is an integer greater than 1; and generate a keypoint list including: locations of the selected X pixels; and the descriptors of the selected X pixels.

Abstract: A system for detecting and describing keypoints in images is described. A camera is configured to capture an image including a plurality of pixels. A fully convolutional network is configured to jointly and concurrently: generate descriptors for each of the pixels, respectively; generate reliability scores for each of the pixels, respectively; and generate repeatability scores for each of the pixels, respectively. A scoring module is configured to generate scores for the pixels, respectively, based on the reliability scores and the repeatability scores of the pixels, respectively. A keypoint list module is configured to: select X of the pixels having the X highest scores, where X is an integer greater than 1; and generate a keypoint list including: locations of the selected X pixels; and the descriptors of the selected X pixels.

Abstract: A method for training, using a plurality of training images with corresponding six degrees of freedom camera pose for a given environment and a plurality of reference images, each reference image depicting an object-of-interest in the given environment and having a corresponding two-dimensional to three-dimensional correspondence for the given environment, a neural network to provide visual localization by: for each training image, detecting and segmenting object-of-interest in the training image; generating a set of two-dimensional to two-dimensional matches between the detected and segmented objects-of-interest and corresponding reference images; generating a set of two-dimensional to three-dimensional matches from the generated set of two-dimensional to two-dimensional matches and the two-dimensional to three-dimensional correspondences corresponding to the reference images; and determining localization, for each training image, by solving a perspective-n-point problem using the generated set of two-dimens