Abstract: An information retrieval training system includes: a training dataset including training data having a feature space; the training data including multiple different types of elements, wherein no labels are provided with the training data; a training module configured to: maintain fixed a pre-trained model configured to receive features of queries; learn sets of pseudo-labels based on the training data; train parameters of adaptor modules for each of the sets of pseudo-labels, respectively, the adaptor modules configured to receive outputs of the pre-trained model, respectively; and train parameters of fusion modules based on neighboring pairs of the training data, the fusion modules configured to fuse together outputs of the adaptor modules, respectively.

Abstract: A system includes: a training dataset including first objects of a first modality and second objects of a second modality different than the first modality, where the second objects include text that is descriptive of the first objects; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: assign ones of the second objects to bins based on distances between the ones of the objects and a query; determine a ranking measure based on a number of the ones of the second objects assigned to the bins; determine losses based on the ranking measure and the first and second matrices; determine a final loss based on the losses; train embedding functions based on the final loss.

Abstract: A training system includes: a training dataset including first objects of a first modality and second objects of a second modality that are associated with the first objects, respectively; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: based on similarities between ones of the second objects, generate a third matrix by selectively adding first additional relevance values to the first matrix; based on the similarities between the ones of the second objects, generate a fourth matrix by selectively adding second additional relevance values to the second matrix; and store the third and fourth matrices in memory of a search module for cross-modal retrieval in response to receipt of search queries.

Abstract: An autonomous system includes: a first semantic segmentation model trained based on a training dataset including images and labels for the images, the first semantic segmentation model configured to generate a first segmentation map based on an image from a camera; a second semantic segmentation model of the same type of semantic segmentation model as the first semantic segmentation model, the second semantic segmentation model configured to generate a second segmentation map based on the image from the camera; an adaptation module configured to selectively adjust one or more first parameters of the second semantic segmentation model; and a reset module configured to: determine a first total number of unique classifications included in the first segmentation map; determine a second total number of unique classifications included in the first segmentation map; and selectively reset the first parameters to previous parameters, respectively, based on the first and second total numbers.

Abstract: Am image retrieval system includes: a neural network (NN) module configured to generate local features based on an input image; an iterative attention module configured to, via T iterations, generate an ordered set of super features in the input image based on the local features, where T is an integer greater than 1; and a selection module configured to select a second image from a plurality of images in an image database based on the second image having a second ordered set of super features that most closely match the ordered set of super features in the input image, where the super features in the set of super features do not include redundant local features of the input image.

Abstract: Methods and systems for training a dimensionality reduction model. Pairs of proximately located training vectors in a higher dimensional space are generated. Lower dimension vector pairs are generated by encoding first and second training vectors using the dimensionality reduction model, and augmented dimension vector pairs are generated by projecting to an augmented dimensional representation space having a greater number of dimensions. A similarity preservation loss and a redundancy reduction loss are computed and used to optimize parameters of the dimensionality reduction model.

Abstract: A method for processing a new sample in a data stream for updating a machine learning model configured for performing a task. The machine learning model is implemented by a processor in communication with a memory storing previous samples. The new sample is received, and the machine learning model is trained using combined samples including the new sample and the previous samples. The new sample is stored or not stored in the memory based on distances between the samples in an embedding space learned by the machine learning model.

Abstract: An interaction module includes: a first text-image interaction module configured to generate a vector representation of a first text-image pair based on an encoded representation of a reference image and an encoded representation of a text modifier, the reference image and the text modifier received from a computing device. A second text-image interaction module is configured to generate a vector representation of a second text-image pair based on the encoded representation of the text modifier and an encoded representation of a candidate target image. A compatibility module is configured to compute, based on the vector representation of the first text-image pair and the vector representation of the second text-image pair, a compatibility score for a triplet including the reference image, the text modifier, and the candidate target image. A ranking module is configured to rank a set of candidate target images including the candidate target image by compatibility scores.

Abstract: Methods for training a neural network model for sequentially learning a plurality of domains associated with a task. At least one set of auxiliary model parameters is determined by simulating at least one first optimization step based on a set of current model parameters and at least one auxiliary domain associated with a primary domain comprising one or more data points. A set of primary model parameters is determined by performing a second optimization step based on the current model parameters and the primary domain and on the at least one set of auxiliary model parameters and the primary domain and/or the auxiliary domain. The model is updated with the set of primary model parameters.

Abstract: A training system includes: an encoder module configured to receive a query image and to generate a first vector representative of one or more features in the query image using an encoder; a mixing module configured to generate a second vector by mixing a third vector, representative of one or more features in a second image that is classified as a negative relative to the query image, with a fourth vector; and an adjustment module configured to train the encoder by selectively adjusting one or more parameters of the encoder based on the first vector and the second vector.

Abstract: A method and system pre-trains a convolutional neural network for image recognition based upon masked language modeling by inputting, to the convolutional neural network, an image; outputting, from the convolutional neural network, a visual embedding tensor of visual embedding vectors; tokenizing a caption to create a list of tokens, at least one token having visual correspondence to the image received by the convolutional neural network; randomly selecting one of the tokens in the list of tokens to be masked, the selected token being taken as ground truth; computing, using a language model neural network, hidden representations of the tokens; using the hidden representation of the masked token, as a query vector, to pool the visual embedding vectors in the visual embedding tensor, attentively; predicting the masked token by mapping the pooled visual embedding vectors to the tokens; determining a prediction loss associated with the masked token; and back-propagating the prediction loss to the convolutional ne

Abstract: A method and system pre-trains a convolutional neural network for image recognition based upon masked language modeling by inputting, to the convolutional neural network, an image; outputting, from the convolutional neural network, a visual embedding tensor of visual embedding vectors; tokenizing a caption to create a list of tokens, at least one token having visual correspondence to the image received by the convolutional neural network; randomly selecting one of the tokens in the list of tokens to be masked, the selected token being taken as ground truth; computing, using a language model neural network, hidden representations of the tokens; using the hidden representation of the masked token, as a query vector, to pool the visual embedding vectors in the visual embedding tensor, attentively; predicting the masked token by mapping the pooled visual embedding vectors to the tokens; determining a prediction loss associated with the masked token; and back-propagating the prediction loss to the convolutional ne

Abstract: A method for re-identification of a subject in an image by pre-training a convolutional neural network to recognize individuals within a closed set of possible identifications and further pre-training the convolutional neural network using classification loss; training the pre-trained convolutional neural network by sequentially processing a plurality of triplet of images, each triplet containing a query image degraded by adding random noise to a region of the query image, a positive image corresponding to an image of a same subject as in the query image, and a negative image corresponding to an image of a different subject as in the query image by (a) ranking the triplets by the triplet loss computed, (b) selecting a subset of triplets among the plurality of triplets, and (c) retraining the pre-trained convolutional neural network on each of the triplets of the subset of triplets.

Abstract: A training system includes: a training dataset including first objects of a first modality and second objects of a second modality that are associated with the first objects, respectively; a first matrix including first relevance values indicative of relevance between the first objects and the second objects, respectively; a second matrix including second relevance values indicative of relevance between the second objects and the first objects, respectively; and a training module configured to: based on similarities between ones of the second objects, generate a third matrix by selectively adding first additional relevance values to the first matrix; based on the similarities between the ones of the second objects, generate a fourth matrix by selectively adding second additional relevance values to the second matrix; and store the third and fourth matrices in memory of a search module for cross-modal retrieval in response to receipt of search queries.

Abstract: A method for re-identification of a subject in an image from a set of images, by pre-training, by a data processor in a first server, a convolutional neural network with ImageNet to recognize individuals within a closed set of possible identifications and further pre-training the convolutional neural network using classification loss to realize person identification; training, by the data processor in the first server, the pre-trained convolutional neural network by sequentially processing a plurality of triplet of images and allowing a different size input for each image, each triplet containing a query image degraded by adding random noise to a region of the query image, a positive image corresponding to an image of a same subject as in the query image, and a negative image corresponding to an image of a different subject as in the query image by (a) ranking the triplets by the triplet loss computed (b) selecting a subset of triplets among the plurality of triplets, the subset of triplets having the largest

Abstract: In a method for detecting an object in an input image, an input image vector representing the input image is generated by performing a regional maximum activations of convolutions (R-MAC) using a convolutional neural network (CNN) applied to the input image and using regions for the R-MAC defined by applying a region proposal network (RPN) to the output of the CNN applied to the input image. Likewise, a reference image vector representing a reference image depicting the object is generated by performing the R-MAC using the CNN applied to the reference image and using regions for the R MAC defined by applying the RPN to the output of the CNN applied to the reference image. A similarity metric between the input image vector and the reference image vector is computed, and the object is detected as present in the input image if the similarity metric satisfies a detection criterion.

Abstract: Similar images are identified by semantically matching human-supplied text captions accompanying training images. An image representation function is trained to produce similar vectors for similar images according to this similarity. The trained function is applied to non-training second images in a different database to produce second vectors. This trained function does not require the second images to contain captions. A query image is matched to the second images by applying the trained function to the query image to produce a query vector, and the second images are ranked based on how closely the second vectors match the query vector, and the top ranking ones of the second images are output as a response to the query image.

Abstract: In a method for detecting an object in an input image, an input image vector representing the input image is generated by performing a regional maximum activations of convolutions (R-MAC) using a convolutional neural network (CNN) applied to the input image and using regions for the R-MAC defined by applying a region proposal network (RPN) to the output of the CNN applied to the input image. Likewise, a reference image vector representing a reference image depicting the object is generated by performing the R-MAC using the CNN applied to the reference image and using regions for the R MAC defined by applying the RPN to the output of the CNN applied to the reference image. A similarity metric between the input image vector and the reference image vector is computed, and the object is detected as present in the input image if the similarity metric satisfies a detection criterion.

Abstract: A method for generating object and part detectors includes accessing a collection of training images. The collection of training images includes images annotated with an object label and images annotated with a respective part label for each of a plurality of parts of the object. Joint appearance-geometric embeddings for regions of a set of the training images are generated. At least one detector for the object and its parts is learnt using annotations of the training images and respective joint appearance-geometric embeddings, e.g., using multi-instance learning for generating parameters of scoring functions which are used to identify high scoring regions for learning the object and its parts. The detectors may be output or used to label regions of a new image with object and part labels.

Abstract: An object detection method includes for each of a set of patches of an image, encoding features of the patch with a non-linear mapping function, and computing per-patch statistics based on the encoded features for approximating a window-level non-linear operation by a patch-level operation. Then, windows are extracted from the image, each window comprising a sub-set of the set of patches. Each of the windows is scored based on the computed patch statistics of the respective sub-set of patches. Objects, if any, can then be detected in the image, based on the window scores. The method and system allow the non-linear operations to be performed only at the patch level, reducing the computation time of the method, since there are generally many more windows than patches, while not impacting performance unduly, as compared to a system which performs non-linear operations at the window level.