Abstract: Methods and systems for Few-Shot Class-Incremental Learning (FSCIL) that utilizes a combination of Session Specific Prompts (SSP) and hyperbolic distance metrics to enhance session-wise learning and representation of image-text pairings across differing classes. The methods and systems include a base training session where both text and image features are projected into hyperbolic space for accurate class pairing using a cross-entropy loss function. Subsequent incremental sessions incorporate previously learned SSPs to retain and augment the separability of classes while minimizing the trainable parameters. This enhances performance in image-text classification tasks by leveraging a minimalistic approach, achieving higher accuracy with fewer trainable parameters compared to traditional models.

Abstract: A method includes splitting an input image into a plurality of patches with each patch corresponding to a distinct region of the input image using a vision transformer. The input image is defined using a model output of a vision model. The method further includes defining a plurality of position embeddings including a position embedding for each of the plurality of patches and for the input image as a whole using the vision transformer, labeling identified regions of the original image based on the estimated loss map to define a labeled image; and outputting a test performance qualifier indicating expected performance of the vision model when the vision model is part of the vision system. The test performance qualifier is calculated using a weighted analysis based on the image loss level and the regional loss level for each patch provided with the labeled image.

Abstract: A method includes encoding a set of hierarchical text prompts to define a set of text embeddings, where the set of hierarchical text prompt defines a primary informative prompt and a secondary informative prompt associated with the primary informative prompt. The method further includes encoding an input image to define a plurality of feature representations, changing a value of one or more identified feature representations among the plurality of feature representations to mask the one or more identified feature representation and define a general feature representation of the input image based on a class-specific threshold indicative of boundary between a class-specific feature and a general feature. The method further includes classifying the input image based on an out-of-distribution (OOD) score determined using a similarity analysis of the general feature representation and the set of text embeddings.

Abstract: A computer-implemented system and method relates to open-vocabulary image segmentation. A set of data pairs is automatically generated using a digital image and a corresponding caption. The set of data pairs include image segments and corresponding text data. The set of data pairs includes (i) a first subset that includes object segments as the image segments and corresponding object data as the text data and (ii) a second subset that includes part segments as the image segments and corresponding part data as the text data. A universal segmentation embedding (USE) model includes an image encoder and a segment embedding head. The image encoder generates patch embeddings based on patches of the digital image. The segment embedding head generates segment embeddings based on the image segments and the patch embeddings. Semantic segmentation data is generated based on the segment embeddings.

Abstract: Methods and systems for Few-Shot Class-Incremental Learning (FSCIL) that utilizes a combination of Session Specific Prompts (SSP) and hyperbolic distance metrics to enhance session-wise learning and representation of image-text pairings across differing classes. The methods and systems include a base training session where both text and image features are projected into hyperbolic space for accurate class pairing using a cross-entropy loss function. Subsequent incremental sessions incorporate previously learned SSPs to retain and augment the separability of classes while minimizing the trainable parameters. This enhances performance in image-text classification tasks by leveraging a minimalistic approach, achieving higher accuracy with fewer trainable parameters compared to traditional models.

Abstract: Methods and system for detecting out-of-distribution data for a neural network. A training dataset includes in-distribution data, for example image data associated with one or more images. The neural network is trained on the in-distribution data, and has a plurality of layers. A subspace of in-distribution data of the training dataset is generated based on a sample of one of the layers trained with the in-distribution data. Input image data associated with a sample image is received, and the neural network is executed on the input image data to determine a gradient associated with the sample image. The gradient is projected into the subspace to derive a projection of the gradient. The image data associated with the sample image is determined to be out of distribution based on a magnitude of the projection of the gradient.

Abstract: A method of performing open world object detection includes receiving object data, that includes embeddings data corresponding to a plurality of embeddings for known objects in a first input image, projecting the embeddings into a hyperbolic embedding space that includes embeddings in a plurality of categories of objects each including one or more classes of objects, regularizing the projected embeddings within the hyperbolic embedding space by moving each of the projected embeddings closer to embeddings in a same category of the plurality of categories and further away from embeddings in different categories of the plurality of categories, receiving an unmatched query corresponding to an object in a second input image, and generating, based on the hyperbolic embedding space including the regularized embeddings, an output signal that indicates whether the object in the second input image corresponds to an unknown object in one of the classes of objects.

Abstract: A method of performing data pre-selection for an object detection system includes receiving a first dataset that includes unlabeled data corresponding to one or more images, providing the first dataset and a plurality of learnable prompt vectors to a pre-training model. The learnable prompt vectors include text inputs. The method further includes generating, using the pre-training model, an unsupervised learning prompt based on the first dataset and the plurality of learnable prompt vectors. The unsupervised learning prompt corresponds to a multi-modal feature of the one or more images of the first dataset. The method further includes extracting features from either of the first dataset and a second dataset based on the unsupervised learning prompt, selecting and labeling a subset of instances of the extracted features, and generating and outputting a labeled dataset based on the labeled subset of instances.

Abstract: A method includes accessing, using at least one processor of an electronic device, a machine learning model. The machine learning model is trained by directing a gradient direction of gradients to one or more flat local minima and using a dynamic learning rate for one or more additional tasks. The method also includes receiving, using the at least one processor, an input from an input source. The method further includes providing, using the at least one processor, the input to the machine learning model. The method also includes receiving, using the at least one processor, an output from the machine learning model. In addition, the method includes instructing, using the at least one processor, at least one action based on the output from the machine learning model.