Abstract: Methods and systems for Few-Shot Class-Incremental Learning (FSCIL) that utilizes a combination of Session Specific Prompts (SSP) and hyperbolic distance metrics to enhance session-wise learning and representation of image-text pairings across differing classes. The methods and systems include a base training session where both text and image features are projected into hyperbolic space for accurate class pairing using a cross-entropy loss function. Subsequent incremental sessions incorporate previously learned SSPs to retain and augment the separability of classes while minimizing the trainable parameters. This enhances performance in image-text classification tasks by leveraging a minimalistic approach, achieving higher accuracy with fewer trainable parameters compared to traditional models.

Abstract: A computer-implemented method and system relate to digital image retrieval and data curation. The data curation may relate to training a machine learning model on at least one specific task. A vocabulary of visual concepts is generated for a specific task using a target dataset. The vocabulary includes a representative image embedding for each visual concept. Precomputed image embeddings are retrieved from a vector database. Each precomputed image embedding is decomposed into a linear combination of the visual concepts. For each precomputed image embedding, a set of weights is generated based on the vocabulary. Each weight is indicative of a prominence of a respective representative image embedding. The set of weights of each precomputed image embedding is stored in an enhanced vector database. A set of digital images is retrievable from the enhanced vector database in response to a query.

Abstract: A computer-implemented system and method relates to language-guided self-supervised semantic segmentation. A modified image is generated by performing data augmentation on a source image. A machine learning model generates first pixel embeddings based on the modified image. First segment embeddings are generated using the first pixel embeddings. A pretrained vision-language model generates second pixel embeddings based on the source image. Second segment embeddings are generated by applying segment contour data from the first pixel embeddings to the second pixel embeddings after the data augmentation is performed on the second pixel embeddings. Embedding consistent loss data is generated by comparing the first segment embeddings in relation to the second segment embeddings. Combined loss data is generated that includes the embedding consistent loss data. Parameters of the machine learning model are updated based on the combined loss data.

Abstract: A method includes splitting an input image into a plurality of patches with each patch corresponding to a distinct region of the input image using a vision transformer. The input image is defined using a model output of a vision model. The method further includes defining a plurality of position embeddings including a position embedding for each of the plurality of patches and for the input image as a whole using the vision transformer, labeling identified regions of the original image based on the estimated loss map to define a labeled image; and outputting a test performance qualifier indicating expected performance of the vision model when the vision model is part of the vision system. The test performance qualifier is calculated using a weighted analysis based on the image loss level and the regional loss level for each patch provided with the labeled image.

Abstract: Methods for developing and managing long-term memory solutions for large language models (LLMs) within a context of providing agents of the LLM as a service are disclosed. Following task-related communications between LLM agents and users of the service, information pertaining to domain knowledge, user preferences, and success or not in completing the requested task is distilled into data samples by a reflections agent of the service. The data samples are then stored into a long-term memory database that is accessible by LLM agents in the future, such that the agents can recall information of previous interactions in order to more efficiently perform new tasks for users.

Abstract: A device repairing method includes: collecting a device-parameter set of a target device; based on the device-parameter set, determining whether a to-be-repaired component exists among a plurality of device components of the target device, to obtain a determination result; in response to the determination result indicating that the to-be-repaired component exists, repairing the to-be-repaired component by operating a preset port in the to-be-repaired component, to obtain a repairment result; and according to the repairment result, determining whether the to-be-repaired component is repaired, and in response to the to-be-repaired component being not repaired, repeatedly executing the step of repairing the to-be-repaired component by operating the preset port in the to-be-repaired component, till the repairment result indicates that the to-be-repaired component is repaired or a repairment time quantity reaches a preset time-quantity threshold.

Abstract: Methods for a machine learning network that provide efficient, scalable, and granular analyses during validation of an object detection model are disclosed. The system described herein is configured to use extraction of visual concepts to provide interpretable metadata to a data slice finding technique. The identified, poor-performing slices are then provided to a user for selection as to which slice or slices to focus on when preparing a subsequent training dataset that is to be used to further refine the object detection model. The system then coordinates with a large language model and with a vision and language foundational model to augment the original validation dataset with supplementary image samples that are determined to be associated with the problems currently causing poor performance of the model.

Abstract: A method includes encoding a set of hierarchical text prompts to define a set of text embeddings, where the set of hierarchical text prompt defines a primary informative prompt and a secondary informative prompt associated with the primary informative prompt. The method further includes encoding an input image to define a plurality of feature representations, changing a value of one or more identified feature representations among the plurality of feature representations to mask the one or more identified feature representation and define a general feature representation of the input image based on a class-specific threshold indicative of boundary between a class-specific feature and a general feature. The method further includes classifying the input image based on an out-of-distribution (OOD) score determined using a similarity analysis of the general feature representation and the set of text embeddings.

Abstract: Methods for a machine-learning network that provide efficient, scalable, and granular analyses during validation of a machine learning model are disclosed. Providing quantitative analysis information to machine learning experts when they are deciding how to proceed with further optimizing their given machine learning model allows for more directed procedures during edge case detection. Following the execution of a principal machine learning model using a validation dataset, a shallow learning model may be trained to provide simulations about how the principal model may be improved, or not, given different re-training scenarios. By using slice-based schemes, the validation dataset is divided into certain problematic data slices, and then, during inference of the shallow learning model, additional quantitative information about the effect on other data slices given a subsequent re-training of the principal model using a certain problematic data slice allows the ML expert to make more informed decisions.

Abstract: Methods for a machine-learning network that provide efficient, scalable, and granular analyses during validation of a machine learning model are disclosed. Validation of models depends upon many factors, including the real-world application of the model, the type of model being trained, and the types of data samples it is being trained on. In order to provide relevant edge case information to users that pertains to their specific model, data slice finding techniques may be used to identify subsets of the dataset that are particularly problematic. By limiting a length of the slice description that the algorithm searches and by configuring the algorithm to target specific types of errors, users are provided with a more granular analysis that then allows them to determine how or if they need to retrain the model.

Abstract: A computer-implemented system and method relates to open-vocabulary image segmentation. A set of data pairs is automatically generated using a digital image and a corresponding caption. The set of data pairs include image segments and corresponding text data. The set of data pairs includes (i) a first subset that includes object segments as the image segments and corresponding object data as the text data and (ii) a second subset that includes part segments as the image segments and corresponding part data as the text data. A universal segmentation embedding (USE) model includes an image encoder and a segment embedding head. The image encoder generates patch embeddings based on patches of the digital image. The segment embedding head generates segment embeddings based on the image segments and the patch embeddings. Semantic segmentation data is generated based on the segment embeddings.

Abstract: Methods and systems for Few-Shot Class-Incremental Learning (FSCIL) that utilizes a combination of Session Specific Prompts (SSP) and hyperbolic distance metrics to enhance session-wise learning and representation of image-text pairings across differing classes. The methods and systems include a base training session where both text and image features are projected into hyperbolic space for accurate class pairing using a cross-entropy loss function. Subsequent incremental sessions incorporate previously learned SSPs to retain and augment the separability of classes while minimizing the trainable parameters. This enhances performance in image-text classification tasks by leveraging a minimalistic approach, achieving higher accuracy with fewer trainable parameters compared to traditional models.

Abstract: A visual analytics workflow and system are disclosed for assessing, understanding, and improving deep neural networks. The visual analytics workflow advantageously enables interpretation and improvement of the performance of a neural network model, for example an image-based objection detection and classification model, with minimal human-in-the-loop interaction. A data representation component extracts semantic features of input image data, such as colors, brightness, background, rotation, etc. of the images or objects in the images. The input image data are passed through the neural network to obtain prediction results, such as object detection and classification results. An interactive visualization component transforms the prediction results and semantic features into interactive and human-friendly visualizations, in which graphical elements encoding the prediction results are visually arranged depending on the extracted semantic features of input image data.

Abstract: An anti-carsickness active suspension robust genetic control method, which establishes a vehicle four-degree-of-freedom suspension model considering wheelbase preview, and the suspension designed by the disclosure meets the constraint conditions that the dynamic stroke does not exceed the maximum allowable stroke, the wheels keep good contact with the ground, the control force should be smaller than the maximum output control force of an actuator, and the like. Meanwhile, an objective function for preventing carsickness of a driver and passengers is established, a state output feedback control gain is solved based on a linear matrix inequality method, and a finite frequency domain robust control method optimized through a genetic algorithm is designed, so that optimal parameters of robust control are obtained. The control performance of the active suspension on road excitation in the motion sickness frequency interval is the best, and the riding comfort and the vehicle driving smoothness are improved.

Abstract: The present disclosure discloses an active suspension control method under vehicle-mounted visual perception preview. It uses a binocular camera combined with multiple visual perception algorithms, and monitors in real time the road surface conditions ahead of the vehicle. By accurately capturing and analyzing the road surface information, based on robust control theory and Lyapunov theory, it designs a matching preview H? controller. The vehicle can effectively reduce bumps and vibrations by timely adjusting the suspension system, providing passengers with a more stable and smooth driving experience. The present disclosure uses a machine vision method to sense in advance the road surface information ahead, improving the time lag problem in the traditional suspension control method, thereby significantly improving the vehicle safety and ride comfort.

Abstract: Methods and system for detecting out-of-distribution data for a neural network. A training dataset includes in-distribution data, for example image data associated with one or more images. The neural network is trained on the in-distribution data, and has a plurality of layers. A subspace of in-distribution data of the training dataset is generated based on a sample of one of the layers trained with the in-distribution data. Input image data associated with a sample image is received, and the neural network is executed on the input image data to determine a gradient associated with the sample image. The gradient is projected into the subspace to derive a projection of the gradient. The image data associated with the sample image is determined to be out of distribution based on a magnitude of the projection of the gradient.

Abstract: A method of performing open world object detection includes receiving object data, that includes embeddings data corresponding to a plurality of embeddings for known objects in a first input image, projecting the embeddings into a hyperbolic embedding space that includes embeddings in a plurality of categories of objects each including one or more classes of objects, regularizing the projected embeddings within the hyperbolic embedding space by moving each of the projected embeddings closer to embeddings in a same category of the plurality of categories and further away from embeddings in different categories of the plurality of categories, receiving an unmatched query corresponding to an object in a second input image, and generating, based on the hyperbolic embedding space including the regularized embeddings, an output signal that indicates whether the object in the second input image corresponds to an unknown object in one of the classes of objects.

Abstract: A method of performing data pre-selection for an object detection system includes receiving a first dataset that includes unlabeled data corresponding to one or more images, providing the first dataset and a plurality of learnable prompt vectors to a pre-training model. The learnable prompt vectors include text inputs. The method further includes generating, using the pre-training model, an unsupervised learning prompt based on the first dataset and the plurality of learnable prompt vectors. The unsupervised learning prompt corresponds to a multi-modal feature of the one or more images of the first dataset. The method further includes extracting features from either of the first dataset and a second dataset based on the unsupervised learning prompt, selecting and labeling a subset of instances of the extracted features, and generating and outputting a labeled dataset based on the labeled subset of instances.

Abstract: A device repairing method includes: collecting a device-parameter set of a target device; based on the device-parameter set, determining whether a to-be-repaired component exists among a plurality of device components of the target device, to obtain a determination result; in response to the determination result indicating that the to-be-repaired component exists, repairing the to-be-repaired component by operating a preset port in the to-be-repaired component, to obtain a repairment result; and according to the repairment result, determining whether the to-be-repaired component is repaired, and in response to the to-be-repaired component being not repaired, repeatedly executing the step of repairing the to-be-repaired component by operating the preset port in the to-be-repaired component, till the repairment result indicates that the to-be-repaired component is repaired or a repairment time quantity reaches a preset time-quantity threshold.

Abstract: A computer-implemented system and method relates to language-guided self-supervised semantic segmentation. A modified image is generated by performing data augmentation on a source image. A machine learning model generates first pixel embeddings based on the modified image. First segment embeddings are generated using the first pixel embeddings. A pretrained vision-language model generates second pixel embeddings based on the source image. Second segment embeddings are generated by applying segment contour data from the first pixel embeddings to the second pixel embeddings after the data augmentation is performed on the second pixel embeddings. Embedding consistent loss data is generated by comparing the first segment embeddings in relation to the second segment embeddings. Combined loss data is generated that includes the embedding consistent loss data. Parameters of the machine learning model are updated based on the combined loss data.