Abstract: Methods for determining black-box representations of machine learning models when information pertaining to internal states or parameters of the models are not accessible are disclosed. By using outputs of the model instead of internal states, the black-box representation is model-agnostic and provides a reliable and robust representation of the model using an external lens. The black-box representation is generated using responses from the model to a series of initialization and elicitation questions that quantify the confidence that the model has in answers it just returned. The black-box representation is then used as a training dataset for a linear classifier in order to learn performance metrics about the model.

Abstract: A method of initializing and training a neural network having a plurality of layers includes defining a first function configured to generate a filter based on a plurality of variance values associated with respective pairs of parameters of the plurality of layers, calculating the plurality of variance values based on depths of respective layers of the plurality of layers such that the variance values increase as the depths increase, calculating a covariance matrix using the first function, the covariance matrix having a block structure and each block of the covariance matrix corresponding to a covariance between a respective parameter and other parameters of the plurality of layers, providing, as input, the covariance matrix to the neural network to initialize the neural network for training, and generating, using the neural network, an output based on the covariance matrix.

Abstract: Methods and systems of using a trained machine-learning model to perform root cause analysis on a manufacturing process. A pre-trained machine learning model is provided that is trained to predict measurements of non-faulty parts. The pre-trained model is trained on training measurement data regarding physical characteristics of manufactured parts as measured by a plurality of sensors at a plurality of manufacturing stations. With the trained model, then measurement data from the sensors is received regarding the manufactured part and the stations. This new set of measurement data is back propagated through the pre-trained model to determine a magnitude of absolute gradients of the new measurement data. The root cause is then identified based on this magnitude of absolute gradients. In other embodiments the root cause is identified based on losses determined between a set of predicted measurement data of a part using the model, and actual measurement data.

Abstract: A system includes a machine learning network input interface configured to receive input data from a sensor, one or more processors collectively programmed to receive an input data from the sensor, wherein the input data is indicative of image of a scene that includes a perturbation from a black-box attack with a physical perturbation at the scene, display an adversarial pattern at the scene, determine an objective function utilizing at least the adversarial pattern and a target classification of the machine-learning network, randomly select a plurality of data points associated with the adversarial pattern and the objective function, wherein the data points are associated with a number of queries of the objective function, obtain a machine-learning model output utilizing the data points displayed in the scene, and in response to meeting a criteria associated with the adversarial pattern and model output, identify a successful attack pattern.

Abstract: A system includes a controller configured to generate an original patch utilizing Bayesian optimization, output the original patch at a display at a scene and determine if the original patch does not meet a success criteria of the machine-learning model, in response to the original patch not meeting the success criteria, upscaling the patch, decompose the upscaled patch into o components, for each of the components, utilize Bayesian optimization to update one of the components of the upscaled patch and freezing the other components to generate an updated patch, in response to the updated patch meeting the success criteria, output the updated upscaled patch, and in response to the updated upscaled patch not meeting the success criteria, iteratively update the unfrozen components and determine if the success criteria is met and if not met, unfreeze the frozen components and iteratively update the unfrozen components until the success criteria is met.

Abstract: Methods and systems for training a machine learning model with measurement data captured during a manufacturing process. Measurement data regarding a physical characteristic of a plurality of manufactured parts is received as measured by a plurality of sensors at various manufacturing stations. A time-series dynamics machine learning model encodes the measurement data into a latent space having a plurality of nodes. Each node is associated with the measurement data of one of the manufactured parts and at one of the manufacturing stations. A batch of the measurement data can be built, the batch include a first node and a first plurality of nodes immediately connected to the first node via first edges, and measured in time earlier than the first node. A prediction machine learning model can predict measurements of a first of the manufactured parts based on the latent space of the batch of nodes.

Abstract: A computer-implemented method that includes receiving a plurality of input images, generating a visual matrix utilizing the plurality of images and an image encoder, wherein the visual matrix includes a list of encoded images, receiving a plurality of text prompts, selecting a text prompt from the plurality of text prompts, send the first one of the text prompts to a language model to generate a candidate list of tokens, selecting tokens, converting the text prompts into updated text prompts via appending the tokens, generating a text matrix utilizing the text prompt and text encoder, and utilizing numerical values assigned at an image-text similarity matrix, determining a score associated with the image-text similarity matrix; and evaluating a criteria and outputting a final token to the updated text prompt in response to identifying a highest score associated with the final token after evaluating each of the plurality of text prompts.

Abstract: A method of initializing and training a transformer neural network including a plurality of self-attention layers configured to operate in accordance with a plurality of matrices includes determining structural relationships between respective parameters of the plurality of matrices, initializing the plurality of matrices based on the determined structural relationships, providing, as input, the plurality of matrices to the transformer neural network to initialize the transformer neural network for training, and generating, using the transformer neural network, an output based on the plurality of matrices.

Abstract: A computer-implemented method for a machine learning (ML) system includes receiving a first image frame and a second frame from a sensor, wherein the first and second image frames are time series data, determining a first flow state and a first latent state of the first image frame, determining a Deep Equilibrium Model (DEQ) based fix point solution via a root finding method based on the first flow state, the first latent state, and a layer function to obtain an estimated flow and latent state, receiving a third image frame, wherein the second and third image frames are time series data, determining the fix point solution via the root finding method based on the estimated flow state, the estimated latent state, and layer function to obtain an updated flow state and updated latent state, and outputting the updated flow state.

Abstract: A system includes a controller configured to generate an original patch utilizing Bayesian optimization, output the original patch at a display at a scene and determine if the original patch does not meet a success criteria of the machine-learning model, in response to the original patch not meeting the success criteria, upscaling the patch, decompose the upscaled patch into o components, for each of the components, utilize Bayesian optimization to update one of the components of the upscaled patch and freezing the other components to generate an updated patch, in response to the updated patch meeting the success criteria, output the updated upscaled patch, and in response to the updated upscaled patch not meeting the success criteria, iteratively update the unfrozen components and determine if the success criteria is met and if not met, unfreeze the frozen components and iteratively update the unfrozen components until the success criteria is met.

Abstract: A system includes a machine learning network input interface configured to receive input data from a sensor, one or more processors collectively programmed to receive an input data from the sensor, wherein the input data is indicative of image of a scene that includes a perturbation from a black-box attack with a physical perturbation at the scene, display an adversarial pattern at the scene, determine an objective function utilizing at least the adversarial pattern and a target classification of the machine-learning network, randomly select a plurality of data points associated with the adversarial pattern and the objective function, wherein the data points are associated with a number of queries of the objective function, obtain a machine-learning model output utilizing the data points displayed in the scene, and in response to meeting a criteria associated with the adversarial pattern and model output, identify a successful attack pattern.

Abstract: A method discloses receiving a plurality of input images, receiving text prompts, generating a visual matrix utilizing the images and an image encoder, generating a text matrix utilizing a text encoder, multiplying the text matrix and the visual matrix to generate an image-text similarity matrix that assigns a numerical value indicating similarities between each of encoded visual descriptors and each of the encoded images, wherein similarities are indicated by entries of the image-text similarity matrix having numerical values that determine a loss function associated with the image-text similarity matrix, identify a gradient of the loss function with respect to parameters associated with the image encoder and parameters associated with the text encoder, utilizing the gradient, update parameters associated with the image encoder or the text encoder, and outputting final updated parameters associated with either the text encoder or image encoder of the machine learning network.

Abstract: Methods and systems are disclosed for generating training data for a machine learning model for better performance of the model. A source image is selected from an image database, along with a target image. An image segmenter is utilized with the source image to generate a source image segmentation mask having a foreground region and a background region. The same is performed with the target image to generate a target image segmentation mask having a foreground region and a background region. Foregrounds and backgrounds of the source image and target image are determined based on the masks. The target image foreground is removed from the target image, and the source image foreground is inserted into the target image to create an augmented image having the source image foreground and the target image background. The training data for the machine learning model is updated to include this augmented image.

Abstract: Methods and systems of training neural networks with federated learning. Machine learning models are sent from a server to clients, yielding local machine learning models. At each client, the models are trained with locally-stored data, including determining a respective cross entropy loss for each of the plurality of local machine learning models. Weights for each local model are updated, and transferred to the server without transferring locally-stored data. The transferred weights are aggregated at the server to obtain an aggregated server-maintained machine learning model. At the server, a distillation loss based on a foundation model is generated. The aggregated server-maintained machine learning is updated to obtain aggregated respective weights, which are transferred to the clients for updating in the local models.

Abstract: A method of initializing and training a neural network having a plurality of layers includes defining a first function configured to generate a filter based on a plurality of variance values associated with respective pairs of parameters of the plurality of layers, calculating the plurality of variance values based on depths of respective layers of the plurality of layers such that the variance values increase as the depths increase, calculating a covariance matrix using the first function, the covariance matrix having a block structure and each block of the covariance matrix corresponding to a covariance between a respective parameter and other parameters of the plurality of layers, providing, as input, the covariance matrix to the neural network to initialize the neural network for training, and generating, using the neural network, an output based on the covariance matrix.

Abstract: Methods and systems of training neural networks with federated learning. Server-maintained machine learning models are sent from a server to clients, yielding local machine learning models. At each client, the models are trained with local data to determine a respective cross entropy loss and a distillation loss based on foundation models. Respective weights are updated at each client for each of the local machine learning model based on the losses. The updated weights are transferred to the server without transferring the locally-stored data, whereupon they are aggregated and transferred back to the clients. At each client, the local machine learning model is updated with the aggregated updated weights.

Abstract: A system including a machine learning network that includes a controller configured to, utilizing numerical values assigned at an image-text similarity matrix, output at the machine learning network including a text encoder and an image encoder, update parameters of a untrained layer of the machine learning network utilizing sparse logistic regression to generate a sparse logistic regression layer, wherein the image-text similarity matrix is associated with a plurality of input images received at the controller, freeze one or more entries of the sparse logistic regression layer that include zero values, run a plurality of input images at both (1) the image encoder and (2) one or more unfrozen entries at the sparse logistic regression layer, and update, in response to the running of the plurality of input images, parameters of the image encoder and parameters associated with one or more unfrozen entries, and output a tuned machine learning model until a threshold is met.

Abstract: A computer-implemented method that includes receiving a plurality of input images, generating a visual matrix utilizing the plurality of images and an image encoder, wherein the visual matrix includes a list of encoded images, receiving a plurality of text prompts, selecting a text prompt from the plurality of text prompts, send the first one of the text prompts to a language model to generate a candidate list of tokens, selecting tokens, converting the text prompts into updated text prompts via appending the tokens, generating a text matrix utilizing the text prompt and text encoder, and utilizing numerical values assigned at an image-text similarity matrix, determining a score associated with the image-text similarity matrix; and evaluating a criteria and outputting a final token to the updated text prompt in response to identifying a highest score associated with the final token after evaluating each of the plurality of text prompts.

Abstract: A computer-implemented method for training a machine-learning network, wherein the network includes receiving an input data from a sensor, wherein the input data includes data indicative of an image, wherein the sensor includes a video, radar, LiDAR, sound, sonar, ultrasonic, motion, or thermal imaging sensor, generating an adversarial version of the input data utilizing an optimizer, wherein the adversarial version of the input data utilizes a subset of the input data, parameters associated with the optimizer, and one or more perturbation tiles, determining loss function value in response to the adversarial version of the input data and a classification of the adversarial version of the input data, determining a perturbation tile in response the loss function value associated with one or more subsets of the adversarial version of the input data, and output a perturbation that includes at least the perturbation tile.

Abstract: A computational method for training a meta-learned, evolution strategy black box optimization classifier. The method includes receiving one or more training functions and one or more initial meta-learning parameters of the optimization classifier. The method further includes sampling a sampled objective function from the one or more training functions and an initial mean of the sampled function. The method also includes computing a set of T number of means by running the meta learned, evolution strategy classifier on the sampled objective function using the initial mean for T number of steps in t=1, . . . , T. The method also includes computing a loss function from the set of T number of means. The method further includes updating the one or more initial meta-learning parameters of the optimization classifier in response to a characteristic of the loss function.