Abstract: A method for offline training of a transformer model for the ramp-up phase of manufacturing processes, including the steps of utilizing a pre-trained transformer model designed to predict measurements, utilizing categorical embeddings linked to local parameters and numerical embeddings tied to global parameters from initial measurement data, training using an optimizer that minimizes sample data loss through an inner loop algorithm configured to iteratively update local parameters and determine an inner loop loss using query sets, utilizing an outer loop optimization algorithm configured to aggregate these losses across production lines, and during the ramp-up phase, updating final local parameters to minimize losses on sample sets to output a fine-tuned transformer model optimized for real-time application in manufacturing environments.

Abstract: A method of utilizing a machine learning model to perform root cause analysis to determine a fault that includes providing a transformer model that is trained to predict measurements of non-faulty parts, receiving, from the plurality of sensors, a first set of measurement data regarding physical characteristics of a first plurality of manufactured parts and an identification of a plurality of manufacturing stations, obtaining one or more categorical embeddings and numerical embeddings, concatenating one or more positional embeddings with the categorical numerical embedding associated with the first set of measurement data to generate a concatenation, outputting one or more embedding vectors in response to passing the concatenation at a self-attention module, and outputting a prediction utilizing a linear layer of the pre-trained transformer model and the one or more embedding vectors as input to the linear layer.

Abstract: A method of generating text-driven prompts and class prediction probabilities using a vision-language model (VLM) includes receiving candidate class names associated with a plurality of candidate classes for images, generating class text tokens based on a text description of the candidate class names, and generating a plurality of context prompt vectors using a prompt generator. The context prompt vectors define context information associated with an image classification task to be performed by the VLM. The method further includes generating prompts for each of the plurality of candidate classes by appending respective class text tokens to the context prompt vectors for each of the plurality of candidate classes, and, using the VLM, generating and outputting a class prediction probability for a sample image based on the plurality of context prompt vectors.

Abstract: A machine learning (ML) system includes a vision language model (VLM) and a diffusion model. The VLM is finetuned prior to training the diffusion model with data pairs. A data pair includes image data displaying an anomaly and text data describing the image data. The finetuned VLM includes an image encoder that generates image embeddings using the image data and a text encoder that generates text embeddings using the text data. Semantic subcode is generated using the image embeddings and the text embeddings. The diffusion model generates stochastic subcode using the image data. The diffusion model generates a reconstructed image using the stochastic and semantic subcodes. A loss is optimized based on an expected value of a difference between predicted noise of a noisy instance of the image data at a particular time and actual noise of that noisy instance. Parameters of the diffusion model are updated using the loss.

Abstract: A computer-implemented method for enabling control or monitoring of a computer-controlled entity operating in an environment by predicting a future state of the computer-controlled entity and/or its environment using sensor data which is indicative of a current state of the computer-controlled entity and/or its environment. The method includes using a first neural network for approximating a drift component of a stochastic differential equation and a second neural network for approximating a diffusion component of the stochastic differential equation, and discretizing the stochastic differential equation into time steps, and obtaining time-evolving mean and covariance functions based on the discretization and determining a probability distribution of a second state of the computer-controlled entity and/or its environment therefrom.

Abstract: A system includes a controller configured to receive, at a trained transformer model, one or more run-time logs indicating information associated with interaction between a unique device and a server, output a user-score associated with the unique device and the one or more run-time logs in response to determining a negative log-likelihood of the one or more run-time logs in a next-log prediction probability distribution modeled by the trained transformer model, output a server-score utilizing at least normal-cluster centers associated with the trained transformer model and the one or more run-time logs, and in response to a sum of the user-score and server-score exceeding a threshold, outputting an indication of a cyber-threat associated with the one or more run-time logs.

Abstract: A computer-implemented method for training neural networks with federated learning that includes sending portions of server-maintained machine learning models to clients, using local models without cluster labels at clients, estimating cluster labels at the server using k-means on latent features from clients, training local models with client data using a global-shared encoder parameter and a cluster-shared prediction head, updating the global encoder parameter at the server by aggregating cross-entropy loss updates from clients, updating cluster-shared prediction heads at the server by aggregating updates from clients within each cluster, sending updated global and cluster-shared model parameters to clients, and outputting a final parameter, including a global-shared encoder and cluster-shared model parameter, after meeting a threshold.

Abstract: A method of training neural networks with federated learning that includes sending portions of server-maintained machine learning models to clients, yielding local models in sync with the server; at each client, training a local model with local data, receiving a model parameter including a global-shared encoder and cluster-shared prediction head from the server, utilizing the cluster-shared prediction head for server aggregating models from clients in the respective cluster; at each client, syncing with the server on its locally updated model; at the server, updating the global-shared encoder by aggregating updates of the cross-entropy loss from clients; at the server, updating cluster-shared prediction heads by aggregating updates from clients in each cluster; at the server, sending updated global and cluster-shared model parameters to clients; repeating steps until a threshold is met; outputting a final parameter including a final global-shared encoder and cluster-shared model parameter for each cluster.

Abstract: A computer-implemented system and method relate to anomaly detection. Latent code of a source image is obtained. The latent code is designated as a target image. Source embedding data is generated form the source image. Text data, which is of a different domain than that of the source image, is obtained. Text embedding data is generated from the text data. Additional embedding data is generated using the source embedding data and the text embedding data. The additional embedding data provides guidance for modifying the source image. A modified image is generated via an iterative process that includes at least one iteration, where each iteration includes at least (i) encoding the target image to generate target embedding data, (ii) generating updated embedding data by combining the target embedding data and the additional embedding data, (iii) decoding the updated embedding data to generate a new image, and (iv) assigning the new image as the target image and the modified image.

Abstract: A computer-implemented system and method relate to anomaly detection. A first source image is obtained from a first image set and a second source image is obtained from a second image set of an in-distribution image dataset. A diffusion model generates a modified image using the first source image and the second source image. A non-anomalous label is automatically generated for the first source image. The non-anomalous label is also generated for the second source image. An anomalous label is generated for the modified image. A training dataset is created. The training dataset includes at least the first source image with the non-anomalous label, the second source image with the non-anomalous label, and the modified image with the anomalous label. A machine learning model is trained or fine-tuned using the training dataset. The machine learning model being configured to perform a task of anomaly detection.

Abstract: A anomalous region detection method includes receiving time series data being grouped in patches, encoding, via parameters of an encoder, the data to obtain local latent representations, determining a representation loss from the local latent representations, transforming the local latent representations associated with each patch, via at least two local neural transformations, to a series of diverse transformed vector representations, determining a dynamic deterministic contrastive loss (DDCL) from the series of diverse transformed vector representations, combining the representation loss and the DDCL to obtain updated parameters, updating the parameters of the encoder with the updated parameters, scoring each of the series of the diverse transformed vector representations, via the DDCL, to obtain a diverse semantic requirement score associated with each patch, smoothing the diverse semantic requirement score to obtain a loss region, masking data associated with the loss region to obtain verified data, and o

Abstract: A simple and highly effective zero-shot anomaly detection approach is disclosed. The approach is compatible with a variety of established anomaly detection methods. The approach relies on training an anomaly detector, such as a neural network, on a meta-set in combination with batch normalization.

Abstract: Optical transmitting apparatuses are disclosed. In in an implementation, an apparatus comprises an array light source that includes M*N light sources. An included angle between any column of light sources in the N columns of light sources and any row of light sources in the M rows of light sources is predetermined. The array light source is located on a first side of a collimating lens, a plane on which the array light source is located is perpendicular to an optical axis of the collimating lens, and a distance between the plane on which the array light source is located and a center point of the collimating lens is a focal length of the collimating lens. An rotatable scanning mirror is located on a second side of the collimating lens, and a center point of a reflective surface of the scanning mirror is on the optical axis of the collimating lens.

Abstract: A system includes a controller configured to receive, at a trained transformer model, one or more run-time logs indicating information associated with interaction between a unique device and a server, output a user-score associated with the unique device and the one or more run-time logs in response to determining a negative log-likelihood of the one or more run-time logs in a next-log prediction probability distribution modeled by the trained transformer model, output a server-score utilizing at least normal-cluster centers associated with the trained transformer model and the one or more run-time logs, and in response to a sum of the user-score and server-score exceeding a threshold, outputting an indication of a cyber-threat associated with the one or more run-time logs.

Abstract: A systems and methods for implementing attention-based neural networks, attention modules, regularization techniques, and unique data encoding such as for sequential tabular data and/or manufacturing data is provided. The attention-based neural networks may include a high dropout and unique softmax regularization. The encoding may attend to missing or undefined data as well as numerous data types common to manufacturing data.

Abstract: A systems and methods for implementing attention-based neural networks, attention modules, regularization techniques, and unique data encoding such as for sequential tabular data and/or manufacturing data is provided. The attention-based neural networks may include a high dropout and unique softmax regularization. The encoding may attend to missing or undefined data as well as numerous data types common to manufacturing data.

Abstract: A systems and methods for implementing attention-based neural networks, attention modules, regularization techniques, and unique data encoding such as for sequential tabular data and/or manufacturing data is provided. The attention-based neural networks may include a high dropout and unique softmax regularization. The encoding may attend to missing or undefined data as well as numerous data types common to manufacturing data.

Abstract: A systems and methods for implementing attention-based neural networks, attention modules, regularization techniques, and unique data encoding such as for sequential tabular data and/or manufacturing data is provided. The attention-based neural networks may include a high dropout and unique softmax regularization. The encoding may attend to missing or undefined data as well as numerous data types common to manufacturing data.

Abstract: An optical zoom module (1000) comprising: a module housing (100); a plurality of lens assemblies (10, 20, 30), which are arranged coaxially along an axis; a straight guiding rod (200), which is parallel to the axis and disposed on a first side (A) of the module housing (100); a plurality of carriers, wherein one lens assembly (10, 20, 30) is mounted in each carrier, at least two of the plurality of carriers are movable carriers (40, 50), and the straight guiding rod (200) passes through the at least two movable carriers (40, 50), such that the at least two movable carriers (40, 50) can move along the straight guide rod (200), respectively; and a second rail located on a second side (B) of the module housing (100), the second side (B) is opposite to the first side (A), the second rail is parallel to the straight guiding rod (200), and an upper surface of the second rail and the lower surface of the movable carriers (40, 50) are supported by the balls (112), wherein further provided is a corresponding portable

Abstract: A computer-implemented method includes converting tabular data to a text representation, generating metadata associated with the text representation of the tabular data, outputting one or more natural language data descriptions indicative of the tabular data in response to utilizing a large language model (LLM) and zero-shot prompting of the metadata and text representation of the tabular data, outputting one or more summaries utilizing the LLM and appending a prompt on the one or more natural language data descriptions, selecting a single summary of the one or more summaries in response to the single summary having a smallest validation rate, receiving a query associated with the tabular data, outputting one or more predictions associated with the query, and in response to meeting a convergence threshold with the one or more predictions generated from the one or more iterations, output a final prediction associated with the query.