Abstract: A method for offline training of a transformer model for the ramp-up phase of manufacturing processes, including the steps of utilizing a pre-trained transformer model designed to predict measurements, utilizing categorical embeddings linked to local parameters and numerical embeddings tied to global parameters from initial measurement data, training using an optimizer that minimizes sample data loss through an inner loop algorithm configured to iteratively update local parameters and determine an inner loop loss using query sets, utilizing an outer loop optimization algorithm configured to aggregate these losses across production lines, and during the ramp-up phase, updating final local parameters to minimize losses on sample sets to output a fine-tuned transformer model optimized for real-time application in manufacturing environments.

Abstract: A method of utilizing a machine learning model to perform root cause analysis to determine a fault that includes providing a transformer model that is trained to predict measurements of non-faulty parts, receiving, from the plurality of sensors, a first set of measurement data regarding physical characteristics of a first plurality of manufactured parts and an identification of a plurality of manufacturing stations, obtaining one or more categorical embeddings and numerical embeddings, concatenating one or more positional embeddings with the categorical numerical embedding associated with the first set of measurement data to generate a concatenation, outputting one or more embedding vectors in response to passing the concatenation at a self-attention module, and outputting a prediction utilizing a linear layer of the pre-trained transformer model and the one or more embedding vectors as input to the linear layer.