Abstract: A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.

Abstract: A device, system, and method is provided for training or prediction using a cluster-connected neural network. The cluster-connected neural network may be divided into a plurality of clusters of artificial neurons connected by weights or convolutional channels connected by convolutional filters. Within each cluster is a locally dense sub-network of intra-cluster weights or filters with a majority of pairs of neurons or channels connected by intra-cluster weights or filters that are co-activated together as an activation block during training or prediction. Outside each cluster is a globally sparse network of inter-cluster weights or filters with a minority of pairs of neurons or channels separated by a cluster border across different clusters connected by inter-cluster weights or filters. Training or predicting is performed using the cluster-connected neural network.

Abstract: Methods of inspecting a product made by additive manufacturing (AM) of multiple layers, computer program products and inspection modules for AM systems are provided. An augmented file is derived from a design file including layer data used to produce the product by AM. For each design layer, the augmented file includes the layer data for the design layer and weighted layer data for design layers beneath the design layer. A machine learning (ML) algorithm (trained on previous images and augmented files) is applied with respect to the derived augmented file onto received optical inspection images of the product during the AM process to detect production errors.

Abstract: A device, system, and method for training or prediction of a neural network. A current value may be stored for each of a plurality of synapses or filters in the neural network. A historical metric of activity may be independently determined for each individual or group of the synapses or filters during one or more past iterations. A plurality of partial activations of the neural network may be iteratively executed. Each partial-activation iteration may activate a subset of the plurality of synapses or filters in the neural network. Each individual or group of synapses or filters may be activated in a portion of a total number of iterations proportional to the historical metric of activity independently determined for that individual or group of synapses or filters. Training or prediction of the neural network may be performed based on the plurality of partial activations of the neural network.

Abstract: Abnormal behavior detection for industrial machines component(s) using large language models based on sequence(s) of log file messages recorded by logger(s) at initial time(s). The log file message sequence may be transformed into an anomaly severity sequence comprising log tokens encoding, times, and/or associated anomaly severity levels. The anomaly severity sequence may be input into the large language model. The large language model may output an anomaly severity histogram predicting M probabilities that the logger(s) will record log file messages encoded by log keys associated with M respective anomaly severity levels at the subsequent time(s). Abnormal behavior may be predicted based on the M probability patterns of the anomaly severity histogram. in the one or more components of the industrial machine logged by the one or more loggers at the one or more subsequent times. A control command may trigger automatically altering the component(s) operation to prevent the abnormal behavior.

Abstract: A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).

Abstract: A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.

Abstract: A multi-sequence transformer predict N tokens, in parallel, for log files in industrial machines. Token patterns derived from N log file token sequences logged by N respective loggers may be input into N respective intra sequence multi-head self-attention layers identifying patterns among tokens within the same log file token sequence generated by the same logger. Token patterns derived from a combination of the N log file token sequences may be input into a same inter sequence multi-head self-attention layer identifying patterns among tokens across multiple different sequences generated by multiple different loggers. N softmax layers may be generated of N distinct probability distributions that each candidate tokens is a next token in each of the N respective log file token sequences. A plurality of N next tokens may be predicted, in parallel, to co-occur in the plurality of N respective sequences of log file tokens.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: A device, system, and method is provided for storing a sparse neural network. A plurality of weights of the sparse neural network may be obtained. Each weight may represent a unique connection between a pair of a plurality of artificial neurons in different layers of a plurality of neuron layers. A minority of pairs of neurons in adjacent neuron layers are connected in the sparse neural network. Each of the plurality of weights of the sparse neural network may be stored with an association to a unique index. The unique index may uniquely identify a pair of artificial neurons that have a connection represented by the weight. Only non-zero weights may be stored that represent connections between pairs of neurons (and zero weights may not be stored that represent no connections between pairs of neurons).

Abstract: An efficient technique of machine learning is provided for training a plurality of convolutional neural networks (CNNs) with increased speed and accuracy using a genetic evolutionary model. A plurality of artificial chromosomes may be stored representing weights of artificial neuron connections of the plurality of respective CNNs. A plurality of pairs of the chromosomes may be recombined to generate, for each pair, a new chromosome (with a different set of weights than in either chromosome of the pair) by selecting entire filters as inseparable groups of a plurality of weights from each of the pair of chromosomes (e.g., “filter-by-filter” recombination). A plurality of weights of each of the new or original plurality of chromosomes may be mutated by propagating recursive error corrections incrementally throughout the CNN. A small random sampling of weights may optionally be further mutated to zero, random values, or a sum of current and random values.

Abstract: A device, system, method and non-transitory computer-readable storage medium for filtering a dataset for training a neural network. Initial instances of recorded input samples of source recording device(s) may be encoded, according to an input map, from the training dataset to respective nodes in the neural network's input layer. The input layer of the neural network may be sparsified by eliminating its nodes during a training phase. The training dataset may be filtered to exclude subsequent instances of recorded input samples from the source recording devices encoded in the eliminated nodes. Subsequent instances of the recorded input samples of the filtered training dataset may be encoded to remaining nodes not eliminated (and not to eliminated nodes) in the input layer to train the neural network in a subsequent training phase or generate a prediction output of the neural network in a prediction phase.

Abstract: An efficient technique of machine learning is provided for training a plurality of convolutional neural networks (CNNs) with increased speed and accuracy using a genetic evolutionary model. A plurality of artificial chromosomes may be stored representing weights of artificial neuron connections of the plurality of respective CNNs. A plurality of pairs of the chromosomes may be recombined to generate, for each pair, a new chromosome (with a different set of weights than in either chromosome of the pair) by selecting entire filters as inseparable groups of a plurality of weights from each of the pair of chromosomes (e.g., “filter-by-filter” recombination). A plurality of weights of each of the new or original plurality of chromosomes may be mutated by propagating recursive error corrections incrementally throughout the CNN. A small random sampling of weights may optionally be further mutated to zero, random values, or a sum of current and random values.

Abstract: A device, system, and method for approximating a neural network comprising N synapses or filters. The neural network may be partially-activated by iteratively executing a plurality of M partial pathways of the neural network to generate M partial outputs, wherein the M partial pathways respectively comprise M different continuous sequences of synapses or filters linking an input layer to an output layer. The M partial pathways may cumulatively span only a subset of the N synapses or filters such that a significant number of the remaining the N synapses or filters are not computed. The M partial outputs of the M partial pathways may be aggregated to generate an aggregated output approximating an output generated by fully-activating the neural network by executing a single instance of all N synapses or filters of the neural network. Training or prediction of the neural network may be performed based on the aggregated output.

Abstract: Methods, computer program products and nozzle monitoring modules are provided, to monitor nozzles in 2D and/or 3D inkjet printing systems. Nozzle monitoring comprises registering an image of a printed product with respect to a corresponding raster file and evaluating nozzle performance and managing nozzles by applying a neural network (NN) trained on a plurality of registered images of the printed product and corresponding raster files. The disclosed NN approach may apply a deep learning model, and has been shown to improve performance over manual analysis of images of printed products (which is the current method of monitoring nozzles). Disclosed methods and modules may be implemented in 2D inkjet printing system and/or in 3D additive manufacturing (AM) inkjet printing system, monitoring nozzles that deposit layers of the 3D product.

Abstract: Methods of inspecting a product made by additive manufacturing (AM) of multiple layers, computer program products and inspection modules for AM systems are provided. An augmented file is derived from a design file including layer data used to produce the product by AM. For each design layer, the augmented file includes the layer data for the design layer and weighted layer data for design layers beneath the design layer. A machine learning (ML) algorithm (trained on previous images and augmented files) is applied with respect to the derived augmented file onto received optical inspection images of the product during the AM process to detect production errors.

Abstract: A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.

Abstract: A device, system, and method is provided for training a new neural network to mimic a target neural network without access to the target neural network or its original training dataset. The target neural network and the new neural network may be probed with input data to generate corresponding target and new output data. Input data may be detected that generate a maximum or above threshold difference between the corresponding target and new output data. A divergent probe training dataset may be generated comprising the input data that generate the maximum or above threshold difference and the corresponding target output data. The new neural network may be trained using the divergent probe training dataset to generate the target output data. The new neural network may be iteratively trained using an updated divergent probe training dataset dynamically adjusted as the new neural network changes during training.

Abstract: A device, system, and method is provided for training a new neural network to mimic a target neural network without access to the target neural network or its original training dataset. The target neural network and the new neural network may be probed with input data to generate corresponding target and new output data. Input data may be detected that generate a maximum or above threshold difference between the corresponding target and new output data. A divergent probe training dataset may be generated comprising the input data that generate the maximum or above threshold difference and the corresponding target output data. The new neural network may be trained using the divergent probe training dataset to generate the target output data. The new neural network may be iteratively trained using an updated divergent probe training dataset dynamically adjusted as the new neural network changes during training.

Abstract: A device, system, and method is provided to mimic a pre-trained target model without access to the pre-trained target model or its original training dataset. A set of random or semi-random input data may be sent to randomly probe the pre-trained target model at a remote device. A set of corresponding output data may be received from the remote device that is generated by applying the pre-trained target model to the set of random or semi-random input data. A random probe training dataset may be generated comprising the set of random or semi-random input data and corresponding output data generated by randomly probing the pre-trained target model. A new model may be trained with the random probe training dataset so that the new model generates substantially the same corresponding output data in response to said input data to mimic the pre-trained target model.