Abstract: Methods, systems, and computer program products for accelerating inference of transformer-based models are provided herein. A computer-implemented method includes obtaining a machine learning model comprising a plurality of transformer blocks, a task, and a natural language dataset; generating a compressed version of the machine learning model based on the task and the natural language dataset, wherein the generating comprises: obtaining at least one set of tokens, wherein each token in the set corresponds to one of the items in the natural language dataset, identifying and removing one or more redundant output activations of different ones of the plurality of transformer blocks for the at least one set of tokens, and adding one or more input activations corresponding to the one or more removed output activations into the machine learning model at subsequent ones of the plurality of the transformer blocks; and outputting the compressed version of the machine learning model to at least one user.

Abstract: Techniques for utilizing model and hyperparameter optimization for multi-objective machine learning are disclosed. In one example, a method comprises the following steps. One of a plurality of hyperparameter optimization operations and a plurality of model parameter optimization operations are performed to generate a first solution set. The other of the plurality of hyperparameter optimization operations and the plurality of model parameter optimization operations are performed to generate a second solution set. At least a portion of the first solution set and at least a portion of the second solution set are combined to generate a third solution set.

Abstract: Methods, systems, and computer program products for accelerating inference of transformer-based models are provided herein. A computer-implemented method includes obtaining a machine learning model comprising a plurality of transformer blocks, a task, and a natural language dataset; generating a compressed version of the machine learning model based on the task and the natural language dataset, wherein the generating comprises: obtaining at least one set of tokens, wherein each token in the set corresponds to one of the items in the natural language dataset, identifying and removing one or more redundant output activations of different ones of the plurality of transformer blocks for the at least one set of tokens, and adding one or more input activations corresponding to the one or more removed output activations into the machine learning model at subsequent ones of the plurality of the transformer blocks; and outputting the compressed version of the machine learning model to at least one user.

Abstract: Methods, systems, and computer program products for accelerating inference of neural network models via dynamic early exits are provided herein. A computer-implemented method includes determining a plurality of candidate exit points of a neural network model; obtaining a plurality of outputs of the neural network model for data samples in a target dataset, wherein the plurality of outputs comprises early outputs of the neural network model from the plurality of candidate exit points and regular outputs of the neural network model; and a set of one or more exit points from the plurality of candidate exits points that are dependent on the target dataset based at least in part on the plurality of outputs.

Abstract: A system, computer program product, and method are presented for performing multi-objective automated machine learning, and, more specifically, to identifying a plurality of machine learning pipelines as Pareto-optimal solutions to optimize a plurality of objectives. The method includes receiving input data directed toward one or more subjects of interest and determining a plurality of objectives to be optimized. The method also includes ingesting at least a portion of the input data through one or more machine learning (ML) models. The method further includes aggregating the plurality of objectives into one or more aggregated single objectives. The method also includes determining a plurality of Pareto-optimal solutions, thereby defining a plurality of ML pipelines that optimize the one or more aggregated single objectives. The method further includes selecting one ML pipeline from the plurality of ML pipelines.

Abstract: One or more computer processors decompose a weight matrix associated with a neural network utilizing a permutation dependent decomposition. The one or more computer processors regenerate a recovered matrix utilizing the decomposed weight matrix. The one or more computer processors reduce an error between the decomposed weight matrix and regenerated recovered matrix.