Abstract: A method includes receiving an input for a large language model (LLM) from a user. The method also includes generating one or more token embeddings based on the input. The method further includes generating one or more prompt embeddings based on the input using a contextual prompt generator (CPG), the one or more prompt embeddings representing new or updated information that is not contained in existing knowledge of the LLM. The method also includes providing the one or more token embeddings and the one or more prompt embeddings to the LLM. In addition, the method includes outputting a prediction based on the one or more token embeddings and the one or more prompt embeddings using the LLM, wherein the prediction reflects the new or updated information represented by the one or more prompt embeddings.

Abstract: An apparatus for detecting unsupported utterances in natural language understanding, includes a memory storing instructions, and at least one processor configured to execute the instructions to classify a feature that is extracted from an input utterance of a user, as one of in-domain and out-of-domain (OOD) for a response to the input utterance, obtain an OOD score of the extracted feature, and identify whether the feature is classified as OOD. The at least one processor is further configured to executed the instructions to, based on the feature being identified to be classified as in-domain, identify whether the obtained OOD score is greater than a predefined threshold, and based on the OOD score being identified to be greater than the predefined threshold, re-classify the feature as OOD.

Abstract: A method includes obtaining, using a first electronic device, a weight matrix associated with a trained transformer model. The method also includes factorizing the weight matrix into a dictionary weight matrix and an intermediate matrix. The method further includes pruning the intermediate matrix to generate a sparse intermediate matrix. The method also includes fine-tuning the sparse intermediate matrix based on a training dataset to generate a fine-tuned sparse intermediate matrix. The method further includes determining an index matrix and a coefficient matrix based on the fine-tuned sparse intermediate matrix. In addition, the method includes deploying the dictionary weight matrix, the index matrix, and the coefficient matrix to a second electronic device without deploying the weight matrix to the second electronic device. A number of parameters in the dictionary weight matrix, the index matrix, and the coefficient matrix is smaller than a number of parameters in the weight matrix.

Abstract: A method includes obtaining a parameter matrix associated with a linear layer of a first machine learning model and containing parameter values for parameters of the linear layer. The method also includes determining importance values corresponding to the parameter values. The method further includes generating factorized matrices such that a product of the importance values and factorized matrices contains approximated parameter values for the parameters of the linear layer. In addition, the method includes generating a second machine learning model representing a compressed version of the first machine learning model. The second machine learning model has first and second linear layers containing parameter values based on the importance values and the factorized matrices. The factorized matrices are generated based on weighted errors between the parameter values for the parameters of the linear layer and the approximated parameter values.

Abstract: A method includes providing, using at least one processing device of an electronic device, input data to a machine learning model. The method also includes extracting, using the at least one processing device, features of the input data. The method further includes performing, using the at least one processing device, a geometric transformation of the features, where the geometric transformation is based on first and second parametric instance-dependent scalar functions. In addition, the method includes producing, using the at least one processing device, a predictive probability distribution based on the transformed features.

Abstract: A method includes receiving one or more training corpora for training a machine learning model having a plurality of encoder blocks, where each encoder block includes an attention layer and a feedforward network. The method also includes using the one or more training corpora to train an attention dictionary shared across the plurality of encoder blocks. Training the attention dictionary may include training attention parameters of the attention layer in each of the plurality of encoder blocks, and the attention parameters for a given encoder block among the plurality of encoder blocks may be a weighted combination of columns from the attention dictionary shared across the plurality of encoder blocks.

Abstract: A method of training a student model includes providing an input to a teacher model that is larger than the student model, where a layer of the teacher model outputs a first output vector, providing the input to the student model, where a layer of the student model outputs a second output vector, determining an importance value associated with each dimension of the first output vector based on gradients from the teacher model and updating at least one parameter of the student model to minimize a difference between the second output vector and the first output vector based on the importance values.

Abstract: An apparatus for detecting unsupported utterances in natural language understanding, includes a memory storing instructions, and at least one processor configured to execute the instructions to classify a feature that is extracted from an input utterance of a user, as one of in-domain and out-of-domain (OOD) for a response to the input utterance, obtain an OOD score of the extracted feature, and identify whether the feature is classified as OOD. The at least one processor is further configured to executed the instructions to, based on the feature being identified to be classified as in-domain, identify whether the obtained OOD score is greater than a predefined threshold, and based on the OOD score being identified to be greater than the predefined threshold, re-classify the feature as OOD.