Abstract: A method includes obtaining a parameter matrix associated with a linear layer of a first machine learning model and containing parameter values for parameters of the linear layer. The method also includes determining importance values corresponding to the parameter values. The method further includes generating factorized matrices such that a product of the importance values and factorized matrices contains approximated parameter values for the parameters of the linear layer. In addition, the method includes generating a second machine learning model representing a compressed version of the first machine learning model. The second machine learning model has first and second linear layers containing parameter values based on the importance values and the factorized matrices. The factorized matrices are generated based on weighted errors between the parameter values for the parameters of the linear layer and the approximated parameter values.

Abstract: In one embodiment, a method includes accessing a batch B of a plurality of images, wherein each image in the batch is part of a training set of images used to train a vision transformer comprising a plurality of attention heads. The method further includes determining, for each attention head A, a similarity between (1) the output of the attention head evaluated using each image in the batch and the (2) output of each attention head evaluated using each image in the batch. The method further includes determining, based on the determined similarities, an importance score for each attention head; and pruning, based on the importance scores, one or more attention heads from the vision transformer.

Abstract: A method of training a neural network model includes generating a positive image based on an original image, generating a positive text corresponding to the positive image based on an original text corresponding to the original image, the positive text referring to an object in the positive image, constructing a positive image-text pair for the object based on the positive image and the positive text, constructing a negative image-text pair for the object based on the original image and a negative text, the negative text not referring to the object, training the neural network model based on the positive image-text pair and the negative image-text pair to output features representing an input image-text pair, and identifying the object in the original image based on the features representing the input image-text pair.

Abstract: A method of training a student model includes providing an input to a teacher model that is larger than the student model, where a layer of the teacher model outputs a first output vector, providing the input to the student model, where a layer of the student model outputs a second output vector, determining an importance value associated with each dimension of the first output vector based on gradients from the teacher model and updating at least one parameter of the student model to minimize a difference between the second output vector and the first output vector based on the importance values.

Abstract: A method includes identifying multiple tokens contained in an input utterance. The method also includes generating slot labels for at least some of the tokens contained in the input utterance using a trained machine learning model. The method further includes determining at least one action to be performed in response to the input utterance based on at least one of the slot labels. The trained machine learning model is trained to use attention distributions generated such that (i) the attention distributions associated with tokens having dissimilar slot labels are forced to be different and (ii) the attention distribution associated with each token is forced to not focus primarily on that token itself.

Abstract: A method includes receiving, from an electronic device, information defining a user utterance associated with a skill to be performed, where the skill is not recognized by a natural language understanding (NLU) engine. The method also includes receiving, from the electronic device, information defining one or more actions for performing the skill. The method further includes identifying, using at least one processor, one or more known skills having one or more slots that map to at least one word or phrase in the user utterance. The method also includes creating, using the at least one processor, a plurality of additional utterances based on the one or more mapped slots. In addition, the method includes training, using the at least one processor, the NLU engine using the plurality of additional utterances.

Abstract: An apparatus for performing image processing, may include at least one processor configured to: input an image to a vision transformer comprising a plurality of encoders that correspond to at least one fixed encoder and a plurality of adaptive encoders; process the image via the at least one fixed encoder to obtain image representations; determine one or more layers of the plurality of adaptive encoders to drop, by inputting the image representations to a policy network configured to determine layer dropout actions for the plurality of adaptive encoders; and obtain a class of the input image using remaining layers of the plurality of adaptive encoders other than the dropped one or more layers.

Abstract: A method includes obtaining, using at least one processor of an electronic device, a base natural language understanding (NLU) model that includes a word embedding layer, where the word embedding layer is associated with at least one training utterance. The method also includes calculating, using the at least one processor, a regularization loss value for use in a determination of an intent detection loss, where the regularization loss value reveals an effect of word embeddings on intent determination of the training utterance. The method further includes retraining, using the at least one processor, the word embedding layer of the base NLU model using the intent detection loss to obtain a retrained NLU model.

Abstract: A method includes obtaining, using at least one processor of an electronic device, a base model trained to perform natural language understanding. The method also includes generating, using the at least one processor, a first model expansion based on knowledge from the base model. The method further includes training, using the at least one processor, the first model expansion based on first utterances without modifying parameters of the base model. The method also includes receiving, using the at least one processor, an additional utterance from a user. In addition, the method includes determining, using the at least one processor, a meaning of the additional utterance using the base model and the first model expansion.

Abstract: Described is a system for providing improved exploration for an interactive recommendation system by leveraging intuitive user feedback. The recommendation system may provide images of recommend items and receive user feedback preferences in the form of a natural language expression. Traditional techniques for interactive recommendation systems typically rely on restricted forms of user feedback such as binary relevance responses, or feedback based on a fixed set of relative attributes. In contrast, the recommendation system described herein introduces a new approach to interactive image recommendation (or image search) that enables users to provide feedback via natural language, allowing for a more natural and effective interaction. The recommendation system may be based on formulating the task of natural-language-based interactive image recommendation as a reinforcement learning problem, and reward the recommendation system for improving the rank of the target image during each iterative interaction.

Abstract: An apparatus for detecting unsupported utterances in natural language understanding, includes a memory storing instructions, and at least one processor configured to execute the instructions to classify a feature that is extracted from an input utterance of a user, as one of in-domain and out-of-domain (OOD) for a response to the input utterance, obtain an OOD score of the extracted feature, and identify whether the feature is classified as OOD. The at least one processor is further configured to executed the instructions to, based on the feature being identified to be classified as in-domain, identify whether the obtained OOD score is greater than a predefined threshold, and based on the OOD score being identified to be greater than the predefined threshold, re-classify the feature as OOD.

Abstract: A method includes determining, by an electronic device, a skill from a first natural language (NL) input. Upon successful determination of the skill, the first NL input is transmitted to a custom skill parser for determination of a skill intent. The custom skill parser is trained based on data including at least a custom training data set. Upon unsuccessful determination of the skill, the first NL input is transmitted to a generic parser for determination of a general intent of the first NL input.

Abstract: A method of a server device is provided. The method of a server device includes retrieving a prediction input and a prediction setting, replacing at least one non-linear activation channel in a neural network with at least one replacement channel based on the received prediction setting, generating a prediction based on the received prediction input based on the neural network with the at least one replacement channel, and outputting the generated prediction.

Abstract: A machine learning method using a trained machine learning model residing on an electronic device includes receiving an inference request by the electronic device. The method also includes determining, using the trained machine learning model, an inference result for the inference request using a selected inference path in the trained machine learning model. The selected inference path is selected based on a highest probability for each layer of the trained machine learning model. A size of the trained machine learning model is reduced corresponding to constraints imposed by the electronic device. The method further includes executing an action in response to the inference result.

Abstract: A method includes determining, by an electronic device, a skill from a first natural language (NL) input. Upon successful determination of the skill, the first NL input is transmitted to a custom skill parser for determination of a skill intent. The custom skill parser is trained based on data including at least a custom training data set. Upon unsuccessful determination of the skill, the first NL input is transmitted to a generic parser for determination of a general intent of the first NL input.

Abstract: A method includes applying, by at least one processor, a natural language understanding (NLU) model to an input utterance in order to obtain initial slot probability distributions. The method also includes performing, by the at least one processor, a confidence calibration by applying a calibration probability distribution to the initial slot probability distributions in order to generate calibrated slot probability distributions. The calibration probability distribution has a higher number of dimensions than the initial slot probability distributions. The method further includes identifying, by the at least one processor, uncertainties associated with words in the input utterance based on the calibrated slot probability distributions. In addition, the method includes identifying, by the at least one processor, a new concept contained in the input utterance that is not recognized by the NLU model based on the identified uncertainties.

Abstract: A continual learning method includes obtaining an input data including a trained model, continual learning (CL) Information, and training data by an electronic device. The method also includes re-training, using the electronic device, the model for a task based on the training data. The method also includes updating, using the electronic device, the CL Information based on the model and the training data. The method further includes selecting a first set of exemplars from the training data based on data associated with the CL Information. The CL Information includes a first group of variables associated with the model and a second group of variables associated with the model that changes to the first group of variables have stronger impact to the model's performance of the task than changes to the second group of variables.

Abstract: A method includes retrieving, at an electronic device, a first natural language (NL) input. An intent of the first NL input is undetermined by both a generic parser and a personal parser. A paraphrase of the first NL input is retrieved at the electronic device. An intent of the paraphrase of the first NL input is determined using at least one of: the generic parser, the personal parser, or a combination thereof. A new personal intent for the first NL input is generated based on the determined intent. The personal parser is trained using existing personal intents and the new personal intent.

Abstract: A method includes obtaining, using at least one processor of an electronic device, an image-query understanding model. The method also includes obtaining, using the at least one processor, an image and a user query associated with the image, where the image includes a target image area and the user query includes a target phrase. The method further includes retraining, using the at least one processor, the image-query understanding model using a correlation between the target image area and the target phrase to obtain a retrained image-query understanding model.

Abstract: An electronic device for training a machine learning model includes at least one memory and at least one processor coupled to the at least one memory. The at least one processor is configured to train a classification layer of the model. To train the classification layer, the at least one processor is configured to receive, by the classification layer, one or more language contexts from an utterance encoder layer and to classify, by the classification layer, at least one portion of an utterance into an information type among a plurality of information types. The at least one processor may be further configured to jointly train a slot filling layer and an intent detection layer of the model.