Abstract: Disclosed is an electronic device including processor and memory operatively connected to the processor and storing language model. The electronic device may enter data into the language model, generate an embedding vector in the input embedding layer, add position information to the embedding vector in the positional encoding layer, branch the embedding vector based on domain information, normalize the branched embedding vectors, enter the normalized embedding vectors into the multi-head attention layer, enter output data of the multi-head attention layer into the first layer, normalize pieces of output data of the first layer, enter the normalized pieces of output data of the first layer into the feed-forward layer, enter output data of the feed-forward layer into the second layer and normalize pieces of output data of the second layer, and enter the normalized pieces of output data of the second layer into the linearization layer and the softmax layer to obtain result data.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires a recognition result of a teacher model and a recognition result of a student model for an input sequence and trains the student model such that the recognition result of the teacher model and the recognition result of the student model are not distinguished from each other.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires a recognition result of a teacher model and a recognition result of a student model for an input sequence and trains the student model such that the recognition result of the teacher model and the recognition result of the student model are not distinguished from each other.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires a recognition result of a teacher model and a recognition result of a student model for an input sequence and trains the student model such that the recognition result of the teacher model and the recognition result of the student model are not distinguished from each other.

Abstract: Disclosed is an electronic device including processor and memory operatively connected to the processor and storing language model. The electronic device may enter data into the language model, generate an embedding vector in the input embedding layer, add position information to the embedding vector in the positional encoding layer, branch the embedding vector based on domain information, normalize the branched embedding vectors, enter the normalized embedding vectors into the multi-head attention layer, enter output data of the multi-head attention layer into the first layer, normalize pieces of output data of the first layer, enter the normalized pieces of output data of the first layer into the feed-forward layer, enter output data of the feed-forward layer into the second layer and normalize pieces of output data of the second layer, and enter the normalized pieces of output data of the second layer into the linearization layer and the softmax layer to obtain result data.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires first output data of a student model for first input data and second output data of a teacher model for second input data and trains the student model such that the first output data and the second output data are not distinguished from each other. The student model and the teacher model have different structures.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires first output data of a student model for first input data and second output data of a teacher model for second input data and trains the student model such that the first output data and the second output data are not distinguished from each other. The student model and the teacher model have different structures.

Abstract: A model training method and apparatus is disclosed, where the model training method acquires a recognition result of a teacher model and a recognition result of a student model for an input sequence and trains the student model such that the recognition result of the teacher model and the recognition result of the student model are not distinguished from each other.

Abstract: A model training method and apparatus, and a data recognition method are provided. The model training method includes determining a loss function by reflecting an error rate between a recognition result of a teacher model and a recognition result of a student model to the loss function, and training the student model based on the loss function.