Abstract: A method for training a model based on multi-modal data joint learning, includes: obtaining multi-modal data; in which the multi-modal data include at least one type of single-modal data and at least one type of Pair multi-modal data; inputting the single-modal data and the Pair multi-modal data into a decoupling attention Transformer network model to generate respectively Token semantic representation features and cross-modal semantic representation features; and training the decoupling attention Transformer network model based on the Token semantic representation features and the cross-modal semantic representation features.

Abstract: The present disclosure discloses a method and apparatus for acquiring a pre-trained model, and relates to natural language processing and deep learning technologies in the field of artificial intelligence technologies. An implementation includes: acquiring training data, the training data including a single-modal language material and a multi-modal language material, and the multi-modal language material including a language material pair formed by a first-modal language material and a second-modal language material; and performing a multi-task training operation on a pre-trained model using the training data, the multi-task including at least one cross-modal contrastive learning task and at least one single-modal learning task; the pre-trained language model obtained in the present disclosure may learn from different forms of language materials, i.e.

Abstract: A cross-modality processing method is related to a field of natural language processing technologies. The method includes: obtaining a sample set, wherein the sample set includes a plurality of corpus and a plurality of images; generating a plurality of training samples according to the sample set, in which each of the plurality of the training samples is a combination of at least one of the plurality of the corpus and at least one of the plurality of the images corresponding to the at least one of the plurality of the corpus; adopting the plurality of the training samples to train a semantic model, so that the semantic model learns semantic vectors containing combinations of the corpus and the images.

Abstract: A document recommendation method based on a semantic tag and a document recommendation device. The method includes: for each document, acquiring a first candidate tag set corresponding to the document, and processing each first candidate tag in the first candidate tag set corresponding to the document to obtain a second candidate tag set corresponding to the document; performing normalization processing on each second candidate tag in the second candidate tag set corresponding to the document to obtain a third candidate tag set corresponding to the document; performing expanding process on each third candidate tag in the third candidate tag set corresponding to the document, and acquiring a fourth candidate tag set corresponding to the document, to form a document library having semantic tags; and recommending a target document obtained from the document library having semantic tags to the user, according to historical semantic tag.

Abstract: A cross-modality processing method is related to a field of natural language processing technologies. The method includes: obtaining a sample set, wherein the sample set includes a plurality of corpus and a plurality of images; generating a plurality of training samples according to the sample set, in which each of the plurality of the training samples is a combination of at least one of the plurality of the corpus and at least one of the plurality of the images corresponding to the at least one of the plurality of the corpus; adopting the plurality of the training samples to train a semantic model, so that the semantic model learns semantic vectors containing combinations of the corpus and the images.

Abstract: The present disclosure provides a document recommendation method based on a semantic tag and a document recommendation device. The method includes: for each document, acquiring a first candidate tag set corresponding to the document, and processing each first candidate tag in the first candidate tag set corresponding to the document to obtain a second candidate tag set corresponding to the document; performing normalization processing on each second candidate tag in the second candidate tag set corresponding to the document to obtain a third candidate tag set corresponding to the document; performing expanding process on each third candidate tag in the third candidate tag set corresponding to the document, and acquiring a fourth candidate tag set corresponding to the document, to form a document library having semantic tags; and recommending a target document obtained from the document library having semantic tags to the user, according to historical semantic tag.