Abstract: A technique is described herein for training a tagging model that is able to successfully interpret queries. The technique trains the tagging model in plural stages. A first stage continues training a pre-trained language model based on a set of queries, to produce a first-stage model. A second stage performs training on the basis of a set of supplemented queries and associated weak labels, to produce a second-stage model. Each supplemented query combines a query with titles of documents that match the query. A third stage performs training on the basis of a set of supplemented queries and associated strong labels, to produce a third-stage model. The third stage also uses adversarial knowledge enhancement that has the effect of making the data presented to the third-stage model more difficult for the third-stage model to interpret. This, in turn, improves the generalization capabilities and robustness of the third-stage model.

Abstract: A technique is described herein for training a tagging model that is able to successfully interpret queries. The technique trains the tagging model in plural stages. A first stage continues training a pre-trained language model based on a set of queries, to produce a first-stage model. A second stage performs training on the basis of a set of supplemented queries and associated weak labels, to produce a second-stage model. Each supplemented query combines a query with titles of documents that match the query. A third stage performs training on the basis of a set of supplemented queries and associated strong labels, to produce a third-stage model. The third stage also uses adversarial knowledge enhancement that has the effect of making the data presented to the third-stage model more difficult for the third-stage model to interpret. This, in turn, improves the generalization capabilities and robustness of the third-stage model.

Abstract: A computing system is disclosed that includes a processor and memory. The memory stores instructions that, when executed by the processor, cause the processor to perform several acts. The acts comprise receiving conversational data indicative of an interaction between a client computing device and a generative model. The conversational data is provided as input into an intent classification module and the intent classification module produces an output indicative of a user intent based upon the conversational data. An anchor generation module generates anchor text indicative of portions of the conversational data correlated with the user intent. A content query based upon the anchor text is generated and content responsive to the content query is obtained and presented at the client computing device.

Abstract: A computing device is provided including a processor configured to execute a transformer including an encoder having a global layer configured to receive tokenized embeddings for each of a plurality of tokens in a local input sequence and compute a global self-attention vector for each of the tokenized embeddings. The encoder further includes a local layer configured to receive each global self-attention vector from the global layer and compute local self-attention for each local input sequence, and add and normalize the global self-attention vector with the local self-attention vector to thereby produce an encoder representation including a self-attention vector for each local input sequence that includes both global self-attention values and local self-attention values. The transformer is configured to output a prediction for the global input sequence based on the encoder representation of each of the local input sequences of the global input sequence.