Abstract: Retrieving content items in response to a query in a way that increases user satisfaction and increases chances of users consuming a retrieved content item is not trivial. One retrieval strategy may include dividing the content items into buckets according to a dimension about the content items and retrieving a top K number of items from different buckets to balance semantic affinity and the dimension. Choosing an optimal K for different buckets for a given query can be a challenge. Reinforcement learning can be used to train and implement an agent model that can choose the optimal K for different buckets.

Abstract: Pre-trained large language models may be trained on a large data set which may not necessarily align with specific tasks, business goals, and requirements. Pre-trained large language models can solve generic semantic relationship or question-answering type problems but may not be suited for content item retrieval or recommendation of content items that are semantically relevant to a query. It is possible to build a machine learning model while using transfer learning to learn from pre-trained large language models. Training data can significantly impact the performance of machine learning models, especially machine learning models developed using transfer learning. The training data can impact a model's performance, generalization, fairness, and adaptation to specific domains. To address some of these concerns, a popularity bucketing strategy can be implemented to debias training data. Optionally, an ensemble of models can be used to generate diverse training data.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items: a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items: a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items; a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate a digital knowledge graph based on a plurality of tutorial content items to generate recommendations of digital resource items. Specifically, the disclosed system extracts a plurality of tasks, subject categories related to the tasks, and context signals related to an environment for the tasks from a plurality of tutorial content items for one or more digital content editing applications. The disclosed system generates a digital knowledge graph including nodes corresponding to the tasks and subject categories connected via edges based on relationships extracted from the tutorial content items. In some embodiments, the disclosed system also includes nodes corresponding to digital resource items in the digital knowledge graph or in a subgraph. The disclosed system utilizes the digital knowledge graph with context data to provide a recommendation of digital resource items for display at a client device.

Abstract: This disclosure describes methods, non-transitory computer readable storage media, and systems that generate a digital knowledge graph based on a plurality of tutorial content items to generate recommendations of digital resource items. Specifically, the disclosed system extracts a plurality of tasks, subject categories related to the tasks, and context signals related to an environment for the tasks from a plurality of tutorial content items for one or more digital content editing applications. The disclosed system generates a digital knowledge graph including nodes corresponding to the tasks and subject categories connected via edges based on relationships extracted from the tutorial content items. In some embodiments, the disclosed system also includes nodes corresponding to digital resource items in the digital knowledge graph or in a subgraph. The disclosed system utilizes the digital knowledge graph with context data to provide a recommendation of digital resource items for display at a client device.