Abstract: Retrieving content items in response to a query in a way that increases user satisfaction and increases chances of users consuming a retrieved content item is not trivial. One retrieval strategy may include dividing the content items into buckets according to a dimension about the content items and retrieving a top K number of items from different buckets to balance semantic affinity and the dimension. Choosing an optimal K for different buckets for a given query can be a challenge. Reinforcement learning can be used to train and implement an agent model that can choose the optimal K for different buckets.

Abstract: Retrieving content items in response to a query in a way that increases user satisfaction and increases chances of users consuming a retrieved content item is not trivial. One content item retrieval system can combine different retrieval strategies. The content item retrieval system can retrieve a number of content items using different retrieval strategies and combining the content items together as the final results of the search. A naïve approach is to show fixed numbers of content items retrieved using the different retrieval strategies for any query. User engagement can be improved if the numbers can be tuned or optimized for a given query. Reinforcement learning can be used to train and implement an agent model that can choose the optimal numbers of content items retrieved using different retrieval strategies for a given query.

Abstract: Training data can significantly impact the performance of machine learning models. Its impact may be more significant in transfer learning. Different data sources can be used to generate training data used in transfer learning. The training data originating from user interaction logs may be subject to presentation bias. The training data originating from model generated labeled data may have false positives. Poor quality training data may cause the machine learning model to perform poorly. To address some of these concerns, a checker having one or more models can check for false positives and for labeled data entries that may have been subject to presentation bias. Such entries may be removed or modified. In some cases, the checker can generate a test that can be used to test the machine learning model and penalize the machine learning model if the model generates an incorrect prediction.

Abstract: Pre-trained large language models may be trained on a large data set which may not necessarily align with specific tasks, business goals, and requirements. Pre-trained large language models can solve generic semantic relationship or question-answering type problems but may not be suited for content item retrieval or recommendation of content items that are semantically relevant to a query. It is possible to build a machine learning model while using transfer learning to learn from pre-trained large language models. Training data can significantly impact the performance of machine learning models, especially machine learning models developed using transfer learning. The training data can impact a model's performance, generalization, fairness, and adaptation to specific domains. To address some of these concerns, a popularity bucketing strategy can be implemented to debias training data. Optionally, an ensemble of models can be used to generate diverse training data.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items: a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for determining a list of recommended items in response to a user query. An embodiment can generate an ordered relevance list of items, and determine an initial reward value based on an array of relevance scores and an array of revenue values corresponding to the ordered relevance list of items, a parameter alpha assigned to the array of relevance scores, and a parameter beta assigned to the array of revenue values. The embodiment can generate a next list of recommended items from an initial list of recommended items, and further calculate a next reward value associated with the next list of recommended items, and determine a list of recommended items in response to the query based on a comparison of the initial reward value and the next reward value.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items: a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for model customization for domain-specific tasks. An embodiment may select a pre-trained embedding model trained with a first dataset. The embodiment may determine a second dataset for a target domain. Based on target embeddings for data indicative of the target domain. The embodiment may transform the second dataset from a first format to a second format associated with the target domain. The embodiment may modify the weights of the pre-trained embedding model based on the transformed second dataset. Based on the modified weights, the embodiment may transform the pre-trained embedding model into a target embedding model for the target domain. The embodiment may then generate an efficacy score for the target embedding model based on a task of the target domain performed by the target embedding model.

Abstract: Disclosed herein are system, apparatus, article of manufacture, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for training a heterogenous graph neural network (GNN) to generate user embeddings corresponding to users and item embeddings corresponding to items. An example embodiment generates a first user interaction graph for a first time window and a second user interaction graph for a second time window, wherein each graph represents users and items as nodes and user-item interactions within the respective time window as edges, samples user-item node pairs from the second user interaction graph, and trains the heterogeneous GNN based on user-item node pairs from the first user interaction graph that correspond to the sampled user-item node pairs from the second user interaction graph. User and item embeddings generated by the trained GNN may be used to determine a relevancy of a given item with respect to a given user.

Abstract: Disclosed are system, method and/or computer program product embodiments that retrieve items for a user based on a query using a two-tower deep machine learning model. An example embodiment provides input to a context tower, wherein the input includes the query and one or more of a query embedding corresponding to the query or a graph user embedding corresponding to the user. The context tower generates a context embedding in a vector space based on the input. The model determines a measure of similarity between the context embedding and each of a plurality of item embeddings in the vector space that are generated by an item tower and represent a plurality of candidate items. A relevancy score is calculated for each candidate item based on the measure of similarity between the context embedding and the corresponding item embedding. The relevancy scores are used for item retrieval and/or ranking.

Abstract: Disclosed herein are system, method and/or computer program product embodiments, and/or combinations and sub-combinations thereof, for generating a prediction based on a query. An embodiment operates by providing a query to a deep machine learning (ML) model. The deep ML model generates a plurality of query projection embeddings by projecting the query into each of a plurality of different query embedding spaces and generates the prediction based at least on the plurality of query projection embeddings. Each of a plurality of query projection embedding layers of the deep ML may generate a corresponding one of the query projection embeddings by applying a hash function associated with the query projection layer to the query to generate a vector representation of the query, and applying a set of weights associated with the query projection layer to the vector representation to generate a query projection embedding in the plurality of query projection embeddings.

Abstract: A content retrieval system may receive a query associated with a plurality of content items in a repository. For each content item of the plurality of content items; a respective first and second similarity score may be generated based on a similarity between embeddings indicative of a first and second data type generated from the query and for the content item; and a respective normalized similarity score may be generated based on a combination of the respective first and second similarity scores. A set of content items with respective normalized similarity scores that satisfy a similarity score threshold may be identified. An exact-match (lexical) search may yield respective mapping scores for content items that may also be ranked. An output indicative of content items that are identified in the set of content items with high-ranking similarity scores and identified in the set of content items with high-ranking mapping scores.

Abstract: Mechanisms are provided to implement a medical coding engine to perform medical coding using a neural network architecture that leverages hierarchical semantics between medical concepts. The medical coding engine configures a medical coding neural network to comprise an first layer of nodes comprising preferred terminology (PT) nodes, a second layer comprising lowest level terminology (LLT) nodes, and a third layer comprising weighted values for each connection between each PT node and each LLT node forming a PT node/LLT node connection. Responsive to receiving an adverse event from a cognitive system, a PT node is identified in the first layer associated with a citation from the adverse event. One or more LLT nodes are identified from the second layer based on the identification PT node and a weight associated with the PT node/LLT node connection. A medical code associated with each the one or more LLT nodes is then output.

Abstract: Mechanisms are provided to implement a medical coding engine to perform medical coding using a neural network architecture that leverages hierarchical semantics between medical concepts. The medical coding engine configures a medical coding neural network to comprise an first layer of nodes comprising preferred terminology (PT) nodes, a second layer comprising lowest level terminology (LLT) nodes, and a third layer comprising weighted values for each connection between each PT node and each LLT node forming a PT node/LLT node connection. Responsive to receiving an adverse event from a cognitive system, a PT node is identified in the first layer associated with a citation from the adverse event. One or more nodes are identified from the second layer based on the identification PT node and a weight associated with the PT node/LLT node connection. A medical code associated with each the one or more LLT nodes is then output.

Abstract: Mechanisms are provided to implement a medical coding engine to perform medical coding using a neural network architecture that leverages hierarchical semantics between medical concepts. The medical coding engine configures a medical coding neural network to comprise an first layer of nodes comprising preferred terminology (PT) nodes, a second layer comprising lowest level terminology (LLT) nodes, and a third layer comprising weighted values for each connection between each PT node and each LLT node forming a PT node/LLT node connection. Responsive to receiving an adverse event from a cognitive system, a PT node is identified in the first layer associated with a citation from the adverse event. One or more nodes are identified from the second layer based on the identification PT node and a weight associated with the PT node/LLT node connection. A medical code associated with each the one or more LLT nodes is then output.