Abstract: A data processing system implements obtaining query parameters for a query for content items in a datastore, the query parameters including attributes of content items for which a search is to be conducted; obtaining a first set of content items from a content datastore based on the query parameters; analyzing the first set of content items using a first machine learning model trained to generate relevant content information that identifies a plurality of relevant content items included in the first set of content items; and analyzing the plurality of relevant content items using a second machine learning model configured to output novel content information, the novel content information including a plurality of content items predicted to be relevant and novel, the novel content information ranking the plurality of content items predicted to be relevant and novel based on a novelty score associated with each respective content item.

Abstract: The interpretation of a graph data structure represented on a computing system in which the connection between a pair of nodes in the graph may be interpreted by which intermediary entity (node or edge) on a path (e.g., a shortest path) between the node pair is most dominant. That is, if the intermediary entity were not present, a detour path is determined. The greater the difference between the detour path and the original path, the more significant that intermediary entity is. The significance of multiple intermediary entities in the original path may be determined in this way.

Abstract: A data processing system implements obtaining query parameters for a query for content items in a datastore, the query parameters including attributes of content items for which a search is to be conducted; obtaining a first set of content items from a content datastore based on the query parameters; analyzing the first set of content items using a first machine learning model trained to generate relevant content information that identifies a plurality of relevant content items included in the first set of content items; and analyzing the plurality of relevant content items using a second machine learning model configured to output novel content information, the novel content information including a plurality of content items predicted to be relevant and novel, the novel content information ranking the plurality of content items predicted to be relevant and novel based on a novelty score associated with each respective content item.

Abstract: A data processing system implements obtaining attention matrices from a first machine learning model that is pretrained and includes a plurality of self-attention layers. The data processing system further implements analyzing the attention matrices to generate a computation graph based on the attention matrices. The computation graph provides a representation of behavior of the first machine learning model across the plurality of self-attention layers. The data processing system is further implements analyzing the computation graph using a second machine learning model. The second machine learning model is trained to receive the computation graph to output model behavior information. The model behavior information identifying which layers of model performed specific tasks associated with generating predictions by the first machine learning model.

Abstract: The interpretation of a graph data structure represented on a computing system in which the connection between a pair of nodes in the graph may be interpreted by which intermediary entity (node or edge) on a path (e.g., a shortest path) between the node pair is most dominant. That is, if the intermediary entity were not present, a detour path is determined. The greater the difference between the detour path and the original path, the more significant that intermediary entity is. The significance of multiple intermediary entities in the original path may be determined in this way.

Abstract: The monitoring of performance of a machine-learned model for use in generating an embedding space. The system uses two embedding spaces: a reference embedding space generated by applying an embedding model to reference data, and an evaluation embedding space generated by applying the embedding model to evaluation data. The system obtains multiple views of the reference embedding space, and uses those multiple views to determine a distance threshold. The system determines a distance between the evaluation and reference embedding spaces, and compares that distance with the fitness threshold. Based on the comparison, the system determines a level of acceptability of the model for use with the evaluation dataset.

Abstract: A computer system is configured to provide an intelligent machine-learning (ML) model catalog containing data associated with multiple ML models. The multiple ML models are trained over multiple training datasets respectively, and the intelligent ML model catalog contains at least multiple training data spaces of embeddings generated based on the multiple ML models and the multiple training datasets. In response to receiving a user dataset, for at least one ML model in the plurality of ML models, the computer system is configured to extract a user data space of embeddings based on the at least one ML model and the user dataset, and evaluate the user data space against the training data space to determine whether the at least one ML model is a good fit for the user dataset.

Abstract: The interpretation of a graph data structure represented on a computing system in which the connection between a pair of nodes in the graph may be interpreted by which intermediary entity (node or edge) on a path (e.g., a shortest path) between the node pair is most dominant. That is, if the intermediary entity were not present, a detour path is determined. The greater the difference between the detour path and the original path, the more significant that intermediary entity is. The significance of multiple intermediary entities in the original path may be determined in this way.

Abstract: A computer implemented method includes obtaining deep learning model embedding for each instance present in a dataset, the embedding incorporating a measure of concept similarity. An identifier of a first instance of the dataset is received. A similarity distance is determined based on the respective embeddings of the first instance and a second instance. Similarity distances between embeddings, represented as points, imply a graph, where each instance's embedding is connected by an edge to a set of similar instances' embeddings. Sequences of connected points, referred to as walks, provide valuable information about the dataset and the deep learning model.

Abstract: Creating a machine learning graph neural network configured to process signals. A method includes identifying a plurality of machine learning graphs where each of the machine learning graphs are for different types of data. The method further includes receiving input identifying shared content of different machine learning graph nodes from different graphs in the plurality of machine learning graphs. The method further includes creating a combined machine learning graph neural network, configured to process signals, using the plurality of machine learning graphs based on the shared content, the combined machine learning graph neural network comprising nodes corresponding to nodes in the plurality of machine learning graphs such that output from the combined machine learning graph neural network comprises outputs generated based on relationships of nodes in the combined machine learning graph corresponding to nodes in different machine learning graphs in the plurality of machine learning graphs.