Abstract: Efficient tagging of content items using content embeddings are provided. In one technique, multiple content items are stored a content embedding for content item is stored. Entity names are also stored along with an entity name embedding for each entity name. For each content item, (1) multiple content embeddings that are associated with the content item are identified; (2) a subset of the entity names is identified; and (3) for each entity name in the subset, (i) an embedding of the entity name is identified, (ii) similarity measures are generated based on the entity name embedding and the multiple content embeddings, (iii), a distribution of the similarity measures is generated, (iv) feature values are generated based on the distribution, (v) the feature values are input into a machine-learned classifier, and (vi) based on output from the classifier, it is determined whether to associate the entity name with the content item.

Abstract: Techniques are provided for using machine learning techniques to associate skills with different content. In one technique, multiple classifications models are trained. Each classification model corresponds to a different skill and is trained based on textual embeddings of a plurality of content items and labels indicating whether each content item is associated with the skill that corresponds to that classification model. A particular content item embedding is generated based on text from a particular content item. The particular content item embedding is applied to the classification models to generate multiple results. One or more results of the multiple results are identified that indicate that one or more corresponding skills are associated with the particular content item. For each result of the one or more results, skill tagging data are stored that associate the particular content item with a particular skill that corresponds to that result.

Abstract: Efficient tagging of content items using content embeddings are provided. In one technique, multiple content items are stored a content embedding for content item is stored. Entity names are also stored along with an entity name embedding for each entity name. For each content item, (1) multiple content embeddings that are associated with the content item are identified; (2) a subset of the entity names is identified; and (3) for each entity name in the subset, (i) an embedding of the entity name is identified, (ii) similarity measures are generated based on the entity name embedding and the multiple content embeddings, (iii), a distribution of the similarity measures is generated, (iv) feature values are generated based on the distribution, (v) the feature values are input into a machine-learned classifier, and (vi) based on output from the classifier, it is determined whether to associate the entity name with the content item.

Abstract: In an example embodiment, a fully automated process is provided for frequent model retraining and redeployment of a machine learned model trained to output a prediction of how likely it is that a candidate is qualified for a particular job posting. Model quality verification is provided by maintaining a snapshot of a baseline model and automatically comparing it to a proposed model by performing various metrics on the models by testing the models using a holdout data set that includes only data that was not used during the training process. Overlap between data in the holdout set used during retraining and the training set used during initial training is prevented by splitting each dataset using a hash on certain fields of the data.

Abstract: In an embodiment, the disclosed technologies include determining a digital identifier, computing, using aggregate digital event data obtained from at least one computing device, digital feature data relating to the digital identifier, inputting the digital feature data relating to the digital identifier into a digital model that has machine-learned correlations between digital feature data and digital propensity prediction values, receiving, from the digital model, a predicted propensity value associated with the digital identifier, determining a propensity score based on the predicted propensity value, causing a digital content item to be displayed on a user interface of a network-based software application associated with the digital identifier if the propensity score satisfies a propensity criterion.

Abstract: Techniques for efficiently matching two sets of video items are provided. In on technique, an embedding is generated for each video item in each set. For the first set of video items, multiple groups are generated. The first set of video items may have a relatively little amount of metadata information for them. Each video item in the first set is assigned to one of the groups. Then, for each video item in the second set, one of the groups is selected based on embedding similarity. For each video item in the selected group, an embedding similarity is determined between that video item in the selected group and the video item in the second set. If the embedding similarity is above a certain threshold, then an association is generated for that pair of video items.

Abstract: In some embodiments, a computer system generates a recommendation for a user of an online service based on user actions that have been performed by the user within a threshold amount of time before the generation of the recommendation. For each user action, the computer system determines an intent classification that identifies an activity of the user and that corresponds to different types of user actions, as well as a preference classification that identifies a target of the activity, and then stores these intent and preference classifications as part of indications of the user actions for use in generating different types of recommendations using different types of recommendation models. Additionally, the computer system may use mini-batches of data from an incoming stream of logged data to train an incremental update to one or more recommendation models.

Abstract: Techniques for implementing a feature generation pipeline for machine learning are provided. In one technique, multiple jobs are executed, each of which computes a different set of feature values for a different feature of multiple features associated with videos. A feature registry is stored that lists each of the multiple features. After the jobs are executed and the feature registry is stored, a model specification is received that indicates a set of features for a model. For each feature in a subset of the set of features, a location is identified in storage where a value for said each feature is found and the value for that feature is retrieved from the location. A feature vector is created that comprises, for each feature in the set of features, the value that corresponds to that feature. The feature vector is used to train the model or as input to the model.

Abstract: Techniques for implementing a learning semantic representations of sparse entities using unsupervised embeddings are disclosed herein. In some embodiments, a computer system accesses corresponding profile data of users indicating at least one entity of a first facet type associated with the user, and generating a graph data structure comprising nodes and edges based on the accessed profile data, with each node corresponding to a different entity indicated by the accessed profile data, and each edge directly connecting a different pair of nodes and indicating a number of users whose profile data indicates both entities of the pair of nodes. The computer system generating a corresponding embedding vector for the entities based on the graph data structure using an unsupervised machine learning algorithm.

Abstract: In an embodiment, the disclosed technologies include determining a digital identifier, computing, using aggregate digital event data obtained from at least one computing device, digital feature data relating to the digital identifier, inputting the digital feature data relating to the digital identifier into a digital model that has machine-learned correlations between digital feature data and digital propensity prediction values, receiving, from the digital model, a predicted propensity value associated with the digital identifier, determining a propensity score based on the predicted propensity value, causing a digital content item to be displayed on a user interface of a network-based software application associated with the digital identifier if the propensity score satisfies a propensity criterion.

Abstract: The video recommendation system provided with an on-line connection system generates on-line video recommendations using collaborative filtering for clusters of member profiles. The recommendation system clusters member profiles using member profile information as clustering criteria. The video recommendations are then generated for a given cluster, based on aggregation of video viewing history recorded for the member profiles that are in the given cluster, using the video similarity matrix. In order to produce video recommendations for a particular member profile, the recommendation system first determines cluster membership for the member profile, retrieves recommendations generated for that cluster, and provides recommendations to the associated member. A user interface including references to one or more recommended videos is rendered on a display device of a viewer.

Abstract: Techniques for processing search queries are described. Consistent with some embodiments, a computer system generates a profile vector representation for each of several user profiles based on the user profile data of the user profiles, and then stores the vector representations. A subsequent query is processed to generate a query vector representation for the query. A neural network is used to generate a similarity score for each pairing of the query vector representation and a profile vector representation. Finally, some number of user profiles having the highest similarity scores are provided as search results.

Abstract: Techniques for classifying videos as standalone or non-standalone are provided. Feature (or attribute) values associated with a particular video are identified. Feature values are extracted from metadata associated with the particular video and/or from within a transcript of the particular video. The extracted feature values of the particular video are input to a rule-based or a machine-learned model and the model scores the particular video. Once a determination pertaining to whether the particular video is standalone is made, information about the particular video being a standalone video is presented to one or more users within the network.

Abstract: The recommendation system provided with an on-line connection system identifies on-line recommendations of videos and generates a user interface (UI) by including into the resulting presentation selected recommendations of videos. The recommendations of videos presented in the UI are organized into groups that are topically coherent, where each group is decorated with a context annotation—an explanation of why the recommendations in a given carousel are relevant for a member. Each video that is being evaluated by the recommendation system with respect to a subject member profile is assigned an annotation that is selected from a plurality of potentially applicable annotations.

Abstract: Techniques for applying learning-to-rank with deep learning models for search are disclosed herein. In some embodiments, a computer system trains a ranking model using training data and a loss function, with the ranking model comprising a deep learning model and being configured to generate similarity scores based on a determined level of similarity between profile data of reference candidates users in the training data and reference query data of reference queries in the training data. The computer system receives a target query comprising target query data from a computing device of a target querying user, and then generates a corresponding score for target candidate users based on a determined level of similarity between profile data of the target candidate users and the target query data using the trained ranking model.

Abstract: The video recommendation system provided with an on-line connection system generates on-line video recommendations using collaborative filtering for clusters of member profiles. The recommendation system clusters member profiles using member profile information as clustering criteria. The video recommendations are then generated for a given cluster, based on aggregation of video viewing history recorded for the member profiles that are in the given cluster, using the video similarity matrix. In order to produce video recommendations for a particular member profile, the recommendation system first determines cluster membership for the member profile, retrieves recommendations generated for that cluster, and provides recommendations to the associated member. A user interface including references to one or more recommended videos is rendered on a display device of a viewer.

Abstract: Techniques for improving online content recommendations using generalized linear mixed models are disclosed herein. In some embodiments, a generalized mixed model, comprising a baseline model, a user-based model, and a course-based model, is used to generate scores for each one of a plurality of candidate online courses. The baseline model is a generalized linear model based on profile information of a target user and metadata of the candidate online course, the user-based model is a random effects model based on a history of online activity by the target user directed towards reference online courses having metadata related to the metadata of the candidate online course, and the course-based model is a random effects model based on a history of online activity directed towards the candidate online course by a plurality of reference users having profile information related to the profile information of the target user.

Abstract: Techniques for implementing a feature generation pipeline for machine learning are provided. In one technique, multiple jobs are executed, each of which computes a different set of feature values for a different feature of multiple features associated with videos. A feature registry is stored that lists each of the multiple features. After the jobs are executed and the feature registry is stored, a model specification is received that indicates a set of features for a model. For each feature in a subset of the set of features, a location is identified in storage where a value for said each feature is found and the value for that feature is retrieved from the location. A feature vector is created that comprises, for each feature in the set of features, the value that corresponds to that feature. The feature vector is used to train the model or as input to the model.

Abstract: Techniques for generating supervised embedding representations for search are disclosed herein. In some embodiments, a computer system receives training data comprising query representations including an entity included in a corresponding search query submitted by a querying user, search result representations for each one of the query representations, and user actions for each one of the query representations, and generates a corresponding embedding vector for each one of the at least one entity using a supervised learning algorithm and the received training data. In some example embodiments, the corresponding search result representations for each one of the query representations represents a plurality of candidate users displayed in response to the search queries based on profile data of the candidate users, and the user actions comprise actions by the querying user directed towards at least one candidate user in the search results.

Abstract: Techniques are provided for using machine learning techniques to associate skills with different content. In one technique, multiple classifications models are trained. Each classification model corresponds to a different skill and is trained based on textual embeddings of a plurality of content items and labels indicating whether each content item is associated with the skill that corresponds to that classification model. A particular content item embedding is generated based on text from a particular content item. The particular content item embedding is applied to the classification models to generate multiple results. One or more results of the results are identified that indicate that one or more corresponding skills are associated with the particular content item. For each result of the one or more results, skill tagging data is stored that associates the particular content item with a particular skill that corresponds to that result.