Abstract: A determination is made that an explanatory data set for a common set of predictions generated by a machine learning model for records containing text tokens is to be provided. Respective groups of related tokens are identified from the text attributes of the records, and record-level prediction influence scores are generated for the token groups. An aggregate prediction influence score is generated for at least some of the token groups from the record-level scores, and an explanatory data set based on the aggregate scores is presented.

Abstract: An algorithm releases answers to very large numbers of statistical queries, e.g., k-way marginals, subject to differential privacy. The algorithm answers queries on a private dataset using simple perturbation, and then attempts to find a synthetic dataset that most closely matches the noisy answers. The algorithm uses a continuous relaxation of the synthetic dataset domain which makes the projection loss differentiable, and allows the use of efficient machine learning optimization techniques and tooling. Rather than answering all queries up front, the algorithm makes judicious use of a privacy budget by iteratively and adaptively finding queries for which relaxed synthetic data has high error, and then repeating the projection. The algorithm is effective across a range of parameters and datasets, especially when a privacy budget is small or a query class is large.

Abstract: A set of clusters from a first set of vector representations (VRs) is identified. A center associated with each cluster from the set of clusters to generate a set of centers is determined. For each VR from the first set of VRs, and to generate a first set of distributions, a distribution of that VR is determined that indicates, for each center from the set of centers, similarity between that VR and that center. For each VR from a second set of VRs, and to generate a second set of distributions, a distribution of that VR is determined that indicates, for each center from the set of centers, similarity between that VR and that center. A set of divergence metrics associated with the first set of VRs and the second set of VRs are computed based on comparing the first set of distributions and the second set of distributions.

Abstract: An algorithm releases answers to very large numbers of statistical queries, e.g., k-way marginals, subject to differential privacy. The algorithm answers queries on a private dataset using simple perturbation, and then attempts to find a synthetic dataset that most closely matches the noisy answers. The algorithm uses a continuous relaxation of the synthetic dataset domain which makes the projection loss differentiable, and allows the use of efficient machine learning optimization techniques and tooling. Rather than answering all queries up front, the algorithm makes judicious use of a privacy budget by iteratively and adaptively finding queries for which relaxed synthetic data has high error, and then repeating the projection. The algorithm is effective across a range of parameters and datasets, especially when a privacy budget is small or a query class is large.

Abstract: Hyperparameters for tuning a machine learning system may be optimized for fairness using Bayesian optimization with constraints for accuracy and bias. Hyperparameter optimization may be performed for a received training set and received accuracy and fairness constraints. Respective probabilistic models for accuracy and bias of the machine learning system may be initialized, then hyperparameter optimization may include iteratively identifying respective values for hyperparameters using analysis of the respective models performed using an acquisition function implementing constrained expected improvement on the respective models, training the machine learning system using the identified values to determine measures of accuracy and bias, and updating the respective models using the determined measures.

Abstract: Views may be generated for bias metrics or feature attribution captured in machine learning pipelines. A request to create a view of bias metrics or feature attribution may be received. The bias metrics or feature attribution may have been determined in a machine learning pipeline as part of executing a training job that specified the bias metrics or the feature attribution. A development application may access a data store that stores the bias metrics or the feature attribution determined in the machine learning pipeline. A view based on the bias metrics or feature attribution may be generated and provided.

Abstract: Feature attribution may be captured as part of a machine learning pipeline. A training job may include a request to determine feature attribution as part of a machine learning pipeline that trains a machine learning model from a training data set. A reference data set for determining the feature attribution of the machine learning model may be identified. The feature attribution may be determined based on the reference data set. The feature attribution of the trained machine learning model may be stored.

Abstract: Bias metrics may be captured at different stages for training a machine learning model. A training job may specify bias metrics to capture at multiple different stages of a machine learning pipeline for a feature of a training data set used to train a machine learning model. The training job may be executed and the bias metrics determined at the stages as specified in the training job. The bias metrics for the different stages may be stored.

Abstract: Bias metrics and feature attribution may be monitored for a machine learning model. A request to enable monitoring for bias metrics or feature attribution may be received. Monitoring may be enabled to evaluate respective performance of inferences of a machine learning model according to the enabled bias metrics or feature attribution. If a divergence from reference data is detected, then a notification indicating the divergence may be sent.

Abstract: Aspects of the present disclosure relate to cryptography. In particular, example embodiments relate to computing a relationship between private data of a first entity and private data of a second entity, while preserving privacy of the entities and preventing inter-entity data sharing. A server includes a first component to compute an intersection of two datasets, without directly accessing either dataset. The server includes a second component to compute a relationship, such as a regression, between data in the first dataset and data in the second dataset, without directly accessing either dataset.

Abstract: In an example, each of a plurality of members of social networking service is mapped to a weighted skill vector, each weighted skill vector including a list of skills for the member with an associated weight indicating strength of the skill. Members of the social networking service that belong to an industry are aggregated to obtain a weighted matrix of members and skills along with compensation vectors indicating compensation for each of the members in the matrix. The weighted matrix of users and skills and corresponding compensation vectors is used to train a machine learning skill monetary value prediction model to output a predicted monetary value for one or more skills contained in a candidate vector fed to the machine learning skill monetary value prediction model.

Abstract: Techniques for implementing a learning semantic representations of sparse entities using unsupervised embeddings are disclosed herein. In some embodiments, a computer system accesses corresponding profile data of users indicating at least one entity of a first facet type associated with the user, and generating a graph data structure comprising nodes and edges based on the accessed profile data, with each node corresponding to a different entity indicated by the accessed profile data, and each edge directly connecting a different pair of nodes and indicating a number of users whose profile data indicates both entities of the pair of nodes. The computer system generating a corresponding embedding vector for the entities based on the graph data structure using an unsupervised machine learning algorithm.

Abstract: A member profile including a vector containing a field for each of a plurality of skills and a rating of one or more of the skills in the vector for a member of a social networking service is obtained. A first distance indicating a vector distance between the vector of the member profile and a vector of a hypothetical member profile representing the perfect job candidate is obtained. A hypothetical member profile for the member is created by combining the vector of the member profile with the indication of how each of the one or more skills is improved through taking the course from course information. A second distance between the member and the hypothetical perfect candidate for the job is obtained, and the difference between the first distance and the second distance is calculated to determine an estimate of how much the course will increase the member's job chances.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains, based on parameters of a search of an online system by a user, counts of attributes required by a set of opportunities within a team. Next, the system determines, based on data retrieved from a data store, multiple sets of candidates for the set of opportunities based on multiple objectives that comprise maximizing coverage of the counts of attributes by a given set of candidates. The system then selects, from the multiple sets of candidates, one or more sets of candidates that best meet one or more combinations of the multiple objectives. Finally, the system outputs, in a user interface of the online system, the one or more sets of candidates as recommendations for filling the set of opportunities in the team.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system estimates parameters of a first distribution of one or more attributes in a first set of candidates selected as recommendations for a recruiting entity in an online system. Next, the system determines a first probability that a candidate belongs to the first distribution based on the estimated parameters and values of the one or more attributes for the candidate. The system then applies a first threshold to the first probability to determine a first classification of the candidate as anomalous or non-anomalous with respect to the first set of candidates. Finally, the system updates a user interface of the online system that outputs the recommendations to the recruiting entity based on the first classification.

Abstract: The disclosed embodiments provide a system for processing data. The system obtains, based on parameters of a search by an entity, a first set of scores representing significances of a set of attributes as qualifications for an opportunity. Next, the system generates a ranking of candidates for the opportunity based on the first set of scores and additional sets of scores representing confidences in possession of the attributes by the candidates. The system outputs the ranking as results of the search during a session with the entity. The system updates the first set of scores based on one or more actions by the entity on one or more candidates in the outputted ranking and one or more additional sets of scores for the candidate(s). Finally, the system updates the outputted ranking based on the updated first set of scores and the additional sets of scores during the session.

Abstract: In an example, one or more member profiles and corresponding elapsed times indicating, for each of the one or more member profiles, how long the corresponding member of a social networking service took to respond to a request for confidential data with a submission of confidential data are obtained. Then a first set of one or more features are extracted from the one or more member profiles. The first set of one or more features and corresponding elapsed times are fed into a machine learning algorithm to train a confidential data response time prediction model to output a predicted time to respond to a request for confidential data for a candidate member profile. A second set of one or more features are obtained from a candidate member profile and fed to the confidential data response time prediction model, outputting the predicted time to respond to a request for confidential data.

Abstract: In an example, one or more job postings, as well as corresponding confidential data values, are obtained from a social networking service. A first set of one or more features are extracted from the one or more job postings. The first set of one or more features and corresponding confidential data values are fed into a machine learning algorithm to train a confidential data value prediction model to output a predicted confidential data value for a candidate job posting. Then, the candidate job posting is obtained and a second set of one or more features are extracted from the candidate job posting. The extracted second set of one or more features is fed to the confidential data value prediction model, outputting the predicted confidential data value.

Abstract: Methods, systems, and computer programs are presented for searching job postings for a member of a social network based on transitions from educational institutions to companies. A method includes determining educational-company transition scores indicating a transition probability from educational institution to company. The method identifies jobs based on a search performed for a first member, with a profile including one or more educational institutions, each job associated with a respective company. A server determines a member-company transition score based on the educational-company transition scores of the educational institutions in the profile. For each job, a job affinity score is determined based on data of the job and the profile of the first member. The server ranks the jobs based on the member-company transition score of the company of the job and the job affinity score. Some of the ranked jobs are presented to the first member based on the ranking.

Abstract: A technical problem of multi-objective optimization of job applications redistribution in an online connection network system is addressed by incorporating monetary value of job applications as a signal into a ranker for ranking jobs with respect to a member profile in job search and recommendations. The monetary value of job applications is used in addition to the relevance signal and is determined by executing a machine learning model that accounts for the covariates that could impact monetary value of an application for a job.