Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a first embedding generated by a topic model from parameters of searches by a first recruiting entity and obtains a set of additional embeddings generated by the topic model from attributes of a set of candidates. Next, the system determines match features that include measures of similarity between the first embedding and each embedding in the set of additional embeddings. The system then applies a machine learning model to the match features and additional features for the candidates to produce a set of scores for the candidates. Finally, the system generates a ranking of the candidates according to the scores and outputs at least a portion of the ranking as search results of a current search by the first recruiting entity.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of feature additions and an evaluation metric for assessing the performance of a statistical model. Next, the system automatically builds treatment versions of the statistical model using a set of baseline features for the statistical model and feature combinations generated using the feature additions. The system then uses a hypothesis test and a fixed set of feature values to compare a baseline value of the evaluation metric for a baseline version of the statistical model that is built using the set of baseline features with additional values of the evaluation metric for the treatment versions. Finally, the system outputs a result of the hypothesis test for use in assessing an impact of the feature combinations on a performance of the statistical model.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a first embedding generated by a topic model from parameters of searches by a first recruiting entity and obtains a set of additional embeddings generated by the topic model from attributes of a set of candidates. Next, the system determines match features that include measures of similarity between the first embedding and each embedding in the set of additional embeddings. The system then applies a machine learning model to the match features and additional features for the candidates to produce a set of scores for the candidates. Finally, the system generates a ranking of the candidates according to the scores and outputs at least a portion of the ranking as search results of a current search by the first recruiting entity.

Abstract: The disclosed embodiments provide a system for performing multi-level ranking for mitigating machine learning model bias. During operation, the system applies a machine learning model to features for qualified candidates that match parameters of a request to produce a first ranking of recommended candidates. Next, the system calculates a distribution of an attribute in the qualified candidates and generates a first reranking of recommended candidates that more accurately reflects the distribution of the attribute in the qualified candidates. The system then applies another machine learning model to the first reranking to produce a second ranking of recommended candidates and generates a second reranking of recommended candidates that more accurately reflects the distribution of the attribute in the qualified candidates. Finally, the system outputs at least a portion of the second reranking in a response to the request.

Abstract: The disclosed embodiments provide a system for quantifying machine learning model bias. During operation, the system obtains a set of qualified candidates that match parameters of a request. Next, the system obtains a ranking of recommended candidates outputted by a machine learning model after the qualified candidates are inputted into the machine learning model. The system then generates a first distribution of an attribute in the ranking of recommended candidates and a second distribution of the attribute in the qualified candidates. The system also calculates, based on the first and second distributions, a skew metric representing a difference between a first proportion of the attribute value in the ranking of recommended candidates and a second proportion of the attribute value in the qualified candidates. Finally, the system outputs the skew metric for use in evaluating bias in the machine learning model.

Abstract: The disclosed embodiments provide a system for achieving fairness across multiple attributes in a ranking. During operation, the system obtains a ranking of recommended candidates outputted by a machine learning model in response to a request. Next, the system obtains target proportions of multiple attribute values in the ranking of recommended candidates. The system then generates, based on the ranking, a set of attribute-specific rankings of recommended candidates, wherein each attribute-specific ranking includes candidates with a common attribute value. The system also generates, based on the attribute-specific rankings and one or more ranking criteria associated with the target proportions, a reranking of recommended candidates. Finally, the system outputs at least a portion of the reranking in a response to the request.

Abstract: The disclosed embodiments provide a system for performing calibration of response rates. During operation, the system obtains a position of a content item in a ranking of content items generated for delivery to a member of an online system and a predicted response rate by the member to the content item. Next, the system determines an updated response rate by the member to the content item based on the position of the content item in the ranking and dimensions associated with the predicted response rate and the ranking. The system then outputs the updated response rate for use in managing delivery of the content item.

Abstract: The disclosed embodiments provide a system that performs pacing for balanced delivery. During operation, the system obtains predicted response rates associated with impressions of a content item delivered within an online system and a cost per action (CPA) for the content item. Next, the system determines an impression-based spending for the content item based on the predicted response rates and the CPA. The system then calculates a pacing score for the content item based on the impression-based spending. Finally, the system adjusts subsequent interactions with the content item based on the pacing score.

Abstract: The disclosed embodiments provide a system for performing dynamic job bidding optimization. During operation, the system obtains historical data containing a time series of interactions with a job. Next, the system uses the historical data to calculate an initial price of a job based on a predicted number of interactions with the job. The system then determines a first dynamic adjustment to the initial price that improves utilization of a budget for the job and a second dynamic adjustment to the initial price that improves a performance of the job. Finally, the system applies the first and second adjustments to the initial price to produce an updated price for the job and delivers the job within an online system based on the updated price.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of feature additions and an evaluation metric for assessing the performance of a statistical model. Next, the system automatically builds treatment versions of the statistical model using a set of baseline features for the statistical model and feature combinations generated using the feature additions. The system then uses a hypothesis test and a fixed set of feature values to compare a baseline value of the evaluation metric for a baseline version of the statistical model that is built using the set of baseline features with additional values of the evaluation metric for the treatment versions. Finally, the system outputs a result of the hypothesis test for use in assessing an impact of the feature combinations on a performance of the statistical model.