Abstract: Techniques for detecting label shift and adjusting training data of predictive models in response are provided. In an embodiment, a first machine-learned model is used to generate a predicted label for each of multiple scoring instances. The first machine-learned model is trained using one or more machine learning techniques based on a plurality of training instances, each of which includes an observed label. In response to detecting a shift in observed labels, for each segment of one or more segments in multiple segments, a portion of training data that corresponds to the segment is identified. For each training instance in a subset of the portion of training data, the training instance is adjusted. The adjusted training instance is added to a final set of training data. The machine learning technique(s) are used to train a second machine-learned model based on the final set of training data.

Abstract: In an example, for each feature of one or more features of a target sample data, feature values for one or more pseudo-samples are generated using, localized stratified sampling. The one or more pseudo-samples are fed into the trained machine learned model to obtain their prediction values. A piecewise linear regression model is trained using the one or more pseudo-samples and their prediction values, the piecewise linear regression model having two coefficients for each feature, a first coefficient describing prediction change when a corresponding feature value is increased and a second coefficient describing prediction change when a corresponding feature value is decreased. A top positive feature influencer is identified based on a feature of the one or more features of the target sample having a greatest magnitude of positive first coefficient or greatest magnitude of negative second coefficient.

Abstract: Techniques for detecting label shift and adjusting training data of predictive models in response are provided. In an embodiment, a first machine-learned model is used to generate a predicted label for each of multiple scoring instances. The first machine-learned model is trained using one or more machine learning techniques based on a plurality of training instances, each of which includes an observed label. In response to detecting a shift in observed labels, for each segment of one or more segments in multiple segments, a portion of training data that corresponds to the segment is identified. For each training instance in a subset of the portion of training data, the training instance is adjusted. The adjusted training instance is added to a final set of training data. The machine learning technique(s) are used to train a second machine-learned model based on the final set of training data.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system determines output of a machine learning model, which includes a score generated by the model based on features inputted into the model and feature importance metrics representing effects of the features on the score. Next, the system maps the features to elements in a feature hierarchy that groups the features under a first level of parent features. The system also generates a ranking of the first level of parent features based on the feature importance metrics. The system then combines, based on the ranking, feature values of the mapped features with a set of insight templates to produce a list of narrative insights, wherein each narrative insight includes a natural language description of a factor that contributes to the model's output. Finally, the system outputs the list of narrative insights in a user interface.

Abstract: In an example, for each feature of one or more features of a target sample data, feature values for one or more pseudo-samples are generated using, localized stratified sampling. The one or more pseudo-samples are fed into the trained machine learned model to obtain their prediction values. A piecewise linear regression model is trained using the one or more pseudo-samples and their prediction values, the piecewise linear regression model having two coefficients for each feature, a first coefficient describing prediction change when a corresponding feature value is increased and a second coefficient describing prediction change when a corresponding feature value is decreased. A top positive feature influencer is identified based on a feature of the one or more features of the target sample having a greatest magnitude of positive first coefficient or greatest magnitude of negative second coefficient.