Abstract: Techniques for tuning a machine learning algorithm using automatically determined optimal hyperparameters are described. An exemplary method includes receiving a request to determine a search space for at least one hyperparameter of a machine learning algorithm; determining, according to the request, optimal hyperparameter values from the search space for at least the one hyperparameter of the machine learning algorithm based on an evaluation of hyperparameters from the same machine learning algorithm on different datasets; and tuning the machine learning algorithm using the determined optimal hyperparameter values for the at least one hyperparameter of the machine learning algorithm to generate a machine learning model.

Abstract: Hyperparameters for tuning a machine learning system may be optimized for fairness using Bayesian optimization with constraints for accuracy and bias. Hyperparameter optimization may be performed for a received training set and received accuracy and fairness constraints. Respective probabilistic models for accuracy and bias of the machine learning system may be initialized, then hyperparameter optimization may include iteratively identifying respective values for hyperparameters using analysis of the respective models performed using an acquisition function implementing constrained expected improvement on the respective models, training the machine learning system using the identified values to determine measures of accuracy and bias, and updating the respective models using the determined measures.