Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for reducing the difference in performance of a model across groups and sub-groups within the same group of users with similar characteristics for providing digital components. Methods can include identifying, a loss function that generates a loss representing a measure of performance the model seeks to optimize during training. The loss function is modified by adding an additional term to the loss function. The model is trained using the modified loss function. A request for digital component is received that includes a user group identifier. The model generates one or more user characteristics based on which one or more digital components are selected and transmitted to the client device of the user.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for providing digital components to a client device. Methods can include assigning a temporary group identifier to a client device that identifies a particular group, from among a plurality different groups, that includes the client device based on a current period of user activity on the client device. A training set is generated for training a machine learning model that generates user characteristics. A request for digital component is received from the client device that includes the temporary group identifier currently assigned to the client device, a subset of activity features and one or more additional features that are based on the client device. The machine learning model generates one or more user characteristics based on which one or more digital components are selected and transmitted to the client device.