Abstract: Methods, systems, and computer-readable media for offline evaluation of machine learning models with noise reduction are disclosed. A trigger computation system generates a plurality of experimental requests. The experimental requests do not represent unmodified requests received from clients in a production environment. At least one parameter value varies for individual ones of the experimental requests. The trigger computation system provides the experimental requests to a first machine learning model and a second machine learning model. The first machine learning model and the second machine learning model produce a set of results based at least in part on the experimental requests. The trigger computation system determines a reduced set of results for which the first machine learning model and the second machine learning model differ. An evaluation of the first machine learning model or the second machine learning model is performed using the reduced set of results.

Abstract: To reduce the computational resources used to generate a response, certain inputs from a first request and corresponding outputs are stored in a data structure that is returned to a client device concurrent with a response. When a subsequent request is received, if an input of the subsequent request matches an input stored in the data structure, the output that corresponds to that input is used in a response rather that recomputing the output. For inputs that do not match an input in the data structure, an output is computed and included in a response. The data structure is then returned to the client device concurrent with the subsequent response. Outputs in the data structure may be periodically validated, or may be associated with an expiration time, to ensure that the data structure includes current and valid data.