Abstract: Off-policy evaluation of a new “target” policy is performed using historical data gathered based on a previous “logging” policy to estimate the performance of the target policy. An estimator may be used, wherein either a quality-based estimator or a quality-agnostic estimator is used to weight the difference between an observed reward in the historical data and an estimated reward generated by the target policy. A quality-agnostic estimator may be used to evaluate an importance weight according to a threshold. In such examples, when the importance weight exceeds the threshold, the quality-agnostic estimator clips the importance weight at the threshold, thereby providing an fixed upper bound irrespective of the quality of the reward predictor. In other examples, a quality-based estimator is used, in which an upper bound incorporates the quality of the reward predictor in order to modify an importance weight used by the estimator.

Abstract: Off-policy evaluation of a new “target” policy is performed using historical data gathered based on a previous “logging” policy to estimate the performance of the target policy. An estimator may be used, wherein either a quality-based estimator or a quality-agnostic estimator is used to weight the difference between an observed reward in the historical data and an estimated reward generated by the target policy. A quality-agnostic estimator may be used to evaluate an importance weight according to a threshold. In such examples, when the importance weight exceeds the threshold, the quality-agnostic estimator clips the importance weight at the threshold, thereby providing an fixed upper bound irrespective of the quality of the reward predictor. In other examples, a quality-based estimator is used, in which an upper bound incorporates the quality of the reward predictor in order to modify an importance weight used by the estimator.

Abstract: Off-policy evaluation of a new “target” policy is performed using historical data gathered based on a previous “logging” policy to estimate the performance of the target policy. An estimator may be used, wherein either a quality-based estimator or a quality-agnostic estimator is used to weight the difference between an observed reward in the historical data and an estimated reward generated by the target policy. A quality-agnostic estimator may be used to evaluate an importance weight according to a threshold. In such examples, when the importance weight exceeds the threshold, the quality-agnostic estimator clips the importance weight at the threshold, thereby providing an fixed upper bound irrespective of the quality of the reward predictor. In other examples, a quality-based estimator is used, in which an upper bound incorporates the quality of the reward predictor in order to modify an importance weight used by the estimator.

Abstract: A computing device can determine a decomposition of data of actions of a first session based at least in part on a first computational model associating the actions of the first session with corresponding state values of the first session. The computing device can determine a second computational model based at least in part on the decomposition and an operation template. The computing device can receive a query via the communications interface, the query associated with the second session. The computing device can determine a state value of the second session based at least in part on the query. The computing device can operate the second computational model to determine at least one response associated with the query based at least in part on the state value of the second session. The computing device can provide an indication of the at least one response via the communications interface.