Abstract: Technologies are disclosed for determining network-accessible content changes based on computed models and providing a long term forecast of user interaction at a network accessible site based upon a short term experiment at the site. A forecast model for a period of time is generated based upon historical data of user interactions at the site. An experiment is run for a short term at the site based upon a potential change at the site. Based upon data obtained during the experiment, scores are generated for a control group (no change) and a treatment group (potential change) and compared. If there are statistically significant differences between the control group and the treatment group scores, the long term forecast may be used to forecast what the long term impact of the experiment would be based upon the short term experiment.

Abstract: Described are techniques for determining long-term effects of an experimental change to a user experience after the end of the experiment. A control state and a treatment state of a statistical hypothesis experiment may be assigned to first and second client devices, respectively, during an experiment time period. Subsequent to the end of the experiment, presentation of the control state may be discontinued. Result data corresponding to the treatment state may be determined during the experiment time period and for a length of time subsequent to the experiment time period. Result data corresponding to the control state may be determined during the experiment time period, and for a length of time prior to assignment of the control state to the first client device.

Abstract: A machine generated service cache that utilizes one or more machine learning classifiers is trained using service requests directed to a human-generated service and service responses generated by the human-generated service in response to the service requests. Once the machine generated service cache has been trained to a predetermined level of performance, the machine generated service cache can be utilized to process actual service requests directed to the human-generated service. The machine generated service cache might be utilized to process service requests for which it is not essential that the returned service response be identical to a response that would be generated by the human-generated service.