Abstract: An optimal sample size for experiments targeting units having specific static criteria is determined, based on one or more iterations of a performed experiment. The sample size to use for subsequent experiments targeting units having the specific static criteria is calculated based on target effect of the performed experiment. Multiple iterations can be performed to refine the sample size. Each iteration uses the previously calculated sample size as a parameter. The calculated sample size is then used to determine an optimal treatment. Separate experiments using separate treatments are performed on separate group sets of the calculated sample size. For each separate performed experiment, a score is calculated for the corresponding specific treatment, taking into account actual effect size, statistical significance and homogeneity of effect. Depending upon the score, the given treatment can be accepted for production usage or discarded, or the experiment can be replicated to validate effect.

Abstract: An optimal sample size for experiments targeting units having specific static criteria is determined, based on one or more iterations of a performed experiment. The sample size to use for subsequent experiments targeting units having the specific static criteria is calculated based on target effect of the performed experiment. Multiple iterations can be performed to refine the sample size. Each iteration uses the previously calculated sample size as a parameter. The calculated sample size is then used to determine an optimal treatment. Separate experiments using separate treatments are performed on separate group sets of the calculated sample size. For each separate performed experiment, a score is calculated for the corresponding specific treatment, taking into account actual effect size, statistical significance and homogeneity of effect. Depending upon the score, the given treatment can be accepted for production usage or discarded, or the experiment can be replicated to validate effect.