Abstract: Techniques are described for disaggregating learning curves by student mastery for refining and accurately evaluating automated tutoring models. A method comprises receiving performance data for users logging whether a correct response was provided for each opportunity to use a particular skill in a tutoring system, determining a plurality of subpopulations from the users by using the performance data to group by number of opportunities needed for the particular skill to reach a mastery threshold, creating disaggregated learning curves for each of the plurality of subpopulations that map performance opportunities to percentages correct, and evaluating the disaggregated learning curves to identify a suitable adaptation for the tutoring system. The suitable adaptation may then be carried out and may include sending a notification of portions of the tutoring system that need attention and/or adjusting parameters of the tutoring system for a projected learning progression of a particular user.