Abstract: Methods and systems for mining large data sets using block model averaging techniques are provided. Example embodiments provide a Block Model Averaging System (“BMAS”), which enables users to build/train, test, deploy, and maintain predictive statistical models that can be used to gain knowledge from both static and dynamic data. In one embodiment, the BMAS incrementally builds predictive models from portions (blocks) of input data using block model averaging techniques, determines a voting population of the predictive models to use as components of an ensemble model, generates an ensemble model with these determined components, and deploys the generated ensemble model to input data to derive answers. One technique for determining the voting population is correctness; another is diversity of response.