Abstract: Embodiments of the disclosed technologies are capable of calculating a boundary value N as a function of a parameter M; for each of M first-level partitions of a set of data records, building an index for use by a downstream application by (i) building N second-level partitions using the key; indexing the N second-level partitions to produce N micro-shards; determining a value of a number of tiers parameter, T, and, for each tier, a value of a partitions per merge parameter PMT, merging the N micro-shards using T tiers and, for each tier, PMT partitions per merge, distributed across a plurality of host machines; where M, N, T, and PMT are each a positive integer and a value of M is determined based on the downstream application.