Abstract: A method, system and non-transitory computer-readable medium for operating a storage engine are disclosed. With respect to a data file, a compaction value is obtained and a counter is retrieved. The compaction value indicates the minimum number of valid records in the data file for a clean-up process to be initiated. In response to the counter satisfying a first criterion, a determination is made, for each record in the data file, as to whether the record is written to another data file based on the record satisfying a second criterion. A write amplification factor of the storage engine is determined based on the compaction value. The counter indicates the number of records in the data file that are invalid. The write amplification factor and a space amplification factor are configurable; the storage engine may be tuned based on workloads, desired write throughput, desired storage utilization, and bandwidth of a storage device.

Abstract: The present teaching relates to a method, system and programming for operating a storage engine. A compaction value to be associated with a data file is obtained and a counter corresponding to the data file is retrieved. In response to the counter satisfying a first criterion, a determinization is made for each record included in the data file, as to whether the record is to be written to another data file based on the record satisfying a second criterion, wherein a write amplification factor of the storage engine is determined based on the obtained compaction value.