Abstract: Techniques for use with at least one processor configured to execute one or more MapReduce applications that cause the at least one processor to function as at least a Mapper in a MapReduce system include accessing data stored in a file system implemented on at least one nonvolatile storage medium. In response to input data being written to the file system by an application other than the one or more MapReduce applications, a set of one or more Map functions applicable to the input data may be accessed. At least one Map function of the one or more Map functions may be executed on the input data via the at least one processor functioning as at least the Mapper in the MapReduce system, and at least one set of [key, value] pairs resulting from execution of the at least one Map function on the received input data may be output.

Abstract: An automated auction-based method of determining price to execute one or more candidate programs on a parallel computing system is disclosed. The parallel computing system includes a plurality of computing resources, each having a price per unit of time. For each candidate program, a plurality of executions are performed using different amounts of computing resources. The number of program outputs completed during each execution is measured. A plurality of bids defining a price for completing a desired number of program outputs in a desired amount of time are received. The amount of computing resources required to fulfill each bid is determined. A price per unit of time for the computing resources for each bid is calculated based on the price associated with the bid and the determined amount of computing resources required to fulfill the bids. The bids are fulfilled based on the calculated price per unit of time.