Abstract: Method and system are provided for writing output from map-reduce jobs to an archive file. The method may include providing an archive manager and exposing an interface to be called from map-reduce jobs to output to an archive file in a map-reduce distributed file system. The method may also include using a buffering database as a temporary cache to buffer updates to the archive file. Handling by the archive manager calls from map-reduce jobs may allow: reading directly from an archive file or from a job index at the buffering database; writing to a job index at the buffering database used as a temporary cache to buffer updates; and serializing updates from the buffering database to the archive file.

Abstract: Method and systems for parallel scoring an ensemble model are provided. Aspects include loading data into a first distributed data structure having a plurality of partitions, each partition having loaded data in the form of a set of pairs of data formed of a record to be scored and a partial score for that record. A component model in the ensemble model is selected and processing of the records carried out in parallel across the partitions including updating the partial score for each record. In response to a partial score for a record not meeting an accuracy threshold, the method retains the record in the first distributed data structure to be scored by a subsequent component model. In response to the partial score for a record meeting the accuracy threshold, the method moves the record and updated partial score to an output result data structure to provide a final score.

Abstract: Method and systems for parallel scoring an ensemble model are provided. Aspects include loading data into a first distributed data structure having a plurality of partitions, each partition having loaded data in the form of a set of pairs of data formed of a record to be scored and a partial score for that record. A component model in the ensemble model is selected and processing of the records carried out in parallel across the partitions including updating the partial score for each record. In response to a partial score for a record not meeting an accuracy threshold, the method retains the record in the first distributed data structure to be scored by a subsequent component model. In response to the partial score for a record meeting the accuracy threshold, the method moves the record and updated partial score to an output result data structure to provide a final score.

Abstract: Method and system are provided for writing output from map-reduce jobs to an archive file. The method may include providing an archive manager and exposing an interface to be called from map-reduce jobs to output to an archive file in a map-reduce distributed file system. The method may also include using a buffering database as a temporary cache to buffer updates to the archive file. Handling by the archive manager calls from map-reduce jobs may allow: reading directly from an archive file or from a job index at the buffering database; writing to a job index at the buffering database used as a temporary cache to buffer updates; and serializing updates from the buffering database to the archive file.

Abstract: A method of generating synthetic data from a model of a dataset comprises the steps of receiving a model of a dataset, extracting information from the received model, constructing a database view from the extracted information, receiving a query to the constructed database view, and generating synthetic data from the constructed database view according to the received query.

Abstract: Method and system are provided for writing output from map-reduce jobs to an archive file. The method may include providing an archive manager and exposing an interface to be called from map-reduce jobs to output to an archive file in a map-reduce distributed file system. The method may also include using a buffering database as a temporary cache to buffer updates to the archive file. Handling by the archive manager calls from map-reduce jobs may allow: reading directly from an archive file or from a job index at the buffering database; writing to a job index at the buffering database used as a temporary cache to buffer updates; and serializing updates from the buffering database to the archive file.

Abstract: Method and systems for parallel scoring an ensemble model are provided. Aspects include loading data into a first distributed data structure having a plurality of partitions, each partition having loaded data in the form of a set of pairs of data formed of a record to be scored and a partial score for that record. A component model in the ensemble model is selected and processing of the records carried out in parallel across the partitions including updating the partial score for each record. In response to a partial score for a record not meeting an accuracy threshold, the method retains the record in the first distributed data structure to be scored by a subsequent component model. In response to the partial score for a record meeting the accuracy threshold, the method moves the record and updated partial score to an output result data structure to provide a final score.

Abstract: Method and systems for parallel scoring an ensemble model are provided. Aspects include loading data into a first distributed data structure having a plurality of partitions, each partition having loaded data in the form of a set of pairs of data formed of a record to be scored and a partial score for that record. A component model in the ensemble model is selected and processing of the records carried out in parallel across the partitions including updating the partial score for each record. In response to a partial score for a record not meeting an accuracy threshold, the method retains the record in the first distributed data structure to be scored by a subsequent component model. In response to the partial score for a record meeting the accuracy threshold, the method moves the record and updated partial score to an output result data structure to provide a final score.

Abstract: Method and system are provided for writing output from map-reduce jobs to an archive file. The method may include providing an archive manager and exposing an interface to be called from map-reduce jobs to output to an archive file in a map-reduce distributed file system. The method may also include using a buffering database as a temporary cache to buffer updates to the archive file. Handling by the archive manager calls from map-reduce jobs may allow: reading directly from an archive file or from a job index at the buffering database; writing to a job index at the buffering database used as a temporary cache to buffer updates; and serializing updates from the buffering database to the archive file.

Abstract: A method of generating synthetic data from a model of a dataset comprises the steps of receiving a model of a dataset, extracting information from the received model, constructing a database view from the extracted information, receiving a query to the constructed database view, and generating synthetic data from the constructed database view according to the received query.