Abstract: A system represents data during a data cleansing application. The system includes a record collection. Each record in the collection includes a list of fields and data contained in each field. The system further includes a predetermined sequence of operations to be performed on the record collection and a plurality of bit-maps representing the record collection. The system still further includes a partitioned sequence of operations for parallel processing of the bit-maps by a plurality of separate devices.

Abstract: A system identifies similarities in data. The system includes a collection of records, a plurality of transform functions, and a cell list structure. Each record in the collection represents an entity and has a list of fields. Data is contained in each field. The plurality of transform functions operates upon the data in each field in each record. The plurality of transform functions generates a set of output values for facilitating comparison of the records and determining whether any of the records represent the same entity. The cell list structure is generated from the output values. The cell list structure has a list of cells for each field and a list of pointers to each cell of the list of cells for each output value generated by the plurality of transform functions.

Abstract: A system identifies similar records. The system includes a collection of records, a set of Boolean rules, and a cell list structure. Each record in the collection has a list of fields and data contained in each field. The set of Boolean rules operate upon the data in each field. The cell list structure is generated from the collection of records. The cell list structure has a list of cells for each field and a list of pointers to each cell of the list of cells for each record. The set of Boolean rules identifies the similar records from the cell list structure.