Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of records from a set of inputs, with each record containing an entity key, a partition key, and one or more attribute-value pairs. For each attribute-value pair in the records, the system maps an attribute name in the attribute-value pair to a unique identifier for the attribute name and replaces the attribute name with the unique identifier. The system then identifies a subset of the records with a matching entity key and a matching partition key and merges the subset of the records into a single record that includes the matching entity key, the matching partition key, and a single field containing a list of attribute-value pairs from the subset of the records. Finally, the system provides the single record and the mapping for use in querying from a centralized source.