Abstract: A method and apparatus for processing (transporting) data, such as in a data warehouse system. In one embodiment, the data are received from a source and compared to data in a lookup cache comprising a subset of data from a first data set (e.g., a dimension table). Instances of the data not present in a lookup cache (that is, new data) are identified. Information corresponding to these instances are generated (e.g., a unique identifier is associated with each of these instances), and the first data set is updated accordingly. The lookup cache is then updated with the new data and the unique identifiers. Accordingly, the information (data) in the lookup cache and in the first data set are in synchronization. The lookup cache does not need to be rebuilt (e.g., to update a second data set such as a fact table).

Abstract: A method for updating a target table of a data mart in response to changes made by a transaction to data stored in a source table of an operational database. Data that was changed in the source table by the transaction is stored in a dynamic image table of a change capture database. Data that was not changed in the source table by the transaction, but which is nevertheless required to be mapped to the target table, is stored in a static image table of the change capture database. The change capture database also contains relevant information regarding the transaction. Once the dynamic and static image tables are properly staged, the changes are propagated from the change capture database to the target tables of the data marts. In other words, data is extracted from the change capture database and subsequently transformed and loaded, thereby minimizing the impact to the operational database. Thereupon, the tables of the change capture database are truncated to discard data which is now no longer needed.