Abstract: A method and apparatus for processing (transporting) data, such as in a data warehouse system. In one embodiment, the data are received from a source and compared to data in a lookup cache comprising a subset of data from a first data set (e.g., a dimension table). Instances of the data not present in a lookup cache (that is, new data) are identified. Information corresponding to these instances are generated (e.g., a unique identifier is associated with each of these instances), and the first data set is updated accordingly. The lookup cache is then updated with the new data and the unique identifiers. Accordingly, the information (data) in the lookup cache and in the first data set are in synchronization. The lookup cache does not need to be rebuilt (e.g., to update a second data set such as a fact table).

Abstract: A method of populating multiple data marts in a single operation from a set of transactional data held in a database in a single aggregation process, in which aggregate values are calculated only once, a determination is made as to which output data marts required the aggregate value, and the aggregate values are output to the appropriate data marts. Dimension data associated with the output aggregate records is also output to the appropriate data marts.

Abstract: An aggregation engine for a data warehouse which provides an indexing technique which allows the measures in a fact table data entry to be added to the appropriate aggregate bucket by mapping the each of the dimension keys to an aggregate index within a level in that dimension and then calculating an overall index using the aggregate index from each dimension which is then mapped onto the aggregate bucket in question. A rolling cache is also provided allowing frequently or recently used aggregate buckets to be represented in memory rather than in a file, and merged with the equivalent bucket in an aggregate file when necessary, so that the slower access to the address file and the aggregate files can be avoided.

Abstract: A method of populating multiple data marts in a single operation from a set of transactional data held in a database in a single aggregation process, in which aggregate values are calculated only once, a determination is made as to which output data marts required the aggregate value, and the aggregate values are output to the appropriate data marts. Dimension data associated with the output aggregate records is also output to the appropriate data marts.

Abstract: A method of populating multiple data marts in a single operation from a set of transactional data held in a database in a single aggregation process, in which aggregate values are calculated only once, a determination is made as to which output data marts required the aggregate value, and the aggregate values are output to the appropriate data marts. Dimension data associated with the output aggregate records is also output to the appropriate data marts.

Abstract: A method of defining aggregate levels to be used in aggregation in a data store having one or more dimensions. Levels are defined corresponding to attributes in the dimension, so that data can be aggregated into aggregates corresponding to values of those attributes. The invention provides for the definition of sub-levels which act as levels but define which detail entries in the associated dimension will contribute to the sub-level. The invention also provides for the definition of level groups. A level group can replace a level in a level cross-product and such a cross product is then expanded before aggregation into a set of cross products, each containing one of the level group entries.