Abstract: Systems and methods for event data processing and provisioning using a multi-dimensional database platform including an Apache Kafka framework and an auxiliary database are provided. Methods may include receiving an event profile from a data producer module, tagging the event profile with a timestamp, a unique event identification code, an event type descriptor, a data producer identifier, and a workflow identifier. The method may also include logging, in the Apache Kafka framework, the timestamp and the event identification code, as part of a sequential stream of events; and storing, in the auxiliary database, the event identification code, the event type descriptor, the data producer identifier, and the workflow identifier.

Abstract: Systems and methods for event data processing and provisioning using a multi-dimensional database platform including a first and second database are provided. Methods may include detecting an event and tagging the event with metadata. The method may also include logging the event in the first database along with a timestamp and an event identification code, as part of a sequential stream of events, and storing the event in the second database along with the event identification code and the metadata. The method may include receiving a request from a data consumer module for a customized workflow diagnostic and generating the customized workflow diagnostic based on the request and on data from the first and second databases as linked via the event identification code. The method may also include transmitting the customized workflow diagnostic to the data consumer module.

Abstract: Systems and methods for event data processing and provisioning using a multi-dimensional database platform including an Apache Kafka framework and an auxiliary database are provided. Methods may include receiving an event profile from a data producer module, tagging the event profile with a timestamp, a unique event identification code, an event type descriptor, a data producer identifier, and a workflow identifier. The method may also include logging, in the Apache Kafka framework, the timestamp and the event identification code, as part of a sequential stream of events; and storing, in the auxiliary database, the event identification code, the event type descriptor, the data producer identifier, and the workflow identifier.

Abstract: A data exchange that provides historical data indexed by date is provided. The data exchange may include a raw data layer, a model data layer, a delta staging layer, a delta database and a plurality of workspaces. The raw data layer may be a landing zone for raw data records. The model data layer may include modeled data records. The delta staging layer may be a landing zone for changed data. The changed data may correspond to changes made to the data records. The delta database may be divided into partitions. Each partition may hold data records that changed during a given time period. A plurality of data records may be continuously transferred from the raw data layer to both the model data layer and the delta staging layer. Once, during a predetermined time period, the contents of the delta staging layer may replace the contents of a partition.

Abstract: A data exchange that provides historical data indexed by date is provided. The data exchange may include a raw data layer, a model data layer, a delta staging layer, a delta database and a plurality of workspaces. The raw data layer may be a landing zone for raw data records. The model data layer may include modeled data records. The delta staging layer may be a landing zone for changed data. The changed data may correspond to changes made to the data records. The delta database may be divided into partitions. Each partition may hold data records that changed during a given time period. A plurality of data records may be continuously transferred from the raw data layer to both the model data layer and the delta staging layer. Once, during a predetermined time period, the contents of the delta staging layer may replace the contents of a partition.

Abstract: Methods for hybrid job processing may include receiving raw data records stored within a plurality of tables from a plurality of systems of record at a raw data layer within a data exchange. Methods may include generating, based on a data model, a list of dependencies between the plurality of tables. Each table included in a second subset of the plurality of tables may be dependent on at least one table included in a first subset of the plurality of tables. Methods may include processing the first subset of the plurality of tables concurrently with one another. The processing includes modeling the raw data records and transmitting the modeled data records to the model data layer. Methods may include processing each table included in the second subset after completion of processing of the table included in the first subset from which the table in the second subset depends on.

Abstract: A data exchange that provides self-service data provisioning is provided. The data exchange may include a raw data layer, a model data layer, a plurality of workspaces and a testing environment. The raw data layer may be a landing zone for raw data records received from systems of record. The raw data layer may receive a plurality of raw data records, model and process the data records and transfer the data records to the model data layer. The model data layer may be a data layer that includes data modeled to data exchange specifications and enables queries to be executed on the data included in the model data layer. Each workspace may be allocated to a consumer. The consumer may query the plurality of data records within the model data layer. The testing environment may test scripts to ensure that the scripts conform to a predetermined set of testing specifications.

Abstract: A data exchange that provides self-service data provisioning is provided. The data exchange may include a raw data layer, a model data layer, a plurality of workspaces and a testing environment. The raw data layer may be a landing zone for raw data records received from systems of record. The raw data layer may receive a plurality of raw data records, model and process the data records and transfer the data records to the model data layer. The model data layer may be a data layer that includes data modeled to data exchange specifications and enables queries to be executed on the data included in the model data layer. Each workspace may be allocated to a consumer. The consumer may query the plurality of data records within the model data layer. The testing environment may test scripts to ensure that the scripts conform to a predetermined set of testing specifications.