Abstract: A data replication method can begin with the detection of an inconsistency between records of a target table and corresponding records of a source table of a relational database management system (RDBMS) performing a parallel apply replication by an improved data replication manager. The target table can be a copy of the source table, both of which include multiple unique constraints and indexes. A timeframe that encompasses the records of the target table having the inconsistency can be determined. The timeframe can utilize a commit timestamp or a log sequence number. Consistency between the target table and the source table can be automatically restored for the determined timeframe through use of a reactive-apply process. Data suppression for updates is automatically restored once the copy is consistent. Transactions performed upon the target table by the reactive-apply process can be performed in parallel. Service at the source table and the target table can be uninterrupted.

Abstract: Provided are techniques for replicating relational transactional log data to a big data platform. Change records contained in change data tables are fetched. A relational change history with transaction snapshot consistency is rebuilt to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier. The consistent change records are stored on the big data platform, and queries are answered on the big data platform using the consistent change records.

Abstract: Provided are techniques for replicating relational transactional log data to a big data platform. Change records contained in change data tables are fetched. A relational change history with transaction snapshot consistency is rebuilt to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier. The consistent change records are stored on the big data platform, and queries are answered on the big data platform using the consistent change records.

Abstract: A method for verifying data consistency between update-in-place data structures and append-only data structures containing change histories associated with the update-in-place data structures is provided. The method includes loading data from an update-in-place data structure to a first set of hash buckets in a processing platform, loading data from append-only data structures to a second set of hash buckets in the processing platform, performing a bucket-level comparison between the data in the first set of hash buckets and the data in the second set of has buckets, and generating a report based on the bucket-level comparison.

Abstract: An integrated data processing system with two-tier data caching system and techniques for use thereof in a hybrid RDBMS and BDS computing environment are provided. In one aspect, the system is RDBMS-centric and uses two caches, one on the RDBMS side (1st tier) and the other on the BDS side (2nd tier). In another aspect, a DRDA wrapper on the BDS side enables the RDBMS to communicate with the BDS as if the BDS is another RDBMS. This is advantageous because the RDBMS already supports the DRDA protocol standard. In yet another aspect, the DRDA wrapper performs the data transformation needed when transferring cached objects between the RDBMS cache and BDS cache because RDBMS and BDS save data objects in different formats. This is advantageous because it offloads the computation from RDBMS to BDS therefore reducing the performance impact on RDBMS for its normal query and transaction processing.

Abstract: A system for transferring data from a Relational Database Management System (“RDBMS”) to a big data platform and methods for making and using the same. The system can acquire a partitioning execution scheme of a selected table from the RDBMS and submitting partitioned queries from the big data platform to each mapper of partitions. The partitioned queries are generated based on the partitioning execution scheme. The partitioning execution scheme can be acquired by submitting a query explain request to an optimizer of the RDBMS to generating a parallel query plan. The partitioning execution scheme can also be acquired by querying statistics from a statistics catalog of the RDBMS or by user inputs. The system can use RDBMS capabilities and statistics for parallel data fetching. Thereby, the system can increase efficiency of the fetching and can avoid straggling when target data is not evenly distributed and can avoid table query-in-serial.

Abstract: A system for providing reliable availability of a general workload and continuous availability of a priority workload over long distances may include a first computing site configured to execute a first instance associated with the priority workload, wherein the first instance is designated as an active instance, a second computing site configured to execute a second instance of the priority workload, wherein the second instance is designated as a standby instance, a third computing site configured to restart a third instance associated with the general workload, and a workload availability module configured to synchronize a portion of data associated with the third instance with a corresponding portion of data associated with the second instance.

Abstract: A method for verifying data consistency between update-in-place data structures and append-only data structures containing change histories associated with the update-in-place data structures is provided. The method includes loading data from an update-in-place data structure to a first set of hash buckets in a processing platform, loading data from append-only data structures to a second set of hash buckets in the processing platform, performing a bucket-level comparison between the data in the first set of hash buckets and the data in the second set of has buckets, and generating a report based on the bucket-level comparison.

Abstract: Several replication subscriptions are defined for a table in a database management system. The table is divided into partitions. Each replication subscription replicates transactions to a range of partitions. Subscriptions are assignable to different consistency groups. Transaction consistency is preserved at the consistency group level. A persistent delay table is created for each of several apply functions. Each apply function processes replication subscriptions for one consistency group, to replicate the table to a target table in parallel. Transactions for a given range of a partition are executed in parallel. When an apply function upon a row of the target table results in an error, the row is stored in the delay table. Application of each row in the delay table is repeatedly retried, and if successful, the row is removed from the delay table.

Abstract: Provided are techniques for transaction consistency query support for replicated data from recovery log to external data stores. An external data store is populated with records using entries of a change data table. The change data table has entries for each transaction that has committed and is to be replicated, and each of the entries stores information for each log entry in a recovery log from a database management system. Each log entry identifies a transactional change of data and a transaction completion indicator of one of commit and abort. In response to receiving a query about a transaction of the transactions, a set of records are retrieved from the external data store for the transaction. From the set of records, records whose sequence identifier values are larger than a maximum transaction commit sequence identifier are removed. From the set of records, remaining records having transaction consistency are returned.

Abstract: Provided are techniques for replicating relational transactional log data to a big data platform. Change records contained in change data tables are fetched. A relational change history with transaction snapshot consistency is rebuilt to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier. The consistent change records are stored on the big data platform, and queries are answered on the big data platform using the consistent change records.

Abstract: A system, method and computer program product for enabling light weight table comparison with high-accuracy (high confidence) of tables where one is a copy of the other, which copy may be maintained synchronized by replication. The method performs database comparison using a sample-based, statistics-based, or materialized query tables-based approaches. The method first identifies a block comprising a sub-set of rows of data of a source database table and a corresponding block from a target database table, and obtains a statistical value associated with each block. Then the statistical values for the corresponding source and target block are compared and a consistency evaluation of source and target database is determined based on comparing results. Further methods enable a determination of the data as being persistent or not in manner that accounts for real-time data modifications to underlying source and target database tables while identified blocks are being compared.

Abstract: A system for providing reliable availability of a general workload and continuous availability of a priority workload over long distances may include a first computing site configured to execute a first instance associated with the priority workload, wherein the first instance is designated as an active instance, a second computing site configured to execute a second instance of the priority workload, wherein the second instance is designated as a standby instance, a third computing site configured to restart a third instance associated with the general workload, and a workload availability module configured to synchronize a portion of data associated with the third instance with a corresponding portion of data associated with the second instance.

Abstract: Provided are techniques for replicating relational transactional log data to a big data platform. Change records contained in change data tables are fetched. A relational change history with transaction snapshot consistency is rebuilt to generate consistent change records by joining the change data tables and a unit of work table based on a commit sequence identifier. The consistent change records are stored on the big data platform, and queries are answered on the big data platform using the consistent change records.

Abstract: A system for providing reliable availability of a general workload and continuous availability of a priority workload over long distances may include a first computing site configured to execute a first instance associated with the priority workload, wherein the first instance is designated as an active instance, a second computing site configured to execute a second instance of the priority workload, wherein the second instance is designated as a standby instance, a third computing site configured to restart a third instance associated with the general workload, and a workload availability module configured to synchronize a portion of data associated with the third instance with a corresponding portion of data associated with the second instance.

Abstract: Techniques for restoring point-in-time and transaction consistency across consistency groups between a first and a second independent database management system (DBMS) for a disaster recovery. Several consistency groups (CGs) are defined for replication. For each CG in the first DBMS data changes are transmitted to a second DBMS. A timestamp representing a most recently received commit log record or a heartbeat during periods of inactivity for a CG is stored in a database table at regular intervals. At regular intervals, the timestamp is compared with timestamps for other CGs to identify a common time at which data to be applied to the CGs in the second DBMS have been received into a recoverable data store. The received data is applied to the CGs in the second DBMS up to the common time.

Abstract: Techniques for restoring point-in-time and transaction consistency across consistency groups between a first and a second independent database management system (DBMS) for a disaster recovery. Several consistency groups (CGs) are defined for replication. For each CG in the first DBMS data changes are transmitted to a second DBMS. A timestamp representing a most recently received commit log record or a heartbeat during periods of inactivity for a CG is stored in a database table at regular intervals. At regular intervals, the timestamp is compared with timestamps for other CGs to identify a common time at which data to be applied to the CGs in the second DBMS have been received into a recoverable data store. The received data is applied to the CGs in the second DBMS up to the common time.

Abstract: A data replication method can begin with the detection of an inconsistency between records of a target table and corresponding records of a source table of a relational database management system (RDBMS) performing a parallel apply replication by an improved data replication manager. The target table can be a copy of the source table, both of which include multiple unique constraints and indexes. A timeframe that encompasses the records of the target table having the inconsistency can be determined. The timeframe can utilize a commit timestamp or a log sequence number. Consistency between the target table and the source table can be automatically restored for the determined timeframe through use of a reactive-apply process. Data suppression for updates is automatically restored once the copy is consistent. Transactions performed upon the target table by the reactive-apply process can be performed in parallel. Service at the source table and the target table can be uninterrupted.

Abstract: Provided are techniques for transaction consistency query support for replicated data from recovery log to external data stores. An external data store is populated with records using entries of a change data table. The change data table has entries for each transaction that has committed and is to be replicated, and each of the entries stores information for each log entry in a recovery log from a database management system. Each log entry identifies a transactional change of data and a transaction completion indicator of one of commit and abort. In response to receiving a query about a transaction of the transactions, a set of records are retrieved from the external data store for the transaction. From the set of records, records whose sequence identifier values are larger than a maximum transaction commit sequence identifier are removed. From the set of records, remaining records having transaction consistency are returned.

Abstract: Techniques are disclosed to determine differences between a source table and a target table in a database environment, as being persistent or transient. A first set of differences between the source table and the target table is determined at a first point in time. A second set of differences between the source table and the target table is determined at a second point in time subsequent to the first point in time. At least one of a set of persistent differences and a set of transient differences is determined. The set of persistent differences includes a set intersection of the first and second sets of differences, the set intersection being filtered based on matching non-key values of the differences. The set of transient differences includes a relative complement of the second set of differences in the first set of differences.