Abstract: Mechanisms are provided for replicating transactions at a source database node on a target database node of the data processing system. A transaction message, for a transaction, is stored in a receive queue data structure of the target database node. An agent thread applies a change to data of the target database node based on a specification of the change in the transaction message. An identifier of the transaction message is stored in a done message data structure of the target database node and the identifier of the transaction message is also stored in a prune queue data structure of the target database node. A prune thread determines if a contiguous range of identifiers of transaction messages equal to or greater than a predetermined batch size is present in the prune queue data structure. If so, then a range of transaction messages is deleted from the done message data structure.

Abstract: A computer readable medium encoded with a computer program for handling transaction messages in asynchronous data replication in a database system is disclosed. The computer program provides a high speed parallel apply of transactional changes to a target node such that the parallel nature of the application of changes does not compromise the integrity of the data. The computer program detects, tracks, and handles dependencies between transaction messages to be applied to the target node. If a transaction message has a dependency on one or more preceding transaction messages whose applications have not yet completed, that transaction message is held until the application completes. In addition, the computer program requires significantly less overhead than conventional approaches.

Abstract: Mechanisms are provided for replicating transactions at a source database node on a target database node of the data processing system. A transaction message, for a transaction, is stored in a receive queue data structure of the target database node. An agent thread applies a change to data of the target database node based on a specification of the change in the transaction message. An identifier of the transaction message is stored in a done message data structure of the target database node and the identifier of the transaction message is also stored in a prune queue data structure of the target database node. A prune thread determines if a contiguous range of identifiers of transaction messages equal to or greater than a predetermined batch size is present in the prune queue data structure. If so, then a range of transaction messages is deleted from the done message data structure.

Abstract: Techniques are disclosed for comparing database tables. In one embodiment, the database tables are partitioned. Queries are generated for retrieving each partition. For each generated query, a stored procedure is invoked, using the respective generated query as an input parameter to the stored procedure. The stored procedure is configured to generate a checksum based on the partition retrieved from executing the respective query. The application compares the generated checksums to determine if the partitions and/or tables are consistent.

Abstract: Techniques for initializing a connection in a peer-to-peer database replication environment. A message is received from an initiator node, indicating that the initiator node will begin replicating changes made to a particular data entity. One or more replicated changes relating to the database entity are received from the initiator node. If an updated instance of the data entity does not currently exist on a target node then one or more load operations are performed to create the updated instance. The stored replicated changes may then be processed against the updated instance of the database entity.

Abstract: Mechanisms are provided for replicating transactions at a source database node on a target database node of the data processing system. A transaction message, for a transaction, is stored in a receive queue data structure of the target database node. An agent thread applies a change to data of the target database node based on a specification of the change in the transaction message. An identifier of the transaction message is stored in a done message data structure of the target database node and the identifier of the transaction message is also stored in a prune queue data structure of the target database node. A prune thread determines if a contiguous range of identifiers of transaction messages equal to or greater than a predetermined batch size is present in the prune queue data structure. If so, then a range of transaction messages is deleted from the done message data structure.

Abstract: Mechanisms are provided for replicating transactions at a source database node on a target database node of the data processing system. A transaction message, for a transaction, is stored in a receive queue data structure of the target database node. An agent thread applies a change to data of the target database node based on a specification of the change in the transaction message. An identifier of the transaction message is stored in a done message data structure of the target database node and the identifier of the transaction message is also stored in a prune queue data structure of the target database node. A prune thread determines if a contiguous range of identifiers of transaction messages equal to or greater than a predetermined batch size is present in the prune queue data structure. If so, then a range of transaction messages is deleted from the done message data structure.

Abstract: Provided are a method, system and program for batching transactions to apply to a database. A transaction manager receives a plurality of transactions to apply to a database and adds the transactions to a work queue. One of a plurality of agents processes a batch of transactions from the work queue. Each agent executes a batch of transactions from the work queue against the database independent of the operations of other of the agents executing batches of transactions. The agent applies the transactions in the batch against the database and commits the transactions in the batch to the database in response to completing all of the transactions.

Abstract: Techniques for initializing a connection in a peer-to-peer database replication environment. A message is received from an initiator node, indicating that the initiator node will begin replicating changes made to a particular data entity. One or more replicated changes relating to the database entity are received from the initiator node. If an updated instance of the data entity does not currently exist on a target node then one or more load operations are performed to create the updated instance. The stored replicated changes may then be processed against the updated instance of the database entity.

Abstract: A first log reader publishes first messages to a plurality of queues. The messages comprise changes for transactions extracted from a log by the first log reader. In response to one of the queues becoming unavailable, a second log reader is launched to read and extract from the log, and to publish second messages comprising changes for transactions extracted from the log by the second log reader to the unavailable queue as a catch-up queue. In response to the catch-up queue becoming available and the second log reader reaching the end of the log, the publishing of the second messages for the catch-up queue is transferred from the second log reader to the first log reader.

Abstract: A system and method to ensure a low-latency read of log records from a Database Management System (“DBMS”) in asynchronous log-based database replication capture from a blocking log read Application Programming Interface (“API”). The system may include a replication server with a log read module to initialize a log read thread. The reading module of the log read thread may read a predefined number of DBMS log records. A log flush module of the log flush thread may be executed by the log read thread and waits for a read complete notification. The update module, in response to a log flush thread time out, may also generate a plurality of loggable transactions such that a number of log records in an internal API log satisfies a DBMS threshold and unblocks the DBMS API. Therefore, the latency time may be constant, low, and predictable ensuring an accurate replication capture.

Abstract: A system and method to ensure a low-latency read of log records from a Database Management System (“DBMS”) in asynchronous log-based database replication capture from a blocking log read Application Programming Interface (“API”). The system may include a replication server with a log read module to initialize a log read thread. The reading module of the log read thread may read a predefined number of DBMS log records. A log flush module of the log flush thread may be executed by the log read thread and waits for a read complete notification. The update module, in response to a log flush thread time out, may also generate a plurality of loggable transactions such that a number of log records in an internal API log satisfies a DMBS threshold and unblocks the DBMS API. Therefore, the latency time may be constant, low, and predictable ensuring an accurate replication capture.

Abstract: A first log reader publishes first messages to a plurality of queues. The messages comprise changes for transactions extracted from a log by the first log reader. In response to one of the queues becoming unavailable, a second log reader is launched to read and extract from the log, and to publish second messages comprising changes for transactions extracted from the log by the second log reader to the unavailable queue as a catch-up queue. In response to the catch-up queue becoming available and the second log reader reaching the end of the log, the publishing of the second messages for the catch-up queue is transferred from the second log reader to the first log reader.

Abstract: A first log reader publishes first messages to a plurality of queues. The first messages comprise changes for transactions extracted from a log by the first log reader. In response to one of the queues becoming unavailable, launching a second log reader to read and extract from the log, and to publish second messages comprising changes for transactions extracted from the log by the second log reader to the unavailable queue as a catch-up queue. In response to the catch-up queue becoming available and the second log reader reaches the end of the log, the publishing of the second messages for the catch-up queue is transferred from the second log reader to the first log reader.

Abstract: An improved method for providing parallel apply in asynchronous data replication in a database system is disclosed. The improved method and system provides a high speed parallel apply of transactional changes to a target node such that the parallel nature of the application of changes does not compromise the integrity of the data. The method and system detects, tracks, and handles dependencies between transaction messages to be applied to the target node. If a transaction message has a dependency on one or more preceding transaction messages whose applications have not yet completed, that transaction message is held until the application completes. In addition, the method and system requires significantly less overhead than conventional approaches.

Abstract: A computer readable medium encoded with a computer program for handling transaction messages in asynchronous data replication in a database system is disclosed. The computer program provides a high speed parallel apply of transactional changes to a target node such that the parallel nature of the application of changes does not compromise the integrity of the data. The computer program detects, tracks, and handles dependencies between transaction messages to be applied to the target node. If a transaction message has a dependency on one or more preceding transaction messages whose applications have not yet completed, that transaction message is held until the application completes. In addition, the computer program requires significantly less overhead than conventional approaches.

Abstract: An automatic initial data load to a new table copy concurrently with active replication to existing table copies in an asynchronous replication group includes: initiating a data load of the new table copy; creating spill queue(s) at the new table copy; loading the data from a source table copy to the new table copy, where changes for the active table copies are applied during the loading, but changes to the new table copy are stored in a spill storage area; applying the changes in the spill storage area to the new table copy after the loading of the data is done; and removing the spill storage area after the changes have been applied to the new table copy. Thus, a new table copy is initialized into the replication group without requiring that replication be suspended at the source or any of the existing table copies during the initializing process.

Abstract: A method for handling transaction messages in asynchronous data replication in a database system, in which the database system includes a source node and a target node, and each transaction message has information concerning a row change to a table copy at the source node. The method includes determining whether a first transaction message has a dependency on a preceding non-completed transaction message. Responsive to the first transaction message having a dependency on the preceding non-completed transaction, the method further includes holding the first transaction message, completing the preceding non-completed transaction message including applying the row change associated with the preceding non-completed transaction message to the table copy at the target node, and responsive to completing the preceding non-completed transaction message, releasing the first transaction message and applying the row change associated with the first transaction message to the table copy at the target node.

Abstract: A method, apparatus, and article of manufacture implementing the method, transparently archives data. One or more data sources store data. A first data mover moves a first subset of data from the one or more data sources to a first federated archive in accordance with a first set of rules. In another aspect of the invention, a query is processed to retrieve a second subset of data, and a portion of the second subset of data is retrieved from the first federated archive. In yet another aspect of the invention, the processing of the query determines whether any data that would satisfy the query is stored in the first federated archive.

Abstract: A globally unique transaction identifier for each transaction is recorded in log entries pursuant to a two-phase commit protocol, wherein multiple log entries in multiple database recovery logs that correspond to one transaction have a same globally unique transaction identifier. A causally ordered, ascending timestamp value is maintained in the log entries pursuant to the two-phase commit protocol. For each of the multiple database recovery logs, all the log entries related to a single transaction are marked identically with a mark unique to the single transaction in a given one of the database recovery logs. Global changes are identified for the single transaction from the log entries in the multiple database recovery logs based on the marking and the globally unique transaction identifier for each transaction by reading log entries based on the causally ordered, ascending timestamp value.