Abstract: The disclosed embodiments provide a system for managing a data store. During operation, the system stores a set of pending updates to a data store in a registry. Next, the system executes an asynchronous process that applies a first subset of updates from the registry as writes to records in the data store without blocking processing of read queries of the data store. Upon completing a write by the asynchronous process at a second portion of the data store, the system updates the registry with an indication of the completed write at the second portion of the data store. During processing of a read query of the data store, the system applies a second subset of updates from the registry to a result of the read query. Finally, the system returns the result in a response to the read query.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a user-defined function (UDF) that is defined using a standard UDF interface for accessing multiple platforms. Next, the system converts data in a platform into a standard form that is supported by the standard UDF interface using a platform-specific wrapper for the UDF and a platform-specific implementation of the standard UDF interface. The system then generates output data in the standard form by applying one or more operations to the data in the standard form. Finally, the system converts the output data into a platform-specific form that is supported by the platform and returns the output data in the platform-specific form.

Abstract: The disclosed embodiments provide a system for processing data. During operation, the system obtains a user-defined function (UDF) that is defined using a standard UDF interface for accessing multiple platforms. Next, the system converts data in a platform into a standard form that is supported by the standard UDF interface using a platform-specific wrapper for the UDF and a platform-specific implementation of the standard UDF interface. The system then generates output data in the standard form by applying one or more operations to the data in the standard form. Finally, the system converts the output data into a platform-specific form that is supported by the platform and returns the output data in the platform-specific form.

Abstract: Techniques are provided for dependency-aware transaction batching for data replication. A plurality of change records corresponding to a plurality of transactions is read. Inter-transaction dependency data is generated, the inter-transaction dependency data including at least one inter-transaction dependency relationship between a plurality of pending transactions. Each inter-transaction dependency relationship indicates that a first transaction is dependent on a second transaction. A batch transaction is generated based on the inter-transaction dependency data. The batch transaction includes at least one pending transaction of the plurality of pending transactions. The batch transaction is assigned to an apply process of a plurality of apply processes configured to apply batch transactions in parallel.

Abstract: Parallel logical replication involves multiple apply threads running on a destination database server applying, in parallel, changes made by source transactions, where the changes of a single source transaction may be applied in parallel by multiple apply threads. An apply transaction for a source transaction may be committed by an apply thread independently of the commitment of any other apply transaction of the source transaction, that is, without coordinating the committing of another apply transaction executed by another apply thread for the source transaction. A configuration language is used to configure parallel logical replication. The language facilitates the configuration of various aspects of parallel logical replication, including the number of apply threads, partitioning schemes for the apply threads for partitioning change records between the apply threads, and various other aspects of parallel logical replication.

Abstract: Techniques are provided for dependency-aware transaction batching for data replication. A plurality of change records corresponding to a plurality of transactions is read. Inter-transaction dependency data is generated, the inter-transaction dependency data including at least one inter-transaction dependency relationship between a plurality of pending transactions. Each inter-transaction dependency relationship indicates that a first transaction is dependent on a second transaction. A batch transaction is generated based on the inter-transaction dependency data. The batch transaction includes at least one pending transaction of the plurality of pending transactions. The batch transaction is assigned to an apply process of a plurality of apply processes configured to apply batch transactions in parallel.

Abstract: Parallel logical replication involves multiple apply threads running on a destination database server applying, in parallel, changes made by source transactions, where the changes of a single source transaction may be applied in parallel by multiple apply threads. An apply transaction for a source transaction may be committed by an apply thread independently of the commitment of any other apply transaction of the source transaction, that is, without coordinating the committing of another apply transaction executed by another apply thread for the source transaction. A configuration language is used to configure parallel logical replication. The language facilitates the configuration of various aspects of parallel logical replication, including the number of apply threads, partitioning schemes for the apply threads for partitioning change records between the apply threads, and various other aspects of parallel logical replication.