Abstract: A cross-region data distribution service enables eventually consistent replication of data objects between geographical regions while maintaining data ordering guarantees. For data replication between two regions, the system leverages a data stream-based replication model where destination region leverages a watch API exposed by origin regions. The origin region watch API provides a resumable data stream for changes happening in the origin region that need to be replicated to destination. The system exposes distribution ability through distribution policies which are global objects that encode the source datasets and desired distribution topology. In order to solve data fanout scalability challenges where same data needs to be replicated to many regions, system leverages caching at origin watch servers. The system maintains data ordering by enforcing ordered replication stream provided by watch servers and using version based conditional optimistic concurrency control writes in destination servers.

Abstract: Systems and methods provide a database-agnostic watch service using a write-ahead log (WAL)-based architecture. The system (e.g., a metadata service platform) receives a watch API call from a watch client to subscribe to receiving change notifications, whereby the watch API call includes filtering criteria. Subsequently, a mutation request is received from a client (e.g., a CRUDL client), that comprises a request to change, update, or delete an object in a database. The system writes the mutation request to a request log and then executes the mutation request. Next, the system generates a response message that includes the mutation request and an execution status. This response message is appended to the response log. Based on the response log, a watch server receives change events and generates and transmits the change notifications to the watch client based on the filtering criteria applied to the change events.

Abstract: Systems and methods provide low-latency, ordered delivery of database change events to a watch client. In response to a database (DB) create, update, or delete (CUD) request, the system writes the request to a database. The database usually provides a change data capture stream of events to a publish/subscribe (pub/sub) system. This results in higher change event delivery latency and is referred to as a slow path. The system leverages a dual write approach, where the system asynchronously writes a corresponding change event directly to a fast path pub/sub system. A watch server consumes change events from the slow and fast path pub/sub systems and merges and orders the change events using a first-in first-out cache. An ordered event stream can be transmitted to the watch client with lower latency. The ordered event stream includes one or more batches, and each batch comprises a most recent change event version for each object that was changed.

Abstract: Systems and methods provide for highly scalable real-time watch sessions. A watch server consumes change events from Kafka topics and a snapshot of a current state from a database. The change events and the snapshot are locally cached at the watch server, which removes a need to consume from the Kafka topics and database for every watch subscription of the watch server. The watch server matches the change events to watch subscriptions using a prefix tree that reflects hierarchical subscription filter criteria. The watch server signals a matching watch subscription with a signal that includes a maximum offset associated with the change events. In response to an API call from the matching watch subscription, the watch server provides the change events to the matching watch subscription. Multiple change events including the change event can be aggregated into a batch, which is transmitted as a data stream to a client associated with the matching watch subscription.

Abstract: Systems and methods provide low-latency, ordered delivery of database change events to a watch client. In response to a database (DB) create, update, or delete (CUD) request, the system writes the request to a database. The database usually provides a change data capture stream of events to a publish/subscribe (pub/sub) system. This results in higher change event delivery latency and is referred to as a slow path. The system leverages a dual write approach, where the system asynchronously writes a corresponding change event directly to a fast path pub/sub system. A watch server consumes change events from the slow and fast path pub/sub systems and merges and orders the change events using a first-in first-out cache. An ordered event stream can be transmitted to the watch client with lower latency. The ordered event stream includes one or more batches, and each batch comprises a most recent change event version for each object that was changed.

Abstract: A cross-region data distribution service enables eventually consistent replication of data objects between geographical regions while maintaining data ordering guarantees. For data replication between two regions, the system leverages a data stream-based replication model where destination region leverages a watch API exposed by origin regions. The origin region watch API provides a resumable data stream for changes happening in the origin region that need to be replicated to destination. The system exposes distribution ability through distribution policies which are global objects that encode the source datasets and desired distribution topology. In order to solve data fanout scalability challenges where same data needs to be replicated to many regions, system leverages caching at origin watch servers. The system maintains data ordering by enforcing ordered replication stream provided by watch servers and using version based conditional optimistic concurrency control writes in destination servers.

Abstract: According to various embodiments, techniques and mechanisms described herein facilitate the identification of fraudulent request messages. In one example, a request message associated with a user identifier is received at a request server in a request system that includes a plurality of such request servers. An estimated message count value associated with the user identifier is identified. The estimated message count value identifies an estimated number of messages associated with the user identifier that were previously received by the request servers. A response message is transmitted when it is determined that the estimated message count is below a designated threshold.