Abstract: Techniques for continuous lock-minimal checkpointing and recovery with a distributed log-based datastore are described. A continuous, fault-tolerant checkpoint process is run on a node in a cluster in a compute-optimized or memory-optimized manner, thereby checkpointing the in-memory replica state of the node to a durable checkpoint datastore. A node can partially restore its replica state by obtaining checkpoint data, which includes an identifier of a low-water mark in a journal shard. The node can attach to the journal shard using the low-mater mark as the point of attachment, enabling the node to finalize the restoration of the replica state to be current.

Abstract: Techniques for multi-stream transactional event processing under ACID semantics in a distributed log-based append-only datastore are described. A transaction coordinator writes events that are part of a transaction to a transaction buffer, where the events can be made visible to clients involved in the transaction while other clients are not aware. Upon committing the transaction, an optimistic concurrency control based technique is utilized to attempt to obtain locks on all events involved in the transaction across one or multiple shards by one or multiple designated writer nodes. When all involved writer nodes indicate that they are able to commit their events, the transaction can be committed.