Abstract: Disclosed herein are system, method, and computer program product embodiments for disaster resilience of applications managed by Kubernetes operators. An embodiment operates by creating an orchestration and worker cluster, where the worker cluster is coupled to the orchestration cluster by a proxy server. Custom resources are deployed to the orchestration cluster and custom resource controllers are deployed to the worker cluster. The proxy server federates these custom resources between the orchestration cluster and the worker cluster. During disasters, the worker cluster is recreated and reconciled to prevent loss of the federated cluster.

Abstract: Disclosed herein are system, method, and computer program product embodiments for disaster resilience of applications managed by Kubernetes operators. An embodiment operates by creating an orchestration and worker cluster, where the worker cluster is coupled to the orchestration cluster by a proxy server. Custom resources are deployed to the orchestration cluster and custom resource controllers are deployed to the worker cluster. The proxy server federates these custom resources between the orchestration cluster and the worker cluster. During disasters, the worker cluster is recreated and reconciled to prevent loss of the federated cluster.

Abstract: A system includes reception of a first write request from a client including a first key and a first one or more stream, payload pairs associated with the first key, copying of the first key and the first one or more stream, payload pairs to a first buffer of a volatile memory, storage of data of the first buffer in one or more blocks of a raw block non-volatile memory device, providing of the first buffer to a stream store server, reception of the first buffer at the stream store server, adding of the first key and the first one or more stream, payload pairs to a second buffer of the volatile memory, in key-order, storage of the data of the second buffer in a filesystem storage device, according to stream, and transmission of an indication of the durability of the key to the tail store server.

Abstract: Disclosed herein are system, method, and computer program product embodiments for quorum-based replication of data records. In one embodiment, a read request for reading a record is received from a user node on a replica node of a cluster of replica nodes. The record is then determined not committed on the replica node. In response to the determination, an update message indicative of whether the number of replica nodes on which the record is durable exceeds a threshold is received on the replica node. In response to the number of replica nodes exceeds the threshold, a value of the record on the replica node is transmitted to the user node.

Abstract: A system includes writing of log data in chunks over a first range of heap log sequence positions of a heap log stored in the non-volatile memory system, and writing of a heap anchor marker in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log, wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

Abstract: A method includes a primary storage unit receiving a first write request including a first key and a first value; persisting the first value in a first non-volatile memory in association with the first key; broadcasting the first write request and a first set of globally-durable keys to secondary storage units; receiving, from the secondary storage units, an acknowledgement of the first write request and a first set of locally-durable keys, each of the first sets of locally-durable keys including the first key; the primary storage unit receiving a second write request including a second key and a second value; persisting the second value in the first non-volatile memory in association with the second key; and broadcasting the second write request and a second set of globally-durable keys to the secondary storage units, the second set of locally-durable keys including the first key. A system is also disclosed.

Abstract: A method and system, the system including a plurality of machines each having a processor and a main memory component; a shared distributed storage facility storing a set of data and accessible by the plurality of machines over a communication network; a controller to select, in response to a state of a query execution plan comprising a plurality of executable jobs for the set of data, which one of a set of scheduling algorithms to execute; an execution engine to execute the selected scheduling algorithm to determine, for each job in the plurality of jobs, which server to schedule to execute the respective job; and providing an indication of the scheduling of the servers determined to be schedules for the execution of the jobs.

Abstract: Disclosed herein are system, method, and computer program product embodiments for quorum-based replication of data records. In one embodiment, a read request for reading a record is received from a user node on a replica node of a cluster of replica nodes. The record is then determined not committed on the replica node. In response to the determination, an update message indicative of whether the number of replica nodes on which the record is durable exceeds a threshold is received on the replica node. In response to the number of replica nodes exceeds the threshold, a value of the record on the replica node is transmitted to the user node.

Abstract: Disclosed herein are system, method, and computer program product embodiments for distributed transaction processing in massively parallel processing (MPP) databases. An embodiment operates by receiving a transaction including a read timestamp corresponding to a version of data of a database associated with the query. It is determined whether a locally stored portion of the data of the database has a local timestamp that is equivalent to or after the read timestamp. A lock may be requested, and if received, the data is updated.

Abstract: A system and method include reception, at a primary storage unit, of a first write request from a client including a first key and a first value, persisting of the first value in a first non-volatile memory of the primary storage unit in association with the first key, broadcast of the first write request and a first set of globally-durable keys to the one or more secondary storage units, reception, from each of the one or more secondary storage units, of an acknowledgement of the first write request and a first set of locally-durable keys, each of the first sets of locally-durable keys including the first key, reception, at the primary storage unit, of a second write request from the client including a second key and a second value, persisting of the second value in the first non-volatile memory in association with the second key, and broadcast of the second write request and a second set of globally-durable keys to the one or more secondary storage units, the second set of locally-durable keys including the

Abstract: A system includes writing of log data in chunks over a first range of heap log sequence positions of a heap log stored in the non-volatile memory system, and writing of a heap anchor marker in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log, wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

Abstract: A system includes reception of a first write request from a client including a first key and a first one or more stream, payload pairs associated with the first key, copying of the first key and the first one or more stream, payload pairs to a first buffer of a volatile memory, storage of data of the first buffer in one or more blocks of a raw block non-volatile memory device, providing of the first buffer to a stream store server, reception of the first buffer at the stream store server, adding of the first key and the first one or more stream, payload pairs to a second buffer of the volatile memory, in key-order, storage of the data of the second buffer in a filesystem storage device, according to stream, and transmission of an indication of the durability of the key to the tail store server.

Abstract: Disclosed herein are system, method, and computer program product embodiments for accessing a distributed shared log that supports a total order over log writes and multiple logical log streams. An embodiment operates by selecting a range of log entries for a logical log stream from the distributed shared log and mapping the selected range of log entries to associated storage units in a plurality of storage units. The selected range of log entries are stored at the associated storage units according to metadata that indexes the selected range of log entries by logical log stream. The selected range of log entries are then requested from the associated storage units based on the mapping. The selected range of log entries for the logical log stream are then returned from the associated storage units based on a scan of the metadata. The received log entries are then reassembled.

Abstract: A method and system, the system including a plurality of machines each having a processor and a main memory component; a shared distributed storage facility storing a set of data and accessible by the plurality of machines over a communication network; a controller to select, in response to a state of a query execution plan comprising a plurality of executable jobs for the set of data, which one of a set of scheduling algorithms to execute; an execution engine to execute the selected scheduling algorithm to determine, for each job in the plurality of jobs, which server to schedule to execute the respective job; and providing an indication of the scheduling of the servers determined to be schedules for the execution of the jobs.

Abstract: Disclosed herein are system, method, and computer program product embodiments for accessing a distributed shared log that supports a total order over log writes and multiple logical log streams. An embodiment operates by selecting a range of log entries for a logical log stream from the distributed shared log and mapping the selected range of log entries to associated storage units in a plurality of storage units. The selected range of log entries are stored at the associated storage units according to metadata that indexes the selected range of log entries by logical log stream. The selected range of log entries are then requested from the associated storage units based on the mapping. The selected range of log entries for the logical log stream are then returned from the associated storage units based on a scan of the metadata. The received log entries are then reassembled.

Abstract: Disclosed herein are system, method, and computer program product embodiments for distributed transaction processing in massively parallel processing (MPP) databases. An embodiment operates by receiving a transaction including a read timestamp corresponding to a version of data of a database associated with the query. It is determined whether a locally stored portion of the data of the database has a local timestamp that is equivalent to or after the read timestamp. A lock may be requested, and if received, the data is updated.