Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Brokering log records so as to prevent log records that are not yet persisted in a persistent log from being disseminated. The log records may be generated as a primary compute system performs operations. Upon receiving a request for a log record, the broker component determines whether the requested log record has been persisted in a persistent log. If the broker component determines that the log record has been persisted in the persistent log, the broker component responds to the request by causing the requested log record to be provided to the requesting entity (e.g., a secondary compute system). On the other hand, if the log record cannot yet determine that the log record has been persisted in the persistent log, the broker component prevents the log record from being provided to the requesting entity. This prevents data from being inconsistent during recovery.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Assembly of a state of user data as of a particular point in time in the context of the data being spread across multiple storage segment servers. Furthermore, the assembly occurs despite the storage segment servers not needing to coordinate a timing of any snapshots. Rather, the storage segment servers perform a snapshot of the portion of the data that it individually stores without having to coordinate with any other storage segment server. The particular point of time may be any arbitrary time, such as a time selected by a user.

Abstract: Assembly of a state of user data as of a particular point in time in the context of the data being spread across multiple storage segment servers. Furthermore, the assembly occurs despite the storage segment servers not needing to coordinate a timing of any snapshots. Rather, the storage segment servers perform a snapshot of the portion of the data that it individually stores without having to coordinate with any other storage segment server. The particular point of time may be any arbitrary time, such as a time selected by a user.

Abstract: Brokering log records so as to prevent log records that are not yet persisted in a persistent log from being disseminated. The log records may be generated as a primary compute system performs operations. Upon receiving a request for a log record, the broker component determines whether the requested log record has been persisted in a persistent log. If the broker component determines that the log record has been persisted in the persistent log, the broker component responds to the request by causing the requested log record to be provided to the requesting entity (e.g., a secondary compute system). On the other hand, if the log record cannot yet determine that the log record has been persisted in the persistent log, the broker component prevents the log record from being provided to the requesting entity. This prevents data from being inconsistent during recovery.