Abstract: Distributed database systems including compute nodes and page servers are described herein that enable separating logical and physical storage of database files in a distributed database system. A distributed database system includes a page server and a compute node, and is configured to store a logical database file that includes data and is associated with a file identifier. Each page server is configurable to store slices (i.e., subportions) of the logical database file. The compute node is coupled to the plurality of page servers and configured to store the logical database file responsive to a received command. In an aspect, such storage may comprise slicing the data comprising the logical database file into a set of slices with each being associated with a respective page server, maintaining an endpoint mapping for each slice of the first set of slices, and transmitting each slice to the associated for storage thereby.

Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Distributed database systems including compute nodes and page servers are described herein that enable separating logical and physical storage of database files in a distributed database system. A distributed database system includes a page server and a compute node and is configured to store a logical database file that includes data and is associated with a file identifier. Each page server is configurable to store slices (i.e., subportions) of the logical database file. The compute node is coupled to the plurality of page servers and configured to store the logical database file responsive to a received command. In an aspect, such storage may comprise slicing the data comprising the logical database file into a set of slices with each being associated with a respective page server, maintaining an endpoint mapping for each slice of the first set of slices, and transmitting each slice to the associated for storage thereby.

Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Distributed database systems including compute nodes and page servers are described herein that enable separating logical and physical storage of database files in a distributed database system. A distributed database system includes a page server and a compute node and is configured to store a logical database file that includes data and is associated with a file identifier. Each page server is configurable to store slices (i.e., subportions) of the logical database file. The compute node is coupled to the plurality of page servers and configured to store the logical database file responsive to a received command. In an aspect, such storage may comprise slicing the data comprising the logical database file into a set of slices with each being associated with a respective page server, maintaining an endpoint mapping for each slice of the first set of slices, and transmitting each slice to the associated for storage thereby.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Data base performance is improved using write-behind optimization of covering cache. Non-volatile memory data cache includes a full copy of stored data file(s). Data cache and storage writes, checkpoints, and recovery may be decoupled (e.g., with separate writes, checkpoints and recoveries). A covering data cache supports improved performance by supporting database operation during storage delays or outages and/or by supporting reduced I/O operations using aggregate writes of contiguous data pages (e.g., clean and dirty pages) to stored data file(s). Aggregate writes reduce data file fragmentation and reduce the cost of snapshots. Performing write-behind operations in a background process with optimistic concurrency control may support improved database performance, for example, by not interfering with write operations to data cache. Data cache may store (e.g., in metadata) data cache checkpoint information and storage checkpoint information. A stored data file may store storage checkpoint information (e.g.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Brokering log records so as to prevent log records that are not yet persisted in a persistent log from being disseminated. The log records may be generated as a primary compute system performs operations. Upon receiving a request for a log record, the broker component determines whether the requested log record has been persisted in a persistent log. If the broker component determines that the log record has been persisted in the persistent log, the broker component responds to the request by causing the requested log record to be provided to the requesting entity (e.g., a secondary compute system). On the other hand, if the log record cannot yet determine that the log record has been persisted in the persistent log, the broker component prevents the log record from being provided to the requesting entity. This prevents data from being inconsistent during recovery.

Abstract: Apparatus and associated methods relate to a compact gas sensor (CGS) including a housing with a central stepped cavity with one or more first lead contact(s) forming a portion of a base plane in a bottom of the cavity and one or more second lead contact(s) forming a portion of a stepped plane higher than the base plane, the cavity sized to receive a chemically based stack of material made up of a bottom diffusion electrode layer, a middle electrolyte gel layer, and a top diffusion electrode layer. The bottom diffusion electrode layer is in electrical contact with the first lead contact(s). The top diffusion electrode layer electrically couples to the second lead contact(s) via an overlaying micro electromechanical system (MEMS) element layer with conductive coating. In an illustrative example, the CGS may provide gas sensing in small spaces.

Abstract: Embodiments described herein detect data corruption in a distributed data set system. For example, a system comprises node(s) for processing queries with respect to a distributed data set comprising a plurality of storage segments. A write transaction resulting from a query with respect to a particular storage segment is logged in a log record that describes a modification to the storage segment. A log service provides the log record to a data server managing a portion of the distributed data set in which the storage segment is included, which performs the write transaction with respect to the storage segment. For redundancy purposes, the data server has replica(s) that manage respective replicas of the portion of the distributed data set managed thereby. For backup purposes, snapshots of the replica(s) are periodically generated. To determine a data corruption, a snapshot of one replica is cross-validated with a snapshot of another replica.

Abstract: Adaptive adjusting of the growth of a persistent log. The persistent log has a log record generator that adds log records to the persistent log. In addition, there are multiple log consumers that consume records from the persistent log. The log consumers publish log processing parameters with respect to the persistent log. The log processing parameters are then used to determine an appropriate adjustment in the growth of the log, which adjustments may then be executed. As an example, the log processing parameter may be a log consumption progress, in which case the log generator may be caused to slow down the generation of log records, thereby slowing the growth of the log.

Abstract: Caching storage segments (e.g., pages) loaded from a remote storage such that, during recovery, the cached loaded storage segments may be at least partially recovered without reloading the storage segments from the remote storage. During normal operation of a computing system, storage segments are loaded from remote storage into local memory of a computing system. At some point, either due to eviction of the storage segment due to aging out of the storage segment, or due to writing of the storage segment, it is determined to write at least some of the loaded storage segments into local persistent storage. In conjunction with this, the corresponding storage segment is written to a respective storage address of the local persistent storage. Also, a correlation between an identifier of the storage segment and the respective address in the persistent storage is recorded in a persistent data structure.

Abstract: An in-memory database system stores an entirety of a database within main memory of the computer system. The database includes a table comprising a plurality of rows including a particular row. A first transaction updating the particular row is executed to create a first version of the particular row. A second transaction updating the particular row is executed to create a second version of the particular row. The first version is visible only to the first transaction, and the second version is visible only to the second transaction. The in-memory database system may associate an index with the database table, the index referencing the plurality of rows, including referencing the first version of the particular row and the second version of the particular row. The in-memory database system may initiate garbage collection of the first version of the particular row and/or the second version of the particular row.

Abstract: The bifurcation of a log into a fixed-size log portion and a growable log portion. Log records are first written to the fixed-size log portion. At some point, the older log records within the fixed-size log portion are destaged to the growable log portion. Destaging may involve copying the log records from the fixed-size log portion and tacking the log record onto the end of the growable log portion. This destaging might occur one sub-portion (e.g., one virtual log file) at a time. The more recent log records are within the fixed-size log portion. Accordingly, new writes to the log occur to the fixed-size log portion. Furthermore, a large portion of the reads are of the more recent log records within the fixed-size log portion. Thus, optimizing performance within the fixed-size log portion results in significant improvement in the performance of the overall log.

Abstract: A storage segment server environment that includes storage segment server(s) that cover a contiguous address space of data. The storage segment server has storage segments that are contiguous and that matches the address space of a respective of storage segments in the cloud store service. Thus, a read of the contiguous address spaces of the data from the cloud store service may instead be handled as a single read of the contiguous address space of the storage segment server. Each storage segment server(s) may be assigned to a range within the total contiguous address space of the data. Each storage segment server(s) includes a respective storage segment write component that is configured to perform write operations to storage segments in the respective storage segment server. Each storage segment server(s) also includes a respective cloud storage write component that initiates corresponding write operations to the cloud store service.

Abstract: The subject disclosure relates to a distributed transaction management technique that ensures synchronization between participating nodes in a global or distributed database transaction. The technique leverages a commit protocol that uses local clocks at the respective participating nodes. Participants in a global transaction are configured to utilize the same commit timestamp and logical read time and can advance their respective local clocks to establish this synchronization. In one embodiment, distributed commit utilizes a modified version of two-phase commit that includes an extra phase to collect commit timestamp votes from participants. Additionally, a heartbeat mechanism can be used to establish loose synchronization between nodes. In another embodiment, a node can respond to a remote transaction request by returning a list of nodes involved in generating the result of the transaction and the types of access used by such nodes in addition to the transaction result itself.

Abstract: The use of log marking (otherwise known as “coloring”) of sub-portions of a log that records actions (e.g., data operations) performed by a computing system. The log is composed of multiple sub-portions, such as virtual log files, which are successively added to the log as the log grows. For instance, the sub-portions may be virtual log files of the log. The principles described herein change the use of log marking depending on which sub-portion of the log is being marked. If the computing system fails, and recovery is needed, the recovery process can thus deterministically identify where the last written log record is.

Abstract: A versioned records management computing system that uses a restart era in order to promote rapid recovery. A persistent store includes a multi-versioned record collection. The records are also associated with a restart era that corresponds to the era of operation of the computing system after a restart. Upon a recovery, the current restart era changes. An object collection media has an object collection that conforms to an object model such that the object model is used to operate upon the records. The object collection media is operable such that the object collection is durable so as to survive restarts of the system to thereby allow for accelerated recovery.

Abstract: Storing an incoming data stream using successive files that are consecutively populated. The appropriate file to populate a given data stream portion into is determined by mapping the data stream offset to a file, and potentially also an address within that file. The successive files may be the same size, so that the file can be identified based on the data stream address (or offset) without the use of an index. Furthermore, the files may be easily named by having that size be some multiple of a binary power of bytes. That way, the files themselves can be automatically and named and identified by using the more significant bit or bits of the data stream offset to uniquely identify the file and establish ordering of the files. Replication may occur from a primary to a secondary store by transmitting the offset, and the actual data to be stored.