Abstract: Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.

Abstract: A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to “slices”, and storing multiple copies (“duplicas”) of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the “primary duplica”. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (“secondary duplicas”) of the same slice.

Abstract: A shared-nothing database system is provided in which parallelism and workload balancing are increased by assigning the rows of each table to “slices”, and storing multiple copies (“duplicas”) of each slice across the persistent storage of multiple nodes of the shared-nothing database system. When the data for a table is distributed among the nodes of a shared-nothing system in this manner, requests to read data from a particular row of the table may be handled by any node that stores a duplica of the slice to which the row is assigned. For each slice, a single duplica of the slice is designated as the “primary duplica”. All DML operations (e.g. inserts, deletes, updates, etc.) that target a particular row of the table are performed by the node that has the primary duplica of the slice to which the particular row is assigned. The changes made by the DML operations are then propagated from the primary duplica to the other duplicas (“secondary duplicas”) of the same slice.

Abstract: Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.

Abstract: Techniques are provided for assigning read requests to storage devices in a manner that reduces the likelihood that any storage device will become overloaded or underutilized. Specifically, a read-request handler assigns read requests that are directed to each particular item among the storage devices that have copies of the item based on how busy each of those storage devices is. Consequently, even though certain storage devices may have copies of the same item, there may be times during which one storage device is assigned a disproportionate number of the reads of the item because the other storage device is busy with read requests for other items, and there may be other times during which other storage device is assigned a disproportionate number of the reads of the item because the one storage device is busy with read request for other items.

Abstract: Techniques are described herein for making a clean file snapshot of a target file. The techniques may be applied to a single target file, to a set of target files, or to an entire database The techniques involve transitioning the target file through a series of states. During each state, particular actions are performed and/or prevented. In the final state of each approach, a clean file snapshot of the target file exists. Transitioning through the states, only one of which does not allow new changes to be made to the target file, allows the database to remain online and available to a greater extent than is possible with an approach that prevents database changes for the duration of the clean file snapshot creation operation.

Abstract: A system and method implementing revocable secure remote keys is disclosed. A plurality of indexed base secrets is stored in a register of a coprocessor of a local node coupled with a local memory. When it is determined that a selected base secret expired, the base secret stored in the register based on the base secret index is changed, thereby invalidating remote keys generated based on the expired base secret. A remote key with validation data and a base secret index is received from a node requesting access to the local memory. A validation base secret is obtained from the register based on the base secret index. The coprocessor performs hardware validation on the validation data based on the validation base secret. Hardware validation fails if the base secret associated with the base secret index has been changed in the register of the selected coprocessor.

Abstract: Systems and methods for reliably using data storage media. Multiple processors are configured to access a persistent memory. For a given data block corresponding to a write access request from a first processor to the persistent memory, a cache controller prevents any read access of a copy of the given data block in an associated cache. The cache controller prevents any read access while detecting an acknowledgment that the given data block is stored in the persistent memory is not yet received. Until the acknowledgment is received, the cache controller allows write access of the copy of the given data block in the associated cache only for a thread in the first processor that originally sent the write access request. The cache controller invalidates any copy of the given data block in any cache levels below the associated cache.

Abstract: Techniques are provided for assigning read requests to storage devices in a manner that reduces the likelihood that any storage device will become overloaded or underutilized. Specifically, a read-request handler assigns read requests that are directed to each particular item among the storage devices that have copies of the item based on how busy each of those storage devices is. Consequently, even though certain storage devices may have copies of the same item, there may be times during which one storage device is assigned a disproportionate number of the reads of the item because the other storage device is busy with read requests for other items, and there may be other times during which other storage device is assigned a disproportionate number of the reads of the item because the one storage device is busy with read request for other items.

Abstract: Techniques for moving data files without interrupting access are described. A first process moves a database file from a first location to a second location while the database file is accessible to one or more other processes for read or write operations. According to one technique, the first process communicates a move status and a copy range into the database file to one or more database server instances executing the one or more other processes. The one or more other processes then perform input/output (IO) operations on the database file based at least in part on the move status and the copy range communicated by the first process.

Abstract: A method and apparatus for sending and receiving messages between nodes on a compute cluster is provided. Communication between nodes on a compute cluster, which do not share physical memory, is performed by passing messages over an I/O subsystem. Typically, each node includes a synchronization mechanism, a thread ready to receive connections, and other threads to process and reassemble messages. Frequently, a separate queue is maintained in memory for each node on the I/O subsystem sending messages to the receiving node. Such overhead increases latency and limits message throughput. Due to a specialized coprocessor running on each node, messages on an I/O subsystem are sent, received, authenticated, synchronized, and reassembled at a faster rate and with lower latency. Additionally, the memory structure used may reduce memory consumption by storing messages from multiple sources in the same memory structure, eliminating the need for per-source queues.

Abstract: Techniques are described herein for making a clean file snapshot of a target file. The techniques may be applied to a single target file, to a set of target files, or to an entire database The techniques involve transitioning the target file through a series of states. During each state, particular actions are performed and/or prevented. In the final state of each approach, a clean file snapshot of the target file exists. Transitioning through the states, only one of which does not allow new changes to be made to the target file, allows the database to remain online and available to a greater extent than is possible with an approach that prevents database changes for the duration of the clean file snapshot creation operation.

Abstract: Systems and methods for reliably using data storage media. Multiple processors are configured to access a persistent memory. For a given data block corresponding to a write access request from a first processor to the persistent memory, a cache controller prevents any read access of a copy of the given data block in an associated cache. The cache controller prevents any read access while detecting an acknowledgment that the given data block is stored in the persistent memory is not yet received. Until the acknowledgment is received, the cache controller allows write access of the copy of the given data block in the associated cache only for a thread in the first processor that originally sent the write access request. The cache controller invalidates any copy of the given data block in any cache levels below the associated cache.

Abstract: Techniques for moving data files without interrupting access are described. A first process moves a database file from a first location to a second location while the database file is accessible to one or more other processes for read or write operations. According to one technique, the first process communicates a move status and a copy range into the database file to one or more database server instances executing the one or more other processes. The one or more other processes then perform input/output (IO) operations on the database file based at least in part on the move status and the copy range communicated by the first process.

Abstract: A system and method implementing revocable secure remote keys is disclosed. A plurality of indexed base secrets is stored in a register of a coprocessor of a local node coupled with a local memory. When it is determined that a selected base secret expired, the base secret stored in the register based on the base secret index is changed, thereby invalidating remote keys generated based on the expired base secret. A remote key with validation data and a base secret index is received from a node requesting access to the local memory. A validation base secret is obtained from the register based on the base secret index. The coprocessor performs hardware validation on the validation data based on the validation base secret. Hardware validation fails if the base secret associated with the base secret index has been changed in the register of the selected coprocessor.

Abstract: A method and apparatus for sending and receiving messages between nodes on a compute cluster is provided. Communication between nodes on a compute cluster, which do not share physical memory, is performed by passing messages over an I/O subsystem. Typically, each node includes a synchronization mechanism, a thread ready to receive connections, and other threads to process and reassemble messages. Frequently, a separate queue is maintained in memory for each node on the I/O subsystem sending messages to the receiving node. Such overhead increases latency and limits message throughput. Due to a specialized coprocessor running on each node, messages on an I/O subsystem are sent, received, authenticated, synchronized, and reassembled at a faster rate and with lower latency. Additionally, the memory structure used may reduce memory consumption by storing messages from multiple sources in the same memory structure, eliminating the need for per-source queues.

Abstract: This disclosure describes solutions for reducing the amount of fragmentation on a computer memory device, such as a hard disk, random access memory device, and/or the like. In an aspect, this disclosure describes systems, methods and software for allocating storage space for variable-sized data chunks in a fashion that reduces or eliminates the need for periodic de-fragmentation of the memory device. In another aspect, this disclosure describes solutions that provide for the dynamic re-allocation of existing data blocks on the memory device to provide contiguous available space that can be allocated for new data blocks.

Abstract: A method and system for reducing overhead associated with recovering after a failure. According to the method, a checkpoint value is maintained that indicates which records of a plurality of records have to be processed after the failure. The plurality of records contain change information that corresponds to a plurality of data blocks. A target checkpoint value is determined based on a desired number of data block reads that will be required during a redo phase of recovery. Changes contained in volatile memory are then written to nonvolatile memory to advance the checkpoint value to at least the target checkpoint value.

Abstract: Techniques are described herein for returning a repository to a prior state. The repository may be, for example a database, and the prior state may be, for example the consistent state that the database had at a particular point in time in the past. When a operator-caused error has been introduced to the database by changes made by an already-committed transaction, the techniques described herein may be used to recover from the error by returning the database to a point in time prior to the commit time of the transaction that introduced the error. The techniques involve the generation of “physical undo” information, and the use of the physical undo information in conjunction with physiological undo and physiological redo to efficiently return a repository to the prior state.

Abstract: Pieces of data are stored among storage devices based on a cycle value, which is computed for each storage device as the total capacity of all storage devices divided by the individual capacity of the storage device. Next, a storage device for a current piece of data is selected to be the storage device with the smallest key value; followed by determination of a new key value based on at least (a) the cycle value and (b) a sequence number of the current piece. After allocation, if the number of storage devices changes, reallocation is done similarly except that selection is first from a preferred set and only if the preferred set is empty then from a remainder set. Storage devices are placed in the preferred set if a piece of data would be outside a shadow (based on cycle value) of another piece pre-existing in the storage device.