Abstract: Techniques are described for offloading remote direct memory operations (RDMOs) to “execution candidates”. The execution candidates may be any hardware capable of performing the offloaded operation. Thus, the execution candidates may be network interface controllers, specialized co-processors, FPGAs, etc. The execution candidates may be on a machine that is remote from the processor that is offloading the operation, or may be on the same machine as the processor that is offloading the operation. Details for certain specific RDMOs, which are particularly useful in online transaction processing (OLTP) and hybrid transactional/analytical (HTAP) workloads, are provided.

Abstract: Data blocks are cached in a persistent cache (“NV cache”) allocated from as non-volatile RAM (“NVRAM”). The data blocks may be accessed in place in the NV cache of a “source” computing element by another “remote” computing element over a network using remote direct memory access (“RMDA”). In order for a remote computing element to access the data block in NV cache on a source computing element, the remote computing element needs the memory address of the data block within the NV cache. For this purpose, a hash table is stored and maintained in RAM on the source computing element. The hash table identifies the data blocks in the NV cache and specifies a location of the cached data block within the NV cache.

Abstract: Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.

Abstract: Herein are high throughput techniques for incorporating cryptographic blockchains into relational data. In an embodiment, a computer indicates, in a database dictionary in a database, that a relational table is for blockchain storage. The relational table contains application column(s). In response to that indication, the relational table is associated with system column(s) that are unmodifiable by administrators and clients of the database. The system column(s) include a cryptographic hash column. A request to store a particular value in a particular application column is received from a client. In response to receiving the request, a cryptographic hash value is calculated for a new row for the relational table. In the relational table, the computer stores the new row that contains: the particular value in the particular application column, and the cryptographic hash value in the cryptographic hash column.

Abstract: Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or “RDMOs”. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

Abstract: Distributed ledgered data is stored within a distributed persistent storage system comprising multiple persistent storage systems as distributed ledgered participants. In various embodiments, the distributed ledgered data is maintained using the native capabilities of a persistent storage system. The distributed ledgered data is replicated as persistent data objects in a “ledgered repository of objects” that are replicated at each of the persistent storage systems. Changes at one persistent storage system are recorded within a block in a distributed blockchain that is distributed across each of the other distributed ledgered participants. The other distributed ledgered participants read the changes from the blockchain and apply the changes to the respective replicas at each of the other distributed ledgered participants. Hence, this approach is referred to as blockchain apply. Blockchain apply may be used to replicate the repository objects of various forms of PSSs.

Abstract: Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or “RDMOs”. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

Abstract: Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or “RDMOs”. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

Abstract: Techniques are provided for enabling a requesting entity to retrieve data that is managed by a database server instance from the volatile memory of a server machine that is executing the database server instance. The techniques allow the requesting entity to retrieve the data from the volatile memory of the host server machine without involving the database server instance in the retrieval operation. Because the retrieval does not involve the database server instance, the retrieval may succeed even when the database server instance has stalled or become unresponsive. In addition, direct retrieval of data using the techniques described herein will often be faster and more efficient than retrieval of the same information through conventional interaction with the database server instance.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Abstract: A storage device uses non-volatile memory devices for caching. The storage device operates in a mode referred to herein as write-back mode. In write-back mode, a storage device responds to a request to write data by persistently writing the data to a cache in a non-volatile memory device and acknowledges to the requestor that the data is written persistently in the storage device. The acknowledgement is sent without necessarily having written the data that was requested to be written to primary storage. Instead, the data is written to primary storage later.

Abstract: Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or “RDMOs”. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

Abstract: Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, d) whether the I/O request is associated with an offloaded filter provided by the database server to the storage system, or e) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Abstract: A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Abstract: No-loss rapid recovery performs resynchronization efficiently while concurrently allowing availability to mirrored data on the storage device. No-loss rapid recovery has two stages and involves storage devices that have both a non-volatile cache and primary storage and that operate as mirror buddies. The first stage is referred to herein as the buddy-retention stage. During the buddy-retention stage, writes to mirrored data are not performed on the offline mirror buddy but are performed on the online mirror buddy. The mirrored data changed in the online mirrored buddy is retained in the non-volatile cache of the retention buddy. The next stage is referred to herein as the rapid resynchronization stage. In this stage, the changed mirrored data retained by the retention buddy for no-loss rapid recovery is used to resynchronize the offline buddy. The storage device is resynchronized using the changed mirrored data retained in the cache of the mirror buddy.

Abstract: A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Abstract: A minimum value (MV) is computed for start timestamps that each correspond to an uncommitted transaction. In an embodiment, the MV is computed for a pluggable database that is open on at least first and second instances of a database. The MV is computed for the first instance as of a first current timestamp (CT). The MV and the first CT are communicated to a second instance that has a second CT. If the first and second CTs are equal, the second instance store the MV. If the first CT is bigger, the second CT also becomes equal to the first CT. If the first CT is smaller, the MV is discarded, and the first CT becomes equal to the second CT. In an embodiment, if the MV remains unchanged for a predetermined time period, a start timestamp corresponding to the MV is advanced to a current or future timestamp.

Abstract: A method, system, and computer program product is disclosed for interacting with a client supported by a client-side cache. Embodiments of a method, a system, and a computer program product are disclosed that retrieve a first snapshot, indicating a state of the database after a last database request by the client, associated with the client, determine any number of invalid cached results for the client based on the first snapshot, and transmit the any number of invalid cached results and a second snapshot, an update for the first snapshot.