Abstract: Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Abstract: A storage device uses non-volatile memory devices for caching. The storage device operates in a mode referred to herein as write-back mode. In write-back mode, a storage device responds to a request to write data by persistently writing the data to a cache in a non-volatile memory device and acknowledges to the requestor that the data is written persistently in the storage device. The acknowledgement is sent without necessarily having written the data that was requested to be written to primary storage. Instead, the data is written to primary storage later.

Abstract: Techniques are provided to allow more sophisticated operations to be performed remotely by machines that are not fully functional. Operations that can be performed reliably by a machine that has experienced a hardware and/or software error are referred to herein as Remote Direct Memory Operations or “RDMOs”. Unlike RDMAs, which typically involve trivially simple operations such as the retrieval of a single value from the memory of a remote machine, RDMOs may be arbitrarily complex. The techniques described herein can help applications run without interruption when there are software faults or glitches on a remote system with which they interact.

Abstract: Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, d) whether the I/O request is associated with an offloaded filter provided by the database server to the storage system, or e) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Abstract: A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Abstract: No-loss rapid recovery performs resynchronization efficiently while concurrently allowing availability to mirrored data on the storage device. No-loss rapid recovery has two stages and involves storage devices that have both a non-volatile cache and primary storage and that operate as mirror buddies. The first stage is referred to herein as the buddy-retention stage. During the buddy-retention stage, writes to mirrored data are not performed on the offline mirror buddy but are performed on the online mirror buddy. The mirrored data changed in the online mirrored buddy is retained in the non-volatile cache of the retention buddy. The next stage is referred to herein as the rapid resynchronization stage. In this stage, the changed mirrored data retained by the retention buddy for no-loss rapid recovery is used to resynchronize the offline buddy. The storage device is resynchronized using the changed mirrored data retained in the cache of the mirror buddy.

Abstract: A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Abstract: A minimum value (MV) is computed for start timestamps that each correspond to an uncommitted transaction. In an embodiment, the MV is computed for a pluggable database that is open on at least first and second instances of a database. The MV is computed for the first instance as of a first current timestamp (CT). The MV and the first CT are communicated to a second instance that has a second CT. If the first and second CTs are equal, the second instance store the MV. If the first CT is bigger, the second CT also becomes equal to the first CT. If the first CT is smaller, the MV is discarded, and the first CT becomes equal to the second CT. In an embodiment, if the MV remains unchanged for a predetermined time period, a start timestamp corresponding to the MV is advanced to a current or future timestamp.

Abstract: A method, system, and computer program product is disclosed for interacting with a client supported by a client-side cache. Embodiments of a method, a system, and a computer program product are disclosed that retrieve a first snapshot, indicating a state of the database after a last database request by the client, associated with the client, determine any number of invalid cached results for the client based on the first snapshot, and transmit the any number of invalid cached results and a second snapshot, an update for the first snapshot.

Abstract: Techniques are described herein for routing queries to particular nodes of a multi-node database system based on the query. A database table is partitioned into a plurality of affinity groups. Each affinity group is assigned a particular node as the master node of the affinity group. A mapping is sent to a query router indicating the master node for each affinity group of the plurality of affinity groups. The query router determines, for a particular query, a target node to which to send the particular query based on the mapping and the particular query.

Abstract: Data blocks are cached in a persistent cache (“NV cache”) allocated from as non-volatile RAM (“NVRAM”). The data blocks may be accessed in place in the NV cache of a “source” computing element by another “remote” computing element over a network using remote direct memory access (“RMDA”). In order for a remote computing element to access the data block in NV cache on a source computing element, the remote computing element needs the memory address of the data block within the NV cache. For this purpose, a hash table is stored and maintained in RAM on the source computing element. The hash table identifies the data blocks in the NV cache and specifies a location of the cached data block within the NV cache.

Abstract: A minimum value (MV) is computed for start timestamps that each correspond to an uncommitted transaction. In an embodiment, the MV is computed for a pluggable database that is open on at least first and second instances of a database. The MV is computed for the first instance as of a first current timestamp (CT). The MV and the first CT are communicated to a second instance that has a second CT. If the first and second CTs are equal, the second instance store the MV. If the first CT is bigger, the second CT also becomes equal to the first CT. If the first CT is smaller, the MV is discarded, and the first CT becomes equal to the second CT. In an embodiment, if the MV remains unchanged for a predetermined time period, a start timestamp corresponding to the MV is advanced to a current or future timestamp.

Abstract: Approaches, techniques, and mechanisms are disclosed for improved caching in database systems that deal with multiple data access patterns, such as in database systems that interface with both OLTP and Data Warehouse clients. A cache is deployed between a database server and a storage system that stores data units. Some of the data units accessed by the database server are buffered within the cache. The data units may be associated with data access patterns, such as a random data access pattern or a scan data access pattern, in accordance with which the database server is or appears to be accessing the data units. A processor selects when to cache data units accessed by the database server, based at least on the associated data access patterns. Recent access counts may also be stored for the data units, and may further be utilized to select when to cache data units.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: Techniques are provided for enabling a requesting entity to retrieve data that is managed by a database server instance from the volatile memory of a server machine that is executing the database server instance. The techniques allow the requesting entity to retrieve the data from the volatile memory of the host server machine without involving the database server instance in the retrieval operation. Because the retrieval does not involve the database server instance, the retrieval may succeed even when the database server instance has stalled or become unresponsive. In addition, direct retrieval of data using the techniques described herein will often be faster and more efficient than retrieval of the same information through conventional interaction with the database server instance.

Abstract: A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Abstract: Techniques related to efficient data storage and retrieval using a heterogeneous main memory are disclosed. A database includes a set of persistent format (PF) data that is stored on persistent storage in a persistent format. The database is maintained on the persistent storage and is accessible to a database server. The database server converts the set of PF data to sets of mirror format (MF) data and stores the MF data in a hierarchy of random-access memories (RAMs). Each RAM in the hierarchy has an associated latency that is different from a latency associated with any other RAM in the hierarchy. Storing the sets of MF data in the hierarchy of RAMs includes (1) selecting, based on one or more criteria, a respective RAM in the hierarchy to store each set of MF data and (2) storing said each set of MF data in the respective RAM.

Abstract: To prioritize repopulation of in-memory compression units (IMCU), a database server compresses, into an IMCU, a plurality of data units from a database table. In response to changes to any of the plurality of data units within the database table, the database server performs the steps of: (a) invalidating corresponding data units in the IMCU; (b) incrementing an invalidity counter of the IMCU that reflects how many data units within the IMCU have been invalidated; (c) receiving a data request that targets one or more of the plurality of data units of the database table; (d) in response to receiving the data request, incrementing an access counter of the IMCU; and (e) determining a priority for repopulating the IMCU based, at least in part, on the invalidity counter and the access counter.

Abstract: Techniques related to efficient data storage and retrieval using a heterogeneous main memory are disclosed. A database includes a set of persistent format (PF) data that is stored on persistent storage in a persistent format. The database is maintained on the persistent storage and is accessible to a database server. The database server converts the set of PF data to sets of mirror format (MF) data and stores the MF data in a hierarchy of random-access memories (RAMs). Each RAM in the hierarchy has an associated latency that is different from a latency associated with any other RAM in the hierarchy. Storing the sets of MF data in the hierarchy of RAMs includes (1) selecting, based on one or more criteria, a respective RAM in the hierarchy to store each set of MF data and (2) storing said each set of MF data in the respective RAM.

Abstract: A minimum value (MV) is computed for start timestamps that each correspond to an uncommitted transaction. In an embodiment, the MV is computed for a pluggable database that is open on at least first and second instances of a database. The MV is computed for the first instance as of a first current timestamp (CT). The MV and the first CT are communicated to a second instance that has a second CT. If the first and second CTs are equal, the second instance store the MV. If the first CT is bigger, the second CT also becomes equal to the first CT. If the first CT is smaller, the MV is discarded, and the first CT becomes equal to the second CT. In an embodiment, if the MV remains unchanged for a predetermined time period, a start timestamp corresponding to the MV is advanced to a current or future timestamp.