Abstract: Techniques are described herein for an integrated in-front database cache (“IIDC”) providing an in-memory, consistent, and automatically managed cache for primary database data. An IIDC comprises a database server instance that (a) caches data blocks from a source database managed by a second database server instance, and (b) performs recovery on the cached data using redo records for the database data. The IIDC instance implements relational algebra and is configured to run any complexity of query over the cached database data. Any cache miss results in the IIDC instance fetching the needed block(s) from a second database server instance managing the source database that provides the IIDC instance with the latest version of the requested data block(s) that is available to the second instance. Because redo records are used to continuously update the data blocks in an IIDC cache, the IIDC guarantees consistency of query results.

Abstract: Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.

Abstract: A method and an apparatus for implementing a buffer cache for a persistent file system in a non-volatile memory is provided. A set of data is maintained in one or more extents in a non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in a dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in the NVRAM. A reference to the first extent in the NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in the NVRAM.

Abstract: Techniques herein store database blocks (DBBs) in byte-addressable persistent memory (PMEM) and prevent tearing without deadlocking or waiting. In an embodiment, a computer hosts a DBMS. A reader process of the DBMS obtains, without locking and from metadata in PMEM, a first memory address for directly accessing a current version, which is a particular version, of a DBB in PMEM. Concurrently and without locking: a) the reader process reads the particular version of the DBB in PMEM, and b) a writer process of the DBMS replaces, in the metadata in PMEM, the first memory address with a second memory address for directly accessing a new version of the DBB in PMEM. In an embodiment, a computer performs without locking: a) storing, in PMEM, a DBB, b) copying into volatile memory, or reading, an image of the DBB, and c) detecting whether the image of the DBB is torn.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: Techniques are provided for managing cached data objects in a mixed workload environment. In an embodiment, a database system receives request to access a target data object. The database system determines whether the request to access the target data object is associated with a first type of workload or a second type of workload. In response to determining that the request is associated with the first type of workload, the target data object replaces a least recently used data object in a cache. In response to determining that the request is associated with the second type of workload, the target data object is cached based on an associated access-level value.

Abstract: A method and apparatus for implementing a buffer cache for a persistent file system in non-volatile memory is provided. A set of data is maintained in one or more extents in non-volatile random-access memory (NVRAM) of a computing device. At least one buffer header is allocated in dynamic random-access memory (DRAM) of the computing device. In response to a read request by a first process executing on the computing device to access one or more first data blocks in a first extent of the one or more extents, the first process is granted direct read access of the first extent in NVRAM. A reference to the first extent in NVRAM is stored in a first buffer header. The first buffer header is associated with the first process. The first process uses the first buffer header to directly access the one or more first data blocks in NVRAM.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of an application and the non-volatile storage of a storage system. The application may be any type of application that uses a storage system to persistently store data. The intermediate cache may be local to the machine upon which the application is executing, or may be implemented within the storage system. In one embodiment where the application is a database server, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which object an item belongs to, the item type of the item, a characteristic of the item, or the type of operation in which the item is involved.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of an application and the non-volatile storage of a storage system. The application may be any type of application that uses a storage system to persistently store data. The intermediate cache may be local to the machine upon which the application is executing, or may be implemented within the storage system. In one embodiment where the application is a database server, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which object an item belongs to, the item type of the item, a characteristic of the item, or the type of operation in which the item is involved.

Abstract: One embodiment of the present invention provides a system for performing selective encryption/decryption in a data storage system. During operation, the system receives a data block from a storage medium at an input/output layer, wherein the input/output layer serves as an interface between the storage medium and a buffer cache. Next, the system determines whether the data block is an encrypted data block. If not, the system stores the data block in the buffer cache. Otherwise, if the data block is an encrypted data block, the system retrieves a storage-key, wherein the storage-key is associated with a subset of storage, which is associated with the encrypted data block. Using the storage-key, the system then decrypts the encrypted data block to produce a decrypted data block. Finally, the system stores the decrypted data block in the buffer cache, wherein the data block remains encrypted in the storage medium.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of a database server and the non-volatile storage of a storage system. The intermediate cache may be local to the machine upon which the database server is executing, or may be implemented within the storage system. In one embodiment, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which database object an item belongs to, the item type of the item, a characteristic of the item; or the database operation in which the item is involved.

Abstract: Techniques are provided for managing cached data objects in a mixed workload environment. In an embodiment, a database system receives request to access a target data object. The database system determines whether the request to access the target data object is associated with a first type of workload or a second type of workload. In response to determining that the request is associated with the first type of workload, the target data object replaces a least recently used data object in a cache. In response to determining that the request is associated with the second type of workload, the target data object is cached based on an associated access-level value.

Abstract: One embodiment of the present invention provides a system that facilitates generating random data-encryption keys for data files. During operation, the system receives a command at a computer system to create a data file that may include encrypted data. This data file includes a wrapped data-encryption key to facilitate encrypting and decrypting data. In response to the command, the system generates a bit pattern to be used as the wrapped data-encryption key. Finally, the system creates the data file, which includes the bit pattern as the wrapped data-encryption key.

Abstract: Techniques are provided for using an intermediate cache between the shared cache of a database server and the non-volatile storage of a storage system. The intermediate cache may be local to the machine upon which the database server is executing, or may be implemented within the storage system. In one embodiment, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which database object an item belongs to, the item type of the item, a characteristic of the item; or the database operation in which the item is involved.

Abstract: One embodiment of the present invention provides a system that facilitates generating random data-encryption keys for data files. During operation, the system receives a command at a computer system to create a data file that may include encrypted data. This data file includes a wrapped data-encryption key to facilitate encrypting and decrypting data. In response to the command, the system generates a bit pattern to be used as the wrapped data-encryption key. Finally, the system creates the data file, which includes the bit pattern as the wrapped data-encryption key.

Abstract: One embodiment of the present invention provides a system for performing selective encryption/decryption in a data storage system. During operation, the system receives a data block from a storage medium at an input/output layer, wherein the input/output layer serves as an interface between the storage medium and a buffer cache. Next, the system determines whether the data block is an encrypted data block. If not, the system stores the data block in the buffer cache. Otherwise, if the data block is an encrypted data block, the system retrieves a storage-key, wherein the storage-key is associated with a subset of storage, which is associated with the encrypted data block. Using the storage-key, the system then decrypts the encrypted data block to produce a decrypted data block. Finally, the system stores the decrypted data block in the buffer cache, wherein the data block remains encrypted in the storage medium.