Abstract: Region summaries of database data are stored in persistent memory of a storage cell. Because the region summaries are stored in persistent memory, when a storage cell is powered off and data in volatile memory is not retained, region summaries are nevertheless preserved in persistent memory. When the storage cell comes online, the region summaries already exist and may be used without the delay attendant to regenerating the region summaries stored in volatile memory.

Abstract: Region summaries of database data are stored in persistent memory of a storage cell. Because the region summaries are stored in persistent memory, when a storage cell is powered off and data in volatile memory is not retained, region summaries are nevertheless preserved in persistent memory. When the storage cell comes online, the region summaries already exist and may be used without the delay attendant to regenerating the region summaries stored in volatile memory.

Abstract: Region summaries of database data are stored in persistent memory of a storage cell. Because the region summaries are stored in persistent memory, when a storage cell is powered off and data in volatile memory is not retained, region summaries are nevertheless preserved in persistent memory. When the storage cell comes online, the region summaries already exist and may be used without the delay attendant to regenerating the region summaries stored in volatile memory.

Abstract: A storage system communicatively coupled to a database management system (DBMS performs storage-side scanning of data sources that are not stored in native database storage format of the DBMS. Data sources for external tables are accessible in a storage system referred to as a distributed data access system (DDAS), e.g. a Hadoop Distributed File System. To execute a query that references an external table, a DBMS first generates an execution plan. The DDAS supplies the DBMS with information that specifies each portion of the data source, and specifies which data node to use to access the portion. The DBMS sends a request for each portion to the respective data node, requesting that the data node generate rows from data in the portion. The request may specify scanning criteria, specifying one or more columns to project and/or filter on, and code modules for the data node to execute to generate records.

Abstract: A shared storage architecture persistently stores database files in non-volatile random access memories (NVRAMs) of computing nodes of a multi-node DBMS. The computing nodes of the multi-node DBMS not only collectively store database data on NVRAMs of the computing nodes, but also host database server instances that process queries in parallel, host database sessions and database processes, and together manage access to a database stored on the NVRAMs of the computing nodes. To perform a data block read operation from persistent storage, a data block may be transferred directly over a network between NVRAM of a computing node that persistently stores the data block to a database buffer in non-volatile RAM of another computing node that requests the data block. The transfer is accomplished using remote direct memory access (“RDMA).

Abstract: Embodiments of the present invention provide a hardware accelerator that assists a host database system in processing its queries. The hardware accelerator comprises special purpose processing elements that are capable of receiving database query/operation tasks in the form of machine code database instructions, execute them in hardware without software, and return the query/operation result back to the host system.

Abstract: Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Abstract: According to embodiments, a derived cache that is derived from a first instance of particular data is used to speed up queries and other operations over a second instance of the particular data. Traditionally, a DBMS generates and uses derived cache data only for the database data from which the derived data was derived. However, according to embodiments, derived cache data associated with a first instance of database data is relocated to the location of a second, newly created, instance of the database data. Since the derived cache data is derived from an identical copy of the database data, the cache data derived for the first instance can successfully be used to speed up applications running over the second instance of the database data.

Abstract: Techniques for optimizing a query with an extrema function are provided. In main memory, a data summary is maintained for a plurality of extents stored by at least one storage server. The data summary includes an extent minimum value and an extent maximum value for one or more columns. A storage server request is received, from a database server, based on a query with an extrema function applied to a particular column of a particular table. The data summaries for a set of relevant extents are processed by maintaining at least one global extrema value corresponding to the extrema function and, for each relevant extent of the set of relevant extents, determining whether to scan records of the relevant extent based on at least one of the global extrema value and an extent summary value of the data summary of the relevant extent.

Abstract: Region summaries of database data are stored in persistent memory of a storage cell. Because the region summaries are stored in persistent memory, when a storage cell is powered off and data in volatile memory is not retained, region summaries are nevertheless preserved in persistent memory. When the storage cell comes online, the region summaries already exist and may be used without the delay attendant to regenerating the region summaries stored in volatile memory.

Abstract: A shared storage architecture persistently stores database files in non-volatile random access memories (NVRAMs) of computing nodes of a multi-node DBMS. The computing nodes of the multi-node DBMS not only collectively store database data on NVRAMs of the computing nodes, but also host database server instances that process queries in parallel, host database sessions and database processes, and together manage access to a database stored on the NVRAMs of the computing nodes. To perform a data block read operation from persistent storage, a data block may be transferred directly over a network between NVRAM of a computing node that persistently stores the data block to a database buffer in non-volatile RAM of another computing node that requests the data block. The transfer is accomplished using remote direct memory access (“RDMA).

Abstract: Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query.

Abstract: Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query.

Abstract: Techniques are described herein for supporting multiple versions of a database server within a database machine comprising a separate database layer and storage layer. In an embodiment, the database layer includes compute nodes each hosting one or more instances of a database server. The storage layer includes storage nodes each hosting one or more instances of a storage server, also referred to herein as a “cell server.” In general, the database servers may receive data requests, such as SQL queries, from client applications and service the requests in coordination with the cell servers of the storage layer.

Abstract: Techniques are described herein for supporting multiple versions of a database server within a database machine comprising a separate database layer and storage layer. In an embodiment, the database layer includes compute nodes each hosting one or more instances of a database server. The storage layer includes storage nodes each hosting one or more instances of a storage server, also referred to herein as a “cell server.” In general, the database servers may receive data requests, such as SQL queries, from client applications and service the requests in coordination with the cell servers of the storage layer.

Abstract: According to embodiments, a derived cache that is derived from a first instance of particular data is used to speed up queries and other operations over a second instance of the particular data. Traditionally, a DBMS generates and uses derived cache data only for the database data from which the derived data was derived. However, according to embodiments, derived cache data associated with a first instance of database data is relocated to the location of a second, newly created, instance of the database data. Since the derived cache data is derived from an identical copy of the database data, the cache data derived for the first instance can successfully be used to speed up applications running over the second instance of the database data.

Abstract: Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Abstract: Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query.

Abstract: Embodiments of the present invention provide hardware-friendly indexing of databases. In particular, forward and reverse indexing are utilized to allow for easy traversal of primary key to foreign key relationships. A novel structure known as a hit list also allows for easy scanning of various indexes in hardware. Group indexing is provided for flexible support of complex group key definition, such as for date range indexing and text indexing. A Replicated Reordered Column (RRC) may also be added to the group index to convert random I/O pattern into sequential I/O of only needed column elements.

Abstract: Embodiments of the present invention provide one or more hardware-friendly data structures that enable efficient hardware acceleration of database operations. In particular, the present invention employs a column-store format for the database. In the database, column-groups are stored with implicit row ids (RIDs) and a RID-to-primary key column having both column-store and row-store benefits via column hopping and a heap structure for adding new data. Fixed-width column compression allow for easy hardware database processing directly on the compressed data. A global database virtual address space is utilized that allows for arithmetic derivation of any physical address of the data regardless of its location. A word compression dictionary with token compare and sort index is also provided to allow for efficient hardware-based searching of text. A tuple reconstruction process is provided as well that allows hardware to reconstruct a row by stitching together data from multiple column groups.