Abstract: Techniques are provided for sharding objects across different compute nodes. In one embodiment, a database server instance generates, for an object, a plurality of in-memory chunks including a first in-memory chunk and a second in-memory chunk, where each in-memory chunk includes a different portion of the object. The database server instance assigns each in-memory chunk to one of a plurality of computer nodes including the first in-memory chunk to a first compute node and a second in-memory chunk to a second local memory of a second compute node. The database server instance stores an in-memory map that indicates a memory location for each in-memory chunk. The in-memory map indicates that the first in-memory chunk is located in the first local memory of the first compute node and that the second in-memory chunk is located in the second local memory of the second compute node.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: Techniques are described herein for distributing distinct portions of a database object across the volatile memories of a plurality of nodes in a clustered database system. The techniques involve establishing a single database server instance located on a node in a multi-node cluster as a load-operation master for a particular data set. The load-operation master determines how the data set may be separated into chunks using a hash function. The load-operation master then broadcasts a small payload of consistency information to other database servers, so each database server may independently execute the hash function and independently load their respectively assigned chunks of data.

Abstract: Techniques are described herein for executing queries on distinct portions of a database object that has been separate into chunks and distributed across the volatile memories of a plurality of nodes in a clustered database system. The techniques involve receiving a query that requires work to be performed on data that resides in a plurality of on disk extents. A parallel query coordinator that is aware of the in-memory distribution divides the work into granules that align with the in-memory separation. The parallel query coordinator then sends each granule to the database server instance with local in memory access to the data required by the granule and aggregates the results to respond to the query.

Abstract: Techniques are described herein for distributing data from one or more partitioned tables across the volatile memories of a cluster. In memory copies of data from partitioned tables are grouped based on the data falling within the same partition criteria. These groups are used for assigning data from corresponding partitions to the same node when distributing data from partitioned tables across the volatile memories of a multi-node cluster. When a query requires a join between rows of partitioned tables, the work for the join query is divided into work granules that correspond to partition-wise join operations. Those partition-wise join operations are assigned to nodes by a query coordinator based on the partition-to-node mapping located in the node of the query coordinator.

Abstract: Techniques are provided for more efficiently using the bandwidth of the I/O path between a CPU and volatile memory during the performance of database operation. Relational data from a relational table is stored in volatile memory as column vectors, where each column vector contains values for a particular column of the table. A binary-comparable format may be used to represent each value within a column vector, regardless of the data type associated with the column. The column vectors may be compressed and/or encoded while in volatile memory, and decompressed/decoded on-the-fly within the CPU. Alternatively, the CPU may be designed to perform operations directly on the compressed and/or encoded column vector data. In addition, techniques are described that enable the CPU to perform vector processing operations on the column vector values.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: For automatic data placement of database data, a plurality of access-tracking data is maintained. The plurality of access-tracking data respectively corresponds to a plurality of data rows that are managed by a database server. While the database server is executing normally, it is automatically determined whether a data row, which is stored in first one or more data blocks, has been recently accessed based on the access-tracking data that corresponds to that data row. After determining that the data row has been recently accessed, the data row is automatically moved from the first one or more data blocks to one or more hot data blocks that are designated for storing those data rows, from the plurality of data rows, that have been recently accessed.

Abstract: Techniques are described herein for storing and processing codes included in dictionary-encoded data. In an embodiment, for each respective code of a plurality of codes in the dictionary-encoded data: a plurality of bits from a first portion of the respective code is contiguously stored. One or more bits from a second portion of the respective code is stored in one or more slices. Each respective slice of the one or more slices stores a bit from the one or more bits with a corresponding bit position in the respective code. In another embodiment, a bit-vector is generated based on at least one slice by loading each respective bit of the plurality of bits into different respective partitions in a register at a bit position corresponding to the at least one slice.

Abstract: A method, apparatus, and system for OZIP, a data compression and decompression codec, is provided. OZIP utilizes a fixed size static dictionary, which may be generated from a random sampling of input data to be compressed. Compression by direct token encoding to the static dictionary streamlines the encoding and avoids expensive conditional branching, facilitating hardware implementation and high parallelism. By bounding token definition sizes and static dictionary sizes to hardware architecture constraints such as word size or processor cache size, hardware implementation can be made fast and cost effective. For example, decompression may be accelerated by using SIMD instruction processor extensions. A highly granular block mapping in optional stored metadata allows compressed data to be accessed quickly at random, bypassing the processing overhead of dynamic dictionaries. Thus, OZIP can support low latency random data access for highly random workloads, such as for OLTP systems.

Abstract: In a database system having a plurality of concurrently executing session processes, the method commences by establishing a master list of sequences, the master list comprising a plurality of sequence objects which in turn define a sequence of values used for numbering and other identification within the database system. To reduce sequence cache latch access contention, multiple tiers of latches are provided. Methods of the system provide a first tier having a first tier “global” latch to serialize access to the master list. A second tier of latches is provided, the second tier having multiple second tier latches to serialize access to corresponding allocated sequences of values such that at any point in time, only one of the concurrently executing session processes is granted access to the allocated sequence.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: For automatic data placement of database data, a plurality of access-tracking data is maintained. The plurality of access-tracking data respectively corresponds to a plurality of data rows that are managed by a database server. While the database server is executing normally, it is automatically determined whether a data row, which is stored in first one or more data blocks, has been recently accessed based on the access-tracking data that corresponds to that data row. After determining that the data row has been recently accessed, the data row is automatically moved from the first one or more data blocks to one or more hot data blocks that are designated for storing those data rows, from the plurality of data rows, that have been recently accessed.

Abstract: Techniques are provided for managing in-memory space and objects. In one embodiment, a set of in-memory objects are maintained within an area in volatile memory that is accessible to a database server. An in-memory object in this context includes a set of one or more in-memory segments where each respective in-memory segment includes a set of in-memory extents and each respective in-memory extent is a contiguous chunk of memory from the area in volatile memory that is accessible to the database server. The area in volatile memory is managed as a set of stripes, where each stripe is a contiguous chunk of in-memory extents. Stripe control blocks are used to locate free in-memory extents for allocation and registration with an in-memory segment.

Abstract: Techniques are provided for granular load and refresh of columnar data. In an embodiment, a particular data object that contains particular data formatted different from column-major format is maintained, the particular data including first data and second data. First and second data objects contain the first and second data, respectively, organized in the column-major format. In response to changes being committed to the first data in the particular data object, invalidating one or more rows of the first data object. In response to a number of invalidated rows of the first data object exceeding a threshold, automatically performing a refresh operation on the first data object independent of any refresh operation on the second data object.

Abstract: Techniques are provided for performing parallel processing on in-memory objects within a database system. In one embodiment, a plurality of in-memory chunks are maintained on a plurality of non-uniform memory access (NUMA) nodes. In response to receiving a query, a set of clusters is determined for the plurality of in-memory chunks. Each respective cluster in the set of clusters corresponds to a particular NUMA node of the plurality of NUMA nodes and includes a set of one or more in-memory chunks from the plurality of in-memory chunks. For each respective cluster in the set of clusters, a query coordinator assigns, to the respective cluster, a set of one or more processes associated with the particular NUMA node that corresponds to the respective cluster.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: Techniques are provided for maintaining data persistently in one format, but making that data available to a database server in more than one format. For example, one of the formats in which the data is made available for query processing is based on the on-disk format, while another of the formats in which the data is made available for query processing is independent of the on-disk format. Data that is in the format that is independent of the disk format may be maintained exclusively in volatile memory to reduce the overhead associated with keeping the data in sync with the on-disk format copies of the data.

Abstract: For automatic data placement of database data, a plurality of access-tracking data is maintained. The plurality of access-tracking data respectively corresponds to a plurality of data rows that are managed by a database server. While the database server is executing normally, it is automatically determined whether a data row, which is stored in first one or more data blocks, has been recently accessed based on the access-tracking data that corresponds to that data row. After determining that the data row has been recently accessed, the data row is automatically moved from the first one or more data blocks to one or more hot data blocks that are designated for storing those data rows, from the plurality of data rows, that have been recently accessed.