Abstract: A method for distributing tables to a cluster of nodes managed by database management system (DBMS), is disclosed. Multiple data placement schemes are evaluated based on a query workload set to select a data placement scheme for the cluster of nodes. Tables, used in join operations in the workload set, are selected for evaluation of data placement schemes. Query execution costs for the workload set are generated based on estimating a query execution cost for each data placement scheme for the tables. The data placement scheme that has least costly estimated execution cost for the workload set is selected as the data placement scheme for the cluster of nodes managed by DBMS.

Abstract: Herein is described a data placement scheme for a distributed query processing systems that achieves load balance amongst the nodes of the system. To identify a node on which to place particular data, a supervisor node performs a placement algorithm over the particular data's identifier, where the placement algorithm utilizes two or more hash functions. The supervisor node runs the placement algorithm until a destination node is identified that is available to store the data, or the supervisor node has run the placement algorithm an established number of times. If no available node is identified using the placement algorithm, then an available destination node is identified for the particular data and information identifying the data and the selected destination node is included in an exception map. Most data may be located by any node in the system based on the node performing the placement algorithm for the required data.

Abstract: Techniques related to version control based on a dual-range validity model are disclosed. In an embodiment, an online analytical processing (OLAP) server stores a plurality of version records describing versions of a data item. A version record may describe any open transactions for a version of the data item. The version record may specify a commit timestamp for the data item at a database and a valid timestamp at least as great as the commit timestamp. The commit timestamp and the valid timestamp may specify a validity range. The version record may also specify an expiration timestamp, which along with the valid timestamp may specify an unresolved range. The OLAP server may also identify a valid version of the data item for a query timestamp that corresponds to a query for particular data in the data item and that falls within either the validity range or the unresolved range.

Abstract: Data recovery for a compute node in a heterogeneous database system is provided. A failure is detected of a particular compute node of a compute cluster comprising a plurality of compute nodes. The compute cluster is configured to store, in memory, data stored by a RDBMS. Particular data of the data stored by the RDBMS is identified that is assigned to the particular compute node. The particular compute node is restored. After restoring the particular compute node, the particular data assigned to the particular compute node is reloaded without taking the particular data offline. During reloading, the particular compute node receives pending modified data comprising data of the particular data that was modified during said reloading.

Abstract: Techniques for performing database operations using vectorized instructions are provided. In one technique, a hash table probe phase involves executing vectorized instructions to determine where in a bucket a particular key is located. This determination may be preceded by one or more vectorized instructions that are used to determine whether the bucket contains the particular key.

Abstract: A database management server manages a distributed data set at least partially stored in volatile memory of a plurality of nodes. The database management server stores transaction logs in persistent storage corresponding to one or more database transactions performed on the distributed data set. In response to a query, a join operation is performed with assistance from the cluster. A snapshot identifier identifies a particular database state that is transactionally consistent. Based on the transactional logs, updated data blocks updated after the snapshot identifier of the query are identified. The cluster is instructed to perform one or more distributed join operations on the distributed data set without processing data in the updated data blocks. The database management server performs a partial join evaluation on the updated data blocks.

Abstract: Techniques related to version control based on a dual-range validity model are disclosed. In an embodiment, an online analytical processing (OLAP) server stores a plurality of version records describing versions of a data item. A version record may describe any open transactions for a version of the data item. The version record may specify a commit timestamp for the data item at a database and a valid timestamp at least as great as the commit timestamp. The commit timestamp and the valid timestamp may specify a validity range. The version record may also specify an expiration timestamp, which along with the valid timestamp may specify an unresolved range. The OLAP server may also identify a valid version of the data item for a query timestamp that corresponds to a query for particular data in the data item and that falls within either the validity range or the unresolved range.

Abstract: A method and apparatus for data recovery for a RDBMS instance in a heterogeneous database system is provided. A failure of a first RDBMS instance is detected in a plurality of RDBMS instances of a shared-disk database system. A compute cluster is configured to store, in memory, one or more tables stored by the shared-disk database system. The first RDBMS instance is configured to modify the one or more tables stored by the shared-disk database system and transfer modified data to the compute cluster to update the one or more tables at the compute cluster. After detecting the failure of the first RDBMS instance, redo records generated by the first RDBMS instance are scanned, pending modified data that was not transferred to the compute cluster before the failure is identified, and the pending modified data is transferred to the compute cluster.

Abstract: Herein is described a data placement scheme for a distributed query processing systems that achieves load balance amongst the nodes of the system. To identify a node on which to place particular data, a supervisor node performs a placement algorithm over the particular data's identifier, where the placement algorithm utilizes two or more hash functions. The supervisor node runs the placement algorithm until a destination node is identified that is available to store the data, or the supervisor node has run the placement algorithm an established number of times. If no available node is identified using the placement algorithm, then an available destination node is identified for the particular data and information identifying the data and the selected destination node is included in an exception map. Most data may be located by any node in the system based on the node performing the placement algorithm for the required data.

Abstract: Data recovery for a compute node in a heterogeneous database system is provided. A failure is detected of a particular compute node of a compute cluster comprising a plurality of compute nodes. The compute cluster is configured to store, in memory, data stored by a RDBMS. Particular data of the data stored by the RDBMS is identified that is assigned to the particular compute node. The particular compute node is restored. After restoring the particular compute node, the particular data assigned to the particular compute node is reloaded without taking the particular data offline. During reloading, the particular compute node receives pending modified data comprising data of the particular data that was modified during said reloading.

Abstract: A method and apparatus for data recovery for a RDBMS instance in a heterogeneous database system is provided. A failure of a first RDBMS instance is detected in a plurality of RDBMS instances of a shared-disk database system. A compute cluster is configured to store, in memory, one or more tables stored by the shared-disk database system. The first RDBMS instance is configured to modify the one or more tables stored by the shared-disk database system and transfer modified data to the compute cluster to update the one or more tables at the compute cluster. After detecting the failure of the first RDBMS instance, redo records generated by the first RDBMS instance are scanned, pending modified data that was not transferred to the compute cluster before the failure is identified, and the pending modified data is transferred to the compute cluster.

Abstract: Techniques are described to evaluate an operation from an execution plan of a query to offload the operation to another database management system for less costly execution. In an embodiment, the execution plan is determined based on characteristics of the database management system that received the query for execution. One or more operations in the execution plan are then evaluated for offloading to another heterogeneous database management system. In a related embodiment, the offloading cost for each operation may also include communication cost between the database management systems. The operations that are estimated to be less costly to execute on the other database management system are then identified for offloading to the other database management system. In an alternative embodiment, the database management system generates permutations of execution plans for the same query, and similarly evaluates each permutation of the execution plans for offloading its one or more operations.

Abstract: A method for distributing tables to a cluster of nodes managed by database management system (DBMS), is disclosed. Multiple data placement schemes are evaluated based on a query workload set to select a data placement scheme for the cluster of nodes. Tables, used in join operations in the workload set, are selected for evaluation of data placement schemes. Query execution costs for the workload set are generated based on estimating a query execution cost for each data placement scheme for the tables. The data placement scheme that has least costly estimated execution cost for the workload set is selected as the data placement scheme for the cluster of nodes managed by DBMS.

Abstract: Techniques for performing database operations using vectorized instructions are provided. In one technique, an aggregation operation involves executing vectorized instructions to update a data value that corresponds to a particular key. The aggregation operation may be one of count, sum, minimum, maximum, or average.

Abstract: A two-level cache to facilitate resolving resource path expressions for a hierarchy of resources is described, which includes a system-wide shared cache and a session-level cache. The shared cache is organized as a hierarchy of hash tables that mirrors the structure of a repository hierarchy. A particular hash table in a shared cache includes information for the child resources of a particular resource. A database management system that manages a shared cache may control the amount of memory used by the cache by implementing a replacement policy for the cache based on one or more characteristics of the resources in the repository. The session-level cache is a single level cache in which information for target resources of resolved path expressions may be tracked. In the session-level cache, the resource information is associated with the entire path expression of the associated resource.

Abstract: A system and method for performing a query operation on a pair of relations in a database system coupled to a heterogeneous system (HS) is disclosed. Assuming that that pair of relations is partitioned and already loaded into the HS, the database system receives a query on the pair of relations and based on the type of query operation computes the cost of performing the query operation on the database alone or the costs of performing the query operation with the assistance of the HS, each of the costs corresponding to a particular algorithm. If the costs indicate that the HS improves the performance of the query operation, then the HS computes portions of the operation, and returns the results back to the database system. If any parts of the relation are out of sync with the database system, the database system performs operations to maintain transactional consistency.

Abstract: A system and method for improving performance of join operations is disclosed. The method in one embodiment includes building a graph of vertices and edges, which represent the tables and join conditions for a particular database schema. The method then traverses the graph to find a set of candidate join operations. The method further includes selecting from the set of subset of join operations that makes the best use of a given amount of memory, which means finding a collection of denormalizations which when residing in memory provide the most benefit and the least cost for the given amount of memory. The method then deploys the denormalizations so that current queries in a query processing system can use them instead of actually performing the joins.

Abstract: Techniques for performing database operations using vectorized instructions are provided. In one technique, a hash table probe phase involves executing vectorized instructions to determine where in a bucket a particular key is located. This determination may be preceded by one or more vectorized instructions that are used to determine whether the bucket contains the particular key.

Abstract: Techniques for performing database operations using vectorized instructions are provided. In one technique, an aggregation operation involves executing vectorized instructions to update a data value that corresponds to a particular key. The aggregation operation may be one of count, sum, minimum, maximum, or average.

Abstract: A database management server manages a distributed data set at least partially stored in volatile memory of a plurality of nodes. The database management server stores transaction logs in persistent storage corresponding to one or more database transactions performed on the distributed data set. In response to a query, a join operation is performed with assistance from the cluster. A snapshot identifier identifies a particular database state that is transactionally consistent. Based on the transactional logs, updated data blocks updated after the snapshot identifier of the query are identified. The cluster is instructed to perform one or more distributed join operations on the distributed data set without processing data in the updated data blocks. The database management server performs a partial join evaluation on the updated data blocks.