Abstract: Approaches, techniques, and mechanisms are disclosed for ensuring that a database command is executed according to a query plan that has been verified to be actually optimal. Except in rare circumstances, a database server does not execute a query plan unless it is first verified by the database server. The database server receives a request to execute a database command. The database server determines an unverified plan is the best plan for satisfying the request. Rather than risk the unknown behavior of an unverified plan, the database server instead satisfies the request according to a verified plan. Subsequently—for example as part of a scheduled job—the database server executes the unverified plan to determine performance statistics. Based at least on the performance statistics, the database server determines whether or not to verify the unverified plan. Techniques for concurrent and optimistic verifications are also disclosed.

Abstract: Techniques are described herein for filtering data from a table during an in-memory scan. Predicates are pushed to in-memory scan to avoid scanning unnecessary columnar units and reduce the overhead of decompressing, row stitching and distributing data during evaluation. Techniques are described herein for generating implied predicates that have conditions on single columns from complex predicates that have multiple conditions on the same column, which can be evaluated during an in-memory scan. Techniques are also described herein to reduce the overhead of a table scan for processing a join query. When redistributing a first table for performing a hash-join, the nodes performing an in-memory scan of the first table may create a filter that tracks unique values from the join key. Data from the second table is only processed and transferred to other nodes in the cluster if the values from the join key pass through the filter.

Abstract: Techniques are provided for determining costs for alternative execution plans for a query, where at least a portion of the data items required by the query are in in-memory compression-units within volatile memory. The techniques involve maintaining in-memory statistics, such as statistics that indicate what fraction of a table is currently present in in-memory compression units, and the cost of decompressing in-memory compression units. Those statistics are used to determine, for example, the cost of a table scan that retrieves some or all of the necessary data items from the in-memory compression-units.

Abstract: Techniques are described herein for filtering data from a table during an in-memory scan. Predicates are pushed to in-memory scan to avoid scanning unnecessary columnar units and reduce the overhead of decompressing, row stitching and distributing data during evaluation. Techniques are described herein for generating implied predicates that have conditions on single columns from complex predicates that have multiple conditions on the same column, which can be evaluated during an in-memory scan. Techniques are also described herein to reduce the overhead of a table scan for processing a join query. When redistributing a first table for performing a hash-join, the nodes performing an in-memory scan of the first table may create a filter that tracks unique values from the join key. Data from the second table is only processed and transferred to other nodes in the cluster if the values from the join key pass through the filter.

Abstract: Techniques are described herein for filtering data from a table during an in-memory scan. Predicates are pushed to in-memory scan to avoid scanning unnecessary columnar units and reduce the overhead of decompressing, row stitching and distributing data during evaluation. Techniques are described herein for generating implied predicates that have conditions on single columns from complex predicates that have multiple conditions on the same column, which can be evaluated during an in-memory scan. Techniques are also described herein to reduce the overhead of a table scan for processing a join query. When redistributing a first table for performing a hash-join, the nodes performing an in-memory scan of the first table may create a filter that tracks unique values from the join key. Data from the second table is only processed and transferred to other nodes in the cluster if the values from the join key pass through the filter.

Abstract: Techniques are described herein for accurately estimating the cost of performing a join against “mirror format” data that is at least partially located in volatile memory. Mirror format data is a copy of on-disk data (a.k.a. persistent format data) stored in a different format in volatile memory. In the preferred embodiment described herein, the mirror format data is stored in a column-major format, while the persistent format data is stored in a row-major format. The cost of performing a join query with a particular join method against mirror format data is different than the cost of performing the same particular join method against persistent format data. Techniques described herein for estimating the cost of multiple join query execution plans, and selecting the most optimized plan.

Abstract: Techniques are described herein for filtering data from a table during an in-memory scan. Predicates are pushed to in-memory scan to avoid scanning unnecessary columnar units and reduce the overhead of decompressing, row stitching and distributing data during evaluation. Techniques are described herein for generating implied predicates that have conditions on single columns from complex predicates that have multiple conditions on the same column, which can be evaluated during an in-memory scan. Techniques are also described herein to reduce the overhead of a table scan for processing a join query. When redistributing a first table for performing a hash-join, the nodes performing an in-memory scan of the first table may create a filter that tracks unique values from the join key. Data from the second table is only processed and transferred to other nodes in the cluster if the values from the join key pass through the filter.

Abstract: Techniques are described herein for accurately estimating the cost of performing a join against “mirror format” data that is at least partially located in volatile memory. Mirror format data is a copy of on-disk data (a.k.a. persistent format data) stored in a different format in volatile memory. In the preferred embodiment described herein, the mirror format data is stored in a column-major format, while the persistent format data is stored in a row-major format. The cost of performing a join query with a particular join method against mirror format data is different than the cost of performing the same particular join method against persistent format data. Techniques described herein for estimating the cost of multiple join query execution plans, and selecting the most optimized plan.

Abstract: Techniques are provided for determining costs for alternative execution plans for a query, where at least a portion of the data items required by the query are in in-memory compression-units within volatile memory. The techniques involve maintaining in-memory statistics, such as statistics that indicate what fraction of a table is currently present in in-memory compression units, and the cost of decompressing in-memory compression units. Those statistics are used to determine, for example, the cost of a table scan that retrieves some or all of the necessary data items from the in-memory compression-units.

Abstract: Approaches, techniques, and mechanisms are disclosed for maintaining a set of baseline query plans for a database command. Except in rare circumstances, a database server may only execute a command according to a baseline plan, even if the database server predicts that a different plan has a lower cost. The set of baseline plans are plans that, for one reason or another, have been determined to provide acceptable actual performance in at least one execution context. When the database server receives a request to execute a particular command, the database server, if possible, always executes the command according to the lowest predicted cost baseline plan. The database server may evolve the plan baseline to include additional plans by generating and testing new plans in response to new requests to execute the database command, or as part of a query optimization or tuning process.

Abstract: Approaches, techniques, and mechanisms are disclosed for maintaining a set of baseline query plans for a database command. Except in rare circumstances, a database server may only execute a command according to a baseline plan, even if the database server predicts that a different plan has a lower cost. The set of baseline plans are plans that, for one reason or another, have been determined to provide acceptable actual performance in at least one execution context. When the database server receives a request to execute a particular command, the database server, if possible, always executes the command according to the lowest predicted cost baseline plan. The database server may evolve the plan baseline to include additional plans by generating and testing new plans in response to new requests to execute the database command, or as part of a query optimization or tuning process.

Abstract: Approaches, techniques, and mechanisms are disclosed for ensuring that a database command is executed according to a query plan that has been verified to be actually optimal. Except in rare circumstances, a database server does not execute a query plan unless it is first verified by the database server. The database server receives a request to execute a database command. The database server determines an unverified plan is the best plan for satisfying the request. Rather than risk the unknown behavior of an unverified plan, the database server instead satisfies the request according to a verified plan. Subsequently—for example as part of a scheduled job—the database server executes the unverified plan to determine performance statistics. Based at least on the performance statistics, the database server determines whether or not to verify the unverified plan. Techniques for concurrent and optimistic verifications are also disclosed.

Abstract: Approaches, techniques, and mechanisms are disclosed for ensuring that a database command is executed according to a query plan that has been verified to be actually optimal. Except in rare circumstances, a database server does not execute a query plan unless it is first verified by the database server. The database server receives a request to execute a database command. The database server determines an unverified plan is the best plan for satisfying the request. Rather than risk the unknown behavior of an unverified plan, the database server instead satisfies the request according to a verified plan. Subsequently—for example as part of a scheduled job—the database server executes the unverified plan to determine performance statistics. Based at least on the performance statistics, the database server determines whether or not to verify the unverified plan. Techniques for concurrent and optimistic verifications are also disclosed.

Abstract: Approaches, techniques, and mechanisms are disclosed for capturing and utilizing information related to query plans exhibiting interesting characteristics. A database server receives a request to execute a command. The database server executes the command according to a query plan. In response to determining that the command matches one or more pre-defined criteria, the database server captures information related to the execution of the first command. The criteria may include, for example, whether or not the command is repeatable, the existence of bind variables, access of a particular object, high resource utilization, receipt from a particular user, client, or application, etc. The information recorded may include, for example, performance statistics collected during execution of the first plan, data indicating the execution context during execution of the first plan, and properties of the first plan.

Abstract: Approaches, techniques, and mechanisms are disclosed for maintaining a history of query plans executed for a database command, along with information related to each query plan. A database server receives a request to execute a particular command. The database server determines a plan for executing the particular command. The database server adds first information to a plan history associated with the particular command. The plan history comprises information related to a plurality of plans that have been generated for the particular command. The first information may include, for example, properties of the plan (including an outline of the plan) as well as statistics collected during execution of the plan. The database server may implement techniques for periodically refreshing information in a plan history. The database server may also implement techniques for purging old or less important plans.

Abstract: In one embodiment, a database server registers one or more functions included in a user-defined index that includes one or more auxiliary properties. The one or more functions are operable to generate index entries of the user-defined index for the one or more auxiliary properties. The one or more auxiliary properties are different than, and in addition to, a primary property of the user-defined index that provides for evaluating a query operator. The database server receives a first statement that defines the user-defined index, where the first statement includes one or more parameters that define the one or more auxiliary properties. The database server invokes the one or more functions in response to the first statement, where the one or more functions when invoked generate and store the index entries of the user-defined index according to the one or more parameters that define the one or more auxiliary properties.

Abstract: Join predicate push down transformations push down a join predicate of an outer query into a view. Among the types of views for which join predicate push down is performed are a view with a GROUP BY or DISTINCT operator, an anti-joined or semi-joined view, and a view that contains one or more nested views. During optimization, join predicate push down may be used to generate many transformed queries for comparison. The number of query transformations performed for comparison is managed.

Abstract: In one embodiment, a database server registers one or more functions included in a user-defined index that includes one or more auxiliary properties. The database server receives a query that specifies one or more parameters. A query optimizer generates a plurality of execution plans for evaluating the query. The query optimizer estimates a plurality of total costs respectively associated with the plurality of execution plans, where estimating a particular total cost associated with a particular execution plan comprises: invoking the one or more functions with one or more function arguments based on the one or more parameters; receiving one or more index costs that are associated with using the one or more auxiliary properties of the user-defined index to evaluate the query; and determining the particular total cost based on the one or more index costs.

Abstract: Techniques are provided for delaying evaluation of expensive expressions in a query. Expensive expressions in the query are established by cost information or by looking up a list of known expensive expressions for a match. After an execution plan is determined by using the early evaluation technique, one or more equivalent execution plans is established. The one or more equivalent execution plans may include both a type of execution plans that delay evaluation of expensive expressions and a type of execution plans that do not. In addition, the one or more equivalent execution plans may include both parallelized and non-parallelized alternatives to the execution plan identified by the early evaluation technique. Finally, based on a set of criteria, which may include comparing cost information among all the equivalent execution plans generated thus far, the best execution plan is chosen for the query.

Abstract: Auto-tuning can be performed by receiving a database query language statement and performance information related to the statement, determining whether one or more performance statistics of the statement are available or missing in the performance information, and determining an auto-tuning hint for each missing statistic.