Abstract: A data processing system is provided that comprises a processor, a random access memory for storing data and programs for execution by the processor, and computer readable instructions stored in the random access memory for execution by the processor to perform a method for clustering data points in a multidimensional dataset in a multidimensional image space. The method comprises generating a multidimensional image from the multidimensional dataset; generating a pyramid of multidimensional images having varying resolution levels by successively performing a pyramidal sub-sampling of the multidimensional image; identifying data clusters at each resolution level of the pyramid by applying a set of perceptual grouping constraints; and determining levels of a clustering hierarchy by identifying each salient bend in a variation curve of a magnitude of identified data clusters as a function of pyramid resolution level.

Abstract: A method for predictable query execution through early materialization is provided. The method deals with the problem of cardinality misestimation in query execution plans, by pre-executing sub-plans on a query execution plan that have questionable estimates and collecting statistics on the output of these sub-plans. If needed, the overall query execution plan is changed in light of these statistics, before optimizing and executing the remainder of the query.

Abstract: Provided is a method for modeling the cost of XML as well as relational operators. As with traditional relational cost estimation, a set of system catalog statistics that summarizes the XML data is exploited; however, the novel use of a set of simple path statistics is also proposed. A new statistical learning technique called transform regression is utilized instead of detailed analytical models to predict the overall cost of an operator. Additionally, a query optimizer in a database is enabled to be self-tuning, automatically adapting to changes over time in the query workload and in the system environment.

Abstract: Disclosed is a method for directing a data processing system, the data processing system and an article of manufacture. The data processing system has memory for storing a database. The method directs the data processing system to configuring the database based on a database workload having a performance metric. The method includes selecting a plurality of database characteristics based upon the database workload, each of the database characteristics having an associated estimated performance metric for the database workload, selecting a set of database characteristics from the plurality of database characteristics for which the associated estimated performance metric is lower than the database workload performance metric, and configuring the database using the selected set of database characteristics.

Abstract: The present invention provides a method, system and program for optimizing compression of a workload processed by a database management system. In an embodiment of the present invention a method of optimizing the compression of database workloads is provided. Initially, an estimate of a cost of execution for each query according to a defined metric such as execution time or memory consumption is determined. A sub-set of queries is then selected from the workload in order of the most costly to least costly relative to the defined metric for compression according to either a predetermined compression threshold percentage or a threshold percentage derived from an allotted workload execution time. Compression is then performed on the selected sub-set of queries (i.e. those that will benefit the most from the compression) to achieve a net beneficial trade-off between the cost of workload compression and the cost of workload execution.

Abstract: A method, system, and computer program product to make query processing more robust in the face of optimization errors. The invention validates the statistics and assumptions used for compiling a query as the query is executed and, when necessary, progressively re-optimizes the query in mid-execution based on the knowledge learned during its partial execution. The invention selectively places a number of CHECK operators in a query execution plan to validate the optimizer's cardinality estimates against actual cardinalities. Errors beyond a threshold trigger re-optimization, and the optimizer decides whether the old plan is still optimal and whether to re-use previously computed results. The invention addresses arbitrary SQL queries whose plans can contain sub-queries, updates, trigger checking, and view maintenance operations.

Abstract: A compilation time estimator provides a quantified estimate of the optimizer compilation time for a given query optimizer. The estimator automates the optimizer to choose the right level of optimization in commercial database systems. The estimator reuses an optimizer's join enumerator to obtain actual number of joins, but bypasses plan generation to save estimation overhead, and maintains a small number of interesting physical properties to estimate the number of plans by using a linear regression model. The estimator uses the number of generated plans to estimate query compilation time.