Abstract: Embodiments of the present invention relate to elimination of blocks such as splits in distributed processing systems such as MapReduce systems using the Hadoop Distributed Filing System (HDFS). In one embodiment, a method of and computer program product for optimizing queries in distributed processing systems are provided. A query is received. The query includes at least one predicate. The query refers to data. The data includes a plurality of records. Each record comprises a plurality of values in a plurality of attributes. Each record is located in at least one of a plurality of blocks of a distributed file system. Each block has a unique identifier. For each block of the distributed file system, at least one value cluster is determined for an attribute of the plurality of attributes. Each value cluster has a range. The predicate of the query is compared with the at least one value cluster of each block.

Abstract: According to one embodiment of the present invention, a method for processing join predicates in full-text indexes is provided. The method includes evaluating local predicates of an outer full text index to generate a first posting list of documents. For each document in the first posting list, the value of a join attribute is determined and an inner full text index is probed to obtain a second posting list of documents containing one of the join attributes determined for each document. Local predicates of an inner full text index are evaluated to generate a third posting list of documents, and the second posting list is merged with the third posting list to generate a merge list of documents. Documents in the first posting list may be paired up with documents in the merge list.

Abstract: According to one embodiment of the present invention, a method for processing join predicates in full-text indexes is provided. The method includes evaluating local predicates of an outer full text index to generate a first posting list of documents. For each document in the first posting list, the value of a join attribute is determined and an inner full text index is probed to obtain a second posting list of documents containing one of the join attributes determined for each document. Local predicates of an inner full text index are evaluated to generate a third posting list of documents, and the second posting list is merged with the third posting list to generate a merge list of documents. Documents in the first posting list may be paired up with documents in the merge list.

Abstract: An apparatus, system, and method are disclosed for optimization of XPath expressions in a database management system configured to accept XML queries. Operations of the method include receiving an XQuery representation and partitioning XPath expressions within the XQuery representation into a plurality of XPath expression clusters. The XPath expression clusters may comprise one or more XPath expressions and those in each cluster may operate on a common document. Furthermore, the XPath expressions in each cluster are hierarchically related to each other such that branch nodes of the cluster are executable independent of nodes in other XPath expression clusters. The method also defines merging the one or more XPath expressions into one or more expression trees for each XPath expression cluster. The method generates one or more query execution plans from the one or more XPath expression blocks.

Abstract: A method and apparatus is disclosed for XML query evaluation using early-outs and multiple passes to evaluate an XML query. A multi-pass evaluation procedure evaluates the XML query one step at a time as needed to complete evaluation. The multi-pass evaluation procedure evaluates XML queries containing logical expressions such as “AND” expressions, “OR” expressions, and implied “AND” expressions within “FOR” clauses. Queries containing logical expressions are often satisfied before every component is evaluated. Thus, executing the multi-pass evaluation procedure allows the evaluation to exit early when the veracity of the query is determined, not necessarily when every component has been evaluated. The multi-pass evaluation procedure executes as long as a descendant axis of the XML query need not be evaluated past a child node. When evaluation of a descendant axis past a child node is required, the multi-pass evaluation procedure may switch to a single-pass evaluation procedure to complete evaluation.

Abstract: An apparatus, system, and method are disclosed for optimization of XPath expressions in a database management system configured to accept XML queries. Operations of the method include receiving an XQuery representation and partitioning XPath expressions within the XQuery representation into a plurality of XPath expression clusters. The XPath expression clusters may comprise one or more XPath expressions and those in each cluster may operate on a common document. Furthermore, the XPath expressions in each cluster are hierarchically related to each other such that branch nodes of the cluster are executable independent of nodes in other XPath expression clusters. The method also defines merging the one or more XPath expressions into one or more expression trees for each XPath expression cluster. The method generates one or more query execution plans from the one or more XPath expression blocks.

Abstract: A method and apparatus is disclosed for XML query evaluation using early-outs and multiple passes to evaluate an XML query. A multi-pass evaluation procedure evaluates the XML query one step at a time as needed to complete evaluation. The multi-pass evaluation procedure evaluates XML queries containing logical expressions such as “AND” expressions, “OR” expressions, and implied “AND” expressions within “FOR” clauses. Queries containing logical expressions are often satisfied before every component is evaluated. Thus, executing the multi-pass evaluation procedure allows the evaluation to exit early when the veracity of the query is determined, not necessarily when every component has been evaluated. The multi-pass evaluation procedure executes as long as a descendant axis of the XML query need not be evaluated past a child node. When evaluation of a descendant axis past a child node is required, the multi-pass evaluation procedure may switch to a single-pass evaluation procedure to complete evaluation.

Abstract: A method for using pre-computed information stored in auxiliary structures to speed up processing of expensive queries on hierarchical documents such as XML documents being queried using XPath. The invention defines a taxonomy of such structures such as indexes and materialized views for storing pre-computed XPath results (PXRs), determines what portion of the query can be evaluated by the structures, and computes the compensation for the results generated by the structures. The invention detects all structures applicable to the query and rewrites the query to use such structures, speeding up the performance of the queries. The invention identifies the matching structures by detecting containment mappings between XPath expressions in the query and the structure. The invention also includes a new representation for XPath expressions that is rich enough to express all features of XPath.

Abstract: An XML query compilation processor (20) includes an execution compiler (42) that transforms an XML query into an executable XML query plan (22). A query rewrite processor (34) performs query transformations on the XML query. Said query transformations include transforming an expression input (60) received by an expression (62) conditional upon a set of items defined by an output (64) of the expression (62) being independent of grouping of items in the expression input (60).

Abstract: An XML query compilation processor (20) includes an execution compiler (42) that transforms an XML query into an executable XML query plan (22). A query rewrite processor (34) performs query transformations on the XML query. Said query transformations include transforming an expression input (60) received by an expression (62) conditional upon a set of items defined by an output (64) of the expression (62) being independent of grouping of items in the expression input (60).

Abstract: A method for using pre-computed information stored in auxiliary structures to speed up processing of expensive queries on hierarchical documents such as XML documents being queried using XPath. The invention defines a taxonomy of such structures such as indexes and materialized views for storing pre-computed XPath results (PXRs), determines what portion of the query can be evaluated by the structures, and computes the compensation for the results generated by the structures. The invention detects all structures applicable to the query and rewrites the query to use such structures, speeding up the performance of the queries. The invention identifies the matching structures by detecting containment mappings between XPath expressions in the query and the structure. The invention also includes a new representation for XPath expressions that is rich enough to express all features of XPath.