Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for handling faults in a distributed key-value storage system. One of the methods includes receiving an indication that a machine storing a primary replica of a first replication chain is inactive, in response to receiving the indication, generating a concatenated replica comprising a first replica of the first replication chain and a second replica of a second replication chain, the second replication chain comprising replicas of a second key segment, the second key segment being adjacent to the first key segment in the multiple key segments of the plurality of keys, and providing, to another machine in the ordered sequence of machines, a notification of availability of the concatenated replica.

Abstract: A method for indexing a plurality of documents, that includes a plurality of duplicate documents, first identifies one or more duplicate groups of documents from among the plurality of documents. Then, one index of content for the duplicate group is created instead of indexing the content from every document within the duplicate group. However, in contrast to the content index, an index of metadata for each of the documents in the duplicate group is created. Thus the content of each duplicate group is indexed only once, while a search engine using such indexing techniques retains the capability to answer queries as if the duplicated content was indexed for each document of the group.

Abstract: A method for storing XML documents in a relational database system while exploiting XML Schema information to capture information about types, inheritance, equivalence classes, and integrity constraints in the generated relational schema, enabling efficient querying. The invention simplifies complex XML schema types, translates the simplified XML schema types into relational tables, and then stores relations corresponding to each XML element in relational table rows. The simplification includes grouping all occurrences of a given element together, assembling two or more element types into element groups if the schema indicates that elements of those element types will occur the same number of times, and applying a number of transformation rules to the element groups. The translation includes constructing a type graph from the simplified schema, building an element graph for each global element in the simplified schema from the type graph, and generating relational tables from the element graph.

Abstract: A system, method, and computer program product for processing XML queries over relational data and meta-data using a relational database system. The invention runs a query to generate a view definition, so a regular query processor can handle queries on a mix of data and meta-data. Names of desired tables are data-dependent, so the invention represents a default XML view as an XML query over the relational tables, gets the table names and table data by recursively invoking the query processor (first by querying the catalog table, and then querying base data according to a new construct). The invention thus processes queries over table names in addition to table values.

Abstract: A method for publishing relational data as XML by translating XML queries into queries against an relational database. Conversion of the relational database into an XML database is not required. Each relational table is mapped to a virtual XML document, and XML queries are issued over these virtual documents. An XML query is parsed and transformed into a language-neutral intermediate representation, which is a sequence of operations describing how the output document is derived from the underlying relational tables. The intermediate representation is then translated into an SQL query over the underlying relational tables and into instructions for a tagger. The SQL query is executed, and the SQL query results are then fed into the tagger, which follows tagger instructions to generate the marked up output.

Abstract: A method for publishing relational data as XML by translating XML queries into queries against a relational database. Conversion of the relational database into an XML database is not required. Each relational table is mapped to a virtual XML document, and XML queries are issued over these virtual documents. An XML query is parsed and transformed into a language-neutral intermediate representation, which is a sequence of operations describing how the output document is derived from the underlying relational tables. The intermediate representation is then translated into an SQL query over the underlying relational tables. The intermediate representation is also used to generate a tagger graph, which the tagger runtime ‘walks’ to generate the tagged, structured XML output. Each of the nodes of the tagger graph are operators which perform processing on the results of the SQL query. The SQL query is executed, and the SQL query results are then provided to the tagger.

Abstract: A system, method, and computer program product for processing XML queries over relational data and meta-data using a relational database system. The invention runs a query to generate a view definition, so a regular query processor can handle queries on a mix of data and meta-data. Names of desired tables are data-dependent, so the invention represents a default XML view as an XML query over the relational tables, gets the table names and table data by recursively invoking the query processor (first by querying the catalog table, and then querying base data according to a new construct). The invention thus processes queries over table names in addition to table values.

Abstract: A method for storing XML documents in a relational database system while exploiting XML Schema information to capture information about types, inheritance, equivalence classes, and integrity constraints in the generated relational schema, enabling efficient querying. The invention simplifies complex XML schema types, translates the simplified XML schema types into relational tables, and then stores relations corresponding to each XML element in relational table rows. The simplification includes grouping all occurrences of a given element together, assembling two or more element types into element groups if the schema indicates that elements of those element types will occur the same number of times, and applying a number of transformation rules to the element groups. The translation includes constructing a type graph from the simplified schema, building an element graph for each global element in the simplified schema from the type graph, and generating relational tables from the element graph.

Abstract: A technique enabling querying of XML documents in a relational database system via a reconstruction view allowing XML documents to be queried as though XML views of relational data. A single query processor can be used with all relational schema generation methods (whether or not using XML schema information) to seamlessly query across XML documents, relational data, and XML views of relational data; no special purpose query processor is required. The technique creates an XML document view, creates relational tables for storing XML documents using relational schema, shreds the XML documents and stores the XML documents as rows in the relational tables according to the relational schema, generates a reconstruction view over the relational tables to define how the shredded documents are to be virtually reconstructed, and processes queries over the stored XML documents as queries over the reconstruction view.

Abstract: A relational data base management system includes a query processor that uses a query operator partition property to perform QEP pruning and to ensure that data input to a query operator is partitioned appropriately for the operation. The partition property indicates the group of network nodes across which a table is distributed. The query processor also makes use of partition classes that are designated “interesting classes” to perform preoptimization planning and query pruning, and to perform look-ahead partitioning based on partition classes that are identified as being of interest to future operations, thereby more efficiently evaluating complex query statements in an MPP, shared-nothing environment.

Abstract: A method for publishing relational data as XML by translating XML queries into queries against a relational database. Conversion of the relational database into an XML database is not required. Each relational table is mapped to a virtual XML document, and XML queries are issued over these virtual documents. An XML query is parsed and transformed into a language-neutral intermediate representation, which is a sequence of operations describing how the output document is derived from the underlying relational tables. The intermediate representation is then translated into an SQL query over the underlying relational tables. The intermediate representation is also used to generate a tagger graph, which the tagger runtime ‘walks’ to generate the tagged, structured XML output. Each of the nodes of the tagger graph are operators which perform processing on the results of the SQL query. The SQL query is executed, and the SQL query results are then provided to the tagger.

Abstract: In a multiprocessing system, multiple concurrently operating task execution units are operated to perform an aggregate task by using incremental and on-demand sub-task allocation. A command is received to perform a machine-executed task divisible into multiple sub-tasks, i.e., an “aggregate task”. A granularity is then established, for dividing the aggregate task into sub-tasks. Preferably, the granularity is not too large to permit potentially uneven sub-task allocation, and not too small to incur excessive overhead in allocating sub-tasks. Having established the granularity, multiple task execution units are independently operated on-demand to sequentially self-allocate and execute sub-tasks of the aggregate tasks. Each sub-task is sized according to the established granularity. Operating “on-demand”, each task execution unit sequentially allocates and executes one sub-task at a time, then proceeding to the next unexecuted sub-task.

Abstract: A system and method for joining a build table to a probe table in response to a query for data includes executing a hash loops join of the build table and the probe table. Matched rows are joined and output when the rows match each other by satisfying a join predicate. In an outer join, unmatched rows in the probe table are joined to a NULL build table field values and output, such that all rows of the probe table are output regardless of whether they have matched rows in the build table. In an early-out join, on the other hand, a “match once” table defines the probe table and in response to a query for unique probe table outputs, the joining of a probe table row, once joined and output a first time, to any other rows in the other table is prevented regardless of whether the row might match other rows. In both the hash loops early-out join and the hash loops outer join, when the build table is larger than main memory, the roles of the build and probe tables are reversed.

Abstract: A system and method for joining a build table to a probe table in response to a query for data includes over partitioning the build table into “N” build partitions using a uniform hash function and writing the build partitions into main memory of a database computer. When the main memory becomes full, one or more partitions is selected as a victim partition to be written to disk storage, and the process continues until all build table rows or tuples have either been written into main memory or spilled to disk. Then, a packing algorithm is used to initially designate never-spilled partitions as “winners” and spilled partitions as “losers”, and then to randomly select one or more winners for prospective swapping with one or more losers. The I/O savings associated with each prospective swap is determined and if any savings would be realized, the winners are designated as losers the losers are designated as winners. The swap determination can be made multiple times, e.g.

Abstract: Query evaluation is optimized using parallel optimization techniques to make repartitioning more efficient. Efficiency is improved by recognizing the possible partitioning requirements for achieving parallelism for a query operation, and by recognizing when the partitioning property of data satisfies the partitioning requirements of a query operation. A data base management system in accordance with the invention uses parallel query processing techniques to optimize data repartitioning, or to avoid it altogether.

Abstract: A system and method for a relational database system for identifying star joins in a query and for breaking the query down for bitmap ANDing. The fact table of the star join is located, and cycles between and within dimension tables are broken. Then, the minimal set of tables necessary to execute the star join is identified, and the dimension tables that should appear in the bitmap ANDing plan are also identified. A bitmap ANDing plan is then generated, or, if the query does not qualify for bitmap ANDing, a conventional execution plan is generated.

Abstract: A relational data base management system includes a query processor that uses a query operator partition property to perform QEP pruning and to ensure that data input to a query operator is partitioned appropriately for the operation. The partition property indicates the group of network nodes across which a table is distributed. The query processor also makes use of partition classes that are designated "interesting classes" to perform preoptimization planning and query pruning, and to perform look-ahead partitioning based on partition classes that are identified as being of interest to future operations, thereby more efficiently evaluating complex query statements in an MPP, shared-nothing environment.

Abstract: Unwieldy star/join queries are performed more efficiently using a filtered fact table. Suitable queries include star/join queries with a large fact table joined with multiple subsidiary dimension tables, where indices exist over fact table join columns. The query is analyzed to prepare a query plan for the dimension table accesses. This plan is supplemented by adding nested loop join operations, where the inner table is a dimension table plan and the outer table is an index scan performed over a fact table index of the join column with the dimension table. The plan is also supplemented by filtering records resulting from the nested loop joins using a sequence of dynamic bit vectors, ultimately yielding a list of probable fact table records. The plan is further supplemented by fetching these records to construct a distilled fact which is used, instead of the large original table, to execute the query in considerably less time.

Abstract: A query optimizer for optimizing join queries in a relational database system by iterative application of dynamic programming (DP) to select optimal subgraph join execution plans. Unlike traditional DP optimization methods, bounds on search space time and space complexity can be established and adjusted by imposing a subgraph threshold. Each bounded subgraph is selected using a greedy heuristic (GH) hill-climbing procedure or other similarly useful technique to build a low-cost execution plan. The low-cost GH subgraph execution plan is then discarded in favor of an optimal DP subgraph execution plan selected by a dynamic programming optimizer for each subgraph identified by the bounded GH optimization process. The complexity bound may be dynamically tuned to improve execution plan quality responsive to changes in query complexity.