Abstract: Systems and associated methods for address standardization and applications related thereto are described. Embodiments exploit a common context in a taxonomy and a given address to detect and correct deviations in the address. Embodiments establish a possible path from a root of the taxonomy to a leaf in the taxonomy that can possibly generate a given address. Given a new address, embodiments use complete addresses, and/or segments or elements thereof, to compute the representations of the elements and find a closest matching leaf in the taxonomy. Embodiments then traverse the path to a root node to detect the agreement and disagreement between the path and the address entry. Taxonomical structured is thus used to detect, segregate and standardize the expected fields.

Abstract: Methods and arrangements for extracting tuples from a streaming XML document. A query twig is applied to the XML document stream, tuples are extracted from the XML document stream based on the query twig, and a quantity of extracted tuples is limited via foregoing extraction of duplicate tuples extraction of tuples that do not satisfy query twig criteria.

Abstract: A storage requirements estimating system estimates the storage required for a proposed multidimensional clustering data by modeling wasted space. The amount of wasted space is modeled by calculating the cardinality of the unique value of the clustering key for the proposed configuration. Cardinality may be determined by estimation techniques. Specific values for wasted space and total space may be determined in response to the determined cardinality. Comparison of estimates for different proposed clustering configurations facilitate a selection among proposed multidimensional clustering data configurations.

Abstract: An information retrieval system and method are provided for minimizing the number of blocks searched in a cell before recording a new record in the table and determining which block can be assigned if a table has space available to store a new record in the case an additional block should be associated with a cell. Dimensions for a table are identified, and at least one block in the table is associated with a dimension value for each dimension, where each block comprises contiguous storage pages. The block can be further associated with a cell; this associated cell has a unique combination of dimension values comprising an dimension value for each of the dimensions. A unique associated bit list for each dimension value for each dimension has a unique corresponding list entry for each block associated with that dimension value, and a unique associated bit list for each cell has a unique corresponding list entry for each block associated with that cell.

Abstract: A computer implemented method, apparatus, and computer usable program code for generating an execution plan graph from a data flow. A metadata representation of the data flow is generated in response to receiving the data flow. A set of code units is generated from the metadata representation. Each code unit in the set of code units is executable on multiple different types of runtime engines. The set of code units is processed to produce the execution plan graph.

Abstract: A learning-based method for estimating costs or statistics of an operator in a continuous query includes a cost estimation model learning procedure and a model applying procedure. The model learning procedure builds a cost estimation model from training data, and the applying procedure uses the model to estimate the cost associated with a given query. The learning procedure uses a feature extractor, a confidence adjustor and a cost estimator. The feature extractor collects relevant training data and obtains feature values. The extracted feature values are associated with costs and used to create the cost estimator. The extracted feature values, the associated costs, the cost estimator, and a user interface are used to create a confidence adjuster.

Abstract: Disclosed is a data processing system implemented method, a data processing system and an article of manufacture for improving execution efficiency of a database workload to be executed against a database. The database includes database tables, and the database workload identifies at least one of the database tables. The data processing system includes an identification module for identifying candidate database tables being identifiable in the database workload, the identified candidate database tables being eligible for organization under a clustering schema, a selection module for selecting the identified candidate tables according to whether execution of the database workload is improved if the selected identified candidate table is organized according to the clustering scheme, and an organization module for organizing the clustering schema of the selected organized identified candidate tables prior to the database workload being execution against the database.

Abstract: A method and system for clustering of XML documents is disclosed. The method operates under specified memory-use constraints. The system implements the method and scans an XML document, assigns edge-weights according to the application workload, and maps clusters of XML nodes to disk pages, all in a single parser-controlled pass over the XML data. Application workload information is used to generate XML clustering solutions that lead to substantial reduction in page faults for the workload under consideration. Several approaches for representing workload information are disclosed. For example, the workload may list the XPath operators invoked during the application along with their invocation frequencies. The application workload can be further refined by incorporating additional features such as query importance or query compilation costs. XML access patterns could be also modeled using stochastic approaches.

Abstract: Disclosed is a data processing system implemented method, a data processing system and an article of manufacture for improving execution efficiency of a database workload to be executed against a database. The database includes database tables, and the database workload identifies at least one of the database tables. The data processing system includes an identification module for identifying candidate database tables being identifiable in the database workload, the identified candidate database tables being eligible for organization under a clustering schema, a selection module for selecting the identified candidate tables according to whether execution of the database workload is improved if the selected identified candidate table is organized according to the clustering scheme, and an organization module for organizing the clustering schema of the selected organized identified candidate tables prior to the database workload being execution against the database.

Abstract: A storage requirements estimating system estimates the storage required for a proposed multidimensional clustering data by modeling wasted space. The amount of wasted space is modeled by calculating the cardinality of the unique value of the clustering key for the proposed configuration. Cardinality may be determined by estimation techniques. Specific values for wasted space and total space may be determined in response to the determined cardinality. Comparison of estimates for different proposed clustering configurations facilitate a selection among proposed multidimensional clustering data configurations.

Abstract: Interoperability is enabled between participants in a network by determining values associated with a value metric defined for at least a portion of the network. Information flow is directed between two or more of the participants based at least in part on semantic models corresponding to the participants and on the values associated with the value metric. The semantic models may define interactions between the participants and define at least a portion of information produced or consumed by the participants. The determination of the values and the direction of the information flow may be performed multiple times in order to modify the one or more value metrics. The direction of information flow may allow participants to be deleted from the network, may allow participants to be added to the network, or may allow behavior of the participants to be modified.

Abstract: A storage requirements estimating system estimates the storage required for a proposed multidimensional clustering data by modeling wasted space. The amount of wasted space is modeled by calculating the cardinality of the unique value of the clustering key for the proposed configuration. Cardinality may be determined by estimation techniques. Specific values for wasted space and total space may be determined in response to the determined cardinality. Comparison of estimates for different proposed clustering configurations facilitate a selection among proposed multidimensional clustering data configurations.

Abstract: A system and method of converting a recursive XML document into a relational schema comprises providing a recursive XML document; parsing an external mapping script specifying a mapping from the recursive XML document to a relational table format; building a recursive shredding tree based on the external mapping script and the relational table format; and shredding the mapped recursive XML document into a relational table. The system and method further comprise detecting whether any of a XML schema and a DTD document is recursive, wherein the detecting comprises building a directed graph comprising element names; corresponding elements names as nodes in the directed graph; forming arcs from every element parent node to every element child node of the element parent node; and checking for cycles in the directed graph. The system and method further comprise identifying all recursive cursor nodes and a recursive degree corresponding to the recursive shredding tree.

Abstract: Methods and systems for implementing a splitter operation in an extract, transform, and load (ETL) process are provided. In one implementation, the method includes receiving a data flow including a splitter operation, and generating an execution plan graph based on the data flow. The execution plan graph includes structured query language (SQL) code for implementing the splitter operation, in which the structured query language (SQL) code is respectively executable among database servers associated with different vendors.

Abstract: A repository index records the position of document entries relative to landmark entries within the document. Landmark entries are selecting using a landmarking policy and their position relative to the document are stored in a landmark directory. During index updates, an edit transcript is generated describing the difference between old and new document versions, and both the document repository index and the landmark directory are updated as needed. Thus, the number of update operations preformed as compared with conventional indexing techniques may be substantially reduced when small, localized changes are made to the document. This is due to fact that the positions of document entries are recorded relative to the landmark entries rather than the document itself. By doing so, the document index becomes more shift-invariant, requiring fewer update operations when entries are added or inserted in localized areas of the document.

Abstract: A computer implemented method, apparatus, and computer usable program code for generating an execution plan graph from a data flow. A metadata representation of the data flow is generated in response to receiving the data flow. A set of code units is generated from the metadata representation. Each code unit in the set of code units is executable on multiple different types of runtime engines. The set of code units is processed to produce the execution plan graph.

Abstract: A computer implemented method, apparatus, and computer usable program code for generating code for an integrated data system. A mixed data flow is received. The mixed data flow contains mixed data flow operators, which are associated with multiple runtime environments. A graph is generated containing logical operators based on the mixed data flow in response to receiving the mixed data flow. The logical operators are independent of the plurality of runtime environments. The graph is converted to a model. The logical operators are converted to model operators associated with the multiple runtime environments. The model operators allow for analysis of operations for the mixed data flow. The model is converted into an execution plan graph. The execution plan graph is executable on different runtime environments.

Abstract: A multi-level locking hierarchy for a relational database includes a locking level applied to a multi-dimensionally clustering table, a locking level applied to blocks within the table, and a locking level applied to rows within the blocks. The hierarchy leverages the multi-dimensional clustering of the table data for efficiency and to reduce lock overhead. Data is normally locked in order of coarser to finer granularity to limit deadlock. When data of finer granularity is locked, data of coarser granularity containing the finer granularity data is also locked. Block lock durations may be employed to ensure that a block remains locked if any contained row remains locked. Block level lock attributes may facilitate detection of at least one of a concurrent scan and a row deletion within a block. Detection of the emptying of a block during a scan of the block may bar scan completion in that block.

Abstract: A block map is associated with a multi-dimensionally clustered (MDC) base table in a database to maintain a “free” or “in use” status for each of the contiguous blocks in the table. Additionally, information may be stored in the block map to indicate whether, after loading data, a given block in the base table requires constraint checking operation or whether, based on changes to the block, a materialized query table dependent upon the table requires a refresh operation. Each time data is loaded into the table, the associated block map is updated. When delayed constraint check or refresh operations are to be performed, the block map is consulted to learn which blocks require the operations. After one of the constraint check or refresh operations is performed, the associated block map is again updated.

Abstract: Multidimensional clustered tables are provided for efficient processing and management in a relational database management system. A multidimensional clustered table is one whose data is simultaneously clustered along one or more independent dimensions, or clustering keys, and physically organized into blocks or pages on disk. When such a table is created, one can specify one or more keys as dimensions along which to cluster the table's data. Each of the dimensions can consist of one or more columns. Further provided are efficient query processing and maintenance techniques for use in conjunction with multidimensional clustered tables.