Abstract: A dynamic split node defined within a calculation model can receive data being operated on by a calculation plan generated based on the calculation model. A partition specification can be applied to one or more reference columns in a table containing at least some of the received data. The applying can cause the table to be split such that a plurality of records in the table are partitioned according to the partition specification. A separate processing path can be set for each partition, and execution of the calculation plan can continue using the separate processing paths, each of which can be assigned to a processing node of a plurality of available processing nodes.

Abstract: Techniques for enabling fault tolerant distributed query processing are provided. In one embodiment, a first processing node in a plurality of processing nodes can execute an instance of a query operator, where the query operator is part of a query plan for a database query that is processed in a distributed manner by the plurality of processing nodes. The first processing node can further generate a snapshot of the instance of the query operator, where the snapshot includes state information usable for moving execution of the query operator from the first processing node to another processing node after a failure of the first processing node. The first processing node can then save the snapshot to a distributed data store.

Abstract: Techniques for storing and querying graph data in a key-value store are provided. A graph statement (e.g., an RDF graph statement) includes a plurality of values, at least two of which correspond to nodes in a graph. A key is generated based on the graph statement. The key may be generated based on concatenating hash values that are generated based on the plurality of values. The key-value store stores the key. The value that corresponds to the key may be a null or empty value. In response to a graph query (e.g., in SPARQL) that includes one or more query patterns, each of which includes one or more values, a key is generated based on the one or more values and sent to the key-value store, which returns one or more other keys, each of which is a superset of the generated key.

Abstract: Techniques for performing dynamic load balancing during distributed query processing are provided. In one embodiment, a first processing node in a plurality of processing nodes can execute an instance of a query operator, where the query operator is part of a query plan for a database query that is processed in a distributed manner by the plurality of processing nodes. The first processing node can further monitor its load level while the executing occurs. If the load level exceeds a threshold, the first processing node can split an amount of remaining work that is associated with the executing into a first portion and a second portion, determine state information for moving execution of the second portion from the first processing node to a second processing node, and save the state information to a distributed data store.

Abstract: Embodiments of the present invention provide query processing for column stores by accumulating table record attributes during application of query plan operators on a table. The attributes and associated attribute values are compacted when said attribute values are to be consumed for an operation in the query plan, during the execution of the query plan. Table column record values are materialized late in query plan execution.

Abstract: A semantic query over an RDF database is received with RDF database statistics and access methods for evaluating triple patterns in the query. The semantic query is expressed as a parse tree containing triple patterns and logical relationships among the triple patterns. The parse tree and access methods create a data flow graph containing a plurality of triple pattern and access method pair nodes connected by a plurality of edges, and an optimal flow tree through the data flow graph is determined such that costs are minimized and all triple patterns in the semantic query are contained in the optimal flow tree. A structure independent execution tree defining a sequence of evaluation through the optimal flow tree is created and is transformed into a database structure dependent query plan. This is used to create an SQL query that is used to evaluate the semantic query over the RDF database.

Abstract: Embodiments of the present invention provide query processing for column stores by accumulating table record attributes during application of query plan operators on a table. The attributes and associated attribute values are compacted when said attribute values are to be consumed for an operation in the query plan, during the execution of the query plan. Table column record values are materialized late in query plan execution.

Abstract: A PSM complier optimizer analyzes a pattern of the PSM program to generate instruction codes, which will be subjected to a bulk operation and makes the instruction codes for the bulk operation in a form of an array of bind parameters to perform a bulk cursor. Accordingly, the bulk DML processing improves the performance of the PSM program.

Abstract: Optimizing the execution of a query in a multi-database system includes identifying a region within a table, the table being referenced in the query. The region is stored on data-storage devices on first and second system databases in the multi-database system. A first access plan for the query is developed, the first access plan comprising accessing the version of the region stored on the first system database. A second access plan for the query is developed, the second access plan comprising accessing the version of the region stored on the second system database. A selection is made between the first access plan and the second access plan to execute the query. The query is executed using the selected access plan to produce a result.

Abstract: A method system and program for performing a query in which context and information including opcodes accompanies an execution plan for the query. The opcodes determine operations to retrieve a set of bind variables. The execution plan and context information shared by a plurality of application programs.

Abstract: A network interface receives stream data and a processor processes the stream data. The processor: constructs an operator graph comprising information regarding a plurality of operators for processing the stream data based on a query; assigns an operator execution order to the plurality of operators based on an inter-operator input and output relation, with respect to all of the operators on the operator graph; extracts a set of the operators that maintains the stream data assigned with a time stamp, a time of which is earlier than or equal to a reference time, in an input queue; and executes processing of the stream data assigned with the time stamp, having the earliest time, in the operator assigned a minimum operator graph execution order among the operators and maintaining the stream data assigned with the time stamp having the earliest time, among the extracted set of the operators.

Abstract: Executing a XML database query. The method may include compiling the XML database query to provide at least two alternative execution plans, wherein the at least two alternative execution plans provide the same response to the XML database query. The method may further include deciding during runtime, which of the at least two alternative execution plans is executed. The method may include executing the selected execution plan.

Abstract: A computer-implemented system and method of executing search queries by a search rule engine is disclosed. Embodiments of the present disclosure comprise a search engine that may evaluate an arbitrary number of rule triggering conditions with constant run-time complexity, which may result in increased efficiency and faster searching relative to traditional search rule engine methods.

Abstract: System and method embodiments are provided for adaptive vector size selection for vectorized query execution. The adaptive vector size selection is implemented in two stages. In a query planning stage, a suitable vector size is estimated for a query by a query planner. The planning stage includes analyzing a query plan tree, segmenting the tree into different segments, and assigning to the query execution plan an initial vector size to each segment. In a subsequent query execution stage, an execution engine monitors hardware performance indicators, and adjusts the vector size according to the monitored hardware performance indicators. Adjusting the vector size includes trying different vector sizes and observing related processor counters to increase or decrease the vector size, wherein the vector size is increased to improve hardware performance according to the processor counters, and wherein the vector size is decreased when the processor counters indicate a decrease in hardware performance.

Abstract: A fair scheduling system with methodology for fairly scheduling queries for execution by a database management system is disclosed. The techniques involve obtaining computer-executable query jobs and cost estimates to execute the query jobs. For example, the cost estimate can be a number of results the query is expected to return. Based on the cost estimates, the fair scheduling system causes the database management system to execute the query jobs as separately executable sub-query tasks in a round-robin fashion which can decrease latency of low cost queries concurrently executing with high cost queries.

Abstract: A low latency query engine for Apache Hadoop that provides real-time or near real-time, ad hoc query capability, while completing batch-processing of MapReduce. In one embodiment, the low latency query engine comprises a daemon that is installed on data nodes in a Hadoop cluster for handling query requests and all internal requests related to query execution. In a further embodiment, the low latency query engine comprises a daemon for providing name service and metadata distribution. The low latency query engine receives a query request via client, turns the request into collections of plan fragments and coordinates parallel and optimized execution of the plan fragments on remote daemons to generate results at a much faster speed than existing batch-oriented processing frameworks.

Abstract: Performing database queries. A method includes receiving a particular database query. The method further includes accessing a query plan based on the particular database query. The query plan has operators and specific operational parameters associated with each of the operators. The association of operators and specific operational parameters is specific to the particular database query. From the query plan, the method further includes instantiating a plurality of compiled code templates. Each code template includes executable code that when executed performs functionality of one of the operators from the query plan with the specific operational parameters applied in the compilation. The method further includes binding the code templates together using programmatic control flow to create a functioning program.

Abstract: Systems and methods are described that generate an execution plan for a query in a relational database system. The systems and methods generate the execution plan by generating one or more initial logical representations of the query, performing an exploration process around each of the one or more initial logical representations of the query, the performing of the exploration process around a particular initial logical representation of the query comprising applying transformation rules to generate one or more additional logical representations of the query that are logically equivalent to the particular initial logical representation of the query and that are within a maximum allowable transformation distance of the particular initial logical representation of the query, generating one or more execution plans for each initial logical representation of the query and each additional logical representation of the query, and selecting an execution plan from among the generated execution plans.

Abstract: Techniques for improving complex database queries are provided. A determination is made whether to adopt a static or dynamic query execution plan based on metrics. When the dynamic query execution plan is used, a request fragment of the request is planned and the corresponding plan fragment is executed. The processed fragment provides feedback related to its processing to the remaining request and the process is repeated on the remaining request until the request is completed.

Abstract: Disclosed herein is parallel processing of a query, which uses inter-query parallelism in posting list intersections. A plurality of tasks, e.g., posting list intersection tasks, are identified for processing in parallel by a plurality of processing units, e.g., a plurality of processing cores of a multi-core system.

Abstract: A query is received and an initial data flow graph comprising a plurality of nodes is generated for executing the query. The initial data flow graph is optimized using a model optimizer that accesses at least one of a plurality of patterns to identify a matching pattern and executes at least one optimization rule associated with a matching pattern. Execution of the query is then initiated using the optimized data flow graph. Related apparatus, systems, techniques and articles are also described.

Abstract: A method and system for providing keyword ranking using a common affix is provided. The method for providing keyword ranking using the common affix includes: extracting a common affix using a keyword set; and providing keyword ranking which includes the extracted common affix.

Abstract: A computer system affords a user the ability to interactively formulate, visualize, and modify query plans that are to be posed to a database. The computer system provides query visualization in the form of decision trees having nodes selectable/deselectable to search for alternative query plans with lower or minimum execution time. The estimated cost and/or the execution time of a subtree of a query plan may be mapped according to a rectangle area metaphore, with the area of a rectangle enclosed by a larger rectangle representing a parent node in the plan tree. Query plans may be saved by the system in order to facilitate their visualization, optimization, and/or modification at a later date.

Abstract: Techniques are provided for sharing expression evaluation across a database management system and a storage layer. According to an embodiment, a plurality of target operators are identified that include a common expression. Each respective target operator corresponds to a step that evaluates the common expression during query execution. The plurality of target operators are modified to enable sharing of an evaluation result of the common expression. The query is executed according to the target operators. During query execution, the evaluation result is shared by the target operators. In other embodiments, the evaluation of the common expression is pushed to a storage layer. During query execution, a database management system retrieves the evaluation result of the common expression from the storage layer.

Abstract: A client accepts 20 an input query, rewrites 22 the query and forwards 24 the query to a database server, which returns 26 results which are then rewritten 28. The database includes classes arranged hierarchically as superclasses and subclasses, and/or properties also arranged hierarchically as superproperties and subproperties. In embodiments, the query is rewritten 22 to replace each instance of each class and each property in the input query with the transitive closure of the respective class or property. The client allows inferencing whether or not this is supported by the database server.

Abstract: A search server system having one or more processors and memory receives a search query including filter criteria. The filter criteria specify operands and a logical relationship. In response to the search query the server identifies index portions sorted in a same sort order. The server identifies matching entities that match the search query by identifying index entries having a first identifier and matching filter(s) of a first operand and identifying, index entries having a second identifier that is sequentially adjacent to the first identifier in the sort order and matching filter(s) of a second operand. When index portion(s) associated with the first operand include index entries having a respective identifier between the first identifier and the second identifier and matching the filter(s) of the first operand, the server marks a respective entity associated with the respective identifier as matching the search query and sends search results to the client.

Abstract: A database query governor determines when a query cannot be executed within a defined time threshold, but still executes the query or some variant of the query so partial results for the query may be returned to the user after executing for less than the defined time threshold, instead of an error message indicating the query was not executed at all. The partial results provide information to the user that may be sufficient for some applications, or may indicate to the user how the query could be modified to run in less time.

Abstract: Provided are techniques for collecting client information for client applications running on a client computer, wherein the client information includes a client application identifier for each client application that issues queries to a database along with text of each of the queries that the client application issues; collecting database monitoring information that includes text of each query issued against the database and performance information for each query; combining the client information and the database monitoring information based on the text of the queries to obtain combined information that provides the client application identifier and the performance information for each of the queries; and identifying a problem query and at least one client application that has issued the problem query using the combined information.

Abstract: An enterprise business intelligence system includes a data access service. In one example, a method comprises receiving, with a data access service, a query and a report specification from an application, wherein the data access service is positioned between the application and a data source; creating a run tree that comprises a sequence of executable decoration nodes; executing the query on the data source, thereby providing a query result set comprising query results drawn from the data source that are responsive to the query; and executing the sequence of decoration nodes on the query result set. The decoration nodes modify the query result set, thereby producing a decorated result set based on the query result set and the decoration nodes. Producing the decorated result set comprises associating members or values in the query results with corresponding positions or calculated members in the report specification.

Abstract: A method of analyzing the performance of a query optimizer includes identifying an event trigger. A reproduction object characterizing operational parameters of the customer computer at the time of the event trigger is populated. The reproduction object is transported from the customer computer to a test computer. The reproduction object is analyzed at the test computer to characterize the performance of a query optimizer.

Abstract: Approaches, techniques, and mechanisms are disclosed for ensuring that a database command is executed according to a query plan that has been verified to be actually optimal. Except in rare circumstances, a database server does not execute a query plan unless it is first verified by the database server. The database server receives a request to execute a database command. The database server determines an unverified plan is the best plan for satisfying the request. Rather than risk the unknown behavior of an unverified plan, the database server instead satisfies the request according to a verified plan. Subsequently—for example as part of a scheduled job—the database server executes the unverified plan to determine performance statistics. Based at least on the performance statistics, the database server determines whether or not to verify the unverified plan. Techniques for concurrent and optimistic verifications are also disclosed.

Abstract: Systems, methods, and media are disclosed herein that can be embodied in a traditional Relational Database Management System (RDBMS) in order to transform it into a Streaming Relational Database Management System (SRDBMS). An SRDBMS may provide functionality such as to manage and populate streams, tables, and archived stream histories and support the evaluation of continuous queries on streams and tables. Both continuous and snapshot queries support the full spectrum of the industry standard, widely used, Structured Query Language. The present technology can support a high number of concurrent continuous queries using a scalable and efficient shared query evaluation scheme, support on-the-fly addition of continuous queries into a mechanism that implements the shared evaluation scheme, reuse RDBMS modules such as relational operators and expression evaluators, and visualize results of continuous queries in real time.

Abstract: Mechanisms for performing database queries are provided. With these mechanisms, in response to a query request, a query plan intended for minimum query response time and a query plan intended for minimum query total time for the query request are obtained execution of the minimum query response time query plan and the minimum query total time query plan is started. Before the execution of the minimum query total time query plan reaches a specified point, an initial query result obtained from the execution of the minimum query response time query plan is output. In response to the execution of the minimum query total time query plan reaching the specified point, continuing the execution of the minimum query total time query plan to output remaining query results.

Abstract: A method for approximating a validity range for a domain of cardinalities of input to an optimal query plan is provided. Such a validity range is iteratively approximated using a modified Newton-Raphson method to find roots of cost functions for optimal and alternative query plans, respectively. The Newton-Raphson method is combined with a method of incrementing roots of cost functions, known as input cardinalities, such that discontinuous and non-differentiable points in cost functions are avoided. In this manner, input cardinalities remain within a domain for which a valid range can be specified. Additionally, a robustness measure is determined by a sensitivity analysis performed on an approximated validity range. Using a robustness measure provided by a sensitivity analysis and resultant validity range and, query plan sub-optimality detection is simplified, re-optimization is selectively triggered, and robustness information is provided to a system or user performing corrective actions.

Abstract: There is provided a computer-executable method of executing an outer join on a parallel database management system. An exemplary method comprises receiving an outer skewed values list (SVL). The outer SVL may comprise values that are indicated to be skewed. The exemplary method further comprises receiving an inner SVL. The inner SVL may comprise values that are indicated to be skewed. Additionally, the exemplary method comprises partitioning the outer table and the inner table across a plurality of join instances, based on the outer SVL and the inner SVL. A missing skew value is identified. The missing skewed value may be a value of the inner SVL that is not found in the inner table. The outer join is performed using the plurality of join instances, based on the missing skewed value.

Abstract: Techniques are described for generating histograms for a multidimensional space. In the presence of large spatial objects, fuzzy splitting techniques are utilized to recursively divide the multidimensional space into partitions, where a single spatial object may belong to multiple partitions. Large spatial objects are essentially broken down into smaller objects that may allow for more efficient partitioning of the multidimensional space. A count of spatial objects in each partition yields a spatial histogram. A spatial object that belongs to multiple partitions may have a weighted count for each of the multiple partitions, based on the extent to which the spatial object overlaps with each partition. Thus, an object that is split among a handful of partitions will only contribute a fraction of a count to each partition. Small partitions having relatively few objects are avoided by refusing to subdivide a partition whose members drop below a threshold number.

Abstract: Database report generation technology for handling a single logical query that requires data that is physically located in two different sources. The system uses intelligent push-down joins, which move data from one source to the other and join data within that source. The system dynamically determines which direction to move data depending on how the data is used in a query.

Abstract: A method, system, and computer program product for managing database statistics are provided. The method, system, and computer program product provide for receiving a query for optimizing, collecting statistics specific to the query prior to generating any access plans for executing the query, and generating an access plan for executing the query based on the collected statistics.

Abstract: In an embodiment, a hit percentage of an expected affinity for a first query is calculated, wherein the expected affinity comprises a first address range in a query plan cache, a hit percentage of a candidate affinity for the first query is calculated, wherein the candidate affinity comprises a second address range in a query plan cache, and if the hit percentage of the candidate affinity is greater than the hit percentage of the expected affinity by more than a threshold amount, query plans in the candidate affinity are swapped with query plans in the expected affinity.

Abstract: Processes, machines, and stored machine instructions are provided for estimating how many rows would be processed by a query. In one embodiment, model aggregating logic may generate a global model based on separate models for separate portions of a set of data. For example, the separate portions may be assigned to separate partitions of the set of data based at least in part on key values in the separate portions. Generating the global model may include generating an uncompressed model that includes components of the separate models before generating a compressed model to estimate the uncompressed model. Query analysis logic may use the global model to estimate how many rows would be processed by queries that, when executed, would access portion(s) of the set of data.

Abstract: A system, method, and computer program product are provided for compiling a stored procedure in a database. The compiling includes the steps of invoking the stored procedure, the stored procedure comprising one or more statements, substituting run-time values for one or more variables in the statements, and creating an access plan for the statements having the substituted run-time values.

Abstract: An information retrieval system implements a search language, through which a querying entity (e.g., a user, a program or process, or the like) formulates a search query. Preferably, a search query is composed of an ordered set of clause definitions, and each clause can have set membership operations applied to it. Each clause includes a clause pipeline, and a time constraint. A clause pipeline includes an ordered set of clause specifications separated by a pipeline operator. A clause specification can be either an expansion operation or a filtering operation. Preferably, a first clause specification in a pipeline operates on an initial universe of all objects, and each subsequent clause specification operates on a set of objects produced from the previous clause specification. The search language is exposed to users (typically, IT administrators), and one or more builder programs within the system (each referred to as a “model builder”) are used internally to present data models to the search language.

Abstract: Embodiments of the present invention provide a method, system and computer program product for query generation from a previous result set. In an embodiment of the invention, a method for query generation from a previous result set is provided. The method includes identifying commonalities in a selection of records of a result set produced by a previously issued query against a database in a database management system executing in memory by at least one processor of a host server. The method further includes deriving a query from the identified commonalities that when executed against the database management system would produce the selection of records as the result set. Finally, the method includes storing the derived query for subsequent use in querying the database.

Abstract: The claimed subject matter provides systems and/or techniques that facilitate and/or effectuate efficient stream sharing for multi-user sensor data collection. The system includes mechanisms that locate within a query a specification that includes spatial regions over which data is required, based on the spatial regions the mechanism determines which sensors are situated within the region or ascertains which of the sensors in the region is actively streaming data and/or has persisted data in order to fulfill the query, constructs or utilizes a directed acyclic graph to obviate redundancy in a functional requirement included in the specification and imposed by the query, and formulates a result in fulfillment of the query that is delivered to an application.

Abstract: A search system architecture is disclosed. The search system architecture includes a search server, which in turn includes a search services module and a search engine adapter. The search services module and the search engine adapter are communicatively coupled to one another. The search services module is configured to communicate with a user interface. The search engine adapter is configured to communicate with a search engine.

Abstract: A method for creating a joined data set from a join input data set is disclosed. The method starts by categorizing the join input data set into a high-skew data set and a low-skew data set. The low-skew data set is distributed to the plurality of CPUs using a first distribution method. The high-skew data set is distributed to the plurality of CPUs using a second distribution method. The plurality of CPUs process the high-skew data set and the low-skew data set to create the joined data set.

Abstract: Disclosed herein are system, method, and computer program product embodiments for performing resource estimation for query optimization. An embodiment operates by generating a subplan for which an optimization process may be invoked, predicting performance and resource consumption for optimizing the subplan by measuring similarity between a hypergraph of the subplan and one or more etalon queries having known performance and resource consumption properties, selecting an algorithm for optimizing the subplan from a plurality of optimization algorithms based on the performance and resource consumption properties, and generating an optimized access plan using the selected algorithm.

Abstract: According to one aspect of the invention, for a database statement that specifies rollup operations, a data distribution key may be selected among a plurality of candidate keys. Numbers of distinct values of the candidate keys may be monitored with respect to a particular set of rows. Hash values may also be generated by column values in the candidate keys. The data distribution key may be determined based on results of monitoring the numbers of distinct values of the candidate keys as well as the frequencies of hash values computed based on column values of the candidate keys. Rollup operations may be shared between different stages of parallel executing processes and data may be distributed between the different stages of parallel executing processes based on the selected data distribution key.

Abstract: According to one aspect of the invention, for a database statement that specifies evaluating ranking or cumulative window functions, an execution strategy based on an extended data distribution key may be used for the database statement. In the execution strategy, each sort operator of multiple parallel processing sort operators computes locally evaluated results of a ranking or cumulative window function based on a subset of rows in all rows used to evaluate the database statement, and sends the first and last rows' locally evaluated results to a query coordinator. The query coordinator consolidates the locally evaluated results received from the multiple parallel processing sort operators and sends consolidated results to the sort operators based on their respective demographics. Each sort operator completes full evaluation of the ranking or cumulative window functions based at least in part on one or more of the consolidated results provided by the query coordinator.

Abstract: A query is received which causes an initial data flow graph that includes a plurality of nodes that are used to execute the query is generated. Thereafter, the initial data flow graph is optimized using a model optimizer that includes an optimizer framework and an application programming interface (API). The optimizer framework provides logic to restructure the initial data flow graph and a rules engine for executing one or more optimization rules. The API allows for registration of new optimization rules to be executed by the rules engine. Execution of the query is then initiated using the optimized data flow graph. Related apparatus, systems, techniques and articles are also described.