Abstract: Relocating data sharing operations for query processing may be implemented when generating plans to perform a query. A query operation that causes a node to share data obtained from a different set of nodes with other nodes in a same set of nodes may be identified. The identified query operation may be replaced with another operation that causes the different set of nodes to share the data directly with the other nodes in the same set of nodes.

Abstract: Provided are methods and systems for generating physical database design tuning recommendations. Given a workload, the system analyzes the workload to identify and recommend a set of rowstore and columnstore indexes optimal for the performance of the workload. The system is designed to estimate the size of the columnstore index (at the granularity of each column) without actually building the index, estimate the improvement in query performance that each columnstore index would result in when built, and automatically derive the workload used for the physical design tuning task by analyzing stored query execution history data. This automatic workload derivation is orthogonal to columnstores and can be used even when columnstore indexes are not being used.

Abstract: An access plan including multiple indexes is selected for accessing data of a database system. A filtering status of each index within the selected access plan is determined based on information available at execution of the selected access plan. The order of the indexes within the selected access plan is modified based on the filtering status of the indexes, and the selected access plan is executed with the modified order of the indexes. The execution of the selected access plan is interrupted in response to accessing a threshold amount of data and the filtering of the indexes of the selected access plan is estimated. A processing order for the indexes is determined based on the estimated filtering. The execution of the selected access plan resumes based on the determined processing order.

Abstract: A database system may receive a query and form a plan for executing the query using a hypothetical index selected based on columns referenced by the query. The database system may compare the hypothetical index to indexes on the computing nodes of the database system. Based on the comparison, the database system may form a new index, based on the hypothetical index, on one of the computing nodes. An additional computing node may be added to the database system for the purpose of maintaining the new index. Subsequent queries having commonality with the first query may be processed on a node maintaining the first index.

Abstract: A large highly parallel database management system includes thousands of nodes storing huge volume of data. The database management system includes multiple query optimizers for determining low cost execution plans for queries. The database management system is adapted to receive a data query. An execution plan generator component of the database management system generates an initial execution plan for the query. The initial execution plan is fed as input to more than one query optimizers. Each optimizer starts with the initial execution plan, generates alternative execution plans, and determines a satisfactory execution plan that incurs the lowest cost. The database management system compares the selected execution plans by the optimizers and selects one with the lowest cost. The multiple query optimizers run in parallel.

Abstract: A system for determining a query ordering includes an input interface and a processor. The input interface is configured to receive a set of commingled data. The commingled data comprises data associated with a plurality of tenants. The input interface is configured to receive a query. The processor is configured to determine an efficient ordering for the query. The efficient ordering for the query is based at least in part on the set of commingled data. The processor is configured to prompt a user for a permission to change an execution ordering for the query in response to the efficient ordering for the query being different from a user indicated ordering for the query.

Abstract: A virtual network function (VNF) management system includes a computer-executable set of instructions to deploy a VNF using one or more resources allocated for use by the VNF in which the one or more resources are allocated by communicating with a resource manager that manages the one or more resources. The system then receives a request from a user interface to un-deploy the VNF, and un-deploys the VNF while keeping the resources allocated for use by the VNF. At a later time, the system can then receive a request to restore the VNF from the user interface, and deploy the VNF again using the allocated resources.

Abstract: Techniques a provided for communication graph tracking in multi system operation in heterogeneous database systems, including retrieving two or more database commit logs from two or more corresponding heterogeneous databases. For each of the commit logs, a subset of information is extracted, including a correlation identifier, and event stream entries are produced based on that information and are stored. The event stream entries are then retrieved and event database entries are created based on the event stream entries. A request for a communication graph related to a particular correlation identifier is received and the event database is searched for matching entries. Those entries are used to generate a displayable communication graph for the particular correlation identifier, which is then provided in response to the request.

Abstract: An index shared between a plurality of tables is used to access a data sharing space (DSS) storing a set of rows from a base table having a plurality of rows with stored data. First data values of the index are identified in response to a query for a table. A subset of rows is then retrieved from the DSS using the index elements corresponding to the first data values. Second data values and some of the first data values are then identified in response to a second query for a table. A second subset of rows are then retrieved from the DSS using the index elements corresponding to the second data values and some of the first data values and the query is then applied to the subset of rows.

Abstract: A system for handling data skew includes a cluster of computing nodes and a processor. The cluster includes one or more first nodes, each of which has a storage, and one or more second nodes, each of which has a storage. The storage of the respective second nodes has a higher access speed than the storage of the respective first nodes. The processor is configured to split input data into partitions of the input data, to detect if any of the partitions has data skew, and to assign ones of the partitions, which are detected as having no data skew, to the first nodes, and ones of the partitions, which are detected as having the data skew, to the second nodes, for parallel processing.

Abstract: A method for rapid data analysis includes receiving and interpreting a first query operating on a first dataset partitioned into shards by a first field; collecting a first data sample from a first set of data shards; calculating a first result to the first query based on analysis of the first data sample; and partitioning a second dataset into shards by a second field based on the first result.

Abstract: A format conversion engine for Apache Hadoop that converts data from its original format to a database-like format at certain time points for use by a low latency (LL) query engine. The format conversion engine comprises a daemon that is installed on each data node in a Hadoop cluster. The daemon comprises a scheduler and a converter. The scheduler determines when to perform the format conversion and notifies the converter when the time comes. The converter converts data on the data node from its original format to a database-like format for use by the low latency (LL) query engine.

Abstract: Techniques for a system capable of performing low-latency database query processing are disclosed herein. The system includes a gateway server and a plurality of worker nodes. The gateway server is configured to divide a database query, for a database containing data stored in a distributed storage cluster having a plurality of data nodes, into a plurality of partial queries and construct a query result based on a plurality of intermediate results. Each worker node of the plurality of worker nodes is configured to process a respective partial query of the plurality of partial queries by scanning data related to the respective partial query that stored on at least one data node of the distributed storage cluster and generate an intermediate result of the plurality of intermediate results that is stored in a memory of that worker node.

Abstract: M×N dispatching in a large scale distributed system is disclosed. In various embodiments, a query is received. A query plan is generated to perform the query. A subset of query processing segments is selected, from a set of available query processing segments, to perform an assigned portion of the query plan. An assignment to perform the assigned portion of the query plan is dispatched to the selected subset of query processing segments.

Abstract: Aspects automatically provision cloud resources for Extract, Transform and Load job execution as a function of optimizer output data. Aspect access paths identify overall cost and individual step operator costs and statistics relative to cloud resources for executing structured English query language command against a relational database. In one aspect operator steps having high step costs are identified and amounts, types or speeds of a cloud resources associated with the step are adjusted to reduce step cost and access path overall cost. In another aspect structured English query language text descriptors and step costs and function statistics are compared to knowledge base criteria for matches to criteria associated inefficient configurations cloud resources for revision to improve corresponding operation efficiency. In another aspect usages of cloud resources by operator steps are monitored and logged and compared to historic data of similar steps to determine under or over-utilization of resources.

Abstract: A method, system and computer program product are provided. A first index scan of records of a database table is performed. A first index used by the first index scan is specified in a DBMS query statement as being ordered by one or more field values of one or more fields of the first index. A second scan of the database table is performed to determine whether each respective record of the second scan is qualified based on one or more fields of the each respective record of the second scan and a predicate. A lookup data structure is built indicating whether each of the respective records of the second scan is qualified. The lookup data structure is used to determine whether a record of the first index scan is qualified when the lookup data structure becomes available.

Abstract: A computer-implemented automated review method for structured query language (SQL) source code files seeking to access a data store comprises the steps of calculating an SQL complexity score; setting an issue flag if the SQL complexity score is greater than a threshold; checking the SQL source code for SQL-92 compliance; setting the issue flag if any of the SQL source code files is not SQL-92 compliant; comparing a type of data in the SQL source code files to be input to the data store with a type of a field in the data store set to receive the input data; setting the issue flag if the type of any input data does not match the type of a corresponding data store field; checking the SQL source code for SQL triggers set to activate when selected SQL commands are executed; and setting the issue flag if there are any SQL triggers.

Abstract: Systems and methods are described for implementation by a computing system having a non-uniform memory access (NUMA) architecture comprising a plurality of NUMA nodes. A query of a document store storing a collection of slices each comprising one or more documents is received from a client. Thereafter, it is determined which of the slices within the document store are required for execution of the query. An execution plan is then generated, using the query, that comprises a plurality of nodes each specifying at least one database operation to execute a portion of the query. The execution plan assigns one of the plurality of NUMA nodes to each slice determined to be required for execution of the query. The database operations specified by the nodes of the execution plan are then executed using the corresponding assigned NUMA nodes for the associated slice. Data responsive to the query is then provided to the client.

Abstract: Embodiments of the present disclosure provide a method for performing search queries in a manner that avoids overloading an indexer cluster or indexers with an unwanted or unauthorized high levels of concurrent searches. The method comprises transmitting a slot request from a search head to a cluster master in response to a query, wherein the cluster master is communicatively coupled with an indexer cluster comprising a plurality of indexers. The method further comprises receiving addresses of active indexers in the indexer cluster and a response to the slot request from the cluster master. Responsive to a grant of a slot by the cluster master, the method comprises using the addresses to transmit the query to the active indexers and receiving results of the query from the active indexers. Subsequently, the method comprises releasing the slot to the cluster master.

Abstract: Methods and systems for optimizing queries in a multi-tenant store are described. In one embodiment, such a method includes retrieving data from a multi-tenant database system having a relational data store and a non-relational data store, receiving a request specifying data to be retrieved, retrieving one or more locations of the data to be retrieved, generating a database query based on the request, in which the database query specifies a plurality of data elements to be retrieved, the plurality of data elements including one or more data elements residing within the non-relational data store and one or more other data elements residing within the relational data store, generating an optimized database query having an optimized query syntax that is distinct from a query syntax of the database query, and executing the optimized database query against the multi-tenant database system to retrieve the data.

Abstract: A system, method, and computer-readable medium for performing a distributed analytics operation. The distributed analytics operation uses interface technologies to de-couple an actual data storage technology from an implementation of distributed analytics. Such a distributed analytics operation obviates requirements to deploy specific computer code onto a data storage platform to specifically target that platform for distributed predictive analytics computations.

Abstract: A database system determines execution plans for database queries by evaluating a number of partial solutions for each database query. The database system determines a partial solutions limit on the number of partial solutions to be evaluated for determining the execution plan of the query. The database system determines a plurality of partial solutions, each partial solution corresponding to a portion of the execution plan for processing the database query. The database system evaluates a number of candidate partial solutions for determining a partial solution. The number of candidate partial solutions evaluated is determined based on the partial solutions limit. The database system combines the plurality of partial solutions to obtain an execution plan for the database query. The database system executes the database query by executing the execution plan.

Abstract: Systems and methods for producing histograms are disclosed. One method for producing a histogram includes processing a database table to produce a grouped result set, and dividing the grouped result set into a plurality of collections with each collection containing one or more intervals of the grouped result set. Further, for each collection containing one or more intervals, the method includes determining one or more statistical parameters characterizing a distribution of the data within the collection. Still further, for each collection, the method includes applying one of a plurality of different histogram-producing algorithms based on the one or more statistical parameters to produce a histogram such that at least two of the different histogram-producing algorithms are used to render a respective different portion of the histogram.

Abstract: A method for optimizing query compilation by tuning memory across a database cluster is provided. The method may include receiving, by a global memory tuner, memory configuration information from a plurality of nodes within the database cluster. The method may also include determining a node within the plurality of nodes having a least available memory value relative to a remainder of nodes within the plurality of nodes based on the received memory configuration information. The method may then include calculating a globally tuned memory value based on a memory value associated with the determined node. The method may further include determining a memory distribution based on the calculated globally tuned memory value. The method may also include sending the determined memory distribution to the plurality of nodes.

Abstract: A Database Query Time Estimator (DQTE) module as a run-time tool to estimate and display the total time to complete a database query while the query is executing. The DQTE module invokes a runtime algorithm to compute the set of rows or blocks in a database that will be resulted at each step of a query's execution plan. The DQTE module may optimize the current execution plan to create one or more parent-child processes and may assign tasks to these processes to monitor the progress of each step in the execution plan. The progress of the child processes is then rolled up to the root parent level to come up with a time value, which is the Estimated Time of Arrival (ETA) to complete the whole execution of the query. The DQTE module may read only the header information of the database objects for a complete snapshot of the object.

Abstract: Systems for database system management. A method embodiment commences upon intercepting a feature or command that is issued from a software application that interfaces with a database management system. Rules are used to identify that the feature or command is of a first type (e.g., an older type). The feature or command is analyzed to determine if it is a candidate to be modified or upgraded to implement a newer feature or command of a second type (e.g., a newer type). Modifications are made to the old feature or command of the first type to implement a new feature or command of a second type. The new feature or command of the second type is sent to the database management system for processing.

Abstract: Examples disclosed herein relate to data provisioning for an analytical process based on lineage metadata. In an example, a value of a parameter related to execution of an analytical process on a remote hub device may be determined based on lineage metadata stored on an edge device, wherein the analytical process is part of an analytical workflow that may be implemented at least in part on the edge device and the remote hub device. In response to a determination that the value of the parameter is above a predefined threshold, the edge device may provide to the remote hub device, input data for a future execution of the analytical process in advance of execution of the analytical process on the remote hub device without a request for the input data by the remote hub device.

Abstract: A repository of key-value data may store a first object value having an internal structure of a hierarchy of sub-objects. The repository may receive a request to modify the first object, expressed as a function, without side effects, of one or more parameters. Values may be assigned to the one or more parameters by mapping from the parameters to locations in the hierarchy of sub-objects. A second object may be formed based on the return value of the function. The first object may be replaced in the repository by the second object.

Abstract: A computing device is provided, comprising a processor configured to select at least one pair of tuples of columns including a source tuple from a first table and a target tuple from a second table. For each pair, the processor may select one or more rows from the source tuple and elements of the target tuple. For each selected row, the processor may programmatically generate a script that, when performed on the source tuple, produces a value consistent with the target tuple. The processor may apply each script to other rows of the source tuple and determine that an output is in the target tuple. For each column of the target tuple, for the script with output that meets a matching criterion, the processor may convey the output and, in response to a signal accepting the script, join the tables at least in part by performing each accepted script.

Abstract: Embodiments of the present disclosure provide a method for performing search queries. The method comprises transmitting a list of active indexers in an indexer cluster from a cluster master for receipt by a first search head, wherein the cluster master is communicatively coupled with an indexer cluster comprising a plurality of indexers and the first search head. The method further comprises receiving a first slot request at the cluster master in response to a query from the first search head, wherein the first search head is operable to transmit the query to the active indexers for execution if granted the slot request. Further, the method comprises evaluating a plurality of policies to determine if the first slot request can be granted and responsive to a positive determination, transmitting an authorization token for a slot to the first search head.

Abstract: A method for document similarity analysis. The method includes generating a reference document content identifier for a reference document, including identifying frequently occurring terms in reference document content, encoding each frequently occurring term in a term identifier and combining the term identifiers to form the reference document content identifier associated with the reference document. The method also includes obtaining at least one document similarity value by comparing the reference document content identifier to a set of archived document content identifiers stored in a document repository.

Abstract: Methods, systems, apparatuses, and computer program products are provided for increasing an efficiency of queries in program code. A plurality of queries is detected in program code. A laziness is extended by which the queries are evaluated in the program code. The queries are decomposed into a plurality of query components. A ruleset that includes a plurality of rules is applied to the query components to generate a functionally equivalent query set to the plurality of queries that evaluates more efficiently relative to the plurality of queries.

Abstract: Disclosed aspects relate to transforming a user-defined table function (UDTF) to a derived table in a database management system (DBMS). It may be detected by the DBMS that a query indicates the UDTF. A derived table may be constructed from the UDTF. The derived table may be derived from a set of statements of the UDTF. The derived table may be introduced to the query to fulfill the UDTF. A query plan may be generated for the query by the DBMS. The query plan may be generated in response to introducing the derived table in the query.

Abstract: A novel distributed graph database is provided that is designed for efficient graph data storage and processing on modern computing architectures. In particular a single node graph database and a runtime & communication layer allows for composing a distributed graph database from multiple single node instances.

Abstract: Mechanisms for improving resource usage of a computing system by properly balancing the timing of performance degradation analysis of an executable action set of a query based on an impact on the workload of that query. The mechanism collects workload metrics for each of multiple queries and their associated executable action sets. Performance degradation occurs when the current executed action set is no longer appropriate for the corresponding query such that more resources are used than are necessary. For each query, the system determines whether to perform degradation analysis of the current executable action set in a manner that is workload aware. That is, queries that occupy more computing resources or that impact the overall workload more will tend to have degradation analysis performed more frequently than those queries that have less impact on the overall workload. This results in improved resource usage for the workload with low overhead.

Abstract: Beneficial query parameterization of one or more databases may be automatically determined. Data associated with a workload of the one or more databases may be automatically gathered and accessed. The gathered data may then be automatically analyzed to determine whether query parameterization would be beneficial to the one or more databases. Based on the analysis one or more query parameterization recommendations may be generated for presentation to a user of the one or more databases. A user response associated with implementing at least one of the one or more query parameterization recommendations may then be received. Upon implementing at least one of the one or more recommendations, the at least one implemented recommendation may be automatically verified whether an expected improvement of the one or more databases has been realized. When the expected improvement has not been realized, the one or more databases may be reverted to a previous state.

Abstract: A computer-implemented method of generating faster and more efficient database query execution plans includes receiving a database query, generating an optimized query execution plan, and initiating execution of the optimized plan. Database can include table columns with associated column dictionaries. Database query can have statements with conditions. Generating the optimized query execution plan involves comparing contents of a column dictionary semantics of the statements, omitting statements which will always be TRUE and omitting conditions which will always be FALSE. Related apparatus, systems, techniques, methods and articles are also described.

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic partition selection. One of the methods includes receiving a representation of a query plan generated for a query, wherein the query plan includes a dynamic scan operator that represents a first computing node obtaining tuples of one or more partitions of a table from storage and transferring the tuples to a second computing node that executes a parent operator of the dynamic scan operator. A partition selector operator is generated corresponding to the dynamic scan operator. A location in the query plan is determined for the partition selector operator. A modified query plan is generated having the partition selector operator at the determined location.

Abstract: A method dynamically selects query execution operators for a database engine. The database engine receives a query and parses the query to form a query execution tree. The engine creates a first executable plan that includes in-memory operators, which execute within the volatile memory. While executing a first in-memory operator, the engine detects insufficient memory to complete the execution and aborts the execution. The engine then recompiles the query execution tree to form a second executable plan, which includes spooling operators. Each spooling operator executes within a fixed volatile memory budget. The engine executes the second executable plan, including the plurality of spooling operators, to identify a set of results from the database that is responsive to the query, and returns the results.

Abstract: A system for generating a query plan is provided. In some implementations, the system performs operations comprising: determining that a first query operation is a full table query operation, the full table query operation comprising a query operation operating on a database table as a whole; determining that a second query operation is a split table query operation, the split table query operation comprising a plurality of parallel query operations each operating on a portion of the database table; and inserting, based on the first query operation being a full table query operation and the second query operation being a split table query operation, a switch operation between the first and the second query operation, the switch operation being configured to distribute data output by the first query operation to the plurality of parallel query operations comprising the second query operation. Related methods and articles of manufacture are also described.

Abstract: Processing a database query. A method includes receiving a database query from a user. The database query includes one or more imperative functions. The one or more imperative functions are converted to one or more declarative query representations. The one or more declarative query representations include standardized relational operators included in a relational query language. Further, the one or more declarative query representations are optimizable by a query optimizer of the database. The database query is optimized at the query optimizer to create a query plan by evaluating any declarative query representation originally in the database query received from the user and the one or more declarative query representations.

Abstract: Methods, systems, and apparatus for data analysis in a distributed computing system by accessing data stored at a first processing zone associated with a distributed data processing job, detecting information identifying a particular child job associated with the distributed data processing job, comparing the identifying information to data stored at a second processing zone, and identifying an additional child job as associated with the distributed data processing job based on a result of the comparison. The methods, systems and apparatus are further for correlating particular output data associated with the particular child job and additional output data associated with the additional child job for the distributed data processing job, determining performance data for the distributed data processing job based on the output data associated with each of the particular child job and the additional child job, and providing for display the performance data for the distributed data processing job.

Abstract: Embodiments relate to aggregation operations. More specifically, embodiments relate to translating a query involving a distinct aggregate(s) into a query that does not involve a distinct aggregate. The translation involves a transformation effectively rewriting the original query containing the distinct aggregate including removal of the distinct aggregate from the query. The translated query enables the query to perform a non-distinct aggregation that effectively provides the results of the removed distinct aggregate(s).

Abstract: The present application discloses methods for storing data, methods for querying data, and apparatus thereof. A method for storing data may include receiving a data record to be stored. When the data record includes a timestamp, a numerical value generated at a time point identified by the timestamp, and a first combination of identification-field values that can identify the numerical value, the method may also include obtaining a target numerical value and a target time satisfying user query needs, and a second combination of identification-field values as a query condition by the preprocessing of the data record based on the user query needs. The method may further include storing a storage identity representing the second combination of identification-field values, the target time, and the target numerical value into a second information table correspondingly.

Abstract: Methods and apparatus are provided for receiving a database query and executing the query against a remote database. The database query is used to generate an execution sequence comprising HTTP requests and data processing operations based on the database query. The HTTP requests are sent to a RESTful API provided by a database system. The HTTP requests are executed by the database system against a database, and the responses from the database system are returned to be processed according to the data processing operations. Finally, a set of query results is returned to the sender of the original database query.

Abstract: A system for translating a query expression for a client-specified model (CSM) to a plurality of query expressions compatible with the object model(s) for a plurality of source systems. The source systems comprise a plurality of databases. The databases comprise source information and X sets of response data. The source information comprises mapping metadata and object model metadata. The X sets of response data is data response to a client query. The client comprises client-specified mapping metadata, a CSM, and a query proxy. The query proxy comprises an import module, a client interface module, and a translator module. The import module imports the source information from the plurality of source systems and determines the mapping metadata and the object model metadata. The client interface module receives the CSM, the client-specified mapping metadata, and a query from a client.

Abstract: As disclosed herein a method, executed by a computer, for enabling a SPARQL explain tool that provides query execution statistics via an explain plan includes initiating a database query comprising a SPARQL query pattern, optimizing the database query according to query semantics and database metadata to produce an optimized query, and generating an explain plan for the optimized query. The explain plan may include a sequence of operations and an execution time, an input, and an output for each operation of the sequence of operations. A computer system, and a computer program product corresponding to the method are also disclosed herein.

Abstract: A method comprising receiving a request to create an import file for a document browser from a Personal Document Collection comprising a topic; retrieving a topic content associated with the topic from a content management system (CMS), wherein the topic content comprises a user defined content applied to a master topic content; generating a content page from the topic content, wherein the content page is configured to be displayed within the document browser; generating a navigation structure based on the topic content and the document browser; and compress the content page and the navigation structure into the import file.

Abstract: The disclosure generally describes methods, software, and systems, including a method for defining and using dependencies and equivalences for data elements in the data model in order to improve query efficiency. At design time of a data model, dependencies and equivalences for data elements in the data model are received over a computer network. The dependencies and equivalences are stored in persistent storage with the data model. A request is received over the computer network, the request to be executed to obtain results from a database associated with the data model. An optimized query is generated using the received request and the dependencies and equivalences. The optimized query is optimized relative to the received request.

Abstract: Methods and apparatus, including computer program products, are provided for split query optimization. In some example embodiments, a method may include: determining, for a query comprising a plurality of operations, a first workload distribution between a first data store and a second data store, the first workload distribution indicating that a first portion of the query is to be performed at the first data store and a second portion of the query is to be performed at the second data store; and determining, based at least on the first workload distribution, a second workload distribution, the determining of the second workload distribution includes pushing down, to the first portion of the query, a first operation from the second portion of the query such that the first operation is performed at the first data store instead of at the second data store.