Abstract: The present disclosure describes an analytical data management system (ADMS) that serves critical dashboards, applications, and internal users. This ADMS has high scalability, and availability through replication and failover, high user query load, and large data volumes. The ADMS provides continuous ingestion and high performance querying with tunable freshness. It further advances the idea of disaggregation by decoupling its architectural components: ingestion, indexing, and querying. As a result, the impact of a slow down in indexing on the query performance is minimized by either trading off data freshness or incurring higher costs.

Abstract: A computer-implemented method for pairing a new document to a document from a plurality of documents. Embodiments include, for each of the new document and the plurality of documents, generating a vector of terms of interest uniquely associated with a document of the new document and the plurality of documents. For each term of interest, an associated element value of the vector is assigned as zero if the term of interest does not occur in the document and one otherwise. The method also includes, for each document from the plurality of documents, determining a similarity between the vectors. The method also includes selecting a document from the plurality of documents as related to the new document if the similarity between the vector for the new document and the vector for the document from the plurality of documents is greater than or equal to a threshold value.

Abstract: Systems and methods are disclosed for managing a multi-store execution environment by applying opportunistic materialized views to improve workload performance and executing a plan on multiple database engines to increase query processing speed by leveraging unique capabilities of each engine by enabling stages of a query to execute on multiple engines, and by moving materialized views across engines.

Abstract: Systems and methods for predicting query execution time for concurrent and dynamic database workloads include decomposing each query into a sequence of query pipelines based on the query plan from a query optimizer, and predicting an execution time of each pipeline with a progress predictor for a progress chart of query pipelines.

Abstract: A system includes a task scheduler that works collaboratively with a flow scheduler; a network-aware task scheduler based on software-defined network, the task scheduler scheduling tasks according to available network bandwidth.

Abstract: Systems and methods for consolidating workload on one or more computing devices, comprising: generating tenant performance profiles for at least one of a buffer memory size and a buffer warmness; assigning one or more tenants to a database management system (DBMS) instance; configuring the one or more tenants to minimize a total server count needed to host the one or more tenants; and reducing the total server count by adjusting of buffer sizes while satisfying Service Level Agreement (SLA) requirements.

Abstract: Systems and methods for prioritizing queries in a cloud based database by determining a priority score in real time under an imprecise query execution time; and incrementally maintaining query priority scores.

Abstract: A system for evolutionary analytics supports three dimensions (analytical workflows, the users, and the data) by rewriting workflows to be more efficient by using answers materialized as part of previous workflow execution runs in the system.

Abstract: A system includes a task scheduler that works collaboratively with a flow scheduler; a network-aware task scheduler based on software-defined network, the task scheduler scheduling tasks according to available network bandwidth.

Abstract: Systems and methods are disclosed to perform performance prediction for cloud-based databases by building on a computer a cloud database performance model using a set of training workloads; and using a learned model on the computer to predict database performance in the cloud for a new workload, wherein for each reference workload r and hardware configuration h, system throughput tr,h, average throughput of ?r and standard deviation ?r, comprising normalizing each throughput as: t r , h _ = t r , h - a r ? r .

Abstract: A method for database consolidation includes generating a model for expected penalty estimation; determining a tenant's value as a function of query arrival rate and SLA penalty; placing a tenant to minimize a total expected cost in the order of the tenant value; and progressively using additional servers to prevent any server from being saturated to guarantee a tenant placement that costs no more than four times the cost of any other placement.

Abstract: Systems and methods are disclosed for operating a software-defined network (SDN) by slicing the SDN into differentiated queues according to different priorities to prioritizes the queries based on the user's request; reserving necessary bandwidth for specific queries to ensure specific performance levels based on the user's request; providing information to a query plan executor; and managing performance of analytical queries in distributed relational databases.

Abstract: Systems and methods are disclosed for selecting a query plan in a database by monitoring network state information and flow information; and selecting an adaptive plan for execution with a query manager that receives the network state information and flow information, including: receiving a query, parsing the query, generating and optimizing a global query plan; dividing the global query plan into local plans; sending the local plans to corresponding data store sites for execution with separate threads; and orchestrating data flows among the data store sites and forwarding a final result to a user.

Abstract: A method for balancing database workloads among a plurality of database servers includes when a new server is available, selecting a set of master and slave databases to be migrated to the new server; and migrating the selected databases to result in a balanced new workload among all servers while minimizing migration cost; and during operation, if workload unbalance is detected in real time from a workload change in a certain database, iteratively selecting one database at a time to migrate to a different server to balance the workload.

Abstract: A method for tenant placement in a multi-tenant system is shown that includes creating a weighted graph of tenants and sharing relationships between tenants, where a weight for each tenant and each sharing relationship represents an associated degree of resource consumption; and adding one or more tenants to a database using a processor based on said weighted graph and a database capacity, such that the combined weight of the added tenants and the sharing relationships belonging to the added tenants are within the database capacity. If a tenant cannot be added to the database without exceeding the database capacity, a new database is created and the one or more tenants are added to the new database, subject to a new database capacity. The adding and creating steps are repeated until all tenants have been added to a database.

Abstract: A computer implemented method of workload scheduling under two different types of Service Level Agreements (SLAs), namely soft and hard SLA which employ both a deadline- and cost-aware scheduler called iCBS-DH. The scheduler is evaluated against deadline and cost performance of various scheduling policies under a large range of SLA cost function and deadline types.

Abstract: Methods and systems for database migration from a multitenant database include taking a snapshot of an original database to be migrated with a hot backup process, such that the database is still capable of answering queries during the hot backup process; maintaining a query log of all queries to the tenant database after the hot backup process begins; initializing a new database at a target server using the snapshot; replaying the query log synchronize the new database with the original database; and answering new queries with the new database and not the original database.

Abstract: There are provided a system and method for predicting query execution time in a database system. A cost model determination device determines a cost model of a database query optimizer for the database system. The cost model models costs of queries applied to the database system. A profiling device determines profiling queries for profiling input/output cost units and processor cost units relating to the database system, and profiles the cost units using the profiling queries to output profiled cost units. A calibrating device calibrates cost units in the cost model responsive to the profiled cost units to output calibrated cost units. A sampling re-estimator samples and re-estimates a cardinality estimation of a final query plan to output an updated cardinality estimation. A predictor applies the calibrated cost units and the updated cardinality estimation in the cost model to generate a prediction of an execution time of a given query.

Abstract: Methods for generating a data fetching plan in a multi-tenant system include placing tenants in consecutively allocated databases according to a weighted graph of tenants and sharing relationships between tenants, where at least one sharing relationship includes multiple accessing tenants accessing a given set of data from a provider tenant. For each sharing relationship, if a current database has one or more accessing tenants and does not have the provider tenant, data is fetched from the latest-allocated database prior to the current database that has accessing tenants, if such a database exists; if a current database has the provider tenant, data is provided to the earliest-allocated database after the current database that has accessing tenants if such a database exists. The fetching and providing steps are repeated for each allocated database.

Abstract: Systems and methods for data sharing include generating at least one sharing plan with a cheapest cost and/or a shortest execution time for one or more sharing arrangements. Admissibility of the one or more sharing arrangements is determined such that a critical time path of the at least one sharing plan does not exceed a staleness level and a cost of the at least one sharing plan does not exceed a capacity. Sharing plans of admissible sharing arrangements are executed while maintaining the staleness level.