Abstract: A serverless query processing system receives a query and determines whether the query is a recurring query or a non-recurring query. The system may predict, in response to determining that the query is the recurring query, a peak resource requirement during an execution of the query. The system may compute, in response to determining that the query is the non-recurring query, a tight resource requirement corresponding to an amount of resources that satisfy a performance requirement over the execution of the query, where the tight resource requirement is less than the peak resource requirement. The system allocates resources to the query based on an applicable one of the peak resource requirement or the tight resource requirement. The system then starts the execution of the query using the resources.

Abstract: Systems, methods, and devices are described for performing scalable data processing operations. A queue that includes a translatable portion comprising indications of data processing operations translatable to data queries and a non-translatable portion comprising indications of non-translatable data processing operations is maintained. A determination that a first data processing operation of a first code block statement is translatable to a database query is made. An indication of the first data processing operation is included in the translatable portion of the queue. Responsive to a determination that a second data processing operation of a second code block statement is undeferrable, the translatable portion of the queue is compiled into a database query. An execution of the database query to be executed by a database engine to generate a query result is caused. A result dataset corresponding to the query result is transmitted to an application configured to analyze the result dataset.

Abstract: Systems, methods, and devices are described for performing scalable data processing operations. A queue that includes a translatable portion comprising indications of data processing operations translatable to data queries and a non-translatable portion comprising indications of non-translatable data processing operations is maintained. A determination that a first data processing operation of a first code block statement is translatable to a database query is made. An indication of the first data processing operation is included in the translatable portion of the queue. Responsive to a determination that a second data processing operation of a second code block statement is undeferrable, the translatable portion of the queue is compiled into a database query. An execution of the database query to be executed by a database engine to generate a query result is caused. A result dataset corresponding to the query result is transmitted to an application configured to analyze the result dataset.

Abstract: Performance degradation of an application that is caused by another computing process that shares infrastructure with the application is detected. The application and the other computing device may execute via different virtual machines hosted on the same computing device. To detect the performance degradation that is attributable to the other computing process, certain storage segments of a data storage (e.g., a cache) shared by the virtual machines is written with data. A pattern of read operations are then performed on the segments to determine whether an increase in read access time has occurred. Such a performance degradation is attributable to another computing process. After detecting the degradation, a metric that quantifies the detected degradation attributable to the other computing process is provided to an ML model, which determines the actual performance of the application absent the degradation attributable to the other computing process.

Abstract: Performance degradation of an application that is caused by another computing process that shares infrastructure with the application is detected. The application and the other computing device may execute via different virtual machines hosted on the same computing device. To detect the performance degradation that is attributable to the other computing process, certain storage segments of a data storage (e.g., a cache) shared by the virtual machines is written with data. A pattern of read operations are then performed on the segments to determine whether an increase in read access time has occurred. Such a performance degradation is attributable to another computing process. After detecting the degradation, a metric that quantifies the detected degradation attributable to the other computing process is provided to an ML model, which determines the actual performance of the application absent the degradation attributable to the other computing process.

Abstract: A serverless query processing system receives a query and determines whether the query is a recurring query or a non-recurring query. The system may predict, in response to determining that the query is the recurring query, a peak resource requirement during an execution of the query. The system may compute, in response to determining that the query is the non-recurring query, a tight resource requirement corresponding to an amount of resources that satisfy a performance requirement over the execution of the query, where the tight resource requirement is less than the peak resource requirement. The system allocates resources to the query based on an applicable one of the peak resource requirement or the tight resource requirement. The system then starts the execution of the query using the resources.

Abstract: Various technologies described herein pertain to controlling geo-scale analytics with bandwidth and regulatory constraints. An analytical query (e.g., a recurrent analytical query, a non-recurrent analytical query, etc.) to be executed over distributed data in data partitions stored in a plurality of data centers can be received. Moreover, a query execution plan for the analytical query can be generated, where the query execution plan includes tasks. Further, replication strategies for the data partitions can be determined. A replication strategy for a particular data partition can specify one or more data centers to which the particular data partition is to be replicated if the particular data partition is to be replicated. The tasks of the query execution plan for the analytical query can further be scheduled to the data centers based on the replication strategies for the data partitions. The analytical query can be part of a workload of analytical queries.

Abstract: A serverless query processing system receives a query and determines whether the query is a recurring query or a non-recurring query. The system may predict, in response to determining that the query is the recurring query, a peak resource requirement during an execution of the query. The system may compute, in response to determining that the query is the non-recurring query, a tight resource requirement corresponding to an amount of resources that satisfy a performance requirement over the execution of the query, where the tight resource requirement is less than the peak resource requirement. The system allocates resources to the query based on an applicable one of the peak resource requirement or the tight resource requirement. The system then starts the execution of the query using the resources.

Abstract: Various technologies described herein pertain to controlling geo-scale analytics with bandwidth and regulatory constraints. An analytical query (e.g., a recurrent analytical query, a non-recurrent analytical query, etc.) to be executed over distributed data in data partitions stored in a plurality of data centers can be received. Moreover, a query execution plan for the analytical query can be generated, where the query execution plan includes tasks. Further, replication strategies for the data partitions can be determined. A replication strategy for a particular data partition can specify one or more data centers to which the particular data partition is to be replicated if the particular data partition is to be replicated. The tasks of the query execution plan for the analytical query can further be scheduled to the data centers based on the replication strategies for the data partitions. The analytical query can be part of a workload of analytical queries.

Abstract: Methods, systems, and computer program products are provided for generating and utilizing materialized graph views. A system according to one embodiment includes a graph database including a graph and schema, a workload analyzer, a view enumerator, a query rewriter and an execution engine. The workload analyzer is configured to receive and analyze queries in a query workload. The view enumerator is configured to use an inference engine to operate on facts derived from the graph and a query, and view templates comprising inference rules to enumerate candidate views. The workload analyzer is further configured to selects a candidate view to materialize, provide the selected view to the execution engine that is configured to generate the materialized view. The workload analyzer may select the at least one candidate view based on factors such as query evaluation cost estimates, candidate view performance improvement estimates, view size estimates and view creation cost estimates.

Abstract: Methods, systems, and computer program products are provided for generating a neural network model. A ML pipeline parser is configured to identify a set of ML operators for a previously trained ML pipeline, and map the set of ML operators to a set of neural network operators. The ML pipeline parser generates a first neural network representation using the set of neural network operators. A neural network optimizer is configured to perform an optimization on the first neural network representation to generate a second neural network representation. A tensor set provider outputs a set of tensor operations based on the second neural network representation for execution on a neural network framework. In this manner, a traditional ML pipeline can be converted into a neural network pipeline that may be executed on an appropriate framework, such as one that utilizes specialized hardware accelerators.

Abstract: The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Abstract: A serverless query processing system receives a query and determines whether the query is a recurring query or a non-recurring query. The system may predict, in response to determining that the query is the recurring query, a peak resource requirement during an execution of the query. The system may compute, in response to determining that the query is the non-recurring query, a tight resource requirement corresponding to an amount of resources that satisfy a performance requirement over the execution of the query, where the tight resource requirement is less than the peak resource requirement. The system allocates resources to the query based on an applicable one of the peak resource requirement or the tight resource requirement. The system then starts the execution of the query using the resources.

Abstract: The cloud-based query workload optimization system disclosed herein the cloud-based query workloads optimization system receives query logs from various query engines to a cloud data service, extracts various query entities from the query logs, parses query entities to generate a set of common workload features, generates intermediate representations of the query workloads, wherein the intermediate representations are agnostic to the language of the plurality of the queries, identifies a plurality of workload patterns based on the intermediate representations of the query workloads, categorizes the workloads in one or more workload type categories based on the workload patterns and the workload features, and selects an optimization scheme based on the category of workload pattern.

Abstract: Described herein is a system and method for ranking and/or taking an action regarding execution of jobs of a shared computing cluster based upon predicted user impact. Information regarding previous executions of a plurality of jobs is obtained, for example, from job execution log(s). Data dependencies of the plurality of jobs are determined. Job impact of each of the plurality of jobs as a function of the determined data dependencies is calculated. User impact of each of the plurality of jobs as a function of the determined data dependencies, the calculated job impact, and time is calculated. The plurality of jobs are ranked in accordance with the calculated user impact. An action is taken in accordance with the ranking of the plurality of jobs. The action can include automatic scheduling of the jobs and/or providing information regarding the rankings to a user.

Abstract: Methods, systems, and computer program products are provided for generating and utilizing materialized graph views. A system according to one embodiment includes a graph database including a graph and schema, a workload analyzer, a view enumerator, a query rewriter and an execution engine. The workload analyzer is configured to receive and analyze queries in a query workload. The view enumerator is configured to use an inference engine to operate on facts derived from the graph and a query, and view templates comprising inference rules to enumerate candidate views. The workload analyzer is further configured to selects a candidate view to materialize, provide the selected view to the execution engine that is configured to generate the materialized view. The workload analyzer may select the at least one candidate view based on factors such as query evaluation cost estimates, candidate view performance improvement estimates, view size estimates and view creation cost estimates.

Abstract: Described herein is a system and method for ranking and/or taking an action regarding execution of jobs of a shared computing cluster based upon predicted user impact. Information regarding previous executions of a plurality of jobs is obtained, for example, from job execution log(s). Data dependencies of the plurality of jobs are determined. Job impact of each of the plurality of jobs as a function of the determined data dependencies is calculated. User impact of each of the plurality of jobs as a function of the determined data dependencies, the calculated job impact, and time is calculated. The plurality of jobs are ranked in accordance with the calculated user impact. An action is taken in accordance with the ranking of the plurality of jobs. The action can include automatic scheduling of the jobs and/or providing information regarding the rankings to a user.

Abstract: Various technologies described herein pertain to controlling geo-scale analytics with bandwidth and regulatory constraints. An analytical query (e.g., a recurrent analytical query, a non-recurrent analytical query, etc.) to be executed over distributed data in data partitions stored in a plurality of data centers can be received. Moreover, a query execution plan for the analytical query can be generated, where the query execution plan includes tasks. Further, replication strategies for the data partitions can be determined. A replication strategy for a particular data partition can specify one or more data centers to which the particular data partition is to be replicated if the particular data partition is to be replicated. The tasks of the query execution plan for the analytical query can further be scheduled to the data centers based on the replication strategies for the data partitions. The analytical query can be part of a workload of analytical queries.