Abstract: Methods, systems, apparatuses, and computer-readable storage mediums described herein are directed to determining and recommending an optimal compute resource configuration for a cloud-based resource (e.g., a server, a virtual machine, etc.) for migrating a customer to the cloud. The embodiments described herein utilize a statistically robust approach that makes recommendations that are more flexible (elastic) and account for the full distribution of the amount of resource usage. Such an approach is utilized to develop a personalized rank of relevant recommendations to a customer. To determine which compute resource configuration to recommend to the customer, the customer's usage profile is matched to a set of customers that have already migrated to the cloud. The compute resource configuration that reaches the performance most similar to the performance of the configurations utilized by customers in the matched set is recommended to the user.

Abstract: The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Abstract: Example solutions provide an artificial intelligence (AI) agent for pre-build configuration of cloud services in order to enable the initial build of a computational resource (e.g., in a cloud service) to minimize the likelihood of excessive throttling or slack. Examples leverage prior-existing utilization data and project metadata to identify similar use cases. The utilization data includes capacity information and resource consumption information (e.g., throttling and slack) for prior-existing computational resources, and the project metadata includes information for hierarchically categorization, to identify similar resources. A pre-build configuration is generated for the customer's resource, which the customer may tune based upon the customer's preferences for a cost and performance balance point.

Abstract: Methods, systems, apparatuses, and computer-readable storage mediums described herein are directed to determining and recommending an optimal compute resource configuration for a cloud-based resource (e.g., a server, a virtual machine, etc.) for migrating a customer to the cloud. The embodiments described herein utilize a statistically robust approach that makes recommendations that are more flexible (elastic) and account for the full distribution of the amount of resource usage. Such an approach is utilized to develop a personalized rank of relevant recommendations to a customer. To determine which compute resource configuration to recommend to the customer, the customer's usage profile is matched to a set of customers that have already migrated to the cloud. The compute resource configuration that reaches the performance most similar to the performance of the configurations utilized by customers in the matched set is recommended to the user.

Abstract: Systems, methods and computer-readable memory devices are provided for greater efficiency in the configuration of a database cluster for performing a query workload. A database cluster configuration system is provided that includes a database cluster comprising one or more compute resources configured to perform database queries. A query workload comprising a plurality of queries is received. An initial workload-level configuration is applied. For each query of the query workload, a query-level configuration is generated using a query configuration model corresponding to each query in a contextual Bayesian optimization with centroid learning while also leveraging the query plan for each executing query for query characterization and including application of virtual operators. Query events are collected and used to update the corresponding query configuration model. The workload-level configuration is updated based on the query events and cached for use during a subsequent execution of the workload.

Abstract: The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Abstract: The present application relates to a network, apparatus, and method for allocating clusters of computing nodes for programming jobs. A network includes a plurality of datacenters including computing resources configurable to instantiate nodes for executing programming jobs on a cluster. The computing resources at one of the datacenters are configured to: provision a live pool including a number of clusters, each cluster in the live pool including a plurality of nodes imaged with a configuration for executing the programming jobs in parallel on the cluster; receive a request from a user to execute a programming job; allocate a cluster from the live pool to the user for the programming job when the cluster is available; evict the cluster from the live pool; and provision a new cluster within the live pool to meet the number of clusters. The number of clusters may be optimized based on linear programming and machine-learning.

Abstract: The cloud-based query workload optimization system disclosed herein the cloud-based query workloads optimization system receives query logs from various query engines to a cloud data service, extracts various query entities from the query logs, parses query entities to generate a set of common workload features, generates intermediate representations of the query workloads, wherein the intermediate representations are agnostic to the language of the plurality of the queries, identifies a plurality of workload patterns based on the intermediate representations of the query workloads, categorizes the workloads in one or more workload type categories based on the workload patterns and the workload features, and selects an optimization scheme based on the category of workload pattern.

Abstract: Learned workload synthesis is disclosed. In an aspect of the present disclosure, a time series dataset corresponding to a target workload is received. A set of performance characteristics is determined from the time series dataset. A call is provided to a prediction model to determine a candidate query sequence based on the determined set of performance characteristics. A synthetic workload is generated based on the determined candidate query sequence. A synthetic workload is generated based on the determined candidate query sequence. A first similarity between a first performance profile of the synthetic workload and a second performance profile of the target workload meets a workload performance threshold condition. A performance insight is determined based on the synthetic workload. In a further aspect, the prediction model is trained to predict performance profiles based on workload profiles generated by executing benchmark queries using hardware and/or software configurations.

Abstract: An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such performance is identified through monitoring various telemetric data of the cloud infrastructure. The tuning service leverages a mix of domain knowledge and principled data-science to capture the essence of our cluster dynamic behavior in a collection of descriptive machine learning (ML) models. The ML models power automated optimization procedures for parameter tuning, and inform administrators in most tactical and strategical engineering/capacity decisions (such as hardware and datacenter design, software investments, etc.).

Abstract: An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such performance is identified through monitoring various telemetric data of the cloud infrastructure. The tuning service leverages a mix of domain knowledge and principled data-science to capture the essence of our cluster dynamic behavior in a collection of descriptive machine learning (ML) models. The ML models power automated optimization procedures for parameter tuning, and inform administrators in most tactical and strategical engineering/capacity decisions (such as hardware and datacenter design, software investments, etc.).

Abstract: Embodiments described herein enable tracking machine learning (“ML”) model data provenance. In particular, a computing device is configured to accept ML model code that, when executed, instantiates and trains an ML model, to parse the ML model code into a workflow intermediate representation (WIR), to semantically annotate the WIR to provide an annotated WIR, and to identify, based on the annotated WIR and ML API corresponding to the ML model code, data from at least one data source that is relied upon by the ML model code when training the ML model. A WIR may be generated from an abstract syntax tree (AST) based on the ML model code, generating provenance relationships (PRs) based at least in part on relationships between nodes of the AST, wherein a PR comprises one or more input variables, an operation, a caller, and one or more output variables.

Abstract: Linguistic schema mapping via semi-supervised learning is used to map a customer schema to a particular industry-specific schema (ISS). The customer schema is received and a corresponding ISS is identified. An attribute in the customer schema is selected for labeling. Candidate pairs are generated that include the first attribute and one or more second attributes which may describe the first attribute. A featurizer determines similarities between the first attribute and second attribute in each generated pair, one or more suggested labels are generated by a machine learning (ML) model, and one of the suggested labels is applied to the first attribute.

Abstract: A system enables tracking machine learning (“ML”) model data provenance. In particular, a computing device is configured to accept ML model code that, when executed, instantiates and trains an ML model, to parse the ML model code into a workflow intermediate representation (WIR), to semantically annotate the WIR to provide an annotated WIR, and to identify, based on the annotated WIR and ML API corresponding to the ML model code, data from at least one data source that is relied upon by the ML model code when training the ML model. A WIR may be generated from an abstract syntax tree (AST) based on the ML model code, generating provenance relationships (PRs) based at least in part on relationships between nodes of the AST, wherein a PR comprises one or more input variables, an operation, a caller, and one or more output variables.

Abstract: Methods, systems, apparatuses, and computer-readable storage mediums described herein are directed to determining and recommending an optimal compute resource configuration for a cloud-based resource (e.g., a server, a virtual machine, etc.) for migrating a customer to the cloud. The embodiments described herein utilize a statistically robust approach that makes recommendations that are more flexible (elastic) and account for the full distribution of the amount of resource usage. Such an approach is utilized to develop a personalized rank of relevant recommendations to a customer. To determine which compute resource configuration to recommend to the customer, the customer’s usage profile is matched to a set of customers that have already migrated to the cloud. The compute resource configuration that reaches the performance most similar to the performance of the configurations utilized by customers in the matched set is recommended to the user.

Abstract: An automated tuning service is used to automatically tune, or modify, the operational parameters of a large-scale cloud infrastructure. The tuning service performs automated and fully data/model-driven configuration based from learning various real-time performance of the cloud infrastructure. Such performance is identified through monitoring various telemetric data of the cloud infrastructure. The tuning service leverages a mix of domain knowledge and principled data-science to capture the essence of our cluster dynamic behavior in a collection of descriptive machine learning (ML) models. The ML models power automated optimization procedures for parameter tuning, and inform administrators in most tactical and strategical engineering/capacity decisions (such as hardware and datacenter design, software investments, etc.).

Abstract: Embodiments described herein enable tracking machine learning (“ML”) model data provenance. In particular, a computing device is configured to accept ML model code that, when executed, instantiates and trains an ML model, to parse the ML model code into a workflow intermediate representation (WIR), to semantically annotate the WIR to provide an annotated WIR, and to identify, based on the annotated WIR and ML API corresponding to the ML model code, data from at least one data source that is relied upon by the ML model code when training the ML model. A WIR may be generated from an abstract syntax tree (AST) based on the ML model code, generating provenance relationships (PRs) based at least in part on relationships between nodes of the AST, wherein a PR comprises one or more input variables, an operation, a caller, and one or more output variables.

Abstract: The description relates to executing an inference query relative to a database management system, such as a relational database management system. In one example a trained machine learning model can be stored within the database management system. An inference query can be received that applies the trained machine learning model on data local to the database management system. Analysis can be performed on the inference query and the trained machine learning model to generate a unified intermediate representation of the inference query and the trained model. Cross optimization can be performed on the unified intermediate representation. Based upon the cross-optimization, a first portion of the unified intermediate representation to be executed by a database engine of the database management system can be determined, and, a second portion of the unified intermediate representation to be executed by a machine learning runtime can be determined.

Abstract: The cloud-based query workload optimization system disclosed herein the cloud-based query workloads optimization system receives query logs from various query engines to a cloud data service, extracts various query entities from the query logs, parses query entities to generate a set of common workload features, generates intermediate representations of the query workloads, wherein the intermediate representations are agnostic to the language of the plurality of the queries, identifies a plurality of workload patterns based on the intermediate representations of the query workloads, categorizes the workloads in one or more workload type categories based on the workload patterns and the workload features, and selects an optimization scheme based on the category of workload pattern.

Abstract: Described herein is a system and method for ranking and/or taking an action regarding execution of jobs of a shared computing cluster based upon predicted user impact. Information regarding previous executions of a plurality of jobs is obtained, for example, from job execution log(s). Data dependencies of the plurality of jobs are determined. Job impact of each of the plurality of jobs as a function of the determined data dependencies is calculated. User impact of each of the plurality of jobs as a function of the determined data dependencies, the calculated job impact, and time is calculated. The plurality of jobs are ranked in accordance with the calculated user impact. An action is taken in accordance with the ranking of the plurality of jobs. The action can include automatic scheduling of the jobs and/or providing information regarding the rankings to a user.