Abstract: Described herein is a system and method for detecting and reusing overlapping computations. Overlapping subgraphs of the query are determined using a normalized signature for a particular subgraph that identifies a particular subgraph across recurring instances of data. A normalized signature for each overlapping subgraph for the determined overlapping subgraphs of the query is provided. For each overlapping subgraph determined to be materialized: whether or not the particular subgraph has been materialized is determined using a precise signature corresponding to a normalized signature of the particular overlapping subgraph. The precise signature identifies a particular subgraph corresponding to the normalized signature within a particular recurring instance of data. When the particular subgraph has not been materialized, the subgraph is materialized and used to respond to the query. When the particular subgraph has been materialized, the materialized subgraph is used to respond to the query.

Abstract: The cloud-based query workload optimization system disclosed herein the cloud-based query workloads optimization system receives query logs from various query engines to a cloud data service, extracts various query entities from the query logs, parses query entities to generate a set of common workload features, generates intermediate representations of the query workloads, wherein the intermediate representations are agnostic to the language of the plurality of the queries, identifies a plurality of workload patterns based on the intermediate representations of the query workloads, categorizes the workloads in one or more workload type categories based on the workload patterns and the workload features, and selects an optimization scheme based on the category of workload pattern.

Abstract: Processing a job request for multiple versions of a distributed computing service. The service processing node does this by at least interleavingly (e.g., via time sharing with rapid context switching, or by actually concurrently) running a first runtime library associated with a first service version of the distributed computerized service and a second runtime library associated with a different service version of the distributed computerized service. While running the first runtime library, job requests of a first service version may be at least partially processed using a first set of one or more executables that interact with the first runtime library. While running the second runtime library, job requests of a second service version may be at least partially processed using a second set of one or more executables that interact with the second runtime library.

Abstract: Processing received job requests for a multi-versioned distributed computerized service. For each received job request, the job request is channeled to an appropriate service processing node that depends on the version of the distributed computing service that is to handle the job request. A version of the distributed computing service is assigned to the incoming job request. A service processing node that runs a runtime library for the assigned service version is then identified. The identified service processing node also has an appropriate set of one or more executables that allows the service processing node to plan an appropriate role (e.g., compiler, scheduler, worker) in the distributed computing service. The job request is then dispatched to the identified service processing node.

Abstract: Processing a job request for multiple versions of a distributed computing service. The service processing node does this by at least interleavingly (e.g., via time sharing with rapid context switching, or by actually concurrently) running a first runtime library associated with a first service version of the distributed computerized service and a second runtime library associated with a different service version of the distributed computerized service. While running the first runtime library, job requests of a first service version may be at least partially processed using a first set of one or more executables that interact with the first runtime library. While running the second runtime library, job requests of a second service version may be at least partially processed using a second set of one or more executables that interact with the second runtime library.

Abstract: Described herein is a system and method for detecting and reusing overlapping computations. Overlapping subgraphs of the query are determined using a normalized signature for a particular subgraph that identifies a particular subgraph across recurring instances of data. A normalized signature for each overlapping subgraph for the determined overlapping subgraphs of the query is provided. For each overlapping subgraph determined to be materialized: whether or not the particular subgraph has been materialized is determined using a precise signature corresponding to a normalized signature of the particular overlapping subgraph. The precise signature identifies a particular subgraph corresponding to the normalized signature within a particular recurring instance of data. When the particular subgraph has not been materialized, the subgraph is materialized and used to respond to the query. When the particular subgraph has been materialized, the materialized subgraph is used to respond to the query.

Abstract: Aspects extend to methods, systems, and computer program products for optimally pipelining result sets with fault tolerance in distributed query execution. Distributed computing jobs are optimized by dividing the distributed computing jobs into one or more bubbles for execution. Each bubble can be independently executed, potentially in parallel with other bubbles, when resources to handle the bubble are available. Intra-bubble communication can be streamed between vertices within a bubble. Inter-bubble communication can be stored to durable storage. Bubbles provide a failure boundary for a job graph and re-executing a bubble along with storage of intermediate results in durable storage can be used to recover from failures. When a vertex inside a bubble fails, computation can resume by rescheduling the execution of the failed bubble from the durable inputs for that bubble. Durable storage provides a light-weight failover to handle non-deterministic behavior.

Abstract: Processing a job request for multiple versions of a distributed computing service. The service processing node does this by at least interleavingly (e.g., via time sharing with rapid context switching, or by actually concurrently) running a first runtime library associated with a first service version of the distributed computerized service and a second runtime library associated with a different service version of the distributed computerized service. While running the first runtime library, job requests of a first service version may be at least partially processed using a first set of one or more executables that interact with the first runtime library. While running the second runtime library, job requests of a second service version may be at least partially processed using a second set of one or more executables that interact with the second runtime library.

Abstract: Aspects extend to methods, systems, and computer program products for optimally pipelining result sets with fault tolerance in distributed query execution. Distributed computing jobs are optimized by dividing the distributed computing jobs into one or more bubbles for execution. Each bubble can be independently executed, potentially in parallel with other bubbles, when resources to handle the bubble are available. Intra-bubble communication can be streamed between vertices within a bubble. Inter-bubble communication can be stored to durable storage. Bubbles provide a failure boundary for a job graph and re-executing a bubble along with storage of intermediate results in durable storage can be used to recover from failures. When a vertex inside a bubble fails, computation can resume by rescheduling the execution of the failed bubble from the durable inputs for that bubble. Durable storage provides a light-weight failover to handle non-deterministic behavior.

Abstract: Processing a job request for multiple versions of a distributed computing service. The service processing node does this by at least interleavingly (e.g., via time sharing with rapid context switching, or by actually concurrently) running a first runtime library associated with a first service version of the distributed computerized service and a second runtime library associated with a different service version of the distributed computerized service. While running the first runtime library, job requests of a first service version may be at least partially processed using a first set of one or more executables that interact with the first runtime library. While running the second runtime library, job requests of a second service version may be at least partially processed using a second set of one or more executables that interact with the second runtime library.

Abstract: Processing received job requests for a multi-versioned distributed computerized service. For each received job request, the job request is channeled to an appropriate service processing node that depends on the version of the distributed computing service that is to handle the job request. A version of the distributed computing service is assigned to the incoming job request. A service processing node that runs a runtime library for the assigned service version is then identified. The identified service processing node also has an appropriate set of one or more executables that allows the service processing node to plan an appropriate role (e.g., compiler, scheduler, worker) in the distributed computing service. The job request is then dispatched to the identified service processing node.