Abstract: A computing device is provided, including a processor that receives a network graph. The processor further receives a specification of a network traffic control heuristic for a network traffic routing problem over the network graph. The processor further constructs a gap maximization problem that has, as a maximization target, a difference between an exact solution to the network traffic routing problem and a heuristic solution generated using the network traffic control heuristic. The processor further generates a Lagrange multiplier formulation of the gap maximization problem. At a convex solver, the processor further computes an estimated maximum gap as an estimated solution to the Lagrange multiplier formulation of the gap maximization problem. The processor further performs a network traffic control action based at least in part on the estimated maximum gap.

Abstract: A computing system identifies mitigation actions in response to failures within a computer network. A service level objective is obtained by the computing system for client-resource data flows traversing the computer network between client-side and resource-side nodes. Indication of a failure event at a network location of the computer network is obtained. For each mitigation action of a set of candidate mitigation actions, an estimated impact to a distribution of the service level objective is determined for the mitigation action by applying simulated client-resource data flows to a network topology model of the computer network in combination with the mitigation action and the failure event. One or more target mitigation actions are identified by the computing system from the set of candidate mitigation actions based on a comparison of the estimated impacts of the set of candidate mitigation actions.

Abstract: A method of updating a trained cardinality estimation model includes receiving a cardinality estimation model with cardinality labels and detecting a drift in underlying data or predicates of the cardinality estimation model. The type of the detected drift is determined and new test queries that mimic test queries for the detected drift are synthesized. A portion of the synthesized test queries is selected to reduce annotation cost and used to update the cardinality estimation model.

Abstract: A computing system identifies mitigation actions in response to failures within a computer network. A service level objective is obtained by the computing system for client-resource data flows traversing the computer network between client-side and resource-side nodes. Indication of a failure event at a network location of the computer network is obtained. For each mitigation action of a set of candidate mitigation actions, an estimated impact to a distribution of the service level objective is determined for the mitigation action by applying simulated client-resource data flows to a network topology model of the computer network in combination with the mitigation action and the failure event. One or more target mitigation actions are identified by the computing system from the set of candidate mitigation actions based on a comparison of the estimated impacts of the set of candidate mitigation actions.

Abstract: Methods, systems, apparatuses, and computer program products are provided for prefetching data. A workload analyzer may identify job characteristics for a plurality of previously executed jobs in a workload executing on a cluster of one or more compute resources. For each job, identified job characteristics may include identification of an input dataset and an input bandwidth characteristic for the input dataset. A future workload predictor may identify future jobs expected to execute on the cluster based at least on the identified job characteristics. A cache assignment determiner may determine a cache assignment that identifies a prefetch dataset for at least one of the future jobs. A network bandwidth allocator may determine a network bandwidth assignment for the prefetch dataset. A plan instructor may instruct a compute resource of the cluster to load data to a cache local to the cluster according to the cache assignment and the network bandwidth assignment.

Abstract: Methods, systems, apparatuses, and computer program products are provided for prefetching data. A workload analyzer may identify job characteristics for a plurality of previously executed jobs in a workload executing on a cluster of one or more compute resources. For each job, identified job characteristics may include identification of an input dataset and an input bandwidth characteristic for the input dataset. A future workload predictor may identify future jobs expected to execute on the cluster based at least on the identified job characteristics. A cache assignment determiner may determine a cache assignment that identifies a prefetch dataset for at least one of the future jobs. A network bandwidth allocator may determine a network bandwidth assignment for the prefetch dataset. A plan instructor may instruct a compute resource of the cluster to load data to a cache local to the cluster according to the cache assignment and the network bandwidth assignment.

Abstract: Methods, systems, apparatuses, and computer program products are provided for prefetching data. A workload analyzer may identify job characteristics for a plurality of previously executed jobs in a workload executing on a cluster of one or more compute resources. For each job, identified job characteristics may include identification of an input dataset and an input bandwidth characteristic for the input dataset. A future workload predictor may identify future jobs expected to execute on the cluster based at least on the identified job characteristics. A cache assignment determiner may determine a cache assignment that identifies a prefetch dataset for at least one of the future jobs. A network bandwidth allocator may determine a network bandwidth assignment for the prefetch dataset. A plan instructor may instruct a compute resource of the cluster to load data to a cache local to the cluster according to the cache assignment and the network bandwidth assignment.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.

Abstract: Methods, systems, apparatuses, and computer program products are provided for prefetching data. A workload analyzer may identify job characteristics for a plurality of previously executed jobs in a workload executing on a cluster of one or more compute resources. For each job, identified job characteristics may include identification of an input dataset and an input bandwidth characteristic for the input dataset. A future workload predictor may identify future jobs expected to execute on the cluster based at least on the identified job characteristics. A cache assignment determiner may determine a cache assignment that identifies a prefetch dataset for at least one of the future jobs. A network bandwidth allocator may determine a network bandwidth assignment for the prefetch dataset. A plan instructor may instruct a compute resource of the cluster to load data to a cache local to the cluster according to the cache assignment and the network bandwidth assignment.

Abstract: The techniques and/or systems described herein are configured to determine a set of update operations to transition a network from an observed network state to a target network state and to generate an update dependency graph used to dynamically schedule the set of update operations based on constraint(s) defined to ensure reliability of the network during the transition. The techniques and/or systems dynamically schedule the set of update operations based on feedback. For example, the feedback may include an indication that a previously scheduled update operation has been delayed, has failed, or has been successfully completed.

Abstract: Various technologies pertaining to scheduling network traffic in a network are described. A request to transfer data from a first computing device to a second computing device includes data that identifies a volume of the data to be transferred and a deadline, where the data is to be transferred prior to the deadline. A long-term schedule is computed based upon the request, wherein the long-term schedule defines flow of traffic through the network over a relatively long time horizon. A short-term schedule is computed based upon the long-term schedule, where devices in the network are configured based upon the short-term schedule.

Abstract: Latency in responding to queries directed to geographically distributed data can be reduced by allocating individual steps, of a multi-step compute operation requested by the query, among the geographically distributed computing devices so as to reduce the duration of shuffling of intermediate data among such devices, and, additionally, by pre-moving, prior to the receipt of the query, portions of the distributed data that are input to a first step of the multistep compute operation, to, again, reduce the duration of the exchange of intermediate data. The pre-moving of input data occurring, and the adaptive allocation of intermediate steps, are prioritized for high-value data sets. Additionally, a threshold increase in a quantity of data exchanged across network communications can be established to avoid incurring network communication usage without an attendant gain in latency reduction.

Abstract: Implementations are presented for utilizing probabilistic predicates (PPs) to speed up searches requiring machine learning inferences. One method includes receiving a search query comprising a predicate for filtering blobs in a database utilizing a user-defined-function (UDF). The filtering requiring analysis of the blobs by the UDF to determine blobs that pass the filtering. Further, the method includes determining a PP sequence of PPs based on the predicate. Each PP is a classifier that calculates a PP-blob probability of satisfying a PP clause. The PP sequence defines an expression to combine the PPs. Further, the method includes operations for performing the PP sequence to determine a blob probability that the blob satisfies the expression, determining which blobs meet an accuracy threshold, discarding the blobs with the blob probability less than the accuracy threshold, and executing the database query over the blobs that have not been discarded. The results are then presented.

Abstract: One or more approximations of query output in a data analytics platform are controlled. The one or more approximations are controlled by generating values of error metrics associated with placements of samplers in one or more query execution plans associated with the query, and injecting a plurality of samplers into the query execution plans, using the determined values of the error metrics, in lieu of storing samples of input to the query prior to execution of the query.

Abstract: The ensuring of predictable and quantifiable networking performance includes adaptively throttling the rate of VM-to-VM traffic flow. A receiving hypervisor can detect congestion and communicate messages for throttling traffic flow to reduce congestion at the receiving hypervisor.

Abstract: Latency in responding to queries directed to geographically distributed data can be reduced by allocating individual steps, of a multi-step compute operation requested by the query, among the geographically distributed computing devices so as to reduce the duration of shuffling of intermediate data among such devices, and, additionally, by pre-moving, prior to the receipt of the query, portions of the distributed data that are input to a first step of the multistep compute operation, to, again, reduce the duration of the exchange of intermediate data. The pre-moving of input data occurring, and the adaptive allocation of intermediate steps, are prioritized for high-value data sets. Additionally, a threshold increase in a quantity of data exchanged across network communications can be established to avoid incurring network communication usage without an attendant gain in latency reduction.

Abstract: Various technologies pertaining to scheduling network traffic in a network are described. A request to transfer data from a first computing device to a second computing device includes data that identifies a volume of the data to be transferred and a deadline, where the data is to be transferred prior to the deadline. A long-term schedule is computed based upon the request, wherein the long-term schedule defines flow of traffic through the network over a relatively long time horizon. A short-term schedule is computed based upon the long-term schedule, where devices in the network are configured based upon the short-term schedule.

Abstract: Latency in responding to queries directed to geographically distributed data can be reduced by allocating individual steps, of a multi-step compute operation requested by the query, among the geographically distributed computing devices so as to reduce the duration of shuffling of intermediate data among such devices, and, additionally, by pre-moving, prior to the receipt of the query, portions of the distributed data that are input to a first step of the multistep compute operation, to, again, reduce the duration of the exchange of intermediate data. The pre-moving of input data occurring, and the adaptive allocation of intermediate steps, are prioritized for high-value data sets. Additionally, a threshold increase in a quantity of data exchanged across network communications can be established to avoid incurring network communication usage without an attendant gain in latency reduction.

Abstract: Various technologies pertaining to scheduling network traffic in a network are described. A request to transfer data from a first computing device to a second computing device includes data that identifies a volume of the data to be transferred and a deadline, where the data is to be transferred prior to the deadline. A long-term schedule is computed based upon the request, wherein the long-term schedule defines flow of traffic through the network over a relatively long time horizon. A short-term schedule is computed based upon the long-term schedule, where devices in the network are configured based upon the short-term schedule.

Abstract: Embodiments for efficient queue management for cluster scheduling and managing task queues for tasks which are to be executed in a distributed computing environment. Both centralized and distributed scheduling is provided. Task queues may be bound by length-based bounding or delay-based bounding. Tasks may be prioritized and task queues may be dynamically reordered based on task priorities. Job completion times and cluster resource utilization may both be improved.