Abstract: Embodiments of the present disclosure provide a region-specific events account that is used as a central place to store the events shared by consumers of shared applications in that region. Use of such an account helps reduce the operational burden of a provider of the applications as they do not need to determine which accounts have shared events for each different consumer. The provider only needs to login to the events account which is in the same region as the consumer and can query all shared events from different applications that are being used in that region. Also, since the provider can designate an events account in each region, the shared events could be directly ingested into the events account without traveling to a different region.

Abstract: Disclosed is an execution information sharing system that writes execution information to a provider target (and other targets) in a secure manner. Execution information generated by an application may be written to a consumer stage, wherein the application is shared by a provider account of a data exchange with a consumer account that executes the application. A consumer exchange service (ES) of the data exchange may send a request to a copy service of the data exchange to copy the execution information from the consumer stage to the provider stage, wherein the consumer ES is a part of the data exchange and is protected from actions of the consumer account. A copy operation may be executed to copy the execution information from the consumer stage to the provider stage using the copy service of the data exchange. The execution information is ingested from the provider stage to a provider table.

Abstract: Disclosed is an execution information sharing system that duplicates execution information to a provider target (and other targets) as it is being loaded to a consumer target. A first log information object and a second log information object are generated. The first and second log information objects comprise information indicating a consumer target and information indicating a provider target respectively where execution information generated by an application shared with a consumer account of a data exchange is written. A first event unloader and a second event unloader are generated based on the first and second log information objects respectively, wherein the first and second event unloaders are both linked to the application using a mapping. In response to receiving execution information from the application, the execution information is forwarded to the consumer target and the provider target using the first event unloader and the second event unloader respectively.

Abstract: There is provided a method and apparatus for vectorized resource scheduling using multi-dimensional hierarchical arrays with a uniform data type. According to embodiments, the hierarchy of resources may be represented as a multi-dimensional array of a uniform data type in a scheduling context. This data structure is known as a tensor. In various embodiments, a tensor may be used to model each of resource usage, requested resources, resource capabilities, tenant resource usage, etc. There is provided tensor scheduling operations (e.g. tensor scheduling algorithms) to make use of this resource model. Some embodiments may exploit specialized hardware, such as GPUs and tensor-processing units, in order to accelerate the tensor scheduling operations.

Abstract: Disclosed is an execution information sharing system that duplicates execution information to a provider target (and other targets) as it is being loaded to a consumer target. A consumer account of a data sharing platform executes an application shared with it by a provider account of the platform. Consumer and provider configurations indicating consumer and provider targets respectively are generated. The consumer configuration and provider configurations are provided to an event context to generate a first and second event unloaders respectively, wherein the event context maintains a mapping linking both the first event unloader and the second event unloader to the application. In response to receiving execution information from the application, the first event unloader and the second event unloader are retrieved. The execution information is then written to the consumer target and the provider target using the first event unloader and the second event unloader respectively.

Abstract: Apparatus and methods for scheduling computing resources is disclosed that facilitate the cooperation of resource managers in the resource layer and workload schedulers in the workload layer working together so that resource managers can efficiently manage and schedule resources for horizontally and vertically scaling resources on physical hosts shared among workload schedulers to run workloads.

Abstract: The disclosed apparatuses and methods are directed to resource scheduling of resource nodes of a computer cluster or a cloud computing platform. The disclosed method comprises receiving node identifiers of nodes of a node set and receiving values of node attributes for each one of node identifiers; receiving a sequence of tasks, each specifying values of task parameters; generating a node graph structure having at least one graph structure vertex mapped to a coordinate space; mapping each task to the coordinate space; determining a first node identifier of a first node by analyzing the at least node graph structure vertex located within a fittable area for each task; and mapping the first node identifier to each task to generate a scheduling scheme.

Abstract: There is provided a method and apparatus for vectorized resource scheduling using multi-dimensional hierarchical arrays with a uniform data type. According to embodiments, the hierarchy of resources may be represented as a multi-dimensional array of a uniform data type in a scheduling context. This data structure is known as a tensor. In various embodiments, a tensor may be used to model each of resource usage, requested resources, resource capabilities, tenant resource usage, etc. There is provided tensor scheduling operations (e.g. tensor scheduling algorithms) to make use of this resource model. Some embodiments may exploit specialized hardware, such as GPUs and tensor-processing units, in order to accelerate the tensor scheduling operations.

Abstract: A method for managing file system replication can include: collecting event data associated with file system events, each of the file system events indicative of a modification of at least one file at a first file system location; for each of the file system events, generating at least one file system action to apply the modification of the at least one file at the first file system location to at least one corresponding file at a second file system location; storing the at least one generated file system action in an action buffer; identifying dependencies between file system actions in the action buffer; and scheduling the file system actions in the action buffer for execution based on the dependencies.

Abstract: Systems and methods are disclosed herein for allocating computing resources for execution of workloads in a distributed computing system with resource acquisition cycles and workload scheduling cycles. One or more resource offer managers transmit resource offers to each of one or more of schedulers. Each scheduler transmits a resource request and/or resource release request in response to each resource offer manager. Each resource offer manager computes a resource allocation for each scheduler after resolving conflicts and assigns the resources to each scheduler according to the resource allocation decisions. The resource acquisition cycle and the workload scheduling cycle are decoupled.

Abstract: Systems and methods are disclosed herein for allocating computing resources for execution of workloads in a distributed computing system with resource acquisition cycles and workload scheduling cycles. One or more resource offer managers transmit resource offers to each of one or more of schedulers. Each scheduler transmits a resource request and/or resource release request in response to each resource offer manager. Each resource offer manager computes a resource allocation for each scheduler after resolving conflicts and assigns the resources to each scheduler according to the resource allocation decisions. The resource acquisition cycle and the workload scheduling cycle are decoupled.

Abstract: In a distributed computing network, requests for allocation of resources to tenant workloads and messages identifying resource availability are received and aggregated. Resources are allocated to the workloads in accordance with a distribution policy defining values for resource entitlements of the tenants. The values include pre-emption quantities. In response to determining that a quantity of resources allocated for workloads of a first tenant is less than the tenant's pre-emption quantity, processing of another workload from a second tenant is interrupted to re-allocate resources from the second tenant's workload to the first tenant's workload.

Abstract: Methods and systems of managing a resource in a distributed resource management system can include: receiving a resource request including data identifying at least one requested resource attribute; identifying one or more candidate resources from a plurality of resources by performing element-wise operations on one or more resource attribute element sets corresponding to the at least one requested resource attribute, each resource attribute element set comprising an element for each of the plurality of resources, each element representing a resource attribute for a respective resource of the plurality of resources; and scheduling a workload associated with the resource request for execution based on the one or more identified candidate resources.

Abstract: Methods and systems of managing workloads and resources for tenants structured as a hierarchical tenant queue defining a resource distribution policy. The tenants are modified and the system responds with dynamic adjustments to the hierarchical tenant queue and workload distribution. A rule-based workload management engine defines rules for actions for the workload to respond to the dynamic update.

Abstract: Systems and methods are disclosed herein for allocating computing resources for execution of workloads in a distributed computing system with resource acquisition cycles and workload scheduling cycles. One or more resource offer managers transmit resource offers to each of one or more of schedulers. Each scheduler transmits a resource request and/or resource release request in response to each resource offer manager. Each resource offer manager computes a resource allocation for each scheduler after resolving conflicts and assigns the resources to each scheduler according to the resource allocation decisions. The resource acquisition cycle and the workload scheduling cycle are decoupled.

Abstract: A method for managing file system replication can include: collecting event data associated with file system events, each of the file system events indicative of a modification of at least one file at a first file system location; for each of the file system events, generating at least one file system action to apply the modification of the at least one file at the first file system location to at least one corresponding file at a second file system location; storing the at least one generated file system action in an action buffer; identifying dependencies between file system actions in the action buffer; and scheduling the file system actions in the action buffer for execution based on the dependencies.

Abstract: The present disclosure is drawn to methods and systems for managing clusters of distributed file systems having cluster files stored thereon. An intermediate layer is provided between user devices having applications running thereon and clusters of distributed file systems for managing and coordinating operation across multiple clusters using metadata about the cluster files.

Abstract: Methods and systems of managing workloads and resources for tenants structured as a hierarchical tenant queue defining a resource distribution policy. The tenants are modified and the system responds with dynamic adjustments to the hierarchical tenant queue and workload distribution. A rule-based workload management engine defines rules for actions for the workload to respond to the dynamic update.

Abstract: Methods and systems of managing a resource in a distributed resource management system can include: receiving a resource request including data identifying at least one requested resource attribute; identifying one or more candidate resources from a plurality of resources by performing element-wise operations on one or more resource attribute element sets corresponding to the at least one requested resource attribute, each resource attribute element set comprising an element for each of the plurality of resources, each element representing a resource attribute for a respective resource of the plurality of resources; and scheduling a workload associated with the resource request for execution based on the one or more identified candidate resources.

Abstract: In a distributed computing network, requests for allocation of resources to tenant workloads and messages identifying resource availability are received and aggregated. Resources are allocated to the workloads in accordance with a distribution policy defining values for resource entitlements of the tenants. The values include pre-emption quantities. In response to determining that a quantity of resources allocated for workloads of a first tenant is less than the tenant's pre-emption quantity, processing of another workload from a second tenant is interrupted to re-allocate resources from the second tenant's workload to the first tenant's workload.