Abstract: Data services for workloads are often provided with a service level agreement specifying various performance guarantees (e.g., latency, availability, scalability, and consistency). Single-master architectures, in which updates t the data set are constrained to a single server, may limit the fulfillment of some performance guarantees. Presented herein are multi-master architectures, in which the server set is partitioned into at least two masters are permitted to update the data set and at least one non-master that is not permitted to update the data set. Non-masters that receive a request to update the data set forward the request to a master server for application to the data set. A master that receives the request applies it to the data set and propagates the update to other master and non-master servers. Conflicting updates may be resolved through a variety of conflict resolution techniques, optionally designating one master server as a conflict resolution server.

Abstract: The disclosure herein describes managing artificial intelligence (AI) workloads in a cloud infrastructure platform. A set of distributed infrastructure resources are integrated into the cloud infrastructure platform via native support interfaces. AI workloads are received from a plurality of tenants, wherein the AI workloads include training workloads and inferencing workloads and resource subsets of the set of distributed infrastructure resources are assigned to the received AI workloads. The received AI workloads are scheduled for execution on the assigned resource subsets and based on the scheduling of the AI workloads, they are executed on the assigned resource subsets. The described cloud infrastructure platform provides efficient, secure execution of AI workloads for many different tenants and enables the flexible use of a wide variety of both third-party and first-party infrastructure resources.

Abstract: The disclosure herein describes scheduling execution of artificial intelligence (AI) workloads in a cloud infrastructure platform. A global scheduler receives AI workloads associated with resource ticket values. The scheduler distributes the AI workloads to nodes based on balancing resource ticket values. Local schedulers of the nodes schedule AI workloads on resources based on the resource ticket values of the AI workloads. Based on scheduling the AI workloads, coordinator services of the local schedulers execute the distributed AI workloads on the infrastructure resources of the nodes. The disclosure further describes scheduling AI workloads based on priority tiers. A scheduler receives AI workloads, and each AI workload is associated with a priority tier indicative of a preemption priority while being executed. The AI workloads are scheduled for execution on a distributed set of nodes based on the priority tiers and then execute based on the scheduling.

Abstract: A server set for a data set may designate a subset of “master” servers that update the data set in order to reduce data version conflicts involving mutually exclusive updates of the data set. Multi-master configurations may fulfill the performance constraints, and the subset of masters may detect and resolve data version conflicts. However, if multiple masters perform conflict resolution for a particular data version conflict, the resolution may produce inefficiency and redundancy (if the masters reach the same outcome) or additional data version conflicts (if the masters reach different outcomes). Instead, among the masters, a merge master may be identified that applies conflict resolution techniques to data version conflicts and forwards the conflict resolution outcome to the other masters for application to the data set to resolve the data version conflict. The other masters may temporarily store updates in a tentative update set until data version conflicts are resolved.

Abstract: Data services for workloads are often provided with a service level agreement specifying various performance guarantees (e.g., latency, availability, scalability, and consistency). Single-master architectures, in which updates to the data set are constrained to a single server, may limit the fulfillment of some performance guarantees. Presented herein are multi-master architectures, in which the server set is partitioned into at least two masters are permitted to update the data set and at least one non-master that is not permitted to update the data set. Non-masters that receive a request to update the data set forward the request to a master server for application to the data set. A master that receives the request applies it to the data set and propagates the update to other master and non-master servers. Conflicting updates may be resolved through a variety of conflict resolution techniques, optionally designating one master server as a conflict resolution server.

Abstract: Data services are often provided with various performance guarantees. Multi-master architectures, in which multiple master servers are permitted to update a portion of the data set, may facilitate some performance requirements, but may also lead to data version conflicts in which different masters have written different versions of one or more data items. Moreover, conflicts involving different data items may have to be resolved using different conflict resolution techniques. Therefore, various data items of the data set may be associated with a conflict resolution technique selected from a conflict resolution technique set, such as manual conflict resolution; a write order policy, such as last writer wins; a conflict resolution logic; and conflict resolution based on data types. A data version conflict may be resolved by identifying and invoking the selected conflict resolution technique that is associated with the data item, and applying the conflict resolution outcome to the data item.

Abstract: Distributed transactions are performed over a collection of servers operating as replicas of a data set, where a successful transaction involves meeting a quorum count of replicas that locally commit the transaction. However, performance constraints of data sets and consuming applications may vary (e.g., sensitivity to latency, scalability, and/or consistency), and the performance characteristics of the server set may be partly determined by the transactional commitment and quorum selection. The distributed transaction may be applied by designating the replicas as a set of followers and a leader that initiates the transaction and receives acknowledgments of local commits by each follower. On condition of the acknowledgments meeting a quorum count for the data set according to the performance characteristics of the application, the leader locally commits the transaction and delivers a result.

Abstract: Processing services are often provisioned by defining and adjusting the performance capabilities of individual servers, and in multitenancy scenarios, servers may allocate computational resources to ensure that a first client workload does not impact a second client workload. However, a reduced performance capability of a server may create a processing jam with respect to an upstream server of the process path of the workload, where the processing rate mismatch creates a risk of failing to fulfill the performance guarantee for the workload. Instead, the downstream server may monitor and compare its performance capability with the performance guarantee. If a performance guarantee failure risk arises, the server may transmit a performance capability alert to the upstream server, which may rate-limit the processing of the workload. Rate-limiting by the first server in the server path may limit workload intake to a volume for which the process path can fulfill the performance guarantee.

Abstract: Data services are often provided with various performance guarantees. Multi-master architectures, in which multiple master servers are permitted to update a portion of the data set, may facilitate some performance requirements, but may also lead to data version conflicts in which different masters have written different versions of one or more data items. Moreover, conflicts involving different data items may have to be resolved using different conflict resolution techniques. Therefore, various data items of the data set may be associated with a conflict resolution technique selected from a conflict resolution technique set, such as manual conflict resolution; a write order policy, such as last writer wins; a conflict resolution logic; and conflict resolution based on data types. A data version conflict may be resolved by identifying and invoking the selected conflict resolution technique that is associated with the data item, and applying the conflict resolution outcome to the data item.

Abstract: A server set for a data set may designate a subset of “master” servers that update the data set in order to reduce data version conflicts involving mutually exclusive updates of the data set. Multi-master configurations may fulfill the performance constraints, and the subset of masters may detect and resolve data version conflicts. However, if multiple masters perform conflict resolution for a particular data version conflict, the resolution may produce inefficiency and redundancy (if the masters reach the same outcome) or additional data version conflicts (if the masters reach different outcomes). Instead, among the masters, a merge master may be identified that applies conflict resolution techniques to data version conflicts and forwards the conflict resolution outcome to the other masters for application to the data set to resolve the data version conflict. The other masters may temporarily store updates in a tentative update set until data version conflicts are resolved.

Abstract: Data services for workloads are often provided with a service level agreement specifying various performance guarantees (e.g., latency, availability, scalability, and consistency). Single-master architectures, in which updates to the data set are constrained to a single server, may limit the fulfillment of some performance guarantees. Presented herein are multi-master architectures, in which the server set is partitioned into at least two masters are permitted to update the data set and at least one non-master that is not permitted to update the data set. Non-masters that receive a request to update the data set forward the request to a master server for application to the data set. A master that receives the request applies it to the data set and propagates the update to other master and non-master servers. Conflicting updates may be resolved through a variety of conflict resolution techniques, optionally designating one master server as a conflict resolution server.

Abstract: Distributed transactions are performed over a collection of servers operating as replicas of a data set, where a successful transaction involves meeting a quorum count of replicas that locally commit the transaction. However, performance constraints of data sets and consuming applications may vary (e.g., sensitivity to latency, scalability, and/or consistency), and the performance characteristics of the server set may be partly determined by the transactional commitment and quorum selection. The distributed transaction may be applied by designating the replicas as a set of followers and a leader that initiates the transaction and receives acknowledgments of local commits by each follower. On condition of the acknowledgments meeting a quorum count for the data set according to the performance characteristics of the application, the leader locally commits the transaction and delivers a result.