Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing dusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: This application relates to load balancing for a number of partitions of a network-based storage service. Each partition includes a number of server devices and/or network devices located in a data center and configured to provide access to storage resources hosted within the data center. User accounts are assigned to a particular partition such that requests related to a particular user account are routed to that partition. Periodically, a load balancing algorithm is executed to re-assign user accounts to different partitions to rebalance resource consumption across the different partitions. The load balancing algorithm can balance resource consumption for any number of resource types by generating a vector of resource utilization parameters for each user account, sorting the plurality of user accounts into clusters based on the vectors, and mapping at least some user accounts to different partitions.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing dusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: Data items such as files or database records associated with particular applications (such as messaging applications and other applications) can be stored in one or more remote locations, such as a cloud storage system, and synchronized with other devices. The remote storage can be configured such that each application executing on a client device can only view data items stored at the remote location to which the application has permission to access. An access manager on each client device enforces application specific access policies. Storage at the remote location can be secured for each application associated with a user or user account, for example, using isolated containers. The cloud storage of data can be anonymized and anonymous group data can be stored in the cloud storage.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, the role of voting is assigned to a number of computing clusters. To make this selection, a task assigning service finds a candidate leader and M computing clusters to be used as voters. This selection may be based on finding latencies between the M computing clusters and the leader.

Abstract: This application relates to load balancing for a number of partitions of a network-based storage service. Each partition includes a number of server devices and/or network devices located in a data center and configured to provide access to storage resources hosted within the data center. User accounts are assigned to a particular partition such that requests related to a particular user account are routed to that partition. Periodically, a load balancing algorithm is executed to re-assign user accounts to different partitions to rebalance resource consumption across the different partitions. The load balancing algorithm can balance resource consumption for any number of resource types by generating a vector of resource utilization parameters for each user account, sorting the plurality of user accounts into clusters based on the vectors, and mapping at least some user accounts to different partitions.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, an indication that a number N of clusters are to be assigned a replica role and a second number M of the replica-role assigned clusters are to be assigned to a voting role. N computing clusters are selected using workload data, and M of the clusters are assigned to a voting role.

Abstract: Methods, systems, and apparatus for automatic sharding and load balancing in a distributed data processing system. In one aspect, a method includes determining workload distribution for an application across worker computers and in response to determining a load balancing operation is required: selecting a first worker computer having a highest load measure relative to respective load measure of the other work computers; determining one or more move operations for a partition of data assigned to the first worker computer and a weight for each move operation; and selecting the move operation with a highest weight the selected move operation.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, the role of voting is assigned to a number of computing clusters. To make this selection, a task assigning service finds a candidate leader and M computing clusters to be used as voters. This selection may be based on finding latencies between the M computing clusters and the leader.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, a leader is selected from among candidate computing clusters. To make this selection, an activity monitor predicts or monitors the workload of one or more clients. Different activities of the workload are given corresponding weights. The delay in performing requested activities, modified by these weights is found, and the candidate leader with the lowest weighted delay is selected as the leader.

Abstract: Replicas are selected in a large distributed network, and the roles for these replicas are identified. In one example, an indication that a number N of clusters are to be assigned a replica role and a second number M of the replica-role assigned clusters are to be assigned to a voting role. N computing clusters are selected using workload data, and M of the clusters are assigned to a voting role.