Abstract: Data sharding partitions data associated with an application into multiple shards in which each shard stores a subset of the data. A shard is further divided into multiple microshards in which each microshard contains a subset of the shard data. A set of microshards are assigned to a specified shard. The assignment of microshards can change, e.g., when new shards are added and microshards are migrated from one shard to another shard. A microshard map, which indicates a mapping between the microshards and shards, is updated automatically when the assignments change. After identifying the shard to which a specified microshard is assigned using the microshard map, a data access request from a client device is automatically forwarded to the appropriate shard. Because data can be accessed efficiently regardless of whether the number of shards changes, the data sharding facilitates provisioning of the shards on a need basis.

Abstract: A request associated with a data item is received. A data grouping corresponding to the data item is identified. The identified data grouping belongs to a data grouping container that has been assigned a plurality of data groupings. Any one data grouping of the plurality of data groupings is individually reassignable to a different data grouping container. At least one data center where the identified data grouping is stored among a plurality of data centers is determined. A processing of the request is managed based on the determined data center that stores the identified data grouping.

Abstract: Technology for migrating database shards from a source deployment to a destination deployment can be accomplished while maintaining a particular level of availability, reliability, and consistency. A process for this migration can include setting up destination shards as followers of shards on the source such that the destination shards receive stored data and have write operations mirrored from the source shards to the destination shards. The process can set an epoch counter for the destination shards higher than a corresponding epoch counter for the source shards. Next, the process can disable writing for the source shards. The process can also perform a special write to determine when the source write pipeline is empty. The process can wait for the shard migration to complete, then configure the destination shards to no longer be followers of the source shards, but instead be primary shards to handle database operations for the stored data.

Abstract: The disclosure is directed to sandboxing data shards in a host server. A shard can store a subset of data associated with an application, and provide read and/or write services to various application services. In a multi-tenant architecture, different application services can share common hardware of a multi-tenant system to improve hardware utilization and efficiency. For example, shards associated with various application services are hosted by a common host server. Accessing of a shard, e.g., by an application service, consumes resources of the host server. A bug or a spike in the usage of the host server by one application service can impact other application services with which the host server is shared. To reduce or eliminate this problem, the shards will be sandboxed in the host server, which can include reserving resources, e.g., computing, networking and/or storage resources, for each of the shards.

Abstract: Systems, methods, and non-transitory computer readable media are configured to determine a request corresponding to a portion of data. A placement configuration associated with the portion of data can be determined. The placement configuration can belong to a set of placement configurations. A datacenter identified by the placement configuration can be selected. Subsequently, the portion of data can be accessed at the selected datacenter.

Abstract: A distributed component model for creating a scalable and available distributed application is disclosed. The distributed component model provides for an application schema to be declaratively defined to include a module having a component. The schema includes a corresponding definition construct in a technology agnostic manner. The corresponding definition construct is declaratively defined to include metadata to control scaling and availability.

Abstract: Data sharding partitions data associated with an application into multiple shards in which each shard stores a subset of the data. A shard is further divided into multiple microshards in which each microshard contains a subset of the shard data. A set of microshards are assigned to a specified shard. The assignment of microshards can change, e.g., when new shards are added and microshards are migrated from one shard to another shard. A microshard map, which indicates a mapping between the microshards and shards, is updated automatically when the assignments change. After identifying the shard to which a specified microshard is assigned using the microshard map, a data access request from a client device is automatically forwarded to the appropriate shard. Because data can be accessed efficiently regardless of whether the number of shards changes, the data sharding facilitates provisioning of the shards on a need basis.

Abstract: A method of creating an application definition for a distributed application is disclosed. Constructs corresponding with a schema of the distributed application are defined in a declarative and technology agnostic manner. The constructs include the application definition, a module definition, and a component definition. Each construct includes metadata.

Abstract: Data sharding partitions data associated with an application into multiple shards in which each shard stores a subset of the data. A shard is further divided into multiple microshards in which each microshard contains a subset of the shard data. A set of microshards are assigned to a specified shard. The assignment of microshards can change, e.g., when new shards are added and microshards are migrated from one shard to another shard. A microshard map, which indicates a mapping between the microshards and shards, is updated automatically when the assignments change. After identifying the shard to which a specified microshard is assigned using the microshard map, a data access request from a client device is automatically forwarded to the appropriate shard. Because data can be accessed efficiently regardless of whether the number of shards changes, the data sharding facilitates provisioning of the shards on a need basis.

Abstract: Systems, methods, and non-transitory computer readable media are configured to determine a request corresponding to a portion of data. A placement configuration associated with the portion of data can be determined. The placement configuration can belong to a set of placement configurations. A datacenter identified by the placement configuration can be selected. Subsequently, the portion of data can be accessed at the selected datacenter.

Abstract: A distributed component model for creating a scalable and available distributed application is disclosed. The distributed component model provides for an application schema to be declaratively defined to include a module having a component. The schema includes a corresponding definition construct in a technology agnostic manner. The corresponding definition construct is declaratively defined to include metadata to control scaling and availability.

Abstract: The disclosure is directed to sandboxing data shards in a host server. A shard can store a subset of data associated with an application, and provide read and/or write services to various application services. In a multi-tenant architecture, different application services can share common hardware of a multi-tenant system to improve hardware utilization and efficiency. For example, shards associated with various application services are hosted by a common host server. Accessing of a shard, e.g., by an application service, consumes resources of the host server. A bug or a spike in the usage of the host server by one application service can impact other application services with which the host server is shared. To reduce or eliminate this problem, the shards will be sandboxed in the host server, which can include reserving resources, e.g., computing, networking and/or storage resources, for each of the shards.

Abstract: Technology for migrating database shards from a source deployment to a destination deployment can be accomplished while maintaining a particular level of availability, reliability, and consistency. A process for this migration can include setting up destination shards as followers of shards on the source such that the destination shards receive stored data and have write operations mirrored from the source shards to the destination shards. The process can set an epoch counter for the destination shards higher than a corresponding epoch counter for the source shards. Next, the process can disable writing for the source shards. The process can also perform a special write to determine when the source write pipeline is empty. The process can wait for the shard migration to complete, then configure the destination shards to no longer be followers of the source shards, but instead be primary shards to handle database operations for the stored data.

Abstract: The disclosure is directed to data replication in a distributed computing system. The system ensures a read-after-write consistency for the replicated data by associating the write requests with unique version numbers, and using them to obtain the data. A server associates a write request with a unique version number, and returns the version number to a client, which can use it in a subsequent read request for the data. On receiving a read request for a first data, the server extracts the version number from the request and ensures a specified write request associated with the version number is processed prior to obtaining the data. If the specified write request is processed, the server obtains the data and returns it to the client, else the server processes the pending write requests up until the version number and then the specified write request, and returns the specified data to the client.

Abstract: Data sharding partitions data associated with an application into multiple shards in which each shard stores a subset of the data. A shard is further divided into multiple microshards in which each microshard contains a subset of the shard data. A set of microshards are assigned to a specified shard. The assignment of microshards can change, e.g., when new shards are added and microshards are migrated from one shard to another shard. A microshard map, which indicates a mapping between the microshards and shards, is updated automatically when the assignments change. After identifying the shard to which a specified microshard is assigned using the microshard map, a data access request from a client device is automatically forwarded to the appropriate shard. Because data can be accessed efficiently regardless of whether the number of shards changes, the data sharding facilitates provisioning of the shards on a need basis.

Abstract: A method of creating an application definition for a distributed application is disclosed. Constructs corresponding with a schema of the distributed application are defined in a declarative and technology agnostic manner. The constructs include the application definition, a module definition, and a component definition. Each construct includes metadata.

Abstract: The disclosure is directed to data replication in a distributed computing system. The data, e.g., received from a client, is replicated to a first set of servers (“sync replica set”) synchronously and to a second set of servers asynchronously (“async tree”). A server can be a primary, secondary or a follower server. A sync replica set includes a primary server and one or more secondary servers. The async tree includes multiple follower servers deployed in a tree structure. A primary server can replicate the data to the secondary servers synchronously, and the secondary servers can replicate the data to one or more follower servers, e.g., a root node of the async tree, asynchronously. The root node then replicates the data to children of the root node, which then replicate to their children and so on until the leaf nodes of the async tree receive the data.

Abstract: A distributed component model for creating a scalable and available distributed application is disclosed. The distributed component model provides for an application schema to be declaratively defined to include a module having a component. The schema includes a corresponding definition construct in a technology agnostic manner. The corresponding definition construct is declaratively defined to include metadata to control scaling and availability.

Abstract: The disclosure is related to a shard manager that manages assignment of shards (data partitions) to application servers. An application service (“app service”) provides a specific service to clients and can be executing on multiple application servers. The dataset managed by the app service can be divided into multiple shards and the shards can be assigned to different app servers. The shard manager can manage the assignment of shards to different app servers based on an assignment policy. The shard assignments can be published to a configuration service. A client can request the configuration service to provide identification information of the app server to which a particular shard the client intends to access is assigned. The shard manager can also provide dynamic load balancing solutions. The shard manager can poll the app servers in runtime to determine the load information and per-shard resource usage, and balance the load by reassigning the shards accordingly.

Abstract: A method of creating an application definition for a distributed application is disclosed. Constructs corresponding with a schema of the distributed application are defined in a declarative and technology agnostic manner. The constructs include the application definition, a module definition, and a component definition. Each construct includes metadata.