Abstract: A computational system determines that a joining node in the federation infrastructure desires to join an identified subfederation within the federation infrastructure. The system accesses the joining node's group table, where the group table includes membership information identifying subfederations within the federation infrastructure. Each node in the federation maintains neighborhood information for each subfederation of which the node is aware. The system determines that no entry for the identified subfederation exists in the joining node's group table and, based on the determination, routes a group join message to a rendezvous point within the federation infrastructure. The rendezvous point checks its group table for an entry corresponding to the identified subfederation.

Abstract: Architecture that applies to a collection of nodes (e.g., a federation) and manages membership of the nodes to the collection via an external datastore. The storage of node state and collection state is stored in the external and highly available datastore. New nodes can now join the collection through the external store. The machines (nodes) in the collection create a lease with the external store that allows the external store to maintain precise knowledge of the set of machines alive in the collection. The external store thus allows the new machines to join the collection while ensuring that there exists only one ring in the collection. In other words, the disclosed architecture protocol ensures that any given point of time there is just one ring for the collection. Moreover, this can be in addition to a current join protocol, which ensures that consistent routing is available.

Abstract: Systems and methods that ensure formation of a single ring from a seed node at any given time. A “bootstrap” phase is included in a node's life cycle, to mitigate adverse affects of split-brain conditions in a network. During such bootstrap phase, if an existing ring is found, the seed node can join the existing ring in a same manner as a non-seed node does. If no ring is detected, the bootstrap phase attempts to elect a seed node to be the “super-seed node”, wherein a new ring can then be formed with such new super-seed node as the first node.

Abstract: Version indicators within an existing range can be associated with a data partition in a distributed data store. A partition reconfiguration can be associated with one of multiple partitions in the data store, and a new version indicator that is outside the existing range can be assigned to the reconfigured partition. Additionally, a broadcast message can be sent to multiple nodes, which can include storage nodes and/or client nodes that are configured to communicate with storage nodes to access data in a distributed data store. The broadcast message can include updated location information for data in the data store. In addition, a response message can be sent to a requesting node of the multiple nodes in response to receiving from that node a message that requests updated location information for the data. The response message can include the requested updated location information.

Abstract: Systems and methods that supply a replication layer/agent that is generic to supporting a plurality of storage configuration as part of a distributed store. Such distributed store employs a Common Availability Substrate (CAS) for data transport and consistency, to render the distributed store scalable and available. Such an arrangement enables continuous operation of the store, while the replication subsystem creates new replicas (e.g., for load balancing, failover, and the like).

Abstract: A primary node is elected from among a plurality of nodes in a replica set of nodes. The primary node accepts and processes client data access requests. The replica set includes the primary node and other secondary nodes. The primary node receives client data access requests and assigns a data sequence number to each client data access request that mutates state in the order the client data access requests are received. The data sequence numbers include a linearized processing order that is followed by each of the nodes in the replica set. The primary node sends the mutating client data access requests including any corresponding data sequence numbers to the secondary nodes. The primary node receives, from a threshold number of secondary nodes, an acknowledgement indicating reception of the client data access request. The primary node commits the data mutating access request.

Abstract: Version indicators within an existing range can be associated with a data partition in a distributed data store. A partition reconfiguration can be associated with one of multiple partitions in the data store, and a new version indicator that is outside the existing range can be assigned to the reconfigured partition. Additionally, a broadcast message can be sent to multiple nodes, which can include storage nodes and/or client nodes that are configured to communicate with storage nodes to access data in a distributed data store. The broadcast message can include updated location information for data in the data store. In addition, a response message can be sent to a requesting node of the multiple nodes in response to receiving from that node a message that requests updated location information for the data. The response message can include the requested updated location information.

Abstract: The present invention extends to methods, systems, and computer program products for facilitating inter-proximity communication within a rendezvous federation. Nodes maintain collateral ring set entry tables that include collateral rings and corresponding entry nodes into the collateral rings. Nodes can exchange collateral ring set entry state to update one another on the configuration of rings within a tree of rings. Nodes can refer to collateral ring set entry tables, as well as to other nodes, to identify entry nodes into rings that are collateral rings of the node. Messages can be sent to entry nodes in collateral rings. A message can include an indication that an entry node in a target proximity ring is to resolve the message to the node in the target proximity ring which has a node ID closest to an indicated destination node.

Abstract: The present invention extends to methods, systems, and computer program products for facilitating inter-proximity communication within a rendezvous federation. Nodes maintain collateral ring set entry tables that include collateral rings and corresponding entry nodes into the collateral rings. Nodes can exchange collateral ring set entry state to update one another on the configuration of rings within a tree of rings. Nodes can refer to collateral ring set entry tables, as well as to other nodes, to identify entry nodes into rings that are collateral rings of the node. Messages can be sent to entry nodes in collateral rings. A message can include an indication that an entry node in a target proximity ring is to resolve the message to the node in the target proximity ring which has a node ID closest to an indicated destination node.

Abstract: In some embodiments, it is determined that a primary node has been elected from among a plurality of nodes in a replica set of nodes. The primary node accepts and processes client data access requests. The replica set includes the primary node and other secondary nodes. The primary node receives client data access requests. The primary node assigns a data sequence number to each client data access request that mutates state in the order the client data access requests are received. The data sequence numbers include a linearized processing order that is to be followed by each of the nodes in the replica set. The primary node sends the mutating client data access requests including any corresponding data sequence numbers to the secondary nodes. The primary node receives, from a threshold number of secondary nodes, an acknowledgement indicating reception of the client data access request. The primary node commits the data mutating access request.

Abstract: Architecture that applies to a collection of nodes (e.g., a federation) and manages membership of the nodes to the collection via an external datastore. The storage of node state and collection state is stored in the external and highly available datastore. New nodes can now join the collection through the external store. The machines (nodes) in the collection create a lease with the external store that allows the external store to maintain precise knowledge of the set of machines alive in the collection. The external store thus allows the new machines to join the collection while ensuring that there exists only one ring in the collection. In other words, the disclosed architecture protocol ensures that any given point of time there is just one ring for the collection. Moreover, this can be in addition to a current join protocol, which ensures that consistent routing is available.

Abstract: Architecture that provides reliable communications of broadcast data (e.g., a message) in a collection of nodes. Each node in the collection is assigned a range of identifiers in a token. The union of the tokens for all nodes is the entire identifier range space. Each node that receives a reliable broadcast message from an originator node acknowledges receipt of the message using its token. One or more intermediate nodes forward the message from the originator node to other nodes with which the originator node has no direct communications (multi-level node structure). The indirect nodes each send acknowledgements back to the parent nodes (which can be an intermediate node) which combine the tokens to ensure the entire range space for the associated assigned token range is covered. The originator node ultimately receives tokens to compute if all nodes have received the message.

Abstract: The present invention extends to methods, systems, and computer program products for allocating and reclaiming resources within a rendezvous federation. Embodiments of the invention include formulating and routing messages in accordance with cached two-way agreements between adjacent nodes on a ring of nodes. Cached two-way agreements divide responsibility for at least unoccupied identifiers between the adjacent nodes on a ring. When a message is received at a node, the node can refer to a cached two-way agreement to determine if it or an adjacent node is responsible for the message. The node then sends the message to the next appropriate component in accordance with the cached two-way agreement.

Abstract: Systems and methods that provide for assignment and recovery of tokens as part of a plurality of nodes and distributed application framework/network. The assignment component assigns numbers and tasks to candidates and facilitates multiple leader election. Moreover, a recovery component can recover a token for a node that leaves the network (e.g., crashes). Such recovery component ensures consistency, wherein only one server is assigned recovery of the token and associated tasks.

Abstract: Systems and methods that supply a global knowledge on what nodes are available in the system, via employing routing tokens that are analyzed by a centralized management component to infer status for the nodes. When nodes fail, the routing tokens associated therewith are acquired by neighboring nodes, and the global knowledge updated. Moreover, upon inferring a failed or down status for a node, a challenge can be sent to a node reporting such failure to verify actual failure(s).

Abstract: The present invention extends to methods, systems, and computer program products for a joining node to join a ring of nodes within a rendezvous federation. Embodiments of the invention include establishing a neighborhood of multiple nodes on the ring of nodes. The joining node indicates its intent to take id-space ownership for a portion of the id-space between the joining node and a selected immediately adjacent node. The joining node initiates a one-way monitoring relationship with the selected immediately adjacent node. The joining node receiving an indication from the selected immediately adjacent node that indicates acceptance of the joining node's intent to take id-space ownership for a portion of the id-space between the joining node and the selected node and indicates establishment of a one-way monitoring relationship between the selected node and the joining node. The joining node agrees to participate in a one-way monitoring relationship with the selected node.

Abstract: Techniques are described herein for storing instances of data among nodes of a distributed store based on hierarchical scale unit values. Hierarchical scale unit values are assigned to the respective nodes of the distributed store. A first instance (e.g., a primary instance) of a data module is stored in a first node having a first hierarchical scale unit value. A primary instance of the data module with respect to a data operation is an instance of the data module at which the data operation with respect to the data module is initiated or initially directed. A second instance (e.g., a primary or secondary instance) of the data module is stored in a second node having a second hierarchical scale unit value based on a magnitude of a difference between the first hierarchical scale unit value and the second hierarchical scale unit value. A secondary instance is essentially a “back-up” instance.

Abstract: Techniques are described herein for managing access to a distributed cache during garbage collection. When garbage collection is to be performed with respect to a node, the node may send a request to a data manager requesting to have an instance of data module(s) included in the node that are in a primary state placed in a secondary state. The data manager may change the state of the data module(s) to the secondary state. The data manager may change the state of another instance of the data module(s) that is included in another node to the primary state. When the garbage collection is complete with respect to the node, the node may send another request to the data manager requesting that the data module(s) that were placed in the secondary state be returned to the primary state. The data manager may return those data module(s) to the primary state.

Abstract: Version indicators within an existing range can be associated with a data partition in a distributed data store. A partition reconfiguration can be associated with one of multiple partitions in the data store, and a new version indicator that is outside the existing range can be assigned to the reconfigured partition. Additionally, a broadcast message can be sent to multiple nodes, which can include storage nodes and/or client nodes that are configured to communicate with storage nodes to access data in a distributed data store. The broadcast message can include updated location information for data in the data store. In addition, a response message can be sent to a requesting node of the multiple nodes in response to receiving from that node a message that requests updated location information for the data. The response message can include the requested updated location information.

Abstract: The present invention extends to methods, systems, and computer program products for broadcasting communication within a rendezvous federation. Embodiments of the invention include inter-ring and intra-ring communication. Inter-ring communication includes sending a message to destination rings included in a node's Collateral Ring Set entry table. When a node identifies a destination ring that has not yet received a message, the node can send a ring notification message. The ring notification message propagates towards a publishing node until a responsible node in the message path to the publishing node is identified. The responsible node updates its entry table to include the ring and forwards the message to the destination ring. Intra-ring communication can include recursively partitioning ranges of nodes within a ring and forwarding the message to nodes included the partitioned ranges.