Abstract: Embodiments are directed to a cluster of nodes configured to run one or more services. A gateway on each node is configured to provide communication between the node and clients. The gateways manage filters designated by the clients, wherein the filters are named based upon a naming format used to name the services. The gateways receive configuration updates broadcast by a failover manager. The configuration updates comprise service endpoint changes for services running on the nodes. The gateways provide notifications to clients when a filter name matches at least a portion of a service name in a configuration update. The configuration updates identify a configuration version, and the gateways may provide notifications with endpoint information from more than one configuration update version. Notifications may be configured to be sent only when the endpoint for a primary replica of a service has changed.

Abstract: Techniques are described herein that are capable of establishing causality order of computer trace records. A computer trace record is information that indicates an event that occurs with regard to execution of a computer program. For instance, machines in a distributed computer system may generate such computer trace records as the machines execute the computer program. The computer trace records may be ordered in an aggregated trace to accurately reflect the causality order of the computer trace records. A causality order of computer trace records is a temporal sequence of the computer trace records in which each cause event is indicated to occur before each effect event that is caused by the cause event. A cause event is an event that causes an effect event. An effect event is an event that is caused by a cause event.

Abstract: In various implementations, methods and systems resource balancing in a distributed computing environment are provided. A client defined resource metric is received that represents a resource of nodes of the cloud computing platform. A placement plan for job instances of service applications is generated. The placement plan includes one or more movements that are executable to achieve a target placement of the job instances on the nodes. It is determined that the placement plan complies with placement rules. Each placement rule dictates whether a given job instance of the job instances is suitable for placement on a given node of the nodes. The placement plan is executed based on determining that the target placement of the job instances improves balance of resources across the nodes of the cloud computing platform based on the resource represented by the client defined resource metric.

Abstract: Techniques are described herein for storing instances of data among nodes of a distributed store based on hierarchical scale unit values. Hierarchical scale unit values are assigned to the respective nodes of the distributed store. A first instance (e.g., a primary instance) of a data module is stored in a first node having a first hierarchical scale unit value. A primary instance of the data module with respect to a data operation is an instance of the data module at which the data operation with respect to the data module is initiated or initially directed. A second instance (e.g., a primary or secondary instance) of the data module is stored in a second node having a second hierarchical scale unit value based on a magnitude of a difference between the first hierarchical scale unit value and the second hierarchical scale unit value. A secondary instance is essentially a “back-up” instance.

Abstract: Obtaining tight knowledge about nodes in a federated ring structure. The tight knowledge includes information about neighbors of a given node. The method includes at a given node, sending a join request, requesting to join a federated ring in a distributed computing environment. The method further includes, at the given node, receiving a join reply. The join reply provides an initial view of at least a portion of a neighborhood of the given node. The neighborhood is a set of nodes logically proximate the given node. The method further includes, at the given node, sending a lock request to nodes in the neighborhood to request that the other nodes in the given node's neighborhood are locked so as to not have overlapping nodes joining the given node's neighborhood simultaneously.

Abstract: Techniques are described herein for storing instances of data among nodes of a distributed store based on hierarchical scale unit values. Hierarchical scale unit values are assigned to the respective nodes of the distributed store. A first instance (e.g., a primary instance) of a data module is stored in a first node having a first hierarchical scale unit value. A primary instance of the data module with respect to a data operation is an instance of the data module at which the data operation with respect to the data module is initiated or initially directed. A second instance (e.g., a primary or secondary instance) of the data module is stored in a second node having a second hierarchical scale unit value based on a magnitude of a difference between the first hierarchical scale unit value and the second hierarchical scale unit value. A secondary instance is essentially a “back-up” instance.

Abstract: Arbitrating node failures. A method includes determining at a first node that communication to a second node has been lost. The method further includes sending an arbitration request to a plurality of arbitrators. The method further includes receiving from the plurality of arbitrators a sufficient number of arbitration reply grant messages to have arbitration reply grant messages from a quorum of the arbitrators. As a result of receiving a sufficient number of arbitration reply grant messages to have arbitration reply grant messages from a quorum of the arbitrators, the method further includes declaring the second node down.

Abstract: The present invention extends to methods, systems, and computer program products for a joining node to join a ring of nodes within a rendezvous federation. Embodiments of the invention include establishing a neighborhood of multiple nodes on the ring of nodes. The joining node indicates its intent to take id-space ownership for a portion of the id-space between the joining node and a selected immediately adjacent node. The joining node initiates a one-way monitoring relationship with the selected immediately adjacent node. The joining node receiving an indication from the selected immediately adjacent node that indicates acceptance of the joining node's intent to take id-space ownership for a portion of the id-space between the joining node and the selected node and indicates establishment of a one-way monitoring relationship between the selected node and the joining node. The joining node agrees to participate in a one-way monitoring relationship with the selected node.

Abstract: Detecting failures of interconnected machines. A method includes establishing a machine to machine lease at a first machine using a first lease agent at the first machine to a second lease agent at a second machine as a result of an application on the first machine requesting to establish a lease with an application on the second machine. Using the machine to machine lease, the method further includes detecting a communication failure between the first machine and the second machine or a machine failure of the second machine.

Abstract: Embodiments include establishing and maintaining a consistent replica set within the federation infrastructure. A replica set, including a primary node and secondary nodes, is established from among nodes in a ring of nodes. An occurrence of a replica set configuration event affecting at least one of the replica set members is indicated. Based on the indication and based on a type of the replica set configuration event, it is determined that the replica set is to be reconfigured. The replica set is reconfigured according to a reconfiguration appropriate for the type of the replica set configuration event.

Abstract: Embodiments are directed to ensuring that only one version of an application is running on a cluster. In one scenario, a computer system includes a configuration agent that generates service instance version queries upon startup to determine which service instance version to activate, maintains a service instance version table that includes a listing of service instance versions currently running on the computer system as part of one or more application instances and activates specified service instances. The computer system also includes a failover manager that manages service instance version queries. The failover manager indicates which service instance version to activate to ensure that the appropriate, current version is activated for each service instance. The computer system further includes a cluster manager that creates, deletes and upgrades service instances as part of an application instance upgrade.

Abstract: Determining if a new federation should be formed. A method includes a voter node determining that it has sufficient votes to form a new federation. The method further includes determining that no node existing in a previous federation already has a quorum of valid global tickets. Each global ticket indicates that a node agrees that a federation should continue to exist. As a result, the voter node forms a new federation.

Abstract: Detecting failures of interconnected machines. A method includes establishing a machine to machine lease at a first machine using a first lease agent at the first machine to a second lease agent at a second machine as a result of an application on the first machine requesting to establish a lease with an application on the second machine. Using the machine to machine lease, the method further includes detecting a communication failure between the first machine and the second machine or a machine failure of the second machine.

Abstract: Embodiments are directed to ensuring that only one version of an application is running on a cluster. In one scenario, a computer system includes a configuration agent that generates service instance version queries upon startup to determine which service instance version to activate, maintains a service instance version table that includes a listing of service instance versions currently running on the computer system as part of one or more application instances and activates specified service instances. The computer system also includes a failover manager that manages service instance version queries. The failover manager indicates which service instance version to activate to ensure that the appropriate, current version is activated for each service instance. The computer system further includes a cluster manager that creates, deletes and upgrades service instances as part of an application instance upgrade.

Abstract: Arbitrating node failures. A method includes determining at a first node that communication to a second node has been lost. The method further includes sending an arbitration request to a plurality of arbitrators. The method further includes receiving from the plurality of arbitrators a sufficient number of arbitration reply grant messages to have arbitration reply grant messages from a quorum of the arbitrators. As a result of receiving a sufficient number of arbitration reply grant messages to have arbitration reply grant messages from a quorum of the arbitrators, the method further includes declaring the second node down.

Abstract: Obtaining tight knowledge about nodes in a federated ring structure. The tight knowledge includes information about neighbors of a given node. The method includes at a given node, sending a join request, requesting to join a federated ring in a distributed computing environment. The method further includes, at the given node, receiving a join reply. The join reply provides an initial view of at least a portion of a neighborhood of the given node. The neighborhood is a set of nodes logically proximate the given node. The method further includes, at the given node, sending a lock request to nodes in the neighborhood to request that the other nodes in the given node's neighborhood are locked so as to not have overlapping nodes joining the given node's neighborhood simultaneously.

Abstract: Systems and methods that restore a failed reconfiguration of nodes in a distributed environment. By analyzing reports from read/write quorums of nodes associated with a configuration, automatic recovery for data partitions can be facilitated. Moreover, a configuration manager component tracks current configurations for replication units and determines whether a reconfiguration is required (e.g., due to node failures, node recovery, replica additions/deletions, replica moves, or replica role changes, and the like.) Reconfigurations of data activated as being replicated from an old configuration to being replicated on a new configuration may be performed in a transactionally consistent manner based on dynamic quorums associated with the new configuration and the old configuration.

Abstract: Version indicators within an existing range can be associated with a data partition in a distributed data store. A partition reconfiguration can be associated with one of multiple partitions in the data store, and a new version indicator that is outside the existing range can be assigned to the reconfigured partition. Additionally, a broadcast message can be sent to multiple nodes, which can include storage nodes and/or client nodes that are configured to communicate with storage nodes to access data in a distributed data store. The broadcast message can include updated location information for data in the data store. In addition, a response message can be sent to a requesting node of the multiple nodes in response to receiving from that node a message that requests updated location information for the data. The response message can include the requested updated location information.

Abstract: Systems and methods that restore a failed reconfiguration of nodes in a distributed environment. By analyzing reports from read/write quorums of nodes associated with a configuration, automatic recovery for data partitions can be facilitated. Moreover, a configuration manager component tracks current configurations for replication units and determines whether a reconfiguration is required (e.g., due to node failures, node recovery, replica additions/deletions, replica moves, or replica role changes, and the like.) Reconfigurations of data activated as being replicated from an old configuration to being replicated on a new configuration may be performed in a transactionally consistent manner based on dynamic quorums associated with the new configuration and the old configuration.

Abstract: Systems and methods that provide for joining a node to a federation. A link component facilitates joining such node thru three phases of join, introduce and advise. A plurality of messages and replies are exchanged between nodes in form of introduce, advise, and response messages. Such messages contain the sending node's full neighborhood.