Abstract: An example method of synchronizing a first inventory of a cross-cluster control plane (xCCP) with a second inventory of a cluster control plane (CCP) includes: receiving, at a replication engine of the xCCP from the CCP, a notification of a CCP operation that modified an object in the second inventory; determining, by the replication engine, a first operation to modify the first inventory with the object; identifying, in a buffer of the replication engine, a second operation to modify the first inventory with a related object associated with the object, the related object included in an earlier CCP notification, received at the xCCP before the notification, but not used to modify the first inventory due to an unresolved dependency; and calling, by the replication engine in response to satisfaction of the unresolved dependency, a service of the xCCP to modify the first inventory by performing the first and second operations.

Abstract: The disclosure provides a method for tracking virtual machines (VMs) associated with a plurality of hosts in an inventory. The method generally includes determining to remove a first host of the plurality of hosts, the first host running a first VM, wherein: the first host and a second host are associated with a first host cluster in the inventory; the first host is the associated-host and the registered-host of the first VM in the inventory; determining the first VM is associated with first host cluster based on the associated-host of the first VM being the first host and the first host being associated with the first host cluster; identifying the second host is associated with the first host cluster in the inventory; altering the associated-host of the first VM to the second host and unsetting the registered-host for the first VM in the inventory; and removing the first host.

Abstract: Methods, apparatus, systems, and articles of manufacture to store cluster information in a distributed datastore are disclosed. An example apparatus includes memory; programmable circuitry; and first instructions to cause the programmable circuitry to: obtain second instructions to create a cluster of first hosts; determine second hosts of the cluster of the first hosts to implement a distributed datastore in the cluster; and cause transmission of third instructions to store cluster information corresponding to the cluster of the first hosts in datastores of the second hosts.

Abstract: Described herein are systems, methods, and software to manage configuration attempts for a service following a failure associated with the service according to an implementation. In one example, a management service monitors hosts in a computing environment and identifies a failure associated with a service on a first host. In response to identifying the failure, the management service identifies one or more criteria to initiate an attempt to configure the service based on the type of failure and initiates the attempt to configure the service when the one or more criteria are satisfied. Additional attempts can be initiated by the management service if the initial configuration attempt fails after a timeout interval.

Abstract: Described herein are systems, methods, and software to manage virtual machine shutdowns in a computing environment. In one example, a management service identifies a virtual machine file in a file system associated with a first virtual machine and one or more additional virtual machines and identifies a lock on the virtual machine file by the first virtual machine. The management service further stops execution of the one or more additional virtual machines in response to identifying the lock.

Abstract: Systems, apparatus, articles of manufacture, and methods are disclosed to manage a deployment of virtual machines in a cluster by, in a first host of a plurality of hosts, monitor, with first control plane services, an availability of second control plane services at a second host of the plurality of hosts, wherein the first control plane services and the second control plane services support implementation of application programming interface (API) requests in association with managing a cluster, after a determination that the second control plane services at the second host is not available, assign the first control plane services at the first host to operate in place of the second control plane services at the second host, and in the first host, assign, via the first control plane services at the first host, resources of one or more hosts in the cluster to support the API request.

Abstract: The present disclosure relates to using maintenance mode to upgrade a distributed system. One method includes determining that a first host of a cluster of a software-defined datacenter (SDDC) is to be upgraded as a part of a rolling upgrade of the hosts of the cluster, wherein the first host is executing a process instance of a cluster store, demoting the process instance to a proxy, creating a replica of the process instance using a different proxy on a second host of the cluster, instructing the first host to enter a maintenance mode, upgrading the first host, and instructing the first host to leave the maintenance mode.

Abstract: A method for maintaining fault tolerance in a storage cluster is provided. Embodiments include receiving, by a management component associated with a distributed data store on a cluster of host machines, a request to place a first host machine of the cluster of host machines in a maintenance mode, wherein the first host machine stores given data of the distributed data store. Embodiments include determining whether a second host machine that does not currently store any data of the distributed data store exists in the cluster of host machines. Embodiments include determining, based on whether the second host machine exists in the cluster of host machines, whether to transfer the given data of the distributed data store from the first host machine to the second host machine. Embodiments include initiating the maintenance mode on the first host machine.

Abstract: The present disclosure relates to health measurement and remediation of distributed systems upgrades. One method includes communicating a request for a health status to a cluster store provided by a cluster of hosts of a software-defined datacenter, wherein the cluster store includes a plurality of process instances undergoing a rolling upgrade, receiving a health status indication in response to the request, the health status indication determined based on a determination of whether the cluster store is available and a determination of whether the cluster store is operational, and taking a remediation action in response to the health status indication exceeding a threshold.

Abstract: A method for maintaining fault tolerance in a storage cluster is provided. Embodiments include receiving, by a management component associated with a distributed data store on a cluster of host machines, a request to place a first host machine of the cluster of host machines in a maintenance mode, wherein the first host machine stores given data of the distributed data store. Embodiments include determining whether a second host machine that does not currently store any data of the distributed data store exists in the cluster of host machines. Embodiments include determining, based on whether the second host machine exists in the cluster of host machines, whether to transfer the given data of the distributed data store from the first host machine to the second host machine. Embodiments include initiating the maintenance mode on the first host machine.

Abstract: A method for reconciling host membership information in a data center. The method generally includes querying, by a remote management device configured to manage one or more host clusters in the data center, one or more cluster stores to retrieve first host membership information for each of the one or more host clusters, each cluster store being associated with a host cluster of the one or more host clusters, comparing second host membership information stored at the remote management device for the one or more host clusters to the first host membership information to identify one or more discrepancies between the second host membership information and the first host membership information, and updating the second host membership information such that the second host membership information is consistent with the first host membership information.

Abstract: A method for reconciling host membership information in a data center. The method generally includes querying, by a remote management device configured to manage one or more host clusters in the data center, one or more cluster stores to retrieve first host membership information for each of the one or more host clusters, each cluster store being associated with a host cluster of the one or more host clusters, comparing second host membership information stored at the remote management device for the one or more host clusters to the first host membership information to identify one or more discrepancies between the second host membership information and the first host membership information, and updating the second host membership information such that the second host membership information is consistent with the first host membership information.

Abstract: Examples described herein include systems and methods for backing up and recovering a software-defined data center (“SDDC”). In one example, entities of the SDDC, such as virtual machines, hosts, and clusters, can coexist with corresponding entity stores. The entity stores can store current state information for each SDDC entity. For example, an identifier or name of a virtual machine can be stored in that virtual machine's corresponding entity store. When recovery of a controller is needed, the controller can rebuild state information that has changed after the controller was backed up, by retrieving state information from entity stores of the various SDDC entities.

Abstract: A method for maintaining fault tolerance in a storage cluster is provided. Embodiments include receiving, by a management component associated with a distributed data store on a cluster of host machines, a request to place a first host machine of the cluster of host machines in a maintenance mode, wherein the first host machine stores given data of the distributed data store. Embodiments include determining whether a second host machine that does not currently store any data of the distributed data store exists in the cluster of host machines. Embodiments include determining, based on whether the second host machine exists in the cluster of host machines, whether to transfer the given data of the distributed data store from the first host machine to the second host machine. Embodiments include initiating the maintenance mode on the first host machine.

Abstract: The present disclosure relates to health measurement and remediation of distributed systems upgrades. One method includes communicating a request for a health status to a cluster store provided by a cluster of hosts of a software-defined datacenter, wherein the cluster store includes a plurality of process instances undergoing a rolling upgrade, receiving a health status indication in response to the request, the health status indication determined based on a determination of whether the cluster store is available and a determination of whether the cluster store is operational, and taking a remediation action in response to the health status indication exceeding a threshold.

Abstract: The present disclosure relates to using maintenance mode to upgrade a distributed system. One method includes determining that a first host of a cluster of a software-defined datacenter (SDDC) is to be upgraded as a part of a rolling upgrade of the hosts of the cluster, wherein the first host is executing a process instance of a cluster store, demoting the process instance to a proxy, creating a replica of the process instance using a different proxy on a second host of the cluster, instructing the first host to enter a maintenance mode, upgrading the first host, and instructing the first host to leave the maintenance mode.

Abstract: Examples described herein include systems and methods for backing up and recovering a software-defined data center (“SDDC”). In one example, entities of the SDDC, such as virtual machines, hosts, and clusters, can coexist with corresponding entity stores. The entity stores can store current state information for each SDDC entity. For example, an identifier or name of a virtual machine can be stored in that virtual machine's corresponding entity store. When recovery of a controller is needed, the controller can rebuild state information that has changed after the controller was backed up, by retrieving state information from entity stores of the various SDDC entities.

Abstract: The present disclosure relates to using maintenance mode to upgrade a distributed system. One method includes determining that a first host of a cluster of a software-defined datacenter (SDDC) is to be upgraded as a part of a rolling upgrade of the hosts of the cluster, wherein the first host is executing a process instance of a cluster store, demoting the process instance to a proxy, creating a replica of the process instance using a different proxy on a second host of the cluster, instructing the first host to enter a maintenance mode, upgrading the first host, and instructing the first host to leave the maintenance mode.

Abstract: Examples described herein include systems and methods for backing up and recovering a software-defined data center (“SDDC”). In one example, entities of the SDDC, such as virtual machines, hosts, and clusters, can coexist with corresponding entity stores. The entity stores can store current state information for each SDDC entity. For example, an identifier or name of a virtual machine can be stored in that virtual machine's corresponding entity store. When recovery of a controller is needed, the controller can rebuild state information that has changed after the controller was backed up, by retrieving state information from entity stores of the various SDDC entities.

Abstract: Examples described herein include systems and methods for backing up and recovering a software-defined data center (“SDDC”). In one example, entities of the SDDC, such as virtual machines, hosts, and clusters, can coexist with corresponding entity stores. The entity stores can store current state information for each SDDC entity. For example, an identifier or name of a virtual machine can be stored in that virtual machine's corresponding entity store. When recovery of a controller is needed, the controller can rebuild state information that has changed after the controller was backed up, by retrieving state information from entity stores of the various SDDC entities.