Abstract: Systems and methods for unified application-level backup and restore using heterogeneous cloud-based backup service providers. An application programming interface is configured to process both data level replication operations as well as application-level operations that are executed to carry out high-level commands between a virtualized computing environment and any one or more of the heterogeneous cloud-based backup service providers. The API receives commands from applications in the virtualized computing environment. The API processes commands from the applications so as to facilitate replication of data to selected one or more cloud-based backup service providers. The commands perform data level replication operations as well as application-level operations for storing content to the cloud-based service provider. After a failure event and/or upon receipt of a restore command, the API initiates application-level operations that restore the application and its constituent entities.

Abstract: Systems for high performance restore of data to storage devices. A method embodiment commences upon identifying a plurality of virtual disks to be grouped together into one or more consistency sets. Storage I/O commands for the plurality of virtual disks of the consistency sets are captured into multiple levels of backup data. On a time schedule, multiple levels of backup data for the virtual disks are cascaded by processing data from one or more higher granularity levels of backup data to one or more lower granularity levels of backup data. A restore operation can access the multiple levels of backup data to construct a restore set that is consistent to a designated point in time or to a designated state. Multiple staging areas can be maintained using lightweight snapshot data structures that each comprise a series of captured I/Os to be replayed over other datasets to generate a restore set.

Abstract: A scalable multi-cluster resource sharing facility. A global witness process runs on a first computing platform that communicates over one or more networks to any number of nodes situated over two or more clusters. The global witness process listens on the network for occurrences of leadership and/or resource requests from nodes of different clusters. The global witness processes a request by retrieving a resource request and a respective last known state value, comparing the last known state value to a global stored state value, then storing a new state value when the respective last known state value is equal to the stored state value. Any number of contemporaneous requests can be processed by the global witness process, however only one request can be granted. The other requestors each receive a rejection of their resource request when their proffered last known state value is not equal to the stored state value.

Abstract: Systems for high performance restore of data to storage devices. A method embodiment commences upon identifying a plurality of virtual disks to be grouped together into one or more consistency sets. Storage I/O commands for the plurality of virtual disks of the consistency sets are captured into multiple levels of backup data. On a time schedule, multiple levels of backup data for the virtual disks are cascaded by processing data from one or more higher granularity levels of backup data to one or more lower granularity levels of backup data. A restore operation can access the multiple levels of backup data to construct a restore set that is consistent to a designated point in time or to a designated state. Multiple staging areas can be maintained using lightweight snapshot data structures that each comprise a series of captured I/Os to be replayed over other datasets to generate a restore set.

Abstract: A scalable multi-cluster resource sharing facility. A global witness process runs on a first computing platform that communicates over one or more networks to any number of nodes situated over two or more clusters. The global witness process listens on the network for occurrences of leadership and/or resource requests from nodes of different clusters. The global witness processes a request by retrieving a resource request and a respective last known state value, comparing the last known state value to a global stored state value, then storing a new state value when the respective last known state value is equal to the stored state value. Any number of contemporaneous requests can be processed by the global witness process, however only one request can be granted. The other requestors each receive a rejection of their resource request when their proffered last known state value is not equal to the stored state value.

Abstract: The disclosed systems emulate high-frequency application-consistent snapshots by forming restore point data sets based on remote site replay of I/O commands. A method embodiment commences upon identifying a primary computing site and a secondary computing site, then identifying an application to be restored from the secondary computing site after a disaster. Prior to the disaster, a group of computing entities of the application to be restored from the secondary computing site are identified. Input/output operations that are performed over any of the computing entities at the primary site are streamed to the secondary site where they are stored. An I/O map that associates a time with an indication of a last received I/O command that had been performed over a changing set of computing entities is sent to the secondary site. An agent at the secondary site accesses the I/O map and the streamed-over I/Os to construct recovery data.

Abstract: Methods, system, and computer program product for implementing an address translation service that uses nondenominational address handles instead of IP addresses between private cloud domain and public cloud domains. The address translation service can be implemented to enable a data-center running in a private cloud domain to communicate with the public cloud domain data-center over load balancers. In addition, the address translation service ensures that all services that need to communicate across data-centers can be reached over load balancers. As such, to avoid conflicting subnets used by a data center from the private cloud domain and the public cloud domain, services in the public cloud domain use a private cloud load balancer to connect with the services in a private cloud domain. Similarly, a public cloud load balancer is used to connect with services in the private cloud domain.

Abstract: Systems and methods for using snapshots to establish operable portions of computing entities on secondary sites for use on the secondary sites before the computing entity is fully transferred to the secondary site. Embodiments commence upon initiating replication of an entity from an originating site to a secondary site. While replication of the entire entity from the originating site to the secondary site is being carried out, the originating site transfers entity metadata to the secondary site. A non-replication user process queries the metadata to determine completion status and/or other characteristics of subcomponents of the entity. After determining the completion status of subcomponents of the entity, the non-replication user process operates on subcomponent data of the entity even before the data of the entity has been completely copied to the secondary site. The subcomponents may comprise certain tables of a database or certain rows of a table of a database.

Abstract: The disclosed systems emulate high-frequency application-consistent snapshots by forming restore point data sets based on remote site replay of I/O commands. A method embodiment commences upon identifying a primary computing site and a secondary computing site, then identifying an application to be restored from the secondary computing site after a disaster. Prior to the disaster, a group of computing entities of the application to be restored from the secondary computing site are identified. Input/output operations that are performed over any of the computing entities at the primary site are streamed to the secondary site where they are stored. An I/O map that associates a time with an indication of a last received I/O command that had been performed over a changing set of computing entities is sent to the secondary site. An agent at the secondary site accesses the I/O map and the streamed-over I/Os to construct recovery data.

Abstract: Systems for restarting a virtual machine in a disaster recovery scenario where a network configuration differs between the failed system and the recovery system. A method commences upon identifying a disaster recovery plan for restarting a virtual machine from a first system on a second system (e.g., a recovery system). A configuration for providing network access at the second system through an adapter present in the second system is stored at a location accessible to the second system. Restarting the virtual machine at the second system upon detection of a failure event at the first system.

Abstract: Systems for storage system rollover and rollback. A data mover agent is installed on a source storage system to capture disaster recovery data and send to a target system. Upon receiving a rollover event signal, a virtualized controller creates one or more replica user virtual machines running on the target system that serve to replicate functions of the user virtual machines from the source storage system. The virtualized controller on the target system converts the target disaster recovery data from a first format to a second format to facilitate use of the target disaster recovery data by the replica user virtual machines. Rollback is initiated when the target system receives a rollback event signal. Differences in the data that have occurred between the rollover event and the rollback signal are calculated and sent to the rollback system. The calculated differences are applied to a registered snapshot on the rollback system.

Abstract: Systems, methods, and computer program products for distributed data storage. A spot defragmentation method commences upon receiving an incoming storage I/O command to access a subject logical object that is composed of subject logical data blocks. The method continues by retrieving a block map that characterizes spatial relationships between the subject logical data blocks and instances of respective subject physical data blocks that store the subject logical object on a storage device. During processing of the incoming storage I/O command, the method determines occurrences of, and locations of one or more fragmented physical data blocks. A defragmentation operation is initiated to coalesce the fragmented physical data blocks to another location. The defragmentation operation is initiated before completing the storage I/O command.

Abstract: Techniques for replay of groups of I/Os for disaster recovery. At a primary site, a plurality of primary site computing entities that are to be restored from the secondary computing site after a disaster recovery event are identified as belonging to an entity group. At the primary site, I/O commands that are performed over any of the entities of the entity group are observed and streamed to the secondary site, where they are persisted into a log. Periodically, the primary site sends the secondary site an I/O map that associates a time with an indication of the last performed I/O commands pertaining to any observed I/O commands that had been performed over any one or more of the computing entities of the entity group. Upon receiving a disaster recovery request, the most recent I/O map and the secondary site log are processed to construct a snapshot for the entity group.

Abstract: A data storage site receives data from different data producer sites. Each of the data producer sites has a particular relationship to the data storage site, and each particular relationship carries corresponding data storage policies, constraints and commitments. When a data storage site receives a data storage request from a data producer, and that particular data is already present from a prior storage operation at the data storage site, the characteristics of the policies, constraints and commitments that were applied when that data was saved by the prior storage operation are reconciled with the policies, constraints and commitments of the requesting data producer. Deduplication logic reconciles different sets of policies, constraints and commitments such that the data can be effectively deduplicated by saving data-producer-specific metadata. Alternatively, the data can be effectively deduplicated by promoting the storage of the data so it covers a broader set of policies, constraints and commitments.

Abstract: A data center communicates with a cloud-based backup system. Client-server roles are established such that a client role is assigned to the data center and a server role is assigned to the cloud-based backup system. On an ongoing basis, backup operations are performed. In the event of disaster or other cause of an outage of the data center, a failover protocol might be invoked such that the cloud-based backup system takes on additional processing operations beyond the aforementioned backup operations. After remediation, the data center issues a data restoration message to the cloud-based backup system. The remediated data center initiates a failback protocol that reverses the client-server roles of the data center and the cloud-based backup system such that the server role is assigned to the data center and the client role is assigned to the cloud-based backup system. After performing system restoration operations, the roles may be reversed again.

Abstract: A method embodiment commences upon identifying an on-premises storage area such as a file or virtual disk that is composed of a set of data blocks. A history of data access patterns over the data blocks is determined by observing accesses to the data blocks over time. When offsite storage is demanded (e.g., such as to a cloud-based storage facility), one or more storage objects are formed such that the blocks in a first one of the one or more storage objects comprise the blocks that have been measured to have the highest access rates (e.g., most recently accessed or most frequently accessed). Metadata that serves as a map is stored with the storage objects. A process or thread can access the storage objects in expectation that the most recently accessed or most frequently accessed blocks will be available in the first ones of the one or more storage objects.

Abstract: Systems for disaster recovery techniques. A method for capturing snapshots in a clustered virtualization environment commences upon identifying user virtual machines to be grouped together to form an application group. A virtualized controller sends or broadcasts messages to the plurality of user virtual machines where the messages request the user virtual machines to quiesce their respective transactions and other application activities. The user virtual machines associated with the application group quiesce their processing, flush their caches, and flush their file data. The virtualized controller waits for confirmations that the application activities associated with the application group have been quiesced. A backup agent commences activities to capture a snapshot of the storage devices associated with quiesced application processes. The application processes can run in conjunction with a hypervisor, or they can run as process containers.

Abstract: Methods, system, and computer program product for implementing an address translation service that uses nondenominational address handles instead of IP addresses between private cloud domain and public cloud domains. The address translation service can be implemented to enable a data-center running in a private cloud domain to communicate with the public cloud domain data-center over load balancers. In addition, the address translation service ensures that all services that need to communicate across data-centers can be reached over load balancers. As such, to avoid conflicting subnets used by a data center from the private cloud domain and the public cloud domain, services in the public cloud domain use a private cloud load balancer to connect with the services in a private cloud domain. Similarly, a public cloud load balancer is used to connect with services in the private cloud domain.

Abstract: A method for indexing virtual machine version snapshots in a virtualization environment commences upon receiving a request (e.g., from an administrator or agent) to initiate a virtual machine version snapshot operation on a subject virtual machine. Processes within or controlled by the subject virtual machine are requested to temporarily suspend transactions and file I/O. When the processes that have been requested to temporarily suspend transactions and file I/O acknowledge quiescence, the method continues by generating a virtual machine version snapshot data structure. An entry in an index is formed from the virtual machine version snapshot data structure. Multiple instances of virtual machine version snapshot data structures can be stored in the index, and the index can be queried to determine the state that a virtual machine had at any of the snapshotted moments in time.

Abstract: A method for performing lockless modifications during a rolling upgrade of a column family from an old column family version to a new column family version in a NoSQL database.