Abstract: Systems and methods for using snapshots to establish operable portions of computing entities on secondary sites for use on the secondary sites before the computing entity is fully transferred to the secondary site. Embodiments commence upon initiating replication of an entity from an originating site to a secondary site. While replication of the entire entity from the originating site to the secondary site is being carried out, the originating site transfers entity metadata to the secondary site. A non-replication user process queries the metadata to determine completion status and/or other characteristics of subcomponents of the entity. After determining the completion status of subcomponents of the entity, the non-replication user process operates on subcomponent data of the entity even before the data of the entity has been completely copied to the secondary site. The subcomponents may comprise certain tables of a database or certain rows of a table of a database.

Abstract: The disclosed systems emulate high-frequency application-consistent snapshots by forming restore point data sets based on remote site replay of I/O commands. A method embodiment commences upon identifying a primary computing site and a secondary computing site, then identifying an application to be restored from the secondary computing site after a disaster. Prior to the disaster, a group of computing entities of the application to be restored from the secondary computing site are identified. Input/output operations that are performed over any of the computing entities at the primary site are streamed to the secondary site where they are stored. An I/O map that associates a time with an indication of a last received I/O command that had been performed over a changing set of computing entities is sent to the secondary site. An agent at the secondary site accesses the I/O map and the streamed-over I/Os to construct recovery data.

Abstract: Systems, methods, and computer program products for distributed data storage. A spot defragmentation method commences upon receiving an incoming storage I/O command to access a subject logical object that is composed of subject logical data blocks. The method continues by retrieving a block map that characterizes spatial relationships between the subject logical data blocks and instances of respective subject physical data blocks that store the subject logical object on a storage device. During processing of the incoming storage I/O command, the method determines occurrences of, and locations of one or more fragmented physical data blocks. A defragmentation operation is initiated to coalesce the fragmented physical data blocks to another location. The defragmentation operation is initiated before completing the storage I/O command.

Abstract: Techniques for replay of groups of I/Os for disaster recovery. At a primary site, a plurality of primary site computing entities that are to be restored from the secondary computing site after a disaster recovery event are identified as belonging to an entity group. At the primary site, I/O commands that are performed over any of the entities of the entity group are observed and streamed to the secondary site, where they are persisted into a log. Periodically, the primary site sends the secondary site an I/O map that associates a time with an indication of the last performed I/O commands pertaining to any observed I/O commands that had been performed over any one or more of the computing entities of the entity group. Upon receiving a disaster recovery request, the most recent I/O map and the secondary site log are processed to construct a snapshot for the entity group.

Abstract: Systems for distributed data storage. A method embodiment commences upon capturing a history of storage I/O activity over a recent time period. A predictive model is derived from the captured storage I/O activity, and the predictive model is then used for predicting future storage I/O activity. A set of snapshot planning parameters comprising objectives (e.g., to minimize costs or to maximize likelihood completing a snapshot activity by a prescribed time) and/or constraints (e.g., don't wait more than one day to start a snapshot) are applied to the predicted storage I/O characteristics to generate a set of feasible snapshot plans. One of the feasible snapshot plans is selected for scheduling so as to begin the planned snapshot activity at a prescribed time. The snapshot planning parameters are normalized based on the predicted storage I/O characteristics.

Abstract: Methods, systems and computer program products for data protection across computing infrastructure comprising a plurality of geographically distant computing clusters. The geographically distant computing clusters form a distributed system comprising clusters that are assigned into availability zones. Disaster recovery policies are maintained in each of the availability zones. A first cluster detects a modification of a disaster recovery policy. The modification of the disaster recovery policy that occurred at the first cluster is to be communicated over a wide area network. Synchronization of the disaster recovery policy is accomplished by carrying out a peer-to-peer communication protocol over the wide area network.

Abstract: A method embodiment commences upon identifying an on-premises storage area such as a file or virtual disk that is composed of a set of data blocks. A history of data access patterns over the data blocks is determined by observing accesses to the data blocks over time. When offsite storage is demanded (e.g., such as to a cloud-based storage facility), one or more storage objects are formed such that the blocks in a first one of the one or more storage objects comprise the blocks that have been measured to have the highest access rates (e.g., most recently accessed or most frequently accessed). Metadata that serves as a map is stored with the storage objects. A process or thread can access the storage objects in expectation that the most recently accessed or most frequently accessed blocks will be available in the first ones of the one or more storage objects.

Abstract: A system and method include reconfiguring original configuration parameters of virtual objects when restoring the virtual objects on a different node. Snapshots of virtual objects running on a first node can be stored on a second node. Upon failure of the first node, the virtual objects can be restored on the second node based on the snapshots. The snapshots can include configuration information of the virtual objects specifying the resource need of the virtual objects. The second node can determine whether the available resources meet the resource needs of the virtual objects being restored. If the resource needs of the virtual objects exceed the available resources, the second node can reconfigure the original configuration parameters of the virtual objects to ensure the virtual objects can be restored and run on the second node.

Abstract: Systems for multi-cluster virtualized computing system management. A method for performing virtual entity replication between source computing clusters and target computing clusters commences upon establishing a virtual entity naming convention that is observed by both the source computing clusters and the target computing clusters. A snapshot from a source cluster is associated with a global snapshot ID before being transmitted to a target computing cluster. At some point in time, the source cluster will initiate acts to replicate a virtual entity to a particular data state that is associated with a particular named snapshot. A second replication protocol then commences. The second replication protocol includes exchanges that serve to determine whether or not the target computing cluster has a copy of a particular named snapshot as named by the global snapshot ID, and if so, to then initiate virtual entity replication at the target computing cluster using the named snapshot.

Abstract: Methods, system, and computer program product for implementing an address translation service that uses nondenominational address handles instead of IP addresses between private cloud domain and public cloud domains. The address translation service can be implemented to enable a data-center running in a private cloud domain to communicate with the public cloud domain data-center over load balancers. In addition, the address translation service ensures that all services that need to communicate across data-centers can be reached over load balancers. As such, to avoid conflicting subnets used by a data center from the private cloud domain and the public cloud domain, services in the public cloud domain use a private cloud load balancer to connect with the services in a private cloud domain. Similarly, a public cloud load balancer is used to connect with services in the private cloud domain.

Abstract: Systems for multi-cluster virtualized computing system management. A method for performing virtual entity replication between source computing clusters and target computing clusters commences upon establishing a virtual entity naming convention that is observed by both the source computing clusters and the target computing clusters. A snapshot from a source cluster is associated with a global snapshot ID before being transmitted to a target computing cluster. At some point in time, the source cluster will initiate acts to replicate a virtual entity to a particular data state that is associated with a particular named snapshot. A second replication protocol then commences. The second replication protocol includes exchanges that serve to determine whether or not the target computing cluster has a copy of a particular named snapshot as named by the global snapshot ID, and if so, to then initiate virtual entity replication at the target computing cluster using the named snapshot.

Abstract: Systems for application configuration backup and restore. Techniques establish high frequency restore points of application configuration metadata. Processing commences upon receiving instructions to periodically take snapshots of application configuration metadata. To avoid incurring unnecessary costs to process snapshots of application configuration metadata when the application configuration metadata is slow-changing, the method performs a low-cost check to get the latest update time of the application configuration metadata. That time is compared to the time of the most recent snapshot of the application configuration metadata. When the time of the latest update time of the application configuration metadata is equal to the time of the most recent snapshot of the application configuration metadata, then rather than incurring the cost of generating a duplicate snapshot, the method waits for a duration without generating a new snapshot.

Abstract: Systems for high performance restore of data to storage devices. A method embodiment commences upon identifying a plurality of virtual disks to be grouped together into one or more consistency sets. Storage I/O commands for the plurality of virtual disks of the consistency sets are captured into multiple levels of backup data. On a time schedule, multiple levels of backup data for the virtual disks are cascaded by processing data from one or more higher granularity levels of backup data to one or more lower granularity levels of backup data. A restore operation can access the multiple levels of backup data to construct a restore set that is consistent to a designated point in time or to a designated state. Multiple staging areas can be maintained using lightweight snapshot data structures that each comprise a series of captured I/Os to be replayed over other datasets to generate a restore set.

Abstract: Systems and methods for using snapshots to establish operable portions of computing entities on secondary sites for use on the secondary sites before the computing entity is fully transferred to the secondary site. Embodiments commence upon initiating replication of an entity from an originating site to a secondary site. While replication of the entire entity from the originating site to the secondary site is being carried out, the originating site transfers entity metadata to the secondary site. A non-replication user process queries the metadata to determine completion status and/or other characteristics of subcomponents of the entity. After determining the completion status of subcomponents of the entity, the non-replication user process operates on subcomponent data of the entity even before the data of the entity has been completely copied to the secondary site. The subcomponents may comprise certain tables of a database or certain rows of a table of a database.

Abstract: Systems for high performance restore of data to storage devices. A method embodiment commences upon identifying a plurality of virtual disks to be grouped together into one or more consistency sets. Storage I/O commands for the plurality of virtual disks of the consistency sets are captured into multiple levels of backup data. On a time schedule, multiple levels of backup data for the virtual disks are cascaded by processing data from one or more higher granularity levels of backup data to one or more lower granularity levels of backup data. A restore operation can access the multiple levels of backup data to construct a restore set that is consistent to a designated point in time or to a designated state. Multiple staging areas can be maintained using lightweight snapshot data structures that each comprise a series of captured I/Os to be replayed over other datasets to generate a restore set.

Abstract: The disclosed systems emulate high-frequency application-consistent snapshots by forming restore point data sets based on remote site replay of I/O commands. A method embodiment commences upon identifying a primary computing site and a secondary computing site, then identifying an application to be restored from the secondary computing site after a disaster. Prior to the disaster, a group of computing entities of the application to be restored from the secondary computing site are identified. Input/output operations that are performed over any of the computing entities at the primary site are streamed to the secondary site where they are stored. An I/O map that associates a time with an indication of a last received I/O command that had been performed over a changing set of computing entities is sent to the secondary site. An agent at the secondary site accesses the I/O map and the streamed-over I/Os to construct recovery data.

Abstract: Systems and methods for “what-if” scenario planning of a distributed data storage system. A scenario planning engine has a user interface to facilitate user interactions to describe “what if” scenarios. A method comprises steps to collect system performance measurements pertaining to measurable characteristics of the distributed storage system. A predictive model is generated and formatted for use as a predictor of one or more predictive model parameters that are derived from the collected system performance measurements and/or any calculated predictions and/or correlations. A user can vary a set of scenario input parameters so as to characterize one or more “what if” scenarios. The user-defined scenario input parameters are formatted and used as predictive model inputs. The predictive model is used to simulate predicted system performance parameters corresponding to respective “what-if” planning scenarios.

Abstract: Techniques for replay of groups of I/Os for disaster recovery. At a primary site, a plurality of primary site computing entities that are to be restored from the secondary computing site after a disaster recovery event are identified as belonging to an entity group. At the primary site, I/O commands that are performed over any of the entities of the entity group are observed and streamed to the secondary site, where they are persisted into a log. Periodically, the primary site sends the secondary site an I/O map that associates a time with an indication of the last performed I/O commands pertaining to any observed I/O commands that had been performed over any one or more of the computing entities of the entity group. Upon receiving a disaster recovery request, the most recent I/O map and the secondary site log are processed to construct a snapshot for the entity group.

Abstract: Systems and methods for scheduling storage management tasks over predicted user tasks in a distributed storage system. A method commences upon receiving a set of historical stimulus records that characterize management tasks that are run in the storage system. A corresponding set of historical response records comprising system metrics associated with execution of the system tasks is also received. A learning model is formed from the stimulus records and the response records and formatted to be used as a predictor. A set of forecasted user tasks is input as new stimulus records to the predictor to determine a set of forecasted system metrics that would result from running the forecasted user tasks. Management tasks are selected so as not to impact the forecasted user tasks. Management tasks can be selected based on non-contentions resource usage between historical management task resource usage and predictions of resource usage by the user tasks.

Abstract: Systems, methods, and computer readable medium for virtualized computing environments. A method for providing a connection between a guest virtual machine and a service virtual machine uses driver code functions to establish a listening port on the service virtual machine without providing a listening port on the guest virtual machine. The guest virtual machine initiates a remote procedure call socket between itself and the service virtual machine over a secure, hardened port. The service virtual machine presents an authority certificate by encoding into the authority certificate identifying information received from the guest virtual machine. The service virtual machine makes available (e.g., as an ISO image) the authority certificate, which is used to establish new secure connections.