Abstract: An embodiment method creates, at a first data center, a first storage volume for an application that is started at the first data center, communicates with a second data center for creating a second storage volume for the application at the second data center to replicate the first storage volume, and sends information identifying the application to the second data center. The second data center creates the second storage volume, and establishes a mapping between the application and the second storage volume using the information identifying the application, where the second data center identifies the second storage volume for the application using the mapping when the application is migrated from the first data center to the second data center.

Abstract: A computer-implemented method for on-demand data replication in a network architecture includes activating an on-demand program to pair a network-based replication agent with a data source. Compute resources for the replication agent are allocated based on one or more hints received from the data source. The one or more hints are indicative of at least a size of an upcoming data traffic transmission from the data source. Upon receiving data from the data traffic transmission, the replication agent is activated via the on-demand program to replicate the received data using the allocated replication resources. The allocated compute resources are released upon completion of data replication of the received data. The on-demand program is a serverless function configured to activate the replication agent on-demand, upon receiving the data from the data traffic transmission.

Abstract: An embodiment method creates, at a first data center, a first storage volume for an application that is started at the first data center, communicates with a second data center for creating a second storage volume for the application at the second data center to replicate the first storage volume, and sends information identifying the application to the second data center. The second data center creates the second storage volume, and establishes a mapping between the application and the second storage volume using the information identifying the application, where the second data center identifies the second storage volume for the application using the mapping when the application is migrated from the first data center to the second data center.

Abstract: A computer-implemented method for on-demand data replication in a network architecture includes activating an on-demand program to pair a network-based replication agent with a data source. Compute resources for the replication agent are allocated based on one or more hints received from the data source. The one or more hints are indicative of at least a size of an upcoming data traffic transmission from the data source. Upon receiving data from the data traffic transmission, the replication agent is activated via the on-demand program to replicate the received data using the allocated replication resources. The allocated compute resources are released upon completion of data replication of the received data. The on-demand program is a serverless function configured to activate the replication agent on-demand, upon receiving the data from the data traffic transmission.

Abstract: Techniques are described for processing multi-page write operations to maintain write level consistency. A multi-page write spanning multiple cache pages is directed to a target device and received on a first data storage system where writes to the target device are synchronously replicated to a second data storage system. On the first data storage system, each of the multiple cache pages may be synchronously replicated to the second data storage system. A lock on each of the cache pages is not released until an acknowledgement is received regarding successful replication of the cache page. On the second data storage system, requests to replicate the multiple cache pages containing write data of the multi-page write are received and processed using locks of corresponding cache pages on the second data storage system. Such techniques also handle concurrent reads and/or writes. Deadlock detection and resolution processing may be performed for concurrent writes.

Abstract: Techniques are described for processing multi-page write operations to maintain write level consistency. A multi-page write spanning multiple cache pages is directed to a target device and received on a first data storage system where writes to the target device are synchronously replicated to a second data storage system. On the first data storage system, each of the multiple cache pages may be synchronously replicated to the second data storage system. A lock on each of the cache pages is not released until an acknowledgement is received regarding successful replication of the cache page. On the second data storage system, requests to replicate the multiple cache pages containing write data of the multi-page write are received and processed using locks of corresponding cache pages on the second data storage system. Such techniques also handle concurrent reads and/or writes. Deadlock detection and resolution processing may be performed for concurrent writes.

Abstract: Ordering data writes includes at least some of a group of local storage devices receiving a first plurality of data writes, waiting for all of the local storage devices to be ready to switch cycles and causing a coordinated and synchronized cycle switch after all of the local storage devices are ready to switch cycles by sending a first command to each of the local storage devices where the first plurality of data writes are associated with a particular cycle on each local storage device in the group. Ordering data writes also includes at least some of the group of local storage devices receiving a second plurality of writes after initiating the coordinated and synchronized cycle switch where all of the second plurality of writes are associated with a cycle different from the particular cycle on each local storage device.

Abstract: A primary storage device maintaining recovery data in connection with ordering data writes includes the primary storage device receiving a plurality of data writes, the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data, and the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data. After completion of all writes associated with the first chunk of data, the primary storage device initiates transfer of writes associated with the first chunk of data to a secondary storage device. The primary storage device maintains a transfer log of data from the first chunk that is successfully transferred to the secondary storage device.

Abstract: A computer-implemented method of ordering data writes includes at least some of a plurality of primary storage devices receiving a first plurality of data writes and causing a cycle switch for the plurality of primary storage devices. The first plurality of data writes are associated with a particular cycle on each primary storage device in the plurality of primary storage devices. At least some of the plurality of primary storage devices receive a second plurality of writes after initiating the cycle switch. All of the second plurality of writes are associated with a cycle different from the particular cycle on each of the plurality of primary storage devices.

Abstract: Maintaining an ordering of transmitted data includes associating a sequence number with each element of the transmitted data, where the sequence number indicates a temporal relationship between different ones of the elements. Maintaining an ordering also includes maintaining a last aborted sequence number and an indicator of whether receipt of information indicating that the last aborted sequence number has been acknowledged and rejecting data associated with a sequence number indicative of a time before a time corresponding to the last aborted sequence number. The last aborted sequence number may be sent until receipt thereof is acknowledged. No data may be transmitted until receipt of the last aborted sequence number is acknowledged. The sequence numbers may be incremented by one.

Abstract: Managing data transferred from a local storage device to a remote storage device includes maintaining the data in chunks in cache memory, where data writes to the local storage device begun during a particular time interval are associated with a chunk of data that is separate from other chunks of data and where data is transferred from the local storage device to the remote storage device by transferring a particular chunk after all data writes associated with the particular chunk have completed and after a previous chunk has been acknowledged as being received by the remote storage device and, in response to use of the cache memory exceeding a first threshold, storing data from a first chunk of the cache memory to a local disk. Data writes to the local storage device may be provided by a host coupled to the local storage device.

Abstract: A primary storage device maintaining recovery data in connection with ordering data writes includes the primary storage device receiving a plurality of data writes, the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data, and the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data. After completion of all writes associated with the first chunk of data, the primary storage device initiates transfer of writes associated with the first chunk of data to a secondary storage device. The primary storage device maintains a transfer log of data from the first chunk that is successfully transferred to the secondary storage device.

Abstract: Managing a cache memory includes providing at least one spillover slot, stored in the cache memory, that contains information about a plurality of cache data slots transferred from the cache memory to a local disk. The spillover slot has a metadata portion with information about cache data slots transferred from the cache memory to the local disk and has a catalog portion with entries that point to different sections of the metadata portion. Each section of the metadata portion corresponding to information about one of the cache data slots transferred from the cache memory to the local disk.

Abstract: Handling ordered writes to a data storage device includes receiving a first chunk of data into slots of a cache of the data storage device and using a hash table to provide a correlation between cache slots containing data from the first chunk of data and locations of a logical storage device of the data storage device. Handling ordered writes may also include maintaining a circularly linked list of pointers to cache slots containing the first chunk of data. Handling ordered writes may also include receiving a second chunk of data into slots of the cache of the data storage device and receiving a message indicating that all data corresponding to the first chunk of data has been transferred to the data storage device. Handling ordered writes may also include, in response to receiving the message, storing data from the first chunk on to the logical storage device.

Abstract: Transitioning from a first data transfer mode to a second data transfer mode that is different from the first data transfer mode includes a primary storage device transitioning to the second data transfer mode by first transferring writes associated with a first chunk of data to a secondary storage device after completion of all writes associated with the first chunk of data and then, after all writes associated with the first chunk of data have been transferred to the secondary storage device, transferring writes associated with a second chunk of data to the secondary storage device using the first data transfer mode. Writes begun after initiating transitioning are provided to the secondary storage device using the second data transfer mode.

Abstract: Recovering data provided in chunks to a plurality of secondary storage devices includes, for each of the secondary storage devices, discarding data corresponding chunks for which all data thereof has not been received, and, for each of the secondary storage devices, restoring a chunk of data thereto where all of the chunks of data restored to the plurality of secondary storage devices correspond to a particular transmission cycle of primary storage devices that provide data to the plurality of secondary storage devices. Recovering data may also include, following discarding and prior to restoring, for each of the plurality of secondary storage devices having two different chunks, waiting for external intervention to indicate whether to restore a particular one of the chunks. The external intervention may be provided by a host computer that is proximate to at least one of the secondary storage devices or may be provided by a host computer that is proximate to at least one of the primary storage computers.

Abstract: Handling ordered writes to a data storage device includes receiving a first chunk of data into slots of a cache of the data storage device and using a hash table to provide a correlation between cache slots containing data from the first chunk of data and locations of a logical storage device of the data storage device. Handling ordered writes may also include maintaining a circularly linked list of pointers to cache slots containing the first chunk of data. Handling ordered writes may also include receiving a second chunk of data into slots of the cache of the data storage device and receiving a message indicating that all data corresponding to the first chunk of data has been transferred to the data storage device. Handling ordered writes may also include, in response to receiving the message, storing data from the first chunk on to the logical storage device.

Abstract: Transferring data from at least one local data storage device to at least one remote data storage device includes subdividing the data into portions, assigning a sequence number to each of the portions, where writes to the at least one local storage device for a first set of portions having a first sequence number are begun before writes for a second set of portions having a second sequence number that is assigned after to the first sequence number, and updating the sequence number in response to a particular sequence number having been used before and there being data to send from the at least one local storage device to the at least one remote storage device. The at least one local storage device may include a plurality of storage entities that form a consistency group.

Abstract: Handling ordered writes to a data storage device includes receiving a first chunk of data into slots of a cache of the data storage device and using a hash table to provide a correlation between cache slots containing data from the first chunk of data and locations of a logical storage device of the data storage device. Handling ordered writes may also include maintaining a circularly linked list of pointers to cache slots containing the first chunk of data. Handling ordered writes may also include receiving a second chunk of data into slots of the cache of the data storage device and receiving a message indicating that all data corresponding to the first chunk of data has been transferred to the data storage device. Handling ordered writes may also include, in response to receiving the message, storing data from the first chunk on to the logical storage device.

Abstract: Ordering data writes includes at least some of a group of primary storage devices receiving a first plurality of data writes, causing a cycle switch for the group of primary storage devices where the first plurality of data writes are associated with a particular cycle on each primary storage device in the group, and at least some of the group of primary storage devices receiving a second plurality of writes after initiating the cycle switch where all of the second plurality of writes are associated with a cycle different from the particular cycle on each primary storage device. Writes to the group begun after initiating the cycle switch may not complete until after the cycle switch has completed. Ordering data writes may also include, after completion of the cycle switch, each of the primary storage devices of the group initiating transfer of the first plurality of writes to a corresponding secondary storage device.