Abstract: A system to optimize scheduling of a data backup and/or restore of a backup data in a data backup/restore environment is presented. The system includes a training module configured to train an artificial intelligence (AI) model based on historical data corresponding to data backup and/or restore of one or more training datasets. The system further includes a time estimator configured to estimate an estimated time taken for the data backup and/or restore of the backup data to a data backup server or a restore location based on the trained AI model and operating data corresponding to operating states of one or more resources in the data backup/restore environment. A related method is also presented.

Abstract: A system for dynamic resource allocation during a data backup and/or restore of a backup data is presented. The system includes a resource allocation map generator configured to generate a resource allocation map for the current data backup and/or restore based on a mathematical model, real-time operating data corresponding to operating states of one or more resources, and historical data corresponding to data back-up and restore of one or more historical datasets. The system further includes a resource allocation recommender configured to generate a recommendation for resource allocation for the current data backup and/or restore based on the resource allocation map and a threshold value corresponding to a particular resource. The system moreover includes a resource allocator configured to dynamically initiate a change in resource allocation based on the generated recommendation. A related method is also presented.

Abstract: A storage cluster includes a group of data nodes having concurrent access to a shared filesystem. The shared filesystem is assigned to a first TLU of a first storage group. Other filesystems of the data nodes associated with the shared filesystem, such as snapshot copies of the shared filesystems and the root/configuration filesystems of the data nodes are assigned to TLUs of a second set of storage groups. The first storage group and the second set of storage groups are all associated with a Remote Data Replication (RDR) group for the storage cluster. An RDR facility is created between a storage array storing the shared filesystem and a backup storage array. The concurrently accessed shared filesystem and the filesystems of all data nodes accessing shared filesystem are replicated on the RDR facility to the backup storage array to enable failover of the storage cluster between sites.

Abstract: A storage node that maintains a replica of a logical volume for use in response to a failover trigger includes a data node with volatile memory in which a filesystem and its metadata and a VDM and its metadata associated with the replica are maintained prior to the failover trigger. The storage node also includes a SAN node in which data associated with the replica is maintained. The data is maintained in a RW (read-write) state by the SAN node prior to the failover trigger. However, the replica is presented in a RO (read-only) state by the storage node prior to the failover trigger. The storage node changes the in-memory state of the filesystem and VDM to RW responsive to the failover trigger. Because the filesystem and its metadata and VDM and its metadata are already in memory and the data is in a RW state in block storage the failover is completed relatively quickly.

Abstract: A data back-up system configured to back-up one or more data sets from one or more devices to a data back-up server by using a proxy pool is presented. The data back-up system includes a load-balancer configured to distribute the one or more data sets across the proxy pool. The load balancer includes a data receiver configured to receive types of files, number of files, and total size of each file in the one or more data sets; a load estimator configured to estimate a weighted average load of each data set based on the number of files, the total size of each file, a compressibility factor for each file type, and an encryption factor for each file type; and a load distributor configured to distribute the one or more data sets as a plurality of workloads across the proxy pool.

Abstract: Snaps are distributed among data nodes for load balancing and overload avoidance. A snap is initially associated with a first VNAS server on a first data node that hosts the primary volume being snapped. A second data node is selected based on loading. The second data node may be in the same or a different cluster. A second VNAS server is instantiated on the second data node. The snap is then mounted on the second VNAS server. The second VNAS server may be configured as the target for all reads to the snap. If the second data node is in a different cluster than the snap may be copied or moved from a first SAN node to a second SAN node. Multiple copies of snaps may be distributed, and individual snaps and copies may be associated with different SLOs.

Abstract: A system for selectively restoring data from a data back-up server is presented. The system includes a data access module configured to access a stateN of the data from a primary data source at a point N. The system further includes a log access module configured to access a log of modified meta-data and data blocks (MMDBs), from the primary data source or the data back-up server, corresponding to a data back-up point previous to the point N. The system furthermore includes a data restore module configured to iteratively perform selective restore of the data, based on the stateN and the MMDBs, from the data back-up server to a restore destination, until the data is restored to a stateRP corresponding to a recovery point (RP), as defined by a user. A related method is also presented.

Abstract: Techniques for performing a restoration may include: creating a first snapshot of a primary file system at a first point in time; and at a second point in time, performing first processing to restore at least a portion of the primary file system from the first point in time. The first processing may include: creating a second snapshot of the first snapshot of the primary file system; exporting the second snapshot as a first target device; mounting the first target device as a read-write checkpoint file system; performing file system consistency processing on the read-write checkpoint file system; and subsequent to performing the file system consistency processing on the read-write checkpoint file system, restoring at least a portion of the primary file system from the second snapshot using the read-write checkpoint file system. The file system consistency processing may include correcting metadata inconsistencies of the read-write checkpoint file system.

Abstract: Software Defined Network Attached Storage data backup is implemented using storage array synchronous data replication. A TLU is created for each user filesystem managed by a Virtual Data Mover (VDM). A VDM configuration filesystem is also created containing metadata describing a mapping of user file system TLU to physical storage resources on the storage array. Both the user filesystem TLUs and the VDM configuration filesystem are mapped to an asynchronous Remote Data Forwarding (RDF) group of the storage array, which is used by the storage array to replicate both the user filesystem TLUs and VDM configuration filesystem. Filesystem ID and export path information are reserved on the primary storage array and backup storage array when the RDF group is set up, to enable transparent failover from the primary to the backup node of selected VDMs.

Abstract: A first Software Defined Network Attached Storage (SDNAS) process on a first storage system forwards snapshots of a file system provided by the first storage system to a second SDNAS process on a second storage system. The snapshots are forwarded by the first SDNAS process on an Internet Protocol (IP) network, to enable the first and second SDNAS processes to maintain redundant access to data of the file system without requiring the first storage system and second storage system to be interoperable. The initial snapshot includes a full version of the file system and subsequent snapshots include changed portions of the file system. Where the snapshot identifies changed portions at the extent level of granularity, the first SDNAS process subdivides the extents into sub-blocks, and only transmits changed sub-blocks of the extents on the IP network.

Abstract: A storage cluster includes a group of data nodes having concurrent access to a shared filesystem. The shared filesystem is assigned to a first TLU of a first storage group. Other filesystems of the data nodes associated with the shared filesystem, such as snapshot copies of the shared filesystems and the root/configuration filesystems of the data nodes are assigned to TLUs of a second set of storage groups. The first storage group and the second set of storage groups are all associated with a Remote Data Replication (RDR) group for the storage cluster. An RDR facility is created between a storage array storing the shared filesystem and a backup storage array. The concurrently accessed shared filesystem and the filesystems of all data nodes accessing shared filesystem are replicated on the RDR facility to the backup storage array to enable failover of the storage cluster between sites.

Abstract: A storage node that maintains a replica of a logical volume for use in response to a failover trigger includes a data node with volatile memory in which a filesystem and its metadata and a VDM and its metadata associated with the replica are maintained prior to the failover trigger. The storage node also includes a SAN node in which data associated with the replica is maintained. The data is maintained in a RW (read-write) state by the SAN node prior to the failover trigger. However, the replica is presented in a RO (read-only) state by the storage node prior to the failover trigger. The storage node changes the in-memory state of the filesystem and VDM to RW responsive to the failover trigger. Because the filesystem and its metadata and VDM and its metadata are already in memory and the data is in a RW state in block storage the failover is completed relatively quickly.

Abstract: A method of dynamic binding and unbinding thin logical storage volumes to data volumes stored in a storage array includes creating a snapshot copy of a user file system, storing the snapshot copy of the user file system in a data volume in the storage array, and binding a logical storage volume to the data volume. The logical storage volume is then used, by a snapshot file system, to access the data volume. If the data volume is not accessed for a period of time, the logical storage volume is unbound from the data volume while maintaining the snapshot copy of the user file system in the data volume. If an access event later occurs on the data volume, a second logical storage volume is bound to the data volume and used by the snapshot file system to access the data volume in connection with the access event.

Abstract: A method for defining local failover within a storage system using storage group containers includes creating a plurality set of storage groups from physical storage resources of a storage array of the storage system, and creating a masking view identifying a first subset of the storage groups and specifying that Thin Logical Units (TLUs) in the first subset of storage groups are visible only to a primary Software Defined Network Attached Storage (SDNAS) process executing on the storage system and to a backup SDNAS process executing on the storage system. The first set of storage groups includes a Virtual Data Mover (VDM) configuration/root TLU and all user data TLUs used by the primary SDNAS process to thereby enable the set of storage groups used by the SDNAS to be used as a container for local failover.

Abstract: Snaps are distributed among data nodes for load balancing and overload avoidance. A snap is initially associated with a first VNAS server on a first data node that hosts the primary volume being snapped. A second data node is selected based on loading. The second data node may be in the same or a different cluster. A second VNAS server is instantiated on the second data node. The snap is then mounted on the second VNAS server. The second VNAS server may be configured as the target for all reads to the snap. If the second data node is in a different cluster than the snap may be copied or moved from a first SAN node to a second SAN node. Multiple copies of snaps may be distributed, and individual snaps and copies may be associated with different SLOs.

Abstract: A storage system includes a storage array having physical storage resources, a Software Defined Network Attached Storage (SDNAS) process running in an emulation on the storage system, a user file system having user file system data stored in the storage array that is accessed by the SDNAS process via a first Thin Logical Unit (TLU) storage volume mounted on an emulation operating system, a snapshot copy of the user file system stored in the storage array, and control logic. In some embodiments, the control logic is configured to associate a logical view of the snapshot copy of the user file system to a second TLU, mount the second TLU on the emulation operating system as a snapshot copy file system, and replay pending transactions and sub-transactions of the user file system on the snapshot copy file system to bring the snapshot copy file system to a stable state.

Abstract: Techniques for performing a restoration may include: creating a first snapshot of a primary file system at a first point in time; and at a second point in time, performing first processing to restore at least a portion of the primary file system from the first point in time. The first processing may include: creating a second snapshot of the first snapshot of the primary file system; exporting the second snapshot as a first target device; mounting the first target device as a read-write checkpoint file system; performing file system consistency processing on the read-write checkpoint file system; and subsequent to performing the file system consistency processing on the read-write checkpoint file system, restoring at least a portion of the primary file system from the second snapshot using the read-write checkpoint file system. The file system consistency processing may include correcting metadata inconsistencies of the read-write checkpoint file system.

Abstract: Disaster recovery is provided for Software Defined Network Attached Storage where the data backup is implemented synchronous data replication. A synchronous replication remote data facility is created between a primary storage array and a backup storage array. User filesystems being handled by a virtual data mover (VDM) and a configuration filesystem describing the location of the user filesystems on the primary storage array are replicated on the remote data facility to the backup storage array. Individual filesystems, individual VDMs, or groups of VDMs may be selectively failed over from the primary storage array to the backup storage array.

Abstract: Data Virtual Data Movers (VDM) are assigned to nodes of the storage cluster and a backup node is assigned for each data VDM. A system VDM on each node collects node statistics including operational parameters of the node and activity levels of the data VDMs on the node. A cluster manager collects the node statistics from each of the system VDMs and uses weighted collected node statistics to assign a node score to each node in the storage cluster. The cluster manager uses the node scores to identify possible data VDM movement combinations within the storage cluster by applying a set of hard rules and a set of soft rules to evaluate the possible data VDM movement combinations. If a VDM movement combination is selected, it is implemented by moving at least some of the data VDMs within the cluster to attempt to equalize node scores within the cluster.

Abstract: Disaster recovery is provided for Software Defined Network Attached Storage where the data backup is implemented using asynchronous remote data replication. An asynchronous replication facility is created between a primary storage array and a backup storage array. User filesystems being handled by a virtual data mover (VDM) and a configuration filesystem describing the location of the user filesystems on the primary storage array are replicated on the remote data facility to the backup storage array. During failover, all filesystems associated with the asynchronous remote data facility are failed over from a primary storage system associated with the primary storage array to a backup storage system associated with the backup storage array. Where an individual filesystem or individual VDM is to be failed over to the backup storage system, a separate asynchronous replication remote data facility is created to separately replicate data for the aspect to be individually failed over.