Abstract: Techniques are provided for implementing a persistent memory storage tier to manage persistent memory of a node. The persistent memory is managed by the persistent memory storage tier at a higher level within a storage operating system storage stack than a level at which a storage file system of the node is managed. The persistent memory storage tier intercepts an operation targeting the storage file system. The persistent memory storage tier retargets the operation from targeting the storage file system to targeting the persistent memory. The operation is transmitted to the persistent memory.

Abstract: Methods, non-transitory machine readable media, and computing devices that facilitate cache rewarming in a failover domain are disclosed. With this technology, a tag is inserted into a local tagstore. The tag includes a location of data in a cache hosted by a failover computing device and is retrieved from a snapshot of a remote tagstore for the cache. An invalidation log for an aggregate received from the failover computing device is replayed subsequent to mounting a filesystem that is associated with the aggregate and comprises the data. The data is retrieved from the cache following determination of the location from the tag in the local tagstore in order to service a received storage operation associated with the data. Takeover nodes do not have to wait for a cache to repopulate organically, and can leverage the contents of a cache of a failover node to thereby improve performance following takeover events.

Abstract: An Updating No Sync (UNS) technique ensures continuous data protection for content driven distribution of data served by storage nodes of a fault tolerant cluster in the event of a degraded cluster state. A storage service implemented in each node includes one or more slice services (SSs) configured to process and store metadata describing the data served by the storage nodes and one or more block services (BSs) configured to process and store the data on storage devices of the node. A bin assignment service may coopt one or more healthy BSs to temporarily store updates of data and metadata received at the SS while the cluster is in the degraded state as an overflow data path (hence the term “Updating No Sync,” which denotes updating without synchronizing, i.e., not distributing the data within the cluster, but only accumulating an overflow of SS information). Once the cluster is no longer degraded, the accumulated overflow SS information at the BSs may be synchronized back to restored BSs, i.e.

Abstract: One or more techniques and/or computing devices are provided for implementing synchronous replication. For example, a synchronous replication relationship may be established between a first storage controller hosting local storage and a second storage controller hosting remote storage (e.g., replication may be specified at a file, logical unit number (LUN), or any other level of granularity). Data operations and offloaded operations may be implemented in parallel upon the local storage and the remote storage. Error handling operations may be implemented upon the local storage and implement in parallel as a best effort on the remote storage, and a reconciliation may be performed to identify any data divergence from the best effort parallel implementation. Storage area network (SAN) operations may be implemented upon the local storage, and upon local completion may be remotely implemented upon the remote storage.

Abstract: Methods, non-transitory machine readable media, and computing devices that manage data storage fabric health are disclosed. With this technology, a network model is applied to network data determined from monitored network traffic exchanged between storage nodes and via network connections over a cluster fabric to generate a health score for the cluster fabric. A cause of health degradation in the cluster fabric is determined based on an analysis of the network data, when the health score indicates that health of the cluster fabric is degrading. A corrective action is automatically initiated based on the cause of health degradation in the cluster fabric. With this technology, cluster fabrics in data storage networks are more effectively monitored for health degradation and, advantageously, corrective actions can be preemptively initiated to improve performance.

Abstract: A technique maintains multiple copies of data served by storage nodes of a cluster during upgrade of a storage node to ensure continuous protection of the data served by the nodes. The data is logically organized as one or more volumes on storage devices of the cluster and includes metadata that describe the data of each volume. A data protection system may be configured to maintain two copies of the data in the cluster during upgrade to a storage node that is assigned to host one of the copies of the data but that is taken offline during the upgrade. As a result, a slice service of the node may become unavailable during the upgrade. In response to the unavailability of the slice service, the technique redirects replicated data targeted to the slice service to a standby slice service according to a degraded redundant metadata (DRuM) service of the cluster.

Abstract: Failover methods and systems for a networked storage environment are provided. A metadata data structure is generated, before starting a replay of entries at a log stored in a non-volatile memory of a second storage node, during a failover operation initiated in response to a failure at a first storage node. The second storage node operates as a partner node of the first storage node, and the metadata structure stores a metadata attribute of each log entry. Furthermore, the metadata attribute of each log entry is persistently stored. The persistently stored metadata attribute is used to respond to a read request received during the replay by the second storage node, while a write request metadata attribute of a write request is used for executing the write request received by the second storage node during the replay.

Abstract: Methods and systems for storage services is provided. One method includes configuring by a processor, a database availability group (DAG) storing an active database copy at a first computing node and a passive database copy of the active database copy at a second computing node; initiating a backup operation for backing up the passive database copy from the second computing node; and interfacing with the first computing node by the second computing node for completing the backup operation. The first computing node identifies logs for the backup operation, backs up the identified logs and provides metadata associated with the backup of the identified logs to the second computing node. The second computing node updates metadata for the backup operation such that a backup copy of the passive database copy points to the second node with metadata received from the first computing node.

Abstract: Presented herein are methods, non-transitory computer readable media, and devices for efficiently administering locks for shared resources, such as data blocks, stored on a storage system. Methods for stamping a plurality of computer data objects are disclosed which include: accessing at least one of the plurality of computer data objects by a first data thread; assigning, by the first data thread, a stamp to the at least one of the plurality of computer data objects, to signify the at least one of the plurality of computer data objects is associated with the first data thread; preventing subsequent access by a second data thread to the stamped at least one of the plurality of computer data objects; and determining the stamp is no longer active, upon an event, effectively releasing the at least one of the plurality of computer data objects.

Abstract: A distributed storage system monitors one or more system performance metrics and one or more client performance metrics related usage of the distributed storage system, including a read latency metric, a write latency metric, a total input/output (I/O) operations per second (IOPS) metric, a read IOPS metric, a write IOPS metric, an I/O size metric, a total bandwidth metric, a read bandwidth metric, a write bandwidth metric, a read/write ratio metric or statistical measures thereof over a period of time. When the distributed storage system is determined to be in an overload condition (e.g., when a system load value, calculated based on the performance metrics, exceeds a threshold), the distributed storage system independently throttles access to one or more components of the distributed storage system by one or more of multiple clients performing I/O operations to the distributed storage system based on their respective contribution to the overload condition.

Abstract: Techniques are provided for deduplicating encrypted data. For example, a device has data to store in an encrypted state within a remote data store. A key is used to encrypt the data to create encrypted data. The data is hashed to create hashed data and the encrypted data is hashed to create hashed encrypted data. A probabilistic data structure of the data is generated. The key is encrypted based upon the data to create an encrypted key. The encrypted data is transmitted to the remote data store, along with metadata comprising the hashed data, the hashed encrypted data, the probabilistic data structure, and the encrypted key. The metadata may be used to implement deduplication for subsequent requests, to store data within the remote data store, with respect to the encrypted data.

Abstract: Techniques are provided for object store mirroring. Data within a storage tier of a node may be determined as being data to tier out to a primary object store based upon a property of the data. A first object is generated to comprise the data. A second object is generated to comprise the data. The first object is transmitted to the primary data store for storage in parallel with the second object being transmitted to a mirror object store for storage. Tiering of the data is designated as successful once acknowledgements are received from both the primary object that the first object was stored and the mirror object store that the second object was stored.

Abstract: Methods, non-transitory computer readable media, and computing devices are disclosed that facilitate efficient storage operations using variable length encoded segment maps for deduplicated objects. A key name and elements of a segment list are received with a request to store an object. Each of the elements corresponds to a segment associated with the object and comprises a segment hash. A segment map is generated for the object. The segment map includes a header and entries. Each of the entries includes at least an expanded offset, one of the segment hashes, and a start offset. The segment map is stored in an object store, as associated with the key name, along with segment data included in at least one of the elements.

Abstract: A request to generate a storage system model is received. The storage system model represents at least a portion of a storage system. In response to receiving the request, a storage system interface configuration is loaded. The storage system interface configuration comprises an attribute of an entity model. The attribute corresponds to an attribute of a storage system entity of the storage system. Further in response to receiving the request, the entity model is identified as representing the storage system entity. In response to identifying the entity model as representing the storage system entity, the entity model is instantiated.

Abstract: One or more techniques and/or computing devices are provided for utilizing snapshots for data integrity validation and/or faster application recovery. For example, a first storage controller, hosting first storage, has a synchronous replication relationship with a second storage controller hosting second storage. A snapshot replication policy rule is defined to specify that a replication label is to be used for snapshot create requests, targeting the first storage, that are to be replicated to the second storage. A snapshot creation policy is created to issue snapshot create requests comprising the replication label. Thus a snapshot of the first storage and a replication snapshot of the second storage are created based upon a snapshot create request comprising the replication label. The snapshot and the replication snapshot may be compared for data integrity validation (e.g., determine whether the snapshots comprise the same data) and/or quickly recovering an application after a disaster.

Abstract: A system, method, and computer program product for a block-based backing up a storage device to an object storage service is provided. This includes the generation of a data object that encapsulates a data of a data extent. The data extent covers a block address range of the storage device. The data object is named with a base name that represents a logical block address (LBA) of the data extent. The base name is appended with an identifier that deterministically identifies a recovery point that the data object is associated with. The base name combined with the identifier represents a data object name for the data object. The named data object is then transmitted to the object storage service for backup of the data extent. At an initial backup, the full storage device is copied. In incremental backups afterwards, only those data extents that changed are backed up.

Abstract: A system, method, and machine-readable storage medium for determining an amount of unique data in a distributed storage system are provided. In some embodiments, a combined efficiency set for a first data set stored in the distributed storage system, such as at a volume, may be generated. The first data set may include a first subset of data and a second subset of data in the distributed storage system. Additionally, a set of efficiency sets for the first subset of data may be generated. A set difference based on the combined efficiency set and the set of efficiency sets may be computed. An amount of memory used for storing unique data of the second subset of data may be estimated based on the set difference. The unique data may be present in the second subset of data but absent from the first subset of data.

Abstract: One or more techniques and/or computing devices are provided for resilient replication of storage operations. For example, a first storage controller may host first storage having a replication relationship with second storage hosted by a second storage controller. To improve resiliency against transient network issues of a network between the storage controllers, the first storage controller may implement a queue and retry mechanism to retry replication operations not acknowledge back by the second storage controller within a threshold time. The second storage controller may maintain a cumulative sequence number of a latest replication operation performed in order, an operation response map of replication operations performed out of order, and an operation finder map identifying currently implemented replication operations, which may be used to process incoming replication operations. Single write semantics, write order consistency, and reduction of write amplification may be provided.

Abstract: One or more techniques and/or systems are provided for identifying configuration inconsistencies between storage virtual machines across storage clusters. For example, a first storage cluster and a second storage cluster may be configured according to a disaster recovery relationship where user data and configuration data of the first storage cluster are replicated to the second storage cluster so that the second storage cluster can takeover for the first storage cluster in the event a disaster occurs at the first storage cluster. Because replication of configuration data (e.g., a name and size of a volume, a backup policy, etc.) may fail for various reasons, configuration of the first storage cluster is compared to configuration of the second storage cluster to identify a configuration difference (e.g., a new size of the volume at the first storage cluster may have failed to be replicated to a replicated volume at the second storage cluster).

Abstract: A technique is configured to maintain multiple copies of data served by storage nodes of a cluster during upgrade of a storage node to ensure continuous protection of the data served by the nodes. The data is logically organized as one or more volumes on storage devices of the cluster and includes metadata that describe the data of each volume. A data protection system may be configured to maintain at least two copies of the data in the cluster during upgrade to a storage node that is assigned to host one of the copies of the data but that is taken offline during the upgrade. As a result, an original slice service of the node may be rendered unavailable during the upgrade. In response, the technique redirects replicated data targeted to the original slice service to a standby pool of slice services in accordance with a degraded redundant metadata service of the cluster.