Abstract: Systems and methods for re-aligning data replication configuration of a cross-site storage solution after a failover are provided. According to one embodiment, after a failover, the new primary distributed storage system orchestrates flipping of the data replication configuration of a peered consistency group (CG) to reestablish zero RPO and zero RTO protections for the peered CG. The primary causes the secondary distributed storage system to perform an atomic database operation on its remote configuration database to (i) delete an existing source configuration that identifies the secondary as a source of data replication; and (ii) persist a new destination configuration identifying the secondary as a destination of data replication.

Abstract: With a forever incremental snapshot configuration and a typical caching policy (e.g., least recently used), a storage appliance may evict stable data blocks of an older snapshot, perhaps unchanged data blocks of the snapshot baseline. If stable data blocks have been evicted, restore of a recent snapshot will suffer the time penalty of downloading the stable blocks for restoring the recent snapshot. Creating synthetic baseline snapshots and refreshing eviction data of stable data blocks can avoid eviction of stable data blocks and reduce the risk of violating a recovery time objective.

Abstract: The disclosed technology relates determining a first subset of a plurality drives having a first zone size and a second subset of the plurality of drives having a second zone size different from the first zone size, within a redundant array of independent disks (RAID) group. A prevailing zone size between the first zone size and the second zone size is determined. One or more logical zones within the determined first subset of the plurality of drives and the determined second subset of the plurality of drives for a received input-output operation is reserved based on the determined prevailing zone size. The received input-output operation is completed within the reserved one or more logical zones within the determined first subset of the plurality of drives and the determined second subset of the plurality of drives.

Abstract: A system, method, and machine-readable storage medium for restoring a data object for a specified active time period are provided. In some embodiments, the method includes receiving, by a storage device from a client, a request specifying an active time period for a data object to remain stored on an accessible tier. The method also includes determining, by the storage device, that the active time period has elapsed. The method further includes responsive to a determination that the active time period has elapsed, sending, by the storage device, a request to a server storing the data object to move the data object from the accessible tier to an archive tier. Data objects that are stored on the accessible tier are accessible by the client, and data objects that are stored on the archive tier are inaccessible by the client.

Abstract: An application may store data to a dataset comprising a plurality of volumes stored on a plurality of storage systems. The application may request a dataset image of the dataset, the dataset image comprising a volume image of each volume of the dataset. A dataset image manager operates with a plurality of volume image managers in parallel to produce the dataset image, each volume image manager executing on a storage system. The plurality of volume image managers respond by performing requested operations and sending responses to the dataset image manager in parallel. Each volume image manager on a storage system may manage and produce a volume image for each volume of the dataset stored to the storage system. If a volume image for any volume of the dataset fails, or a timeout period expires, a cleanup procedure is performed to delete any successful volume images.

Abstract: Systems and methods for automated remediation of issues arising in a data management storage system are provided. Deployed assets of a storage solution vendor may deliver telemetry data to the vendor on a regular basis. The received telemetry data may be processed by an AIOps platform to perform predictive analytics and arrive at “community wisdom” from the vendor's installed user base. In one embodiment, an insight-based approach is used to facilitate risk detection and remediation including proactively addressing issues before they turn into more serious problems. For example, based on continuous learning based on the community wisdom and making one or both of a rule set and a remediation set derived therefrom available for use by cognitive computing co-located with a customer's storage system, a risk to which the storage system is exposed may be determined and a corresponding remediation may be deployed to address or mitigate the risk.

Abstract: Systems and methods for automated remediation of issues arising in a data management storage system are provided. Deployed assets of a storage solution vendor may deliver telemetry data to the vendor on a regular basis. The received telemetry data may be processed by an AIOps platform to perform predictive analytics and arrive at “community wisdom” from the vendor's installed user base. In one embodiment, an insight-based approach is used to facilitate risk detection and remediation including proactively addressing issues before they turn into more serious problems. For example, based on continuous learning based on the community wisdom and making one or both of a rule set and a remediation set derived therefrom available for use by cognitive computing co-located with a customer's storage system, a risk to which the storage system is exposed may be determined and a corresponding remediation may be deployed to address or mitigate the risk.

Abstract: Techniques are provided for implementing a unified object format. The unified object format is used to format data in a performance tier (e.g., infrequently accessed data, snapshot data, etc.) into objects that are stored into an object store for low cost, scalable, long term storage compared to storage of the performance tier. With the unified object format, compression of the data may be retained when the data is stored as the objects into the object store. Additional compression may also be provided for the data in the objects. The unified object format includes slot header metadata used to track the location of the data within the object notwithstanding the data being compressed and/or stored at non-fixed boundaries. The slot header metadata may be cached at the performance tier for improved read performance and may be repaired by a repair subsystem (a slot header repair subsystem).

Abstract: Asynchronous snapshot invalidation techniques are described. According to various such techniques, an enhanced file handle structure may be defined that includes a snapshot generation ID that is to comprise a value that singularly identifies a snapshot performed at a particular point in time. In some embodiments, when a snapshot ID assigned to that snapshot is reused at a subsequent point in time, a different snapshot generation ID may be assigned to that subsequent snapshot. With respect to an in-core cache, the differing snapshot generation IDs may eliminate unacceptable ambiguity regarding respective file information sets corresponding to the initial and subsequent snapshots sharing the same snapshot ID. As a result, obsolete file information sets may be cleared from the in-core cache asynchronously, enabling improved performance. The embodiments are not limited in this context.

Abstract: A method performed by one or more processing resources of one or more computer systems is disclosed. The method comprises receiving an object at a first of a plurality of nodes from a second of the plurality of storage nodes within a cluster switch fabric, examining a value associated included within the received object, wherein the value is associated with a clock value of the second node and updating a clock operating at the first node with the received value.

Abstract: According to one embodiment, a computer implemented method comprises providing multiple channels between a first storage node and a second storage node with each channel having a separate network connection for packets of a transport layer session, assigning packets from each channel to a group of receive queues of the second storage node, continuously monitoring whether two or more channels of the multiple channels share a same receive queue of the second storage node, and sending a communication via a channel to the first storage node to indicate a dynamic change in a hash input field (e.g., a source port, a destination port, a source internet protocol (IP) address, and a destination IP address) when two or more channels of the multiple channels share a same receive queue of the second storage node.

Abstract: A method performed by one or more processing resources of one or more computer systems is disclosed. The method comprises receiving an object at a first of a plurality of nodes from a second of the plurality of storage nodes within a cluster switch fabric, examining a value associated included within the received object, wherein the value is associated with a clock value of the second node and updating a clock operating at the first node with the received value.

Abstract: Techniques are provided for combining data block and checksum block I/O into a single I/O operation. Many storage systems utilize checksums to verify the integrity of data blocks stored within storage devices managed by a storage stack. However, when a storage system reads a data block from a storage device, a corresponding checksum must also be read to verify integrity of the data in the data block. This results in increased latency because two read operations are being processed through the storage stack and are being executed upon the storage device. To reduce this latency and improve I/O operations per second, a single combined I/O operation corresponding to a contiguous range of blocks including the data block and the checksum block is processed through the storage stack instead of two separate I/O operations. Additionally, I/O operation may be combined into a single request that is executed upon the storage device.

Abstract: The instant disclosure provides a data structure store system and a method of managing data in the store. The method includes receiving, by a data structure store management system, a request for storing data from a client. In the created data structure, each data element includes a portion of the data. On receiving a read request for at least part of the data, the data structure store management system provides at least part of the data to a recipient device. The data elements are stored in persistent memory in the form of one or more non-volatile random access devices, wherein during a time interval between receiving the storing request and providing the at least part of the data by the data structure store management system to the recipient device, the data structure store management system provides no portion of the data for writing to a hard disk drive.

Abstract: Systems and methods for performing single I/O writes are provided. According to one embodiment, responsive to receipt of a write operation from a client by a file system layer of a node of a distributed storage system and a data payload of the operation having been determined to meet a compressibility threshold, an intermediate storage layer of the node logically interposed between the file system layer and a block storage media is caused to perform a single input/output (I/O) write operation that persists the compressed data payload and corresponding metadata to support asynchronous journaling of the write operation. The single I/O write operation coupled with the use of a new pool file that maintains a list of available blocks for single I/O write operations and a modified node crash recovery approach allows the write operation to be acknowledged to the client while the journaling is performed asynchronously.

Abstract: Techniques are provided for maintaining and recomputing reference counts in a persistent memory file system of a node. Primary reference counts are maintained for pages within persistent memory of the node. In response to receiving a first operation to link a page into a persistent memory file system of the persistent memory, a primary reference count of the page is incremented before linking the page into the persistent memory file system. In response to receiving a second operation to unlink the page from the persistent memory file system, the page is unlinked from the persistent memory file system before the primary reference count is decremented. Upon the node recovering from a crash, the persistent memory file system is traversed in order to update shadow reference counts for the pages with correct reference count values, which are used to overwrite the primary reference counts with the correct reference count values.

Abstract: Systems and methods are provided for sharing ephemeral storage of a virtual machine (VM) for use as victim caches for virtual storage appliances running on the VM. According to one embodiment, a central service may run within the VM and be responsible for managing allocation and reclamation of ephemeral storage space of the VM to/from the virtual storage appliances. Responsive to startup of a new virtual storage appliance on the VM, the new virtual storage appliance may request space from the central service to inform creation of its victim cache. In connection with servicing the request, the central service may take into consideration various factors including one or more of the total aggregate size of multiple local ephemeral drives associated with the VM, remaining available ephemeral storage space, the number of active virtual storage appliances, and the SLO of the virtual storage appliance seeking to establish its victim cache.

Abstract: In one embodiment, distributed data storage systems and methods integrate a change tracking manager with scalable databases. According to one embodiment, a computer implemented method comprises integrating change tracking of storage objects into the distributed object storage database that includes a first database of a first type and one or more chapter databases of a second type with the distributed object storage database supporting a primary lookup index and a secondary lookup index in order to locate a storage object. The method includes recording in a header of a chapter database a network topology for connecting a bucket having the chapter database to a first peer bucket when a new mirror to the first peer bucket is being established, and recording a first directive into the header of the chapter database to express a type of content to be mirrored from the bucket to the first peer bucket.

Abstract: Methods, non-transitory machine readable media, and computing devices that provide more efficient hierarchical propagation in tree structures are disclosed. With this technology, a first delta record for a first interior node is created optionally in an atomic transaction along with updating a first tally record for a leaf node based on a first value. The transaction is in response to an action associated with the leaf node and the first interior node is a parent of the leaf node in a hierarchical tree. A timer associated with the first delta record is then set. A second value is updated in a second tally record for the first interior node based on the first value, when the timer has expired. Accordingly, this technology advantageously maintains recursive properties or values throughout a hierarchical tree continually, with reduced cost, even in a distributed network and in hierarchical trees with large numbers of nodes.

Abstract: Techniques are provided for implementing garbage collection and bin synchronization for a distributed storage architecture of worker nodes managing distributed storage composed of bins of blocks. As the distributed storage architecture scales out to accommodate more storage and worker nodes, garbage collection used to free unused blocks becomes unmanageable and slow. Accordingly garbage collection is improved by utilizing heuristics to dynamically speed up or down garbage collection and set sizes for subsets of a bin to process instead of the entire bin. This ensures that garbage collection does not use stale information about what blocks are in-use, and ensures garbage collection does not unduly impact client I/O processing or conversely falls behind on garbage collection. Garbage collection can be incorporated into a bin sync process to improve the efficiency of the bin sync process so that unused blocks are not needlessly copied by the bin sync process.