Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during an automatic unplanned failover (AUFO). A computer-implemented method includes monitoring, with a second cluster, heartbeat information received at ultra-short time intervals from a first connection of one or more storage objects of the first cluster, determining, with the second cluster, whether the heartbeat information from the first connection is received during an ultra-short time interval, and intelligently routing heartbeat information from the one or more storage objects of the first cluster from the first connection to a second connection when the heartbeat information from the first connection is not received during the ultra-short time interval.

Abstract: A system, method, and machine-readable storage medium for forming a community based on a common set of attributes are provided. In some embodiments, the method includes creating a list of entities associated with a plurality of users, each entity included in the list of entities being accessed by a user of the plurality of users. The method also includes identifying a first entity accessed by a group of users of the plurality of users. The method further includes determining a first set of entities accessed by each user of the group of users, the first set of entities being included in the list of entities. The method also includes removing the first entity and the first set of entities from the list of entities. The method further includes forming a first community including the group of users, the first entity, and the first set of entities.

Abstract: Techniques are provided for tiering snapshots to archival storage in remote object stores. A restore time metric, indicating that objects comprising snapshot data of snapshots created within a threshold timespan are to be available within a storage tier of a remote object store for performing restore operations, may be identified. A scanner may be executed to evaluate snapshots using the restore time metric to identify a set of candidate snapshots for archival from the storage tier to an archival storage tier of the remote object store. For each candidate snapshot within the set of candidate snapshots, the scanner may evaluate metadata associated with the candidate snapshot to identity one or more objects eligible for archival from the storage tier to the archival storage tier, and may archive the one or more objects from the storage tier to the archival storage tier.

Abstract: Techniques are provided for caching data during an on-demand restore using a cloud block map. A client may be provided with access to an on-demand volume during a restore process that copies backup data from a snapshot within a remote object store to the on-demand volume stored within local storage. In response to receiving a request from the client for a block of the backup data not yet restored from the snapshot to the on-demand volume, the block may be retrieved from the snapshot in the remote object store. The block may be cached within a cloud block map stored within the local storage as a cached block. The client may be provided with access to the cached block.

Abstract: One or more techniques and/or computing devices are provided for managing an arbitrary set of storage items using a granset. For example, a storage controller may host a plurality of storage items and/or logical unit numbers (LUNs). A subset of the storage items are grouped into a consistency group. A granset is created for tracking, managing, and/or providing access to the storage items within the consistency group. For example, the granset comprises application programming interfaces (APIs) and/or properties used to provide certain levels of access to the storage items (e.g., read access, write access, no access), redirect operations to access either data of an active file system or to a snapshot, fence certain operations (e.g., rename and delete operations), and/or other properties that apply to each storage item within the consistency group. Thus, the granset provides a persistent on-disk layout used to manage an arbitrary set of storage items.

Abstract: Techniques are provided for on-demand creation and/or utilization of containers and/or serverless threads for hosting data connector components. The data connector components can be used to perform integrity checking, anomaly detection, and file system metadata analysis associated with objects stored within an object store. The data connector components may be configured to execute machine learning functionality to perform operations and tasks. The data connector components can perform full scans or incremental scans. The data connector components may be stateless, and thus may be offlined, upgraded, onlined, and/or have tasks transferred between data connector components. Results of operations performed by the data connector components upon base objects may be stored within sibling objects.

Abstract: Redistribution of files in a containerized distributed file system is disclosed. Containers each have an engine and a scanner and each of the containers stores files and parameters for characteristics of files stored on the container. A first engine in a first container monitors characteristics of files stored on the first container and, responsive to determining that the parameters for files on the first container exceed one or more predetermined thresholds, communicates with a second engine in a second container to determine a destination container for one or more files from the first container. The second engine in the second container indicates to the first engine in the first container whether the second container is available to receive one or more files from the first container. The first engine triggers file system scanning by the scanner of the first container to identify files to be moved to the second container.

Abstract: Systems and methods are provided for learning normal behavior for user roles of an application running within a cluster of container orchestration platform and based thereon proactively taking action responsive to suspicious events. According to one embodiment, an event data stream is created by an API server of the cluster. The data for each event includes information regarding a request made to an API exposed by the API server with which the event is associated and a user of the application by which the event was initiated. The data is augmented with a role associated with the user and an anomaly threshold for the role. Normal behavior is learned by an ML algorithm of respective user roles by processing the augmented data. When an anomaly score associated with a particular event is output by the ML algorithm that exceeds the anomaly threshold, a predefined or configurable action is triggered.

Abstract: Techniques are provided for volume group backup, volume group restore, and volume group garbage collection for volume groups backed up to an object store. A volume group workflow is implemented to orchestrate individual consistent volume workflows that are separately and individually implemented by nodes hosting constituent volumes of a volume group. The volume group workflow and the individual consistent volume workflows are performed to back up the volume group to the object store, restore a volume group backup from the object store to a restore destination, and/or perform garbage collection on slots of objects storing data unique to a volume group backup to delete.

Abstract: One or more techniques and/or computing devices are provided for resynchronization. For example, a request may be received to create pseudo snapshots of a first consistency group, hosted by a first storage controller, and a second consistency group, hosted by a second storage controller, having a synchronous replication relationship with the first consistency group. Incoming client write requests are logged within an intercept tracking log at the first storage controller. After a first drain without hold of incoming write requests is performed, a first pseudo common snapshot of the second consistency group is created. After a second drain without hold of incoming write operations is performed, a second pseudo common snapshot of the first consistency group and the intercept tracking log is created. The pseudo snapshots and the intercept tracking log (e.g., indicating a delta between the pseudo snapshots) are used to resynchronize the first and second consistency groups.

Abstract: Systems and methods for automated tuning of Quality of Service (QoS) settings of volumes in a distributed storage system are provided. According to one embodiment, one or more characteristics of a workload of a client to which a storage node of multiple storage nodes of the distributed storage system is exposed are monitored. After a determination has been made that a characteristic meets or exceeds a threshold, (i) information regarding multiple QoS settings assigned to a volume of the storage node utilized by the client is obtained, (ii) a new value of a burst IOPS setting of the multiple QoS settings is calculated by increasing a current value of the burst IOPS setting by a factor dependent upon a first and a second QoS setting of the multiple QoS settings, and (iii) the new value of the burst IOPS setting is assigned to the volume for the client.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during an automatic unplanned failover (AUFO). A computer-implemented method includes determining, with a second storage cluster, whether heartbeat information from one or more storage objects of a CG of a first set of CGs is received during a time period, determining an out of sync state for a data replication relationship between the CG of the first set of CGs and a mirrored CG of a second set of CGs when the heartbeat information is not received during the time period and sending a single bulk role change call with a cluster identifier from the second cluster to an external mediator to provide a role change from follower to leader in the second set of CGs.

Abstract: Systems and methods for an improved High-Availability (HA) resource reservation approach are provided. According to one embodiment, for a given cluster of greater than two nodes in which a number (f) of concurrent node failures are to be tolerated, more efficient utilization of resources for an HA system may be achieved by distributing HA reserved capacity across more than f nodes of the cluster rather than naively concentrating the HA reserved capacity in f nodes. As node failures are not a common occurrence, those of the nodes of the cluster having HA reserved capacity may allow for some bursting of one or more units of compute executing thereon unless or until f concurrent node failures occur, thereby promoting more efficient utilization of node resources.

Abstract: Methods and systems for co-locating journaling and data storage are provided. Separate journal and volume partitions may be maintained within each logical storage unit (e.g., Logical Unit Number (LUN)) of a distributed storage system. Journaling of metadata associated with write requests received from one or more clients may be distributed by identifying a destination logical storage unit to which data associated with a given write request is to be stored and causing the data and metadata to be persisted to disk by journaling the metadata and the data to respective portions of an active log within the journal partition of the destination logical storage unit. By using the same logical storage unit for both journaling of write requests and writing the data associated with such write requests, the bottleneck due to there being only a single device or storage unit handling all metadata for all write requests can be avoided.

Abstract: Systems and methods for supporting dynamic disk growth within a storage appliance are provided. According to one embodiment, a portion of a logical size of each of multiple disks (e.g., hyperscale disks or Logical Unit Numbers (LUNs)) are provisioned for use by a storage system as backing for respective file system disks. To accommodate growth, block numbers for the file system disks are pre-allocated within a sparse space of a contiguous sequence of block numbers corresponding to a number of blocks represented by the logical size. Metadata is maintained for the file system disks regarding a range of the pre-allocated block numbers that are available for use. Responsive to a triggering condition, the provisioned portion of a disk is increased and subsequently, responsive to detecting a change in a size of the disk by the storage system, a size of the corresponding file system disk is updated within the metadata.

Abstract: Systems and methods for preserving storage efficiency during restoration of data from the cloud are provided. In one embodiment, a CBMAP is maintained that maps cloud block numbers (CBNs) to respective corresponding block numbers of a volume of a data storage system in which previously restored data has been stored by a previously restored file. By making use of the CBMAP during the restoration process, storage of duplicate file data blocks on the volume may be avoided by sharing with a current file being restored a reference to the corresponding file data block previously stored on the volume and associated with the previously restored file. In addition to preserving storage efficiency, use of the CBMAP facilitates avoidance of repeated GET operations for data associated with CBNs previously retrieved from the cloud and stored to the volume, thereby reducing data access costs as well as latency of the restore operation.

Abstract: One or more systems, devices, computer program products, and/or computer-implemented methods provided herein to use a redundant array of disks. A system can comprise a memory that stores computer executable components, and a processor that executes the computer executable components stored in the memory, wherein the computer executable components can comprise a control component that directs, for n physical drives of a redundant array of disks (RAID) storing data for at least n logical volumes, log-structured writing of data of each logical volume of the at least n logical volumes vertically across chunks of only a single physical drive of the n physical drives, wherein the control component further directs writing of parity data at each of the physical drives, which parity data at each physical drive of the subset respectively corresponds to other ones of the physical drives of the n physical drives.

Abstract: Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.

Abstract: According to one embodiment, a computer implemented method comprises providing multiple channels between a first storage node and a second storage node with each channel having a separate network connection for packets of a transport layer session, assigning packets from each channel to a group of receive queues of the second storage node, continuously monitoring whether two or more channels of the multiple channels share a same receive queue of the second storage node, and sending a communication via a channel to the first storage node to indicate a dynamic change in a hash input field (e.g., a source port, a destination port, a source internet protocol (IP) address, and a destination IP address) when two or more channels of the multiple channels share a same receive queue of the second storage node.

Abstract: Systems and methods for reducing read application in a virtual storage system are provided. According to one embodiment, heuristic data may be tracked and utilized in real-time by a file system of the virtual storage system at the level of granularity of a volume, thereby allowing a fast path flag to be enabled/disabled at a volume level during various phases of operation of a workload. The heuristic data for a given volume may be indicative of a correlation between (i) data blocks stored on the given volume being located within a compressible zone of a zoned checksum scheme and (ii) the respective data blocks containing compressed data and a corresponding checksum. Based on the heuristic data, read requests may be selectively directed to the read path (e.g., a fast path or a slow path) expected to mitigate read amplification when data compression is enabled for a zoned checksum scheme.