Abstract: Systems and methods for verifying an executable portion of a published cloud image represents an unaltered version of an executable portion of a corresponding original cloud image are provided. In one embodiment, modification of a predefined portion of a cloud image by a cloud provider prior to its publication via a marketplace of the cloud provider is proactively addressed as part of (i) an automated signing process performed by a software publisher on the original cloud image prior to delivery to the cloud provider and (ii) a corresponding background verification process performed on the published cloud image on behalf of users by a management platform. The signing and verification processes are operable to exclude the predefined portion when creating their respective digests, thereby allowing the signed digest created prior to the modification to remain useful as part of a subsequent digest comparison performed by the verification process.

Abstract: Data traffic management in a computing environment utilizing direct memory access functionality is disclosed. A management agent is configured to operate within a storage node. The management agent includes a storage interface to communicate with a first set of storage devices, a management memory interface to communicate with a first management memory, and an interconnect (IC) interface to communicate with a remote peer node. The management agent controls data traffic between the storage node and the peer node to provide at least mirroring of the first management memory to the peer node and mirroring of a second management memory on the peer node to the storage node. The management agent further controls the data traffic using a traffic control approach selected based on at least a performance evaluation of an IC fabric accessible via the IC interface.

Abstract: Techniques are provided for a recovery process with selective ordering and concurrent operations in order to recover from a failure. Representations of active log structures are rebuilt within memory according to ordering values assigned to I/O operations logged within the active log structures. Representation of certain active log structures may be concurrently rebuilt based upon the active log structures comprising I/O operations that are non-overlapping within a distributed file system, have no dependencies, relate to different services, and/or target independent files. Representation of stale log structures are concurrently rebuilt within memory. While rebuilding the log structures and executing the I/O operations, a key value map is concurrently rebuilt within the memory for locating data of the I/O operations. Concurrent operations during the recovery process reduces the time to complete the recovery process, and thus reduces client downtime during the recovery process.

Abstract: Techniques are provided for implementing additional compression for existing compressed data. Format information stored within a data block is evaluated to determine whether the data block is compressed or uncompressed. In response to the data block being compressed according to a first compression format, the data block is decompressed using the format information. The data block is compressed with one or more other data blocks to create compressed data having a second compression format different than the first compression format.

Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Abstract: Systems, methods, and machine-readable media are disclosed for determining an optimal location for creating a new volume (or volumes) on a storage platform and placing the volume at the determined location. A resource tracker collects resource use and/or limits data and stores it in a database. A volume placement service receives a volume deployment specification having constraints for creating a new volume, particularly including instructions relating to relationships between volumes to be created. The volume placement service retrieves the data from the database. The volume placement service identifies an optimal location for the volume based at least in part on given constraints from the specification and the resource usage data. The system places the requested volume at the determined location.

Abstract: Methods, non-transitory machine readable media, and computing devices that manage storage operations directed to dual-port solid state disks (SSDs) coupled to multiple hosts are disclosed. With this technology, context metadata comprising a checksum is retrieved based on a first physical address mapped, in a cached zoned namespace (ZNS) mapping table, to a logical address. The logical address is extracted from a request to read a portion of a file. A determination is made when the checksum is valid based on a comparison to identification information extracted from the request and associated with the file portion. At least the first physical address is replaced in the cached ZNS mapping table with a second physical address retrieved from an on-disk ZNS mapping table, when the determination indicates the checksum is invalid. The file portion retrieved from a dual-port SSD using the second physical address is returned to service the request.

Abstract: Multi-site distributed storage systems and computer-implemented methods are described for improving a resumption time of input/output (I/O) operations during a common snapshot process for storage objects. A computer-implemented method comprises performing a baseline transfer from at least one storage object of a first storage node to at least one replicated storage object of a second storage node, starting the common snapshot process including stop processing of I/O operations, performing a snapshot create operation on the primary storage site for the at least one storage object of the first storage node, resuming processing of I/O operations, and assigning a new universal unique identifier (UUID) to the at least one storage object of the second storage node after resuming processing of I/O operations with the new UUID to identify when file system contents are different than the baseline transfer.

Abstract: Approaches and mechanisms for cloning a file are described. A first node requests a clone of a file at a time when it also requests an exclusive delegation of the original file from a second node where the original file is stored. The second node marks the original file as delegated to the first node and the second node records an intent to create the clone file and a delegation record for the clone file. The second node creates the clone file. The delegation of and the identity of the clone file are returned to the first node. The first node marks in the delegation record that the clone file was committed in response to modification. If the clone file was committed the delegation is cleared and the clone file is kept, and if the clone file was not committed, the delegation is cleared, and the clone file is deleted.

Abstract: Techniques are provided for input/output operations per second (IOPS) and throughput monitoring for dynamic and/or optimal resource allocation. These techniques provide automated monitoring of resources, such as memory and processor utilization by a container accessing a volume. The automated monitoring is performed in order to generate and execute intelligent recommendations for improved resource utilization. Resource allocations can be scaled up to meet I/O load demand and satisfy service level agreements (SLAs). Resource allocations can be scaled down or adjusted to conserve resources, such as by consolidating containers or pods hosted in multiple virtual machines into a single virtual machine and decommissioning virtual machines no longer hosting containers or pods.

Abstract: Systems, methods, and data structures for providing a file system with object versioning support are provided. Rather than adding object records for each version of an object to a chapter record, in one example, the chapter record may be limited to a single object record for the object including: (i) an object name of the object; (ii) an object file handle identifying an index of a file containing data of a current version of the multiple versions of the object; and (iii) a version table file handle identifying an index of a file containing a version table. In this manner, enumeration of objects associated with a given chapter may be performed more efficiently and prior versions of objects may be maintained within the version table without disproportionate growth of chapter records and without increasing the search depth with objects that are not referenced by the search at issue.

Abstract: Techniques are provided for implementing a distributed control plane to facilitate communication between a container orchestration platform and a distributed storage architecture. The distributed storage architecture hosts worker nodes that manage distributed storage that can be made accessible to applications within the container orchestration platform through the distributed control plane. The distributed control plane includes control plane controllers that are each paired with a single worker node of the distributed storage architecture. The distributed control plane is configured to selectively route commands to control plane controllers that are paired with worker nodes that are current owners of objects targeted by the commands. If ownership of an object has changed from one worker node to another worker node, then subsequent commands will be re-routed to a control plane controller paired with the other worker node now owning the object.

Abstract: Systems and methods for scaling application and/or storage system functions of a distributed storage system based on a heterogeneous resource pool are provided. According to one embodiment, the distributed storage system has a composable, service-based architecture that provides scalability, resiliency, and load balancing. The distributed storage system includes a cluster of nodes each potentially having differing capabilities in terms of processing, memory, and/or storage. The distributed storage system takes advantage of different types of nodes by selectively instating appropriate services (e.g., file and volume services and/or block and storage management services) on the nodes based on their respective capabilities. Furthermore, disaggregation of these services, facilitated by interposing a frictionless layer (e.g.

Abstract: In one embodiment, distributed data storage systems and methods are described for integrating a change tracking manager with scalable databases. According to one embodiment, a computer implemented method comprises managing storage of objects and continuously tracking changes of the objects in a distributed object storage database, creating a record for an object having an object name, the object being stored in a bucket of the distributed object storage database, linking the bucket to a peer bucket based on a directive, generating a peer marker field for the record to store one peer marker of multiple different peer markers depending on a relationship between the bucket and the peer bucket; and automatically adding a work item for the object to the secondary index of a chapter database based on the record being created in the bucket and the peer marker for the peer bucket.

Abstract: Systems and methods that make use of cluster-level redundancy within a distributed storage system to address various node-level error scenarios are provided. Rather than making use of a generalized one-size-fits-all approach in an effort to reduce complexity, an approach tailored to the node-level error scenario at issue may be performed to avoid doing more than necessary. According to one embodiment, responsive to identification of a failed RAID stripe by a node of a cluster of a distributed storage management system, for each block ID of multiple block IDs associated with the failed RAID stripe, a data block is restored corresponding to the block ID by reading the data block from another node of the cluster having a redundant copy of the data block; and writing the redundant copy of the data block to a storage area of the node that is unaffected by the failed RAID stripe.

Abstract: Systems and methods for automated remediation of deviations from best practices in the context of a data management storage system are provided. Deployed assets of a storage solution vendor may periodically deliver telemetry data to the vendor. The telemetry data may be processed by an AIOps platform to perform predictive analytics and arrive at “community wisdom” from the vendor's installed base. In one embodiment, an insight-based approach is used to facilitate risk detection and remediation including proactively addressing deviations from best practices before they turn into more serious problems. Based on the community wisdom and making a rule set and a remediation set derived therefrom available for use by auto-healing service associated with a customer's storage system, a risk (e.g., a deviation from a best practice) to which the storage system is exposed may be determined and a corresponding remediation may be deployed to address or mitigate the risk.

Abstract: One or more techniques and/or computing devices are provided for resynchronization. For example, a request may be received to create pseudo snapshots of a first consistency group, hosted by a first storage controller, and a second consistency group, hosted by a second storage controller, having a synchronous replication relationship with the first consistency group. Incoming client write requests are logged within an intercept tracking log at the first storage controller. After a first drain without hold of incoming write requests is performed, a first pseudo common snapshot of the second consistency group is created. After a second drain without hold of incoming write operations is performed, a second pseudo common snapshot of the first consistency group and the intercept tracking log is created. The pseudo snapshots and the intercept tracking log (e.g., indicating a delta between the pseudo snapshots) are used to resynchronize the first and second consistency groups.

Abstract: Techniques are provided for compacting indirect blocks. For example, an object is represented as a structure comprising data blocks within which data of the object is stored and indirect blocks comprising block numbers of where the data blocks are located in storage. Block numbers within a set of indirect blocks are compacted into a compacted indirect block comprising a base block number, a count of additional block numbers after the base block number in the compacted indirect block, and a pattern of the block numbers in the compacted indirect block. The compacted indirect block is stored into memory for processing access operations to the object. Storing compacted indirect blocks into memory allows for more block numbers to be stored within memory. In this way, the block numbers are read from memory is faster than loading the block numbers from disk.

Abstract: Techniques are provided for combining data block and checksum block I/O into a single I/O operation. Many storage systems utilize checksums to verify the integrity of data blocks stored within storage devices managed by a storage stack. However, when a storage system reads a data block from a storage device, a corresponding checksum must also be read to verify integrity of the data in the data block. This results in increased latency because two read operations are being processed through the storage stack and are being executed upon the storage device. To reduce this latency and improve I/O operations per second, a single combined I/O operation corresponding to a contiguous range of blocks including the data block and the checksum block is processed through the storage stack instead of two separate I/O operations. Additionally, I/O operation may be combined into a single request that is executed upon the storage device.

Abstract: The disclosed technology enables quicker initialization of a new master node for a cluster when a previous master node fails by tracking node state in the cluster prior to being designated the new master node. In a particular example, a method includes, in a first node, designated as a current master node for the cluster, managing the cluster based on states of the nodes determined by the first node. While the first node is designated the master node, the method includes each of the nodes collecting, and storing locally, the states of the nodes. In response to a failure of the first node, the method includes selecting a second node of the nodes a new master node. Upon being designated the new master node, the method includes the second node managing the cluster of nodes based on the states of the nodes that the second node collected and stored locally.