Abstract: An illustrative method includes receiving a request to create a storage volume; identifying, based on the request and on a configuration file propagated among a plurality of storage nodes in a cluster, status indicators of the storage nodes; identifying, based on the request, a size of the storage volume; identifying, based on the request, a replication factor representing a number of storage nodes that are to be replicated within a cluster for the storage volume; identifying, based on one or more characteristics associated with the storage volume, a traffic priority for the storage volume, the traffic priority representing a hierarchy that determines and prioritizes which traffic is allocated to available hardware and network resources in a particular order; creating, based on the status indicators, the size, the replication factor, and the traffic priority, the storage volume on one or more of the plurality of storage nodes.

Abstract: A virtual storage volume may be implemented as one or more chunks stored on a set of storage nodes. The virtual storage volume may be dynamically adjusted by adjusting the one or more chunks that make up the virtual storage volume, without taking the virtual storage volume offline. Such dynamic volume adjustment may allow for increasing volume size without moving data unnecessarily. In addition or alternatively, such dynamic volume adjustment may free up storage space in a node with minimum movement of data.

Abstract: An illustrative method includes receiving a write request to write payload data to a virtual storage volume; transmitting the write request to a plurality of storage nodes each storing a replica of the virtual storage volume; acknowledging the write request only after a quorum of the storage nodes has stored the payload in their respective kernel memory; and flushing the payloads stored in each kernel memory to persistent storage only after a threshold number of outstanding write requests that have been acknowledged, but not yet flushed, has been reached, the flushing configured to optimize performance for synchronous workloads.

Abstract: An illustrative method includes determining whether a virtual storage volume is successfully mounted to a mount path associated with a compute node, the mount path being marked as read-only, marking, if the determining includes determining that the virtual storage volume is successfully mounted to the mount path, the mount path as writable, and maintaining, if the determining includes determining that the virtual storage volume is unsuccessfully mounted to the mount path, the mount path as read-only.

Abstract: An illustrative method includes receiving a request to create a storage volume; identifying, based on the request and on a configuration file propagated among a plurality of storage nodes in a cluster, status indicators of the storage nodes; identifying, based on the request, a size of the storage volume; identifying, based on the request, a replication factor representing a number of storage nodes that are to be replicated within a cluster for the storage volume; identifying, based on one or more characteristics associated with the storage volume, a traffic priority for the storage volume, the traffic priority representing a hierarchy that determines and prioritizes which traffic is allocated to available hardware and network resources in a particular order; creating, based on the status indicators, the size, the replication factor, and the traffic priority, the storage volume on one or more of the plurality of storage nodes.

Abstract: A virtual storage volume that includes storage space on potentially many different physical disks may be implemented as one or more chunks. This allows for dynamic volume storage adjustment. Dynamic volume storage adjustment allows for increasing volume size without moving data unnecessarily. In addition, dynamic volume storage adjustment also allows freeing up storage space in a node with minimum movement of data.

Abstract: Distributed storage systems are implemented with rule based rebalancing mechanisms. Methods includes steps for creating a set of rules for rebalancing data storage space in a storage node cluster, as well as steps for performing a rebalance operation across the storage node cluster using the set of rules. The distributed storage systems include one or more labels for storage pools and storage volumes.

Abstract: A controller for a storage system manages distribution of snapshots for one or more replicas of a dataset. In certain embodiments, the controller may perform steps including: determining, within a container system, a set of storage resources, where a volume is replicated on each storage resource of the set of storage resources, and wherein the set of storage resources comprises a first number of storage resources associated with a replication factor for the volume; determining, based on a replication policy and in response to an indication to generate a snapshot of the volume, a subset of storage resources, where the subset of storage resources comprises a second number of storage resources that is less than the replication factor; and generating, in accordance with the replication policy, a snapshot of the volume on each storage resource of the subset of storage resources.

Abstract: An illustrative computing system may include a master node that includes a prioritization module. The master node may be in communication with each of a plurality of containerized application nodes. The prioritization module may be configured to determine node prioritization information indicating a relative prioritization of the containerized application nodes for instantiating a designated containerized application, wherein the determining node prioritization information includes assigning a priority score to a candidate containerized application node, wherein higher priority is given based on a number of volumes stored on the candidate containerized application node that is designated for access by the designated containerized application.

Abstract: A clustered storage system may include potentially many different nodes. A node may mount a virtual storage volume for the use of a container application at the node. The node may receive a request from a different node and respond by indicating whether the virtual storage volume is in active use. In this way, the clustered storage system may safely but forcibly unmount a virtual storage volume having a failed or hanging mount point so that the volume may be mounted on a different node.

Abstract: An example method includes identifying, based on a request to write data to a volume provided by storage nodes in a storage node cluster, a cluster hierarchy associated with the storage node cluster, the cluster hierarchy identifying storage characteristics of the storage nodes, wherein the cluster hierarchy is based on data characterizing performance and storage capabilities of the storage nodes; based on the request, selecting a storage node for storing the data in the volume, the selecting being based, at least in part, on the storage characteristics identified by the cluster hierarchy and on one or more data distribution parameters associated with the storage volume; and transmitting the data to the storage node.

Abstract: In some embodiments, storage path routing in a container system includes: determining, within the container system, a storage operation directed to a storage resource; determining one or more conditions associated with a plurality of network paths to the storage resource, where the plurality of network paths includes a first network path through a storage area network and a second network path through a container system communication network; and routing, based on the one or more conditions, the storage operation to the storage resource using the first network path.

Abstract: A clustered storage system may include potentially many different nodes, each including a storage driver and a scheduler agent. A node may mount a virtual storage volume for the use of a container application at the node. When a master node receives a request to instantiate a new containerized application, a scheduler at the master node may determine prioritization information for the new containerized application. The prioritization information may prioritize one or more nodes at which a virtualized storage volume employed by the application container instance is mounted.

Abstract: An illustrative distributed storage system detects a request to resize an online volume that provides persistent storage for a containerized application running in a container system, wherein the online volume is backed by a virtual block device created in device namespace using a device mapper virtualization of data blocks of one or more storage devices. The distributed storage system performs, in response to the request, a resize process to resize the online volume. The resizing of the online volume includes resizing the virtual block device. In some examples, a file system implemented on the virtual block device is also resized while the online volume is online. The storage system may manage inflight IOs over the virtual block device and the online volume such that the resize process is performed without error.

Abstract: An illustrative method includes receiving a write request to write payload data to a virtual storage volume; transmitting the write request to a plurality of storage nodes each storing a replica of the virtual storage volume; acknowledging the write request only after a quorum of the storage nodes has stored the payload in their respective kernel memory; and flushing the payloads stored in each kernel memory to persistent storage only after a threshold number of outstanding write requests that have been acknowledged, but not yet flushed, has been reached, the flushing configured to optimize performance for synchronous workloads.

Abstract: Systems, devices, and methods are disclosed herein for containerized scalable storage applications. Methods may include instantiating an application instance based on a plurality of application instance parameters, the application instance being configured to utilize a plurality of storage volumes implemented in a storage cluster. Methods may also include enumerating a plurality of unattached storage volumes included in the cluster associated with the application instance, the plurality of unattached storage volumes having a plurality of underlying physical storage devices, and the plurality of unattached storage volumes being identified based on a plurality of application instance parameters. The methods may further include attaching at least some of the plurality of unattached storage volumes to the application instance, wherein the attaching enables the application instance to access data stored in the attached storage volumes.

Abstract: Mechanisms are provided for adjusting a configuration of data stored in a storage system. According to various embodiments, a storage module may be configured to store a configuration of data. A processor may be configured to identify an estimated performance level for the storage system based on a configuration of data stored on the storage system. The processor may also be configured to transmit an instruction to adjust the configuration of data on the storage system to meet the service level objective when the estimated performance level fails to meet a service level objective for the storage system.

Abstract: A synchronous write operation is decomposed into an asynchronous write operation and a separate flush operation. The payload of the write operation is sent a plurality of storage nodes hosting replicas of a virtual storage volume. The write operation is only acknowledged after the payload is stored in kernel memory in a quorum of storage nodes hosting replicas. Write operations are only flushed to persistent storage after a predetermined window of time has transpired or after a threshold number of outstanding write operations that have been acknowledged, but not yet flushed, has been reached.

Abstract: Techniques and mechanisms described herein facilitate the execution of a software program container having a plurality of layers. Each layer may include a plurality of files. For instance, a base layer may include a version of an operating system, while an upper layer may include a software program configured to run within the operating system. Different layers may store, create, or modify the same file. However, the precedence of the version of the data for the file may be based on an ordering of the layers. For example, the version of the file in a higher layer should supersede the version of the file in a lower layer. In some implementations, a privileged storage container that runs atop a virtualization layer may manage storage resources for other containers. The privileged storage container may store each file in a software program container as a snapshotable object.

Abstract: An illustrative storage system provides distributed storage for use by containerized applications running in a container system. In some implementations, this includes converting data blocks of storage resources to distributed persistent storage such as thin-provisioned volumes for use by the containerized applications. In some implementations, the conversion includes using a device mapper to virtualize the data blocks of the storage resources to create a virtual storage pool backing datastore in which the thin-provisioned volumes are created.