Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include configuring distributed storage system resources for a distributed storage system. Examples of the storage system resources include a plurality of network segments, one or more network devices coupled to the network, and multiple nodes coupled to the network, the nodes including both frontend and backend nodes. Upon receiving, by a given frontend node in the distributed storage system, an input/output (I/O) request, one or more of the distributed storage system resources required to process the I/O request are identified, and a respective load that the I/O request will generate on each of the identified distributed storage system resources is calculated. The distributed storage system processes the I/O request upon detecting that the respective loads are less than respective available capacities of the identified distributed storage system resources.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include defining a plurality of failure domains for sets of storage devices in a storage facility, and defining, using the failure domains, one or more limitations for distributing data on the storage devices. Upon identifying a data distribution configuration for a software defined storage system that is compliant with the one or more limitations, the identified data distribution configuration can be presented to a user. The failure domains may include physical failure domains, logical failure domains, or a combination of physical and logical failure domains, and the limitations may include mandatory limitations or a combination of mandatory and non-mandatory limitations.

Abstract: Methods, systems, and computer program product embodiments for controlling congestion in a storage grid, by a processor device, are provided. In a storage grid, a storage request transmit queue length of storage requests submitted by a plurality of nodes is monitored, the storage request transmit queue comprising a queue holding the storage requests unable to be transmitted through a current transmission control protocol/internet protocol (TCP/IP) socket buffer of the storage grid. Upon reaching at least one of a certain threshold, a watermarked message indicating a specific one of the plurality of nodes is congested is transmitted to upper layer grid nodes in a hierarchy of the plurality of nodes transmitting the storage requests, and the upper layer grid nodes each alter the storage requests based upon the watermarked message.

Abstract: A technique for supporting in-place updates in a data storage system includes in response to garbage collection for a logical block address (LBA) being indicated, determining whether an in-place update to the LBA is pending. In response to one or more in-place updates to the LBA being pending prior to the garbage collection for the LBA being indicated, the garbage collection for the LBA is initiated following completion of the one or more in-place updates to the LBA. In response to the one or more in-place updates to the LBA not being pending prior to the garbage collection for the LBA being indicated, the garbage collection for the LBA is completed prior to any subsequent in-place update to the LBA that occurs subsequent to initiation and prior to completion of the garbage collection for the LBA.

Abstract: A storage system (system) includes two storage devices (first device and second device). The first device stores encrypted user data prior to being enrolled with an external key server. The system generates a device access key (DAK) and a device encryption key (DEK) used to encrypt such user data and encrypts the DEK with the DAK to generate an encrypted DEK (DEK?). The system stores DEK? in the second device and stores DAK in the first device. The system enrolls the first device with the key server and receives a secure encryption key (SEK). The system obtains DEK? and DAK, which are subsequently deleted from the first and second storage device, respectively. A new DAK? is generated utilizing SEK and a first device identifier. The DEK is encrypted utilizing DAK? to form DEK?. The system indicates DAK? is an externally derived key and saves DEK? to the second device.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include initializing a plurality of first layer software defined storage (SDS) clusters, each of the first layer SDS clusters including multiple storage nodes executing in separate independent virtual machines on respective separate independent servers. A second layer SDS cluster including a combination of the first layer SDS clusters is defined, and using a distributed management application, the second layer SDS cluster is managed, the distributed management application including multiple management nodes executing on all of the servers. In one embodiment, each of the storage nodes executes within a separate independent virtual machine, and each of the virtual machines executes a given management node. In another embodiment, each of the storage nodes executes within a separate independent first virtual machine, and each server includes a second virtual machine that executes a given management node.

Abstract: In one embodiment of the present invention, a server is created, and a computer readable storage media is included in the server. An adapter is connected to the server, a set of packets is received by the adapter from a network, and the network and a repository are interfaced by the adapter. A firmware is stored on the adapter, and the adapter is controlled by the firmware. A management component is stored on the computer readable storage media, and the server is managed by the management component. A driver is stored on the management component, and the driver communicates with the adapter. A set of filters that controls transmission of the set of packets is created, and each filter in the set of filters has a set of filter rules. The set of filters is stored on the server.

Abstract: Embodiments for discovering read failures in a data storage system. Data read from underlying media of the data storage system is validated by initializing a prefix of a buffer used to read the data with a first sequence of bytes of a predefined number. The first sequence of bytes is compared to a sequence of actual bytes of the data read into the buffer. When the first sequence of bytes returns an equal value to the sequence of the actual bytes of the data read, a second sequence of bytes of the predefined number is generated and compared to a subsequent data read to determine the validity.

Abstract: In one embodiment of the present invention, a server is created, and a computer readable storage media is included in the server. An adapter is connected to the server, a set of packets is received by the adapter from a network, and the network and a repository are interfaced by the adapter. A firmware is stored on the adapter, and the adapter is controlled by the firmware. A management component is stored on the computer readable storage media, and the server is managed by the management component. A driver is stored on the management component, and the driver communicates with the adapter. A set of filters that controls transmission of the set of packets is created, and each filter in the set of filters has a set of filter rules. The set of filters is stored on the server.

Abstract: In one embodiment of the present invention, a server is created, and a computer readable storage media is included in the server. An adapter is connected to the server, a set of packets is received by the adapter from a network, and the network and a repository are interfaced by the adapter. A firmware is stored on the adapter, and the adapter is controlled by the firmware. A management component is stored on the computer readable storage media, and the server is managed by the management component. A driver is stored on the management component, and the driver communicates with the adapter. A set of filters that controls transmission of the set of packets is created, and each filter in the set of filters has a set of filter rules. The set of filters is stored on the server.

Abstract: A mega cluster storage system includes clusters of multiple storage modules. Each module is able to access a portion of the data within the mega cluster and serves as a proxy in order for another storage module to access the remaining portion of the data. A cluster is assigned to a unique cluster volume and all the data within the cluster volume is accessible by all of the modules within the cluster. Each host connection to the mega cluster is associated with a particular cluster volume. A module that receives a host I/O request determines whether the I/O request should be satisfied by a module within its own cluster or be satisfied by a module within a different cluster. The module may forward the I/O request to a module within a different cluster as indicated by a distribution data structure that is allocated and stored within each storage module.

Abstract: A technique for supporting in-place updates in a data storage system includes in response to garbage collection for a logical block address (LBA) being indicated, determining whether an in-place update to the LBA is pending. In response to one or more in-place updates to the LBA being pending prior to the garbage collection for the LBA being indicated, the garbage collection for the LBA is initiated following completion of the one or more in-place updates to the LBA. In response to the one or more in-place updates to the LBA not being pending prior to the garbage collection for the LBA being indicated, the garbage collection for the LBA is completed prior to any subsequent in-place update to the LBA that occurs subsequent to initiation and prior to completion of the garbage collection for the LBA.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include initializing multiple computing nodes to execute one or more software applications that provide a computing service, each of the computing nodes having a respective set of configuration data. Examples of the services include storage services, computational services and web service. Subsequent to deploying the multiple computing nodes as a software defined computing (SDC) cluster, an additional computing node is initialized using a given set of configuration data, and the additional computing node is added to the SDC cluster. In some embodiments, the additional node can be initialized and added to the SDC cluster in response to detecting a condition in the SDC cluster. In alternative embodiments, the additional node can be initialized and added to the SDC cluster in response to receiving a request to add an additional service to the SDC cluster.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include defining, for a computer system, one or more system tests, and performing each of the system tests on the computer system. Each of the system tests includes one or more system commands, and in some embodiments, a given system test may be performed by executing one or more system commands while the computer system simultaneously processes additional system commands that perform input/output operations. Upon detecting a failure of a given system command in a given system test to execute correctly, the computer system can be configured to suppress execution of the given system command. In some embodiments, the computer system includes a hypervisor controlling a virtual machine configured as a node of a software defined storage system, and the failure can be a disruption of the hypervisor or the virtual machine.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include initializing multiple computing nodes to execute one or more software applications that provide a computing service, each of the computing nodes having a respective set of configuration data. Examples of the services include storage services, computational services and web service. Subsequent to deploying the multiple computing nodes as a software defined computing (SDC) cluster, an additional computing node is initialized using a given set of configuration data, and the additional computing node is added to the SDC cluster. In some embodiments, the additional node can be initialized and added to the SDC cluster in response to detecting a condition in the SDC cluster. In alternative embodiments, the additional node can be initialized and added to the SDC cluster in response to receiving a request to add an additional service to the SDC cluster.

Abstract: Embodiments for disaster recovery (DR) configuration management in a Software-defined Storage (SDS) environment. An orchestration mechanism is used to automate a deployment and/or a configuring of two or more storage clusters for DR by arranging, in one step, a mirroring session between the two or more storage clusters in the SDS environment.

Abstract: Embodiments for network management in a Software-defined Storage (SDS) environment. Remote mirroring connectivity and data transfer is optimized in the SDS environment by establishing each of a plurality of nodes in a local storage array and a remote storage array as a gateway node. Data residing on a local gateway node is transferred directly to a remote gateway node owning the data, thereby avoiding interconnect data hops and improving datacenter network utilization.

Abstract: Methods, systems, and computer program product embodiments for controlling congestion in a storage grid, by a processor device, are provided. In a storage grid, a storage request transmit queue length of storage requests submitted by a plurality of nodes is monitored, the storage request transmit queue comprising a queue holding the storage requests unable to be transmitted through a current transmission control protocol/internet protocol (TCP/IP) socket buffer of the storage grid. Upon reaching at least one of a certain threshold, a watermarked message indicating a specific one of the plurality of nodes is congested is transmitted to upper layer grid nodes in a hierarchy of the plurality of nodes transmitting the storage requests, and the upper layer grid nodes each alter the storage requests based upon the watermarked message.

Abstract: Improving storage device efficiency during data replication. An asynchronous data replication process is initiated between a source computer and a target computer over a network. Performance of the target computer and a storage device of the target computer is monitored. At least one portion of data stored on the source computer is identified for the data replication, wherein the at least one portion of the data is organized in a data block. Whether to modify a size of the data block is determined, based on the monitored performance of the target computer. Responsive to determining to modify the size of the data block, based on the monitored performance of the target computer, the size of the data block is modified.

Abstract: Methods, computing systems and computer program products implement embodiments of the present invention that include configuring distributed storage system resources for a distributed storage system. Examples of the storage system resources include a plurality of network segments, one or more network devices coupled to the network, and multiple nodes coupled to the network, the nodes including both frontend and backend nodes. Upon receiving, by a given frontend node in the distributed storage system, an input/output (I/O) request, one or more of the distributed storage system resources required to process the I/O request are identified, and a respective load that the I/O request will generate on each of the identified distributed storage system resources is calculated. The distributed storage system processes the I/O request upon detecting that the respective loads are less than respective available capacities of the identified distributed storage system resources.