Abstract: Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Abstract: Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Abstract: A data location table master system generates a master data location table storing associations of tokens with storage nodes for varying responsibility levels. When the master data location table is updated, the data location table master system updates storage nodes affected by the update as well as other storage nodes and application nodes in the system. Then, the storage nodes and the application nodes store a copy of the master data location table. A token migration and synchronization process reallocates data object storage among the storage nodes based on the updated master data location table.

Abstract: Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Abstract: A write request for the data object is received. The data object is stored at one or more of a plurality of storage nodes according to the write request. A storage manager catalog that maps a data object identification (DOID) for the data object is updated with an actual storage location of the data object, the DOID for a corresponding data object being calculated based on content of the data object, a plurality of data objects being organized into one or more storage volumes. The storage manager catalog includes a count of a number of instances of a data object that is currently referenced for each of the one or more storage volumes.

Abstract: A stream verification system for a distributed message queue system with metric collectors on each producer and consumer. A producer time stamp allows correlation of sent and received messages. Verification reports are organized by a message topic. A cumulative checksum allows detection of missing or corrupted messages. Verification messages are used to determine if a zero message report means no messages were sent, or rather that the messages weren't received.

Abstract: A stream verification system for a distributed message queue system with metric collectors on each producer and consumer. A producer time stamp allows correlation of sent and received messages. Verification reports are organized by a message topic. A cumulative checksum allows detection of missing or corrupted messages. Verification messages are used to determine if a zero message report means no messages were sent, or rather that the messages weren't received.

Abstract: A stream verification system for a distributed message queue system with metric collectors on each producer and consumer. A producer time stamp allows correlation of sent and received messages. Verification reports are organized by a message topic. A cumulative checksum allows detection of missing or corrupted messages. Verification messages are used to determine if a zero message report means no messages were sent, or rather that the messages weren't received.

Abstract: A storage cluster in a computing network can operate according to a service-level agreement (e.g., a set of performance conditions). The network can receive a request to register a new storage volume with the storage cluster. In response, the network may evaluate the current extent of replication of data in the storage cluster and how adding the new volume may affect the performance of the cluster to determine whether to register the new volume with the storage cluster.

Abstract: A data location table master system generates a master data location table storing associations of tokens with storage nodes for varying responsibility levels. When the master data location table is updated, the data location table master system updates storage nodes affected by the update as well as other storage nodes and application nodes in the system. Then, the storage nodes and the application nodes store a copy of the master data location table. A token migration and synchronization process reallocates data object storage among the storage nodes based on the updated master data location table.

Abstract: A storage cluster in a computing network can operate according to a service-level agreement (e.g., a set of performance conditions). The network can receive a request to register a new storage volume with the storage cluster. In response, the network may evaluate the current extent of replication of data in the storage cluster and how adding the new volume may affect the performance of the cluster to determine whether to register the new volume with the storage cluster.

Abstract: A write request for the data object is received. The data object is stored at one or more of a plurality of storage nodes according to the write request. A storage manager catalog that maps a data object identification (DOID) for the data object is updated with an actual storage location of the data object, the DOID for a corresponding data object being calculated based on content of the data object, a plurality of data objects being organized into one or more storage volumes. The storage manager catalog includes a count of a number of instances of a data object that is currently referenced for each of the one or more storage volumes.

Abstract: New storage volumes are registered to a data storage environment. Registering new storage volumes is controlled based on the performance requirements of the storage volumes compared to the capacity of the data storage environment.

Abstract: Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Abstract: A stream verification system for a distributed message queue system with metric collectors on each producer and consumer. A producer time stamp allows correlation of sent and received messages. Verification reports are organized by a message topic. A cumulative checksum allows detection of missing or corrupted messages. Verification messages are used to determine if a zero message report means no messages were sent, or rather that the messages weren't received.

Abstract: A data location table master system generates a master data location table storing associations of tokens with storage nodes for varying responsibility levels. When the master data location table is updated, the data location table master system updates storage nodes affected by the update as well as other storage nodes and application nodes in the system. Then, the storage nodes and the application nodes store a copy of the master data location table. A token migration and synchronization process reallocates data object storage among the storage nodes based on the updated master data location table.

Abstract: Storage space is reclaimed by cleaning and compacting data objects where data objects are stored by immutable storage. A storage area of which space needs to be reclaimed is identified. Active and stale data objects stored in a storage area are identified, and only active data objects are transferred to a shadow storage area from the storage area when recovering storage space. I/O operations can be fulfilled from the storage area and the shadow storage area. Compaction requests and I/O requests are throttled according to QOS parameters. Recovery of storage space does not cause a failure to meet performance requirements for any storage volume.

Abstract: Data objects are stored on storage devices, taking into account service level agreements or other quality of service parameters. In one aspect, data objects grouped into storage volumes. In addition, the storage devices are classified into different level storage tiers, where higher level storage tiers have higher performance and lower level storage tiers have lower performance. Ranks for the data objects are calculated, based on both a data usage pattern for the data object (e.g., recency and frequency) and on quality of service (QOS) parameters for the storage volume containing the data object. Examples of QOS parameters include service level agreements, priority, minimum and maximum input/output operations per second. The data objects are then stored on storage devices, based on the data objects' ranks and the storage devices' storage tiers.

Abstract: New storage volumes are registered to a data storage environment. Registering new storage volumes is controlled based on the performance requirements of the storage volumes compared to the capacity of the data storage environment.

Abstract: An input/output (I/O) request is dispatched. A determination is made regarding a storage volume to service. A determination is made regarding whether an actual disk throughput exceeds a first threshold rate. The first threshold rate exceeds a reserved disk throughput. Responsive to determining that the actual disk throughput exceeds the first threshold rate, a first storage volume is selected based on credits or based on priority. Responsive to determining that the actual disk throughput does not exceed the first threshold rate, a second storage volume is selected based on guaranteed minimum I/O rate. An I/O request queue associated with the determined storage volume is determined. An I/O request is retrieved from the determined I/O request queue. The retrieved I/O request is sent to a persistence layer that includes the selected storage volume.