Abstract: Methods for estimating cost savings in a storage system using an external host system. One method includes accessing over a communication network data from a unit of storage of a data storage system, wherein each of the blocks of data is uncompressed. A plurality of blocks is parsed from the data. A plurality of fingerprints is generated from the blocks using a hash algorithm. A deduplication ratio is estimated for the plurality of blocks stored in the unit of storage using a hyperloglog algorithm and a first plurality of buckets compartmentalizing the plurality of blocks, wherein the first plurality of buckets is defined by precision bits of the plurality of fingerprints. An effective compression ratio is estimated for the plurality of blocks stored in the unit of storage using the hyperloglog algorithm and a second plurality of buckets compartmentalizing the plurality of blocks, wherein the second plurality of buckets is defined by ranges of compression ratios.

Abstract: Methods for estimating cost savings in a storage system using an external host system. One method includes accessing over a communication network data from a unit of storage of a data storage system, wherein each of the blocks of data is uncompressed. A plurality of blocks is parsed from the data. A plurality of fingerprints is generated from the blocks using a hash algorithm. A deduplication ratio is estimated for the plurality of blocks stored in the unit of storage using a hyperloglog algorithm and a first plurality of buckets compartmentalizing the plurality of blocks, wherein the first plurality of buckets is defined by precision bits of the plurality of fingerprints. An effective compression ratio is estimated for the plurality of blocks stored in the unit of storage using the hyperloglog algorithm and a second plurality of buckets compartmentalizing the plurality of blocks, wherein the second plurality of buckets is defined by ranges of compression ratios.

Abstract: Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Abstract: According to examples, a data storage system may include a plurality of storage arrays of a cloud volume provider (CVP), in which the plurality of storage arrays is a plurality of logical volumes. The data storage system may also include a CVP portal to link a first compute instance of a first cloud service provider (CSP) with a first logical volume over a network. A first application executing on the first compute instance may access the first logical volume for storage and the first CSP may provide at least one compute instance for a corresponding entity.

Abstract: Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Abstract: Methods, systems, and computer programs are presented for dynamic adaptive compression in a storage device. One method includes operations for setting a percentage factor for utilizing a first and a second compression algorithms, and for receiving incoming blocks in the memory of the storage device. The incoming blocks are compressed before being sent to permanent storage, where a portion of the incoming blocks are compressed with the first compression algorithm based on the percentage factor, and the remainder is compressed with the second compression algorithm. Further, the method includes determining that a processor utilization rate, of a processor in the storage device, is below a first predetermined threshold, and decreasing, in response to the determining, the percentage factor to decrease the portion of the incoming blocks that are compressed with the first compression algorithm, while the remainder of the incoming blocks is compressed with the second compression algorithm.

Abstract: Methods, systems, and computer programs are presented for dynamic adaptive compression in a storage device. One method includes operations for setting a percentage factor for utilizing a first and a second compression algorithms, and for receiving incoming blocks in the memory of the storage device. The incoming blocks are compressed before being sent to permanent storage, where a portion of the incoming blocks are compressed with the first compression algorithm based on the percentage factor, and the remainder is compressed with the second compression algorithm. Further, the method includes determining that a processor utilization rate, of a processor in the storage device, is below a first predetermined threshold, and decreasing, in response to the determining, the percentage factor to decrease the portion of the incoming blocks that are compressed with the first compression algorithm, while the remainder of the incoming blocks is compressed with the second compression algorithm.

Abstract: A cache system for a storage device includes a solid state drive (SSD), a random access memory (RAM), and a cache control device. The cache control device is configured to: retrieve data from the storage device in response to a request to read data from the storage device, store at least some of the data in one or both of (i) the SSD and (ii) the RAM, when storing the at least some of the data to the RAM, write to the RAM non-sequentially with respect to a memory space of the RAM, and when storing the at least some of the data in the SSD, write to the SSD sequentially with respect to a memory space of the SSD. The cache control device comprises an SSD interface device configured to allocate memory for storing data in the SSD sequentially with respect to the memory space of the SSD.

Abstract: Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Abstract: A memory system includes a cache module configured to store data. A duplicate removing filter module is separate from the cache module. The duplicate removing filter module is configured to receive read requests and write requests for data blocks to be read from or written to the cache module, selectively generate fingerprints for the data blocks associated with the write requests, selectively store at least one of the fingerprints as stored fingerprints and compare a fingerprint of a write request to the stored fingerprints.

Abstract: Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Abstract: Methods and systems are presented for allocating CPU cycles among processes in a storage system. One method includes operations for maintaining segments in a first memory, each segment including blocks, and for maintaining a block temperature for each block in a second memory. The first memory is a read-cache where one segment is written at a time, and each block is readable from the first memory without reading the corresponding complete segment. The block temperature is based on the frequency of access to the respective block, and a segment temperature is based on the block temperature of its blocks. Additionally, the segment with the lowest segment temperature is selected for eviction from the second memory, and blocks in the selected segment with a block temperature greater than a threshold temperature are identified. The selected segment is evicted, and a segment with the identified blocks is written to the first memory.

Abstract: A cache system for a storage device includes a solid state drive (SSD), a random access memory (RAM), and a cache control device. The cache control device is configured to: retrieve data from the storage device in response to a request to read data from the storage device, store at least some of the data in one or both of (i) the SSD and (ii) the RAM, when storing the at least some of the data to the RAM, write to the RAM non-sequentially with respect to a memory space of the RAM, and when storing the at least some of the data in the SSD, write to the SSD sequentially with respect to a memory space of the SSD. The cache control device comprises an SSD interface device configured to allocate memory for storing data in the SSD sequentially with respect to the memory space of the SSD.

Abstract: Systems, methods, and other embodiments associated with a lookup structure for a large block cache are described. According to one embodiment, at least two blocks of data are stored in a cache. A lookup entry is constructed that describes the at least two blocks of data. The lookup entry includes block specific information that describes individual blocks of the at least two blocks of data. The lookup entry is stored in the lookup structure.

Abstract: A system including a write module to receive first data for writing over second data stored on a first member of a stripe of a RAID. A read module reads the second data and first parity from a SSD or the RAID. Before receiving third data for writing over fourth data stored on a second member of the stripe, the read module reads the fourth data from the second member and stores the fourth data in the SSD. A parity module generates second parity based on the first data, the second data, and the first parity. The write module writes the second parity on the SSD. On receiving the third data, the parity module generates the third parity based on the third data, the fourth data, and the second parity.

Abstract: A cache system for a storage device includes (i) one or more solid state drives (SSDs), (ii) one or more random access memories (RAMs), and (iii) a cache control device. The cache control device caches at least some of first data that is to be written to the storage device, and caches at least some of second data that is retrieved from the storage device. When caching first data or second data in one of the one or more RAMs, the cache control device writes to the one RAM non-sequentially with respect to a memory space of the one RAM. When caching first data or second data in one of the one or more SSDs, the cache control device writes to the one SSD sequentially with respect to a memory space of the one SSD.

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, can perform data caching. In some implementations, a method and system include receiving information that includes a logical address, allocating a physical page in a non-volatile memory structure, mapping the logical address to a physical address of the physical page, and writing, based on the physical address, data to the non-volatile memory structure to cache information associated with the logical address. The logical address can include an identifier of a data storage device and a logical page number.

Abstract: Methods, apparatus, and systems, including computer programs encoded on a computer storage medium, manage an address space. In some implementations, a method includes managing an allocation data structure for a memory, wherein the allocation data structure indicates groupings of memory space, each of the groupings having a different associated integer, and each of the groupings serving memory space in portions equal to a unit of memory space allocation times the associated integer for that grouping; receiving a request for allocation within the memory, wherein the request has an associated number of the unit of memory space allocation; and selecting one of the groupings from which to serve the request for allocation within the memory based on the associated number in comparison with values obtained using the different associated integers as an exponent.

Abstract: A system includes a write module, a read module, and a parity module. The write module is configured to modify first user data stored on a first member of a redundant array of independent disks (RAID) using second user data. The read module is configured to read the first user data and first parity data corresponding to the first user data from a solid-state disk associated with the RAID if at least one of the first user data and the first parity data are stored on the solid-state disk, or from the RAID if the at least one of the first user data and the first parity data are not stored on the solid-state disk. The parity module is configured to generate second parity data based on the first user data, the second user data, and the first parity data.