Abstract: Techniques for improving data compression of a storage system using coarse and fine grained similarity are described herein. According to one embodiment, region sketches for a plurality of regions of the set of data are generated, each region storing a plurality of data chunks. A region sketch index having a plurality of entries is maintained, each corresponding to one of the region sketches of the regions. The entries of the region sketch index are sorted based on the sketches of the regions, such that regions with an identical region sketch are positioned adjacent to each other within the region sketch index, representing similar regions. The data chunks of the similar regions that are identified based on the sorted entries of the region sketch index are reorganized to improve data compression of the data chunks of the similar regions.

Abstract: Processing and memory resources are optimized in a data storage system by reading a region of compressed data containing desired data is read from primary storage, writing the compressed data to a memory page, selectively decompressing the compressed data to retrieve the desired data, and the writing the decompressed data back to the same page. State information about the start of the compressed data and the end of the decompressed data on the page is maintained to enable decompression to be halted and resumed on demand.

Abstract: Techniques for determining vulnerability of disks are described herein. According to one embodiment, for each of a plurality of disks representing a redundant array of independent disks (RAID), a reallocated sector count associated with the disk is obtained, the reallocated sector count representing a number of sectors that have been reallocated due to an error of a storage transaction to the disk. A failure probability of the disk given the obtained reallocated sector count is determined using a predictive model, wherein the predictive model was generated based on history operating data of a set of known disks. Thereafter, a failure probability of at least two of the disks in the RAID is determined based on the failure probability of each of the disks to determine vulnerability of the RAID.

Abstract: Techniques for improving data compression of a deduplicated storage system are described herein. According to one embodiment, the similarity of a plurality of data chunks stored in one or more first storage areas of the storage system is determined based on a plurality of sketches, each describing characteristics of one of the data chunks. The data chunks are grouped into a plurality of groups of similar data chunks based on the similarity of the data chunks. The groups of similar data chunks are compressed, such that similar data chunks are compressed close to each other.