Abstract: Accessing additional storage space of a storage system includes reading a physical data fragment that is an incremental subset of a physical storage unit of the storage system, accessing metadata corresponding to logical blocks stored on the physical data fragment to determine a sidebar storage portion of the physical data fragment that is unused by the logical blocks, and accessing data of the sidebar storage portion. Accessing data of the sidebar storage portion may include providing data from the sidebar storage portion to a calling process. Accessing data of the sidebar storage portion may include modifying a portion of data from the physical data fragment and writing the physical data fragment back to the physical storage device. The physical data fragment may be locked prior to reading the physical data fragment and the physical data fragment may be unlocked after writing the physical data fragment.

Abstract: Accessing additional storage space of a storage system includes reading a physical data fragment that is an incremental subset of a physical storage unit of the storage system, accessing metadata corresponding to logical blocks stored on the physical data fragment to determine a sidebar storage portion of the physical data fragment that is unused by the logical blocks, and accessing data of the sidebar storage portion. Accessing data of the sidebar storage portion may include providing data from the sidebar storage portion to a calling process. Accessing data of the sidebar storage portion may include modifying a portion of data from the physical data fragment and writing the physical data fragment back to the physical storage device. The physical data fragment may be locked prior to reading the physical data fragment and the physical data fragment may be unlocked after writing the physical data fragment.

Abstract: Techniques for processing data may include: receiving source physical storage allocation units that include valid data and one or more holes of unused physical storage, wherein each source physical storage allocation unit has an associated timestamp denoting a most recent destaging time of any data stored on the source physical storage allocation unit; determining an age for each source physical allocation unit based on the associated timestamp of the source physical allocation unit; for each source physical storage allocation unit, determining one of multiple age buckets for the source physical allocation unit based on the age of the source physical allocation unit; and for a first of the age buckets including two source physical allocation units, performing first processing including: relocating at least some valid data from the two source physical storage allocation units to a target physical storage allocation unit.

Abstract: Techniques for processing data may include: receiving source physical storage allocation units that include valid data and one or more holes of unused physical storage, wherein each source physical storage allocation unit has an associated timestamp denoting a most recent destaging time of any data stored on the source physical storage allocation unit; determining an age for each source physical allocation unit based on the associated timestamp of the source physical allocation unit; for each source physical storage allocation unit, determining one of multiple age buckets for the source physical allocation unit based on the age of the source physical allocation unit; and for a first of the age buckets including two source physical allocation units, performing first processing including: relocating at least some valid data from the two source physical storage allocation units to a target physical storage allocation unit.

Abstract: The system, devices, and methods disclosed herein relate to data ratio reduction technology adapted to reduce storage costs by weeding out duplicative data write operations. The techniques and systems disclosed achieve deduplication benefits of smaller block, finer granularity, deduplication systems while simultaneously reducing the enormous metadata costs traditionally associated with small block deduplication. According to an exemplary 128 K block deduplication system and method, we disclose deduplication rates nearly equivalent to 4 K deduplication systems.

Abstract: The system, devices, and methods disclosed herein relate to data ratio reduction technology adapted to reduce storage costs by weeding out duplicative data write operations. The techniques and systems disclosed achieve deduplication benefits by reducing the size of hash values stored hash tables used to compare unwritten data blocks to data that has already been written and stored somewhere in physical storage. The data deduplication systems, methods, and products facilitate deduplication at the block level as well as for misaligned data chunks within data blocks, that is an unwritten data block that has been stored sequentially in two different physical locations. The deduplication teachings herein are amenable to varying data block sizes as well as data chunk sizes within blocks. Our embodiments enhance computer performance by substantially reducing computational speeds and storage requirements attendant to deduplication systems using larger hash table data sizes.

Abstract: Allocation of physical storage capacity within a storage array may be managed in standard size allocation units, e.g. 128 kb tracks, and smaller sub-allocation units, e.g. 32 kb quarter tracks. Data associated with sub-allocation units that match a known pattern may be represented by a hint rather than data in storage. The hint may be used to mitigate overhead associated with accessing backend storage. The hint may also mitigate the need for performing compression/decompression of the corresponding sub-allocation unit if the data is compressed. In response to a read or write request associated with an allocation unit organized into sub-allocation units, only those sub-allocation units that are required to service the read or write request and fail to match a pattern are accessed, decompressed or compressed.

Abstract: Techniques for processing I/O operations includes: determining whether a current amount of unused physical storage is greater than a threshold; and responsive to determining the current amount of unused physical storage is greater than the threshold, performing normal write processing, and otherwise performing alternative write processing. The alternative write processing includes: initializing a counter; determining whether a physical storage allocation is needed or potentially needed for a write I/O operation; responsive to determining that no physical storage allocation is needed for the write I/O operation, performing the normal write processing. Responsive to determining that a physical storage allocation is needed or potentially needed for the write I/O operation, determining a first amount of one or more credits needed to service the write I/O operation; and responsive to determining the counter does not include at least the first amount of one or more credits, failing the write I/O operation.

Abstract: A storage array presents a logical production volume that is backed by tangible data storage devices. The production volume is organized into fixed size front end allocation units. The tangible data storage devices are organized into discrete size back end allocation units of a plurality of different sizes. Data associated with each one of the front end allocation units is stored on only one of the back end allocation units. For example, compressed data may be stored on a back end allocation unit that is smaller than a front end allocation unit while maintaining a 1-to-1 relationship between the front end allocation unit and the back end allocation unit.

Abstract: The system, devices, and methods disclosed herein relate to data ratio reduction technology adapted to reduce storage costs by weeding out duplicative data write operations. The techniques and systems disclosed achieve deduplication benefits by reducing the size of hash values stored hash tables used to compare unwritten data blocks to data that has already been written and stored somewhere in physical storage. The data deduplication systems, methods, and products facilitate deduplication at the block level as well as for misaligned data chunks within data blocks, that is an unwritten data block that has been stored sequentially in two different physical locations. The deduplication teachings herein are amenable to varying data block sizes as well as data chunk sizes within blocks. Our embodiments enhance computer performance by substantially reducing computational speeds and storage requirements attendant to deduplication systems using larger hash table data sizes.

Abstract: The system, devices, and methods disclosed herein relate to data ratio reduction technology adapted to reduce storage costs by weeding out duplicative data write operations. The techniques and systems disclosed achieve deduplication benefits of smaller block, finer granularity, deduplication systems while simultaneously reducing the enormous metadata costs traditionally associated with small block deduplication. According to an exemplary 128 K block deduplication system and method, we disclose deduplication rates nearly equivalent to 4 K deduplication systems.

Abstract: A technique manages compression based on host data initially residing on a source storage array. The technique involves providing, based on source segments of the host data while the host data initially resides on the source storage array, compressibility labels (or tags) that label the source segments of the host data. Each compressibility label indicates a projected compression level for a respective source segment of the host data. The technique further involves generating a compression profile based on the compressibility labels that label the source segments of the host data. The technique further involves providing, based on the compression profile, a compression tier configuration that defines a prearrangement of compression tiers on a target storage array that receives the host data from the source storage array, the prearrangement of compression tiers providing storage units of allocation having different predefined sizes.

Abstract: Determining whether to compress data of a virtual storage unit based at least in part on: an I/O activity value of a virtual storage unit; a compressibility value of the virtual storage unit; and/or a capacity utilization of the storage system or a component thereof. For example, decision logic may be configured based on one or more of such parameters such that virtual storage units with relatively high I/O activities are rarely or never compressed, e.g., to avoid the disproportionately high increases in CPU and bandwidth resource consumption and I/O latency this could create. Decision logic may be configured such that, in general, the likelihood that a virtual storage unit will be compressed increases as: the I/O activity of the virtual storage unit decreases; the system capacity utilization increases; and/or as the compressibility value of the virtual storage unit increases.

Abstract: For a data copying operation, data compression using constant number-of-track-groups and a thinly provisioned target device facilitates incremental updates where the size of the compressed data on the target device changes. Compressed data is written to the same LBA as the beginning of the source device chunk cluster LBA (1:1 mapping of data start). A termination string or other demarking device is used to identify space freed on the target device resulting from compression. During an incremental update only changed chunk clusters are changed, and freed space is changed accordingly if necessary.

Abstract: Determining authorization for actions includes defining a plurality of groups, defining a plurality of action types and corresponding levels of authorization for each of the groups, for at least a subset of the action types, defining a plurality of devices on which corresponding actions may be performed, wherein at least some of the devices correspond to portions of a data storage device, and, for the at least one of the groups, determining authorization for a requested action, where if the action corresponds to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group and by examining the plurality of devices corresponding to the requested action and where if the action does not correspond to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group. The action types may include system calls to the data storage device.

Abstract: A computer system may include one or more hosts and a plurality of data storage devices for providing multihop system calls. The data storage devices are interconnected and also connected to the one or more hosts. Each data storage device classifies a data operation as a system call, a remote system call, or a multihop system call. Also described is a multipath multihop system call in which one or more communication paths may be selected using predetermined and/or dynamic communication path selection techniques. The number of communication paths determined may be in accordance with parameters that are included in a multipath multihop system call, tunable system parameters, or a combination of the foregoing.

Abstract: Determining authorization for actions includes defining a plurality of groups, defining a plurality of action types and corresponding levels of authorization for each of the groups, for at least a subset of the action types, defining a plurality of devices on which corresponding actions may be performed, wherein at least some of the devices correspond to portions of a data storage device, and, for the at least one of the groups, determining authorization for a requested action, where if the action corresponds to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group and by examining the plurality of devices corresponding to the requested action and where if the action does not correspond to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group. The action types may include system calls to the data storage device.

Abstract: Determining authorization for actions includes defining a plurality of groups, defining a plurality of action types and corresponding levels of authorization for each of the groups, for at least a subset of the action types, defining a plurality of devices on which corresponding actions may be performed, wherein at least some of the devices correspond to portions of a data storage device, and, for the at least one of the groups, determining authorization for a requested action, where if the action corresponds to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group and by examining the plurality of devices corresponding to the requested action and where if the action does not correspond to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group. The action types may include system calls to the data storage device.

Abstract: A computer system may include one or more hosts and a plurality of data storage devices for providing multihop system calls. The data storage devices are interconnected and also connected to the one or more hosts. Each data storage device classifies a data operation as a system call, a remote system call, or a multihop system call. Also described is a multipath multihop system call in which one or more communication paths may be selected using predetermined and/or dynamic communication path selection techniques. The number of communication paths determined may be in accordance with parameters that are included in a multipath multihop system call, tunable system parameters, or a combination of the foregoing.

Abstract: Determining authorization for actions includes defining a plurality of groups, defining a plurality of action types and corresponding levels of authorization for each of the groups, for at least a subset of the action types, defining a plurality of devices on which corresponding actions may be performed, wherein at least some of the devices correspond to portions of a data storage device, and, for the at least one of the groups, determining authorization for a requested action, where if the action corresponds to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group and by examining the plurality of devices corresponding to the requested action and where if the action does not correspond to one of the devices, authorization is determined by examining the levels of authorization for action types corresponding to the at least one group. The action types may include system calls to the data storage device.