Abstract: A request is received to access at least one data unit of a larger data object by an entity within a local host, which is then queried to determine if the requested data unit is present. If the requested data unit is present in the local cache, it is fetched from the local cache. If the requested data unit is not present in the local cache, however, a respective cache within at least one target host, which is different from the local host, is queried to determine if the requested data unit is present remotely and, if so, the data unit is fetched from there instead. If the requested data unit is not present in the local cache or the cache of the target host, the data unit is fetched from a common data storage pool.

Abstract: Techniques for monitoring and sending network connectivity information for connection pathways to host machines is provided. A storage node comprises a plurality of attached network interfaces and is operatively coupled to storage devices that are accessible to the host machines through network interfaces attached to the storage node. Each of the network interfaces send messages to the host machines. The storage node, receives host messages from at least one host machine. The storage node determines connectivity information about connectivity between network interfaces and host machines. The storage node receives a request for data that is stored on the one or more storage devices, from a host machine, at a target network interface, where the target node was chosen by the host machine based upon the connectivity information.

Abstract: Techniques for performing data encryption on data to be stored within a storage system are provided. A client application executing on a host machine may generate a data storage write request to write data to a storage system. A host-side module, executing on the host machine receives the write request. The host-side module is configured to generate one or more fingerprints for the data corresponding to the write request, where the one or more fingerprints are unique identifiers used to identify data blocks that make up the data. The host-side module generates encrypted data by encrypting the data blocks using an encryption technique. The encrypted data is then sent to a storage node within the storage system. Deduplication may be performed on the encrypted data using the one or more generated fingerprints.

Abstract: In a Distributed Virtual Array data storage system, data chunks making up data containers and identified by keys, which are independent of physical storage locations, are written in storage devices according to layouts specified in a layout data structure. When any of the storage devices becomes inaccessible, the chunks nominally designated to be written in the inaccessible storage devices are instead written to alternate devices. Information indicating writing to an alternate device may be included in or along with such chunks so as to make them easily identifiable during later reconstruction or re-writing to the intended storage device when it again becomes available.

Abstract: A data storage system has a plurality of hosts that request writes of data to a relatively high-performance storage device (RHPSD) such as NVRAM, which may be one of a plurality of RHPSDs in a plurality of storage nodes. A storage management system receives the write requests and writes received data to the RHPSD. According to an allocation policy, the storage management system indicates to at least one of the hosts that it should limit its use of RHPSD and it then discards data upon indication from that host to do so. Before being discarded from the RHPSD, the data may be written to a persistent storage device.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: A data storage system includes a plurality of hosts, each of which includes at least one processor and communicates over a network with a plurality of storage nodes, at least one of which has at least one storage device, at least one storage controller, and at least one non-volatile memory. At least one process within a host issues data storage read/write requests. At least one of the hosts has a cache for caching data stored in at least one of the storage nodes. The host writes data corresponding to a write request to at least one remote non-volatile memory and carries out at least one storage processing function; data in the written-to node may then be made available for subsequent reading by a different one of the hosts. Examples of the storage processing function include compression, ECC computation, deduplicating, garbage collection, write logging, reconstruction, rebalancing, and scrubbing.

Abstract: Techniques for copying forward live data within storage regions used by a file system to new storage regions are provided. Storage regions refer to ranges of persistent storage within a storage medium. One or more live-clump-identifying workers identify a set of live-clump-references. The live-clump references are references to live clumps that belong to live objects. Live clumps represent a unit of data. After identifying the set of live-clump-references, one or more container-identifying workers identify a set of storage regions that contain clumps that correspond to the set of live-clump references. After identifying the set of storage regions corresponding to the set of live-clump-references, a plurality of copy-forward task workers perform copy-forward operations on the set of storage regions to copy live data to new storage regions.

Abstract: A data storage system has a plurality of hosts that request writes of data to a relatively high-performance storage device (RHPSD) such as NVRAM, which may be one of a plurality of RHPSDs in a plurality of storage nodes. A storage management system receives the write requests and writes received data to the RHPSD. According to an allocation policy, the storage management system indicates to at least one of the hosts that it should limit its use of RHPSD and it then discards data upon indication from that host to do so. Before being discarded from the RHPSD, the data may be written to a persistent storage device.

Abstract: In a processing system in which at least one entity issues data read and write requests to at least one storage system that stores data as data units, pluralities of data units are grouped. Each group is written as a respective cache line in a cache, which is deduplicated. Before evicting a selected one of the cache lines, a caching component determines whether at least one of the data units in the selected cache line is still active; if so, then the still active data unit(s) in the selected cache line is read and written to a different cache line.

Abstract: A distributed storage system has at least one host and at least one storage node. Hosts write data to and read data from storage devices within the storage nodes. A host upgrade manager (HUM) is installed in at least one of the hosts; a secondary upgrade manager (SUM) is installed in at least one of the nodes; and a primary upgrade manager (PUM) is installed within the distributed storage system itself, for example, in one of the nodes or hosts. Host and node storage system software is upgraded by transferring, via the respective HUM and SUM, upgraded versions of the software from the PUM to at least one host or node detected to be running an outdated version. The host and node storage system software components comprise different, cooperating portions of overall storage management software.

Abstract: In a Distributed Virtual Array data storage system, a storage pool receives, from at least one writing entity within a host, a request to write a container of data that is identified by a unique key that is independent of physical storage identifiers and that comprises a set of data chunks. The key indicates a plurality of storage devices in which to store the data chunks, which are accordingly stored in those storage devices. Different combinations, each comprising a layout map, of storage devices are represented in a layout table. Information in the container key is used to select which layout map to use for the chunks of each container. An error-correction chunk is preferably also stored along with the chunks of each container so as to enable data container reconstruction in case of storage device failure.

Abstract: Data in a storage system is deduplicated after receiving from at least one writing entity requests for a plurality of write operations for a corresponding plurality of data blocks in a storage object. The received blocks are buffered and sorted in order and a sequence of clumps is created from the buffered blocks, where each clump comprises a grouping of at least one of the sorted, buffered blocks. A boundary is determined between at least one pair of clumps based at least in part on the content of at least one of the buffered blocks, and it is then determined whether at least one of the clumps is a duplicate of a previously stored clump.

Abstract: A request is received to access at least one data unit of a larger data object by an entity within a local host, which is then queried to determine if the requested data unit is present. If the requested data unit is present in the local cache, it is fetched from the local cache. If the requested data unit is not present in the local cache, however, a respective cache within at least one target host, which is different from the local host, is queried to determine if the requested data unit is present remotely and, if so, the data unit is fetched from there instead. If the requested data unit is not present in the local cache or the cache of the target host, the data unit is fetched from a common data storage pool.

Abstract: In a processing system in which at least one entity issues data read and write requests to at least one storage system that stores data as data units, pluralities of data units are grouped. Each group is written as a respective cache line in a cache, which is deduplicated. Before evicting a selected one of the cache lines, a caching component determines whether at least one of the data units in the selected cache line is still active; if so, then the still active data unit(s) in the selected cache line is read and written to a different cache line.

Abstract: A distributed storage system has at least one host and at least one storage node. Hosts write data to and read data from storage devices within the storage nodes. A host upgrade manager (HUM) is installed in at least one of the hosts; a secondary upgrade manager (SUM) is installed in at least one of the nodes; and a primary upgrade manager (PUM) is installed within the distributed storage system itself, for example, in one of the nodes or hosts. Host and node storage system software is upgraded by transferring, via the respective HUM and SUM, upgraded versions of the software from the PUM to at least one host or node detected to be running an outdated version. The host and node storage system software components comprise different, cooperating portions of overall storage management software.

Abstract: A data storage system has a plurality of hosts that request writes of data to a relatively high-performance storage device (RHPSD) such as NVRAM, which may be one of a plurality of RHPSDs in a plurality of storage nodes. A storage management system receives the write requests and writes received data to the RHPSD. According to an allocation policy, the storage management system indicates to at least one of the hosts that it should limit its use of RHPSD and it then discards data upon indication from that host to do so. Before being discarded from the RHPSD, the data may be written to a persistent storage device.

Abstract: In a Distributed Virtual Array data storage system, data chunks making up data containers are stored in a key order in storage devices according to layouts specified in a layout data structure. When any of the storage devices becomes inaccessible, the storage devices iteratively return, in storage order, their stored chunks. Chunks belonging to the same container are identified and, if that container had a chunk in the inaccessible storage device, are used to reconstruct the inaccessible chunk. The chunks of the container are then stored according to an updated layout. The keys are independent of physical storage locations.

Abstract: In a Distributed Virtual Array data storage system, a storage pool receives, from at least one writing entity within a host, a request to write a container of data that is identified by a unique key that is independent of physical storage identifiers and that comprises a set of data chunks. The key indicates a plurality of storage devices in which to store the data chunks, which are accordingly stored in those storage devices. Different combinations, each comprising a layout map, of storage devices are represented in a layout table. Information in the container key is used to select which layout map to use for the chunks of each container. An error-correction chunk is preferably also stored along with the chunks of each container so as to enable data container reconstruction in case of storage device failure.

Abstract: In a Distributed Virtual Array data storage system, data chunks making up data containers are stored in a key order in storage devices according to layouts specified in a layout data structure. When any of the storage devices becomes inaccessible, the storage devices iteratively return, in storage order, their stored chunks. Chunks belonging to the same container are identified and, if that container had a chunk in the inaccessible storage device, are used to reconstruct the inaccessible chunk. The chunks of the container are then stored according to an updated layout. The keys are independent of physical storage locations.