Abstract: A backup copy of a production device is used to quantify suitability of host application data for placement on individual managed drives and virtualized managed drives based on storage capabilities associated with those drives. A data analysis program on a data backup storage array may generate block backup statistics to indicate that a production device or certain chunks, blocks or volumes of host application data are highly compressible or reducible via deduplication. The block backup statistics are sent from the data backup storage array to the primary storage array. The primary storage array uses the block backup statistics to select a particular storage resource with suitable storage capabilities for the data. Highly compressible data may be stored on a storage virtualization storage array with data compression capability, and data that is neither highly compressible nor reducible with deduplication may be stored on local resources.

Abstract: Embodiments are described for a executing a processing job using one or more nodes of a storage area network using computing resources on the SAN that are predicted to be idle. A predictive model is generated by monitoring idle states of resources of nodes of the SAN and using machine learning to build the predictive model. A scheduler executes jobs on one or more nodes of the SAN with sufficient predicted idle resources to process the job, in accordance with resource requirements and job attributes in a manifest of the job. If a job cannot be completed during a window of time that the necessary resources are predicted to be idle, or if one or more resources become unavailable, the job can be paused and resumed, migrated to another node, or restarted at a later time when the required resources to complete the job are predicted to be idle.

Abstract: Systems and methods for scheduling multiple flows in a computing system or for allocating resources to the flows in the computing system. Each flow may be associated with a target priority and a target performance. A current priority can be determined for each flow and resources are allocated based on the highest current priority. Over time, the current priorities change and each flow will receive resources as the flows are scheduled for execution. No flow is starved of resources.

Abstract: Embodiments are described for a multi-node file system, such as a clustered or distributed file system, with a file system buffer cache and an additional host-side tier non-volatile storage cache such as 3DXP storage. Cache coherency can be maintained by one of three models: (i) host-side tier management, (ii) file system management, or (iii) storage array management. performing a storage tier-specific file system action in a file system that comprises a namespace that spans multiple tiers of storage.

Abstract: Embodiments are described for a multi-node file system, such as a clustered or distributed file system, with a file system buffer cache and an additional host-side tier non-volatile storage cache such as 3DXP storage. Cache coherency can be maintained by one of three models: (i) host-side tier management, (ii) file system management, or (iii) storage array management. performing a storage tier-specific file system action in a file system that comprises a namespace that spans multiple tiers of storage.

Abstract: Systems and methods are disclosed for associating one or more storage-based services with a storage unit accessible by a primary “tier 1” storage device. A storage-based service can include deduplication, compression, data conversion, statistical analysis of the data to be stored, or other storage-based service. A storage unit can be a disk, a file, a virtual disk, or a logical unit of storage (LUN). A virtual machine within the primary “tier 1” storage can perform the one or more storage-based services associated with the storage unit.

Abstract: A backup engine of a first storage system receives a request to perform a backup session from the first storage system to a second storage system based on a backup service level objective (SLO) that has been configured for the backup session. In response to the request, it is determined that a first backup resource allocated for the backup session by the first storage system cannot satisfy the SLO based on statistics of prior backup sessions in view of characteristics of the backup session to be performed. A dynamic resource allocation (DRA) module is to dynamically perform a first DRA to modify the first backup resource to satisfy the SLO. The backup engine then initiates the backup session by transmitting backup data from the first storage system to the second storage system using the modified first backup resource.

Abstract: A backup engine of a first storage system receives a request to perform a backup session from the first storage system to a second storage system based on a backup service level objective (SLO) that has been configured for the backup session. In response to the request, it is determined that a first backup resource allocated for the backup session by the first storage system cannot satisfy the SLO based on statistics of prior backup sessions in view of characteristics of the backup session to be performed. A dynamic resource allocation (DRA) module is to dynamically perform a first DRA to modify the first backup resource to satisfy the SLO. The backup engine then initiates the backup session by transmitting backup data from the first storage system to the second storage system using the modified first backup resource.

Abstract: According to one embodiment, a secondary primary storage system monitors replicated data received from a primary storage system, where the primary storage system provides primary storage services to a host device and the second storage system operates as a redundant storage system. In response to determining that no replicated data has been received from the primary storage system over a first predetermined period of time, the secondary primary storage system determines an amount of read/write data requests have been received from the host device for a second period of time. The secondary primary storage system transitions into a second primary storage system in response to determining the amount of read/write data requests received from the host exceeds a predetermined threshold.

Abstract: Resources are optimized in an integrated data protection system having a production host, primary storage and a backup appliance for backup and restore of lost data to a selected restore point. Data tracks in primary storage having changed data are pushed to backup storage with a changed-tracks bitmap mapping the changed tracks. A selected restore capacity is reserved on a restore device, but the restore device is only provisioned for a capacity sufficient to store changed tracks corresponding to the selected restore point, thus obviating the creation and storing of metadata for the unprovisioned capacity. This conserves significant storage and memory capacity and reduces processing requirements.

Abstract: A method in a backup engine is described. The method includes receiving a data access request for a data element in a logical storage volume for one of a read or write operation, determining an affinity group associated with the data element based on an affinity group list indicating a physical storage volume for the data element based on the data address range for the data element, determining a first frontend adapter identifier (ID) from the first affinity group associated with the data element, the first frontend adapter ID identifying a first frontend adapter associated with a first physical storage volume, and transmitting the data access request to the first frontend adapter based on the first frontend adapter ID to execute the data access request on the first physical storage volume.

Abstract: A computer-implemented method for indexing content stored in a cache memory device is disclosed. The method starts with maintaining a fingerprint index having a plurality of fingerprint entries, each mapping a fingerprint to a storage location of a cache memory device, where the cache memory device caches some of data blocks stored in a persistent storage device of a storage system, and where the fingerprint index is a partial index indexing a portion of data stored in the cache memory device. In response to receiving a request to insert a new fingerprint, the method continues with evicting one of the fingerprint entries according to a predetermined eviction algorithm and inserting the new fingerprint into the evicted fingerprint entry.

Abstract: A computer-implemented method for indexing content stored in a cache memory device is disclosed. The method starts with maintaining a file index having a plurality of extent entries, each extent entry corresponding to one of a plurality of file extents stored in a cache memory device that caches data stored in a persistent storage device of a storage system. The method continues with maintaining a fingerprint index having a plurality of fingerprint entries, each mapping a fingerprint to a data region of a file indexed in the file index, wherein each fingerprint indexed in the fingerprint index is retrieved from metadata stored in the persistent storage device of the storage system when one or more corresponding data chunks were accessed, and deduplicating and accessing the file extents stored in the cache memory device using the file index and the fingerprint index.

Abstract: A computer-implemented method for caching content in a cache memory device is disclosed. The method starts with receiving a request for accessing a first data block associated with a first file, and a file manager provides access of the first data block in a persistent storage device of a storage system. The file manager then caches the first data block in a cache memory device including deduplicating the first data block, wherein at least some of data blocks stored in the cache memory device are deduplicated data blocks, and wherein at least one of the data blocks is referenced by different regions of an identical file or different files.

Abstract: A computer-implemented method for caching content in a cache memory device is disclosed. The method starts with receiving, at a cache manager, one or more data chunks to be cached in a cache memory device, where the one or more data chunks are retrieved from a persistent storage disk of a storage system in response to a read request of a region of a file. Then the one or more data chunks of a file extent is compressed using a predetermined compression algorithm, and the file extent is packed into a write-evict unit (WEU) maintained in a random-access memory (RAM) that has been open to store a plurality of file extents. In response to determining that the WEU is full, the cache manager writes the WEU from the RAM into the cache memory device.

Abstract: A computer-implemented method for indexing content stored in a cache memory device is disclosed. The method starts with maintaining a file index having a plurality of extent entries, each extent entry corresponding to one of a plurality of file extents stored in a cache memory device that caches data stored in a persistent storage device of a storage system. In response to receiving a request to read a first file region of a first file, the method continues with retrieving first data block from the persistent storage device that contains the first data block, caching the first data block at a first storage location of the cache memory device; and creating a first extent entry in the file index having at least a first node, where the first node includes an address of the first storage location and a first bitmap indicating which data blocks are valid.

Abstract: A computer-implemented method for indexing content stored in a cache memory device is disclosed. The method starts with in response to receiving a first request for caching a first file extent associated with a first file in a cache memory device, generating a first fingerprint based on content of the first file extent. Then the method continues with searching in a fingerprint index based on the first fingerprint to determine whether the first file extent has been stored in the cache memory. In response to determining that a fingerprint entry matching the first fingerprint is found, the method then continues with associating a first identifier identifying the first file extent and the first file with a storage location of the cache memory device obtained from the matching fingerprint entry, without storing the first file extent in the cache memory device.

Abstract: According to various embodiments of the invention, a system and method for controlling a file system. In some embodiments, a control plane interposes between a data plane user and a data plane, intercepts file system operations, and performs control plane operations upon the file system operations. In one such embodiment, the system and method is implemented between a data plane user that is a local file system user and a data plane that is a local file system. In another such embodiment, the system and method is implemented between a data plane user that is a client and a data plane that is a file server. Furthermore, for an embodiment where the control plane that interposes between a client and a file server, the control plane can be implemented as a file system proxy. Control plane operations include, but are not limited to, observation, verification, and transformation of a file system operation.

Abstract: According to various embodiments of the invention, a system and method for controlling a file system. In some embodiments, a control plane interposes between a data plane user and a data plane, intercepts file system operations, and performs control plane operations upon the file system operations. In one such embodiment, the system and method is implemented between a data plane user that is a local file system user and a data plane that is a local file system. In another such embodiment, the system and method is implemented between a data plane user that is a client and a data plane that is a file server. Furthermore, for an embodiment where the control plane that interposes between a client and a file server, the control plane can be implemented as a file system proxy. Control plane operations include, but are not limited to, observation, verification, and transformation of a file system operation.