Abstract: Systems and methods for determining an access pattern in a computing system. Accesses to a file may contain random accesses and sequential accesses. The file may be divided into multiple regions and the accesses to each region are tracked. The access pattern for each region can then be determined independently of the access patterns of other regions of the file.

Abstract: Systems and methods for determining an access pattern in a computing system. Accesses to a file may contain random accesses and sequential accesses. The file may be divided into multiple regions and the accesses to each region are tracked. The access pattern for each region can then be determined independently of the access patterns of other regions of the file.

Abstract: According to some embodiment, a backup storage system receives a request from a client to read a data segment associated with a file object stored in a storage system. In response to the request, the system performs a lookup operation in a first index stored in a memory to identify a first index entry based on a fingerprint of the requested data segment to obtain a first write-evict unit (WEU) identifier (ID) identifying a first WEU storing the requested data segment. The system accesses a solid state device (SSD) operating as a cache memory device to retrieve the data segment from the first WEU. The system extracts and decompresses a compressed data segment retrieved from the first WEU and returns the decompressed data segment to the client without accessing a storage unit for retrieving the same data segment.

Abstract: An example method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, identifying, in the stream, random I/O requests and sequential I/O requests, queuing the sequential I/O requests in a priority/processing queue and processing the sequential I/O requests immediately without waiting for loading of any metadata, and processing a first one of the random I/O requests and a second one of the random I/O request. When the first and second random I/O requests each depend on the same range of metadata, the first and second random I/O requests are processed in the order received, and when the first and second random I/O requests each depend on different respective ranges of metadata, the first and second random I/O requests are processed at the same time.

Abstract: Embodiments are described for reducing memory usage and disk I/O's for reading and writing streams in a deduplication storage system. Generations of backups of a stream can be tracked for the stream. In the first generation (first backup), stream data is ingested, segmented, and written to disk without looking up a fingerprint of each segment, to save disk I/O's and memory. After the first generation, an amount of stream data processed is tracked, and at dynamically tunable increments of data processed, a fingerprint index lookup is performed to disk. The fingerprint index is read into memory and subsequent fingerprint lookups are preformed to cache, until a next dynamically tunable fingerprint lookup to disk. The dynamically tunable increment of processed data can be randomly reset.

Abstract: Systems and methods for determining an access pattern in a computing system. Accesses to a file may contain random accesses and sequential accesses. The file may be divided into multiple regions and the accesses to each region are tracked. The access pattern for each region can then be determined independently of the access patterns of other regions of the file.

Abstract: According to some embodiment, a backup storage system determines whether one or more data segments associated with a file object have been updated, where the data segment(s) are stored in a first segment level in a multi-level hierarchy. For each updated data segment, the system applies a reference-based hierarchical data structure to track metadata associated with the updated data segment, where the metadata is stored in a second segment level in the multi-level hierarchy. The system determines whether the reference-based hierarchical data structure has reached a specific size. The system writes the metadata associated with the updated data segment(s) to a solid state device (SSD) operating as a memory cache device responsive to determining that the reference-based hierarchical data structure has reached the specific size.

Abstract: According to some embodiment, a backup storage system receives, by a first phase of a data deduplication pipeline, a request from a client for reading or writing a data segment associated with a data stream stored in or to a storage system. In response to the request, the system retrieves, by the first phase, load parameters associated with a second phase in the data deduplication pipeline. For each of the load parameters associated with the second phase, the system determines, by the first phase, whether the load parameter has exceeded a load threshold associated with the second phase. The system throttles, by the first phase, performance of a specific job in the data deduplication pipeline by the second phase in response to a determination that at least one of the load parameters associated with the second job phase has exceeded the load threshold associated with the second phase.

Abstract: An example method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, identifying, in the stream, random I/O requests and sequential I/O requests, queuing the sequential I/O requests in a priority/processing queue and processing the sequential I/O requests immediately without waiting for loading of any metadata, and processing a first one of the random I/O requests and a second one of the random I/O request. When the first and second random I/O requests each depend on the same range of metadata, the first and second random I/O requests are processed in the order received, and when the first and second random I/O requests each depend on different respective ranges of metadata, the first and second random I/O requests are processed at the same time.

Abstract: In one example, a method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, and identifying, in the stream, random I/O requests and sequential I/O requests. Next, the stream is split into an incoming queue containing the sequential I/O requests and a priority/processing queue containing the random I/O requests. A metadata load queue is then defined by assigning each of the random I/O requests to a respective range of metadata. The metadata of each range is loaded, and the random I/O requests re-queued. Using the loaded metadata, a random I/O operation is performed that corresponds to one of the random I/O requests. Finally, the random I/O operation and a sequential I/O operation are performed concurrently with each other.

Abstract: Methods, systems, and apparatus for optimizing a cache memory device of a storage system are described. In one embodiment, a first base segment tree representing a first full backup including data and metadata describing the data is cached in a cache memory device. Subsequently, a plurality of incremental segment trees representing incremental backups to the first full backup are cached in the cache memory device. Each of incremental segment trees corresponding to the changes to the first full backup, without modifying the first base segment tree in response to the changes. At least two of the incremental segment trees are merged into an updated incremental segment tree to reduce a storage space of the cache memory device to store the incremental segment trees. The updated incremental segment tree comprises data and metadata represented by two or more incremental segment trees.

Abstract: A method, article of manufacture, and apparatus for performing backup operations using a deduplicated storage system is presented. A data segment can be received at a deduplicated storage system. A fingerprint can be calculated for the data segment. The fingerprint and the offset can be stored in a reference cache on the deduplicated storage system. A metadata segment can be loaded from the deduplicated storage system. The metadata segment can be updated to include the fingerprint. Backup operations may include storage and management using a tree structure.

Abstract: Methods, systems, and apparatus for providing data storage services of a storage system are described. In one embodiment, a first file representing a first full backup including data and metadata describing the data is cached as a first segment tree having a plurality of layers of nodes in a tree hierarchy. A second file representing an incremental backup of the first full backup is cached as a second segment tree in the cache memory device. The second segment tree describes changes of the data and the metadata of the incremental backup in view of the data and the metadata of the first full backup without caching any of nodes of the first segment tree again. The first and second segment trees are collectively used to represent a second full backup based on the incremental backup and the first full backup.

Abstract: A method, article of manufacture, and apparatus for backup operations. A read request is received for a data segment at a deduplicated storage system. The data segment is determined not to be in a data cache on the deduplicated storage system. A data segment reference for the data segment is determined to be in a data segment reference cache on the deduplicated storage system. The data segment is read from a persistent storage using the data segment reference.

Abstract: Embodiments are directed to a method of minimizing latency and input/output (I/O) operations in a data storage system by defining a sparse metadata segment tree to identify changed data blocks, wherein a full version of the tree is stored in a memory and modified versions of the tree are stored in cache memory, and using the sparse metadata segment tree to perform at least one data storage application including file verification, file replication, file restores, and file system snapshots.