Abstract: Techniques described herein relate to systems and methods of data storage, and more particularly to providing layering of file system functionality on an object interface. In certain embodiments, file system functionality may be layered on cloud object interfaces to provide cloud-based storage while allowing for functionality expected from a legacy applications. For instance, POSIX interfaces and semantics may be layered on cloud-based storage, while providing access to data in a manner consistent with file-based access with data organization in name hierarchies. Various embodiments also may provide for memory mapping of data so that memory map changes are reflected in persistent storage while ensuring consistency between memory map changes and writes. For example, by transforming a ZFS file system disk-based storage into ZFS cloud-based storage, the ZFS file system gains the elastic nature of cloud storage.

Abstract: Implementations described and claimed herein provide a system and methods for prioritizing data in a cache. In one implementation, a priority level, such as critical, high, and normal, is assigned to cached data. The priority level dictates how long the data is cached and consequently, the order in which the data is evicted from the cache memory. Data assigned a priority level of critical will be resident in cache memory unless heavy memory pressure causes the system to reclaim memory and all data assigned a priority state of high or normal has been evicted. High priority data is cached longer than normal priority data, with normal priority data being evicted first. Accordingly, important data assigned a priority level of critical, such as a deduplication table, is kept resident in cache memory at the expense of other data, regardless of the frequency or recency of use of the data.

Abstract: Implementations described and claimed herein provide systems and methods for allocating and managing resources for a deduplication table. In one implementation, an upper limit to an amount of memory allocated to a deduplication table is established. The deduplication table has one or more checksum entries, and each checksum entry is associates a checksum with unique data. A new checksum entry corresponding to new unique data is prevented from being added to the deduplication table where adding the new checksum entry will cause the deduplication table to exceed a size limit. The new unique data has a checksum that is different from the checksums in the one or more checksum entries in the deduplication table.

Abstract: Aspects of the present disclosure disclose systems and methods for managing a level-two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. In particular, any data that is scheduled to be evicted or otherwise removed from a level-one cache is stored in the level-two cache with corresponding metadata in a manner that is quickly retrievable. The data contained within the level-two cache is managing using a cache list that manages and/or maintains data chunk entries added to the level-two cache based on a temporal access of the data chunk.

Abstract: Aspects of the present disclosure involve a level two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. In particular, any data that is scheduled to be evicted or otherwise removed from a level-one cache is stored in the level-two cache with corresponding metadata in a manner that is quickly retrievable.

Abstract: Implementations described and claimed herein provide systems and methods for estimating migration progress. In one implementation, a target file system is initialized to which to migrate existing data from a source file system. An initial amount of data to be migrated to the target file system is estimated based on an examination of in-use space at a root of the source file system. Any mount points for nested file systems in the source file system are identified. An amount of data for each of the nested file systems is estimated based on an examination of in-use space at the mount point for the nested file system. An estimated total amount of data to be migrated from the source file system to the target file system is determined based on the initial amount of data to be migrated and the amount of data for each of the nested file systems.

Abstract: Implementations described and claimed herein provide a coordination of interdependent asynchronous reads. In one implementation, an input/output request for a target data block stored on a block device at a virtual address is received. A highest level indirect block from which the target data block depends in a hierarchical data structure pointing to the virtual address of the target data block is identified. The highest level indirect block is uncached. A context item is recorded to an input/output structure for the highest level indirect block. The context item indicates that an ultimate objective of a read request for the highest level indirect block is to retrieve the target data block. The input/output request is asynchronously reissued for the target data block upon receipt of the read request for the highest level indirect block.

Abstract: Implementations of the present disclosure involve a system and/or method for joining read requests for the same data block sent to a storage appliance. The system and method is configured to receive the first read request for the data block at an I/O layer of the storage appliance. The I/O layer is configured to manage obtaining data blocks from one or more storage devices on the storage appliance. The system and method may then receive a second read request for the data block at the I/O layer of the storage appliance. The first and second read request may then be joined at I/O layer and only a single copy of the data block is returned to a cache in response to the first and second read requests.

Abstract: Implementations described and claimed herein provide systems and methods for estimating migration progress. In one implementation, a target file system is initialized to which to migrate existing data from a source file system. An initial amount of data to be migrated to the target file system is estimated based on an examination of in-use space at a root of the source file system. Any mount points for nested file systems in the source file system are identified. An amount of data for each of the nested file systems is estimated based on an examination of in-use space at the mount point for the nested file system. An estimated total amount of data to be migrated from the source file system to the target file system is determined based on the initial amount of data to be migrated and the amount of data for each of the nested file systems.

Abstract: Implementations described and claimed herein provide a system and methods for prioritizing data in a cache. In one implementation, a priority level, such as critical, high, and normal, is assigned to cached data. The priority level dictates how long the data is cached and consequently, the order in which the data is evicted from the cache memory. Data assigned a priority level of critical will be resident in cache memory unless heavy memory pressure causes the system to reclaim memory and all data assigned a priority state of high or normal has been evicted. High priority data is cached longer than normal priority data, with normal priority data being evicted first. Accordingly, important data assigned a priority level of critical, such as a deduplication table, is kept resident in cache memory at the expense of other data, regardless of the frequency or recency of use of the data.

Abstract: Aspects of the present disclosure disclose systems and methods for managing a level-two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. Any data stored in the level-two cache may be stored in a particular version or format of data known as “raw” data, in contrast to storing the data in a “cooked” version, as is typically stored in a level-one cache.

Abstract: Implementations of the present disclosure involve a system and/or method for replication size estimation and progress monitoring for a file system residing on a computing system. The replication progress monitoring system obtains a first snapshot of a file system for a first point in time and a second snapshot of the file system for a second point in time. The system may then calculate the difference between the first snapshot size from the second snapshot size and add to the difference the size a released data size. The released data size includes the size of any blocks of data included in the first snapshot and released before the second snapshot was taken. The replication transfer size may then be estimated by adding the snapshot size difference with the released size estimate.

Abstract: Implementations described and claimed herein provide a system and methods for prioritizing data in a cache. In one implementation, a priority level, such as critical, high, and normal, is assigned to cached data. The priority level dictates how long the data is cached and consequently, the order in which the data is evicted from the cache memory. Data assigned a priority level of critical will be resident in cache memory unless heavy memory pressure causes the system to reclaim memory and all data assigned a priority state of high or normal has been evicted. High priority data is cached longer than normal priority data, with normal priority data being evicted first. Accordingly, important data assigned a priority level of critical, such as a deduplication table, is kept resident in cache memory at the expense of other data, regardless of the frequency or recency of use of the data.

Abstract: Aspects of the present disclosure disclose systems and methods for managing a level-two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. Any data stored in the level-two cache may be stored in a particular version or format of data known as “raw” data, in contrast to storing the data in a “cooked” version, as is typically stored in a level-one cache.

Abstract: Implementations described and claimed herein provide systems and methods for allocating and managing resources for a deduplication table. In one implementation, an upper limit to an amount of memory allocated to a deduplication table is established. The deduplication table has one or more checksum entries, and each checksum entry is associates a checksum with unique data. A new checksum entry corresponding to new unique data is prevented from being added to the deduplication table where adding the new checksum entry will cause the deduplication table to exceed a size limit. The new unique data has a checksum that is different from the checksums in the one or more checksum entries in the deduplication table.

Abstract: Aspects of the present disclosure disclose systems and methods for providing a level-two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. In particular, any data that is scheduled to be evicted or otherwise removed from a level-one cache is stored in the level-two cache with corresponding metadata in a manner that is quickly retrievable.

Abstract: Aspects of the present disclosure disclose systems and methods for recognizing multiple and distinct references within a cache that identify or otherwise provide access to empty blocks of data. Multiple references identifying empty blocks of data are associated with a single block of empty data permanently stored in the cache. Subsequently, each time an empty block of data is added to the cache, a reference corresponding to the empty block is mapped to a generic empty block of data stored in the cache. When a reference is removed or deleted from the cache, only the reference is deleted; the single generic block of empty data continues to reside in the cache.

Abstract: Aspects of the present disclosure disclose systems and methods for managing a level-two persistent cache. In various aspects, a solid-state drive is employed as a level-two cache to expand the capacity of existing caches. In particular, any data that is scheduled to be evicted or otherwise removed from a level-one cache is stored in the level-two cache with corresponding metadata in a manner that is quickly retrievable. The data contained within the level-two cache is managing using a cache list that manages and/or maintains data chunk entries added to the level-two cache based on a temporal access of the data chunk.

Abstract: Implementations of the present disclosure involve a system and/or method for joining read requests for the same data block sent to a storage appliance. The system and method is configured to receive the first read request for the data block at an I/O layer of the storage appliance. The I/O layer is configured to manage obtaining data blocks from one or more storage devices on the storage appliance. The system and method may then receive a second read request for the data block at the I/O layer of the storage appliance. The first and second read request may then be joined at I/O layer and only a single copy of the data block is returned to a cache in response to the first and second read requests.

Abstract: In general, embodiments of the invention relate to a disk, which includes a plurality of files and rectory associated with the plurality of files comprising a plurality of directory entries. Further, each of the plurality of directory entries is associated one of the plurality of files and each of the plurality of directory entries is associated with a collision differentiator (CD). In one aspect of the invention, a hash value calculated for each of the plurality of directory entries is used to determine the CD associated with each of the plurality of directory entries.