Abstract: A method, non-transitory computer readable medium and storage server computing device that determines when a generated storage operation corresponds with one of a set of predefined storage operations. The storage operation is executed and a result of the execution of the storage operation is withheld from a file system, when the determining indicates that the storage operation corresponds with one of the set of predefined storage operations. A determination is made when a consistency point operation has completed. The result of the execution of the storage operation is presented to the file system, when the determining indicates that a consistency point operation has completed. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations in a journal following a recovery.

Abstract: A method, non-transitory computer readable medium, and device that assists with caching filesystem metadata to a partner non-volatile random-access memory (NVRAM) includes caching metadata related to an incoming data modifying operation generated by a client computing device to at least one storage controller device in a cluster. A service interruption event that makes a data block present in the storage device of a hosting storage node inaccessible to the client computing device is determined for during the caching. The requested metadata block from the at least one NVRAM is retrieved when the service interruption event is determined. The cache is warmed using the retrieved metadata block from the at least one NVRAM.

Abstract: Methods, non-transitory computer readable media, and devices for dynamically changing a number of partitions at runtime in a hierarchical data partitioning model include determining a number of coarse mapping objects, determining a number of fine mapping objects, and setting a number of coarse partitions and a number of fine partitions based on the determined number of coarse mapping object and the determined number of fine mapping objects.

Abstract: A method, non-transitory computer readable medium and storage server computing device that determines when a generated storage operation corresponds with one of a set of predefined storage operations. The storage operation is executed and a result of the execution of the storage operation is withheld from a file system, when the determining indicates that the storage operation corresponds with one of the set of predefined storage operations. A determination is made when a consistency point operation has completed. The result of the execution of the storage operation is presented to the file system, when the determining indicates that a consistency point operation has completed. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations in a journal following a recovery.

Abstract: A method, non-transitory computer readable medium, and device that assists with caching filesystem metadata to a partner non-volatile random-access memory (NVRAM) includes caching metadata related to an incoming data modifying operation generated by a client computing device to at least one storage controller device in a cluster. A service interruption event that makes a data block present in the storage device of a hosting storage node inaccessible to the client computing device is determined for during the caching. The requested metadata block from the at least one NVRAM is retrieved when the service interruption event is determined. The cache is warmed using the retrieved metadata block from the at least one NVRAM.

Abstract: Methods, non-transitory computer readable media, and devices for dynamically changing a number of partitions at runtime in a hierarchical data partitioning model include determining a number of coarse mapping objects, determining a number of fine mapping objects, and setting a number of coarse partitions and a number of fine partitions based on the determined number of coarse mapping object and the determined number of fine mapping objects.

Abstract: A modular block allocator receives a cleaner message requesting dirty buffers associated with an inode be cleaned. The modular block allocator provides at least one bucket cache comprising a plurality of buckets, wherein each bucket represents a plurality of free data blocks. The dirty buffers are cleaned by allocating the data blocks of one of the buckets to the dirty buffers. The allocated data blocks are mapped to a stripe set and when the stripe set is full, the stripe set is sent to a storage system. In one embodiment of the invention, a modular block allocator includes a front end module and a back end module communicating with each other via an application programming interface (API). The front end module contains write allocation policies that define how blocks are laid out on disk. The back end module creates data structures for execution of the policies.

Abstract: One or more techniques and/or systems are provided for incremental transfer with unused data block reclamation. For example, source volume of a source storage filer may have a physical replication relationship with a destination volume of a destination storage filer. Snapshots of a file system of the source volume may be used to identify new data blocks (e.g., data blocks allocated or reallocated to a new snapshot) and/or unused data blocks (e.g., data blocks of a deleted snapshot) used to incrementally update the destination volume so that the destination volume is a mirrored replication of the source volume. In this way, new data blocks may be updated and unused data blocks may be punched out (e.g., deallocated) of the destination volume during execution of an incremental update.

Abstract: Systems, methods, and computer program products implementing hybrid file structures for data storage are provided. One embodiment of a method performed in a computer-based storage system includes writing a file as data blocks in an array of storage devices. The method includes associating the data blocks with metadata related to at least one location in the array of storage devices for later access to the data blocks. The file is represented as a hierarchical data structure having a plurality of nodes. A first portion of nodes has a first span type, and a second portion of nodes has a second span type. The data structure includes a buftree. The first span type includes a fixed-span type. The second span type includes a variable-span type.

Abstract: Techniques to clone a writeable data object in non-persistent memory are disclosed. The writeable data object is stored in a storage structure in non-persistent memory that corresponds to a portion of a persistent storage. The techniques enable cloning of the writeable data object without having to wait until the writeable data object is saved to the persistent storage and without needing to quiesce incoming operations (e.g., reads and writes) to the writeable data object.

Abstract: Systems and methods which provide an absent allocation technique with respect to absent data of a data structure for which data migration in a time-displaced data migration operation has not completed are disclosed. Through use of absent allocated data block states of an absent allocation technique of embodiments, dependency between source and destination data stores may be broken. An absent allocated data block state of embodiments stored within the data structure provides a suggestive indication that data of a data block of a data structure is absent from the data structure. A corresponding absent allocated data block state of embodiments stored external to the data structure provides a determinative indication that the data of the data block of the data structure is absent from the data structure. The absent data block determinative indictor of embodiments is updated as data continues to be migrated.

Abstract: A modular block allocator receives a cleaner message requesting dirty buffers associated with an Mode be cleaned. The modular block allocator provides at least one bucket cache comprising a plurality of buckets, wherein each bucket represents a plurality of free data blocks. The dirty buffers are cleaned by allocating the data blocks of one of the buckets to the dirty buffers. The allocated data blocks are mapped to a stripe set and when the stripe set is full, the stripe set is sent to a storage system. In one embodiment of the invention, a modular block allocator includes a front end module and a back end module communicating with each other via an application programming interface (API). The front end module contains write allocation policies that define how blocks are laid out on disk. The back end module creates data structures for execution of the policies.

Abstract: A modular block allocator receives a cleaner message requesting dirty buffers associated with an inode be cleaned. The modular block allocator provides at least one bucket cache comprising a plurality of buckets, wherein each bucket represents a plurality of free data blocks. The dirty buffers are cleaned by allocating the data blocks of one of the buckets to the dirty buffers. The allocated data blocks are mapped to a stripe set and when the stripe set is full, the stripe set is sent to a storage system. In one embodiment of the invention, a modular block allocator includes a front end module and a back end module communicating with each other via an application programming interface (API). The front end module contains write allocation policies that define how blocks are laid out on disk. The back end module creates data structures for execution of the policies.

Abstract: It is determined that a first data block contains the same data as a second data block. The first data block is associated with a first extent and the second data block is associated with a second extent. In response to determining that the first data block contains the same data as the second data block, the second data block is associated with the first extent and the first data block is disassociated with the second extent.

Abstract: Systems, methods, and computer program products implementing hybrid file structures for data storage are provided. One embodiment of a method performed in a computer-based storage system includes writing a file as data blocks in an array of storage devices. The method includes associating the data blocks with metadata related to at least one location in the array of storage devices for later access to the data blocks. The file is represented as a hierarchical data structure having a plurality of nodes. A first portion of nodes has a first span type, and a second portion of nodes has a second span type. The data structure includes a buftree. The first span type includes a fixed-span type. The second span type includes a variable-span type.

Abstract: A modular block allocator includes a front end module and a back end module communicating with each another via an application programming interface (API). The front end module receives cleaner messages requesting dirty buffers associated with the cleaner messages be cleaned. The back end module provides low and high level data structures which are formed by examining bitmaps associated with data storage devices. A stripe set data structure mapping to the low level data structures are formed. The front end module cleans the dirty buffers by allocating data blocks in the high level data structures to the dirty buffers. The low level data structures are used to map the allocated data blocks to the stripe set and when the stripe set is full it is sent to the data storage devices.

Abstract: Techniques to clone a writeable data object in non-persistent memory are disclosed. The writeable data object is stored in a storage structure in non-persistent memory that corresponds to a portion of a persistent storage. The techniques enable cloning of the writeable data object without having to wait until the writeable data object is saved to the persistent storage and without needing to quiesce incoming operations (e.g., reads and writes) to the writeable data object.

Abstract: Techniques to clone a writeable data object in non-persistent memory are disclosed. The writeable data object is stored in a storage structure in non-persistent memory that corresponds to a portion of a persistent storage. The techniques enable cloning of the writeable data object without having to wait until the writeable data object is saved to the persistent storage and without needing to quiesce incoming operations (e.g., reads and writes) to the writeable data object.

Abstract: A request is received to remove duplicate data. A log data container associated with a storage volume in a storage server is accessed. The log data container includes a plurality of entries. Each entry is identified by an extent identifier in a data structures stored in a volume associated with the storage server. For each entry in the log data container, a determination is made if the entry matches another entry in the log data container. If the entry matches another entry in the log data container, a determination is made of a donor extent and a recipient extent. If an external reference count associated with the recipient extent equals a first predetermined value, block sharing is performed for the donor extent and the recipient extent. A determination is made if the reference count of the donor extent equals a second predetermined value. If the reference count of the donor extent equals the second predetermined value, the donor extent is freed.

Abstract: A filesystem can be converted to a different version by creating a new data structure according to a new format of the different version and transforming the data from the filesystem to the new data structure. Transforming the data can include changing the format of the data in the filesystem to be compatible with the new data structure format. The data may be incorporated into the new data structure by copying the data, or creating indirect reference mechanisms to point to the original data.