Abstract: Techniques are provided for multi-tier write allocation. A storage system may store data within a multi-tier storage environment comprising a first storage tier (e.g., storage devices maintained by the storage system), a second storage tier (e.g., a remote object store provided by a third party storage provider), and/or other storage tiers. A determination is made that data (e.g., data of a write request received by the storage system) is to be stored within the second storage tier. The data is stored into a staging area of the first storage tier. A second storage tier location identifier, for referencing the data according to a format utilized by the second storage tier, is assigned to the data and provided to a file system hosting the data. The data is then destaged from the staging area into the second storage tier, such as within an object stored within the remote object store.

Abstract: A method, non-transitory computer readable medium and storage server computing device that determines when a generated storage operation corresponds with one of a set of predefined storage operations. The storage operation is executed and a result of the execution of the storage operation is withheld from a file system, when the determining indicates that the storage operation corresponds with one of the set of predefined storage operations. A determination is made when a consistency point operation has completed. The result of the execution of the storage operation is presented to the file system, when the determining indicates that a consistency point operation has completed. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations in a journal following a recovery.

Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Abstract: Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.

Abstract: Techniques are provided for multi-tier write allocation. A storage system may store data within a multi-tier storage environment comprising a first storage tier (e.g., storage devices maintained by the storage system), a second storage tier (e.g., a remote object store provided by a third party storage provider), and/or other storage tiers. A determination is made that data (e.g., data of a write request received by the storage system) is to be stored within the second storage tier. The data is stored into a staging area of the first storage tier. A second storage tier location identifier, for referencing the data according to a format utilized by the second storage tier, is assigned to the data and provided to a file system hosting the data. The data is then destaged from the staging area into the second storage tier, such as within an object stored within the remote object store.

Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

Abstract: A method, non-transitory computer readable medium, and device that assists with caching filesystem metadata to a partner non-volatile random-access memory (NVRAM) includes caching metadata related to an incoming data modifying operation generated by a client computing device to at least one storage controller device in a cluster. A service interruption event that makes a data block present in the storage device of a hosting storage node inaccessible to the client computing device is determined for during the caching. The requested metadata block from the at least one NVRAM is retrieved when the service interruption event is determined. The cache is warmed using the retrieved metadata block from the at least one NVRAM.

Abstract: A file system layout apportions an underlying physical volume into one or more virtual volumes of a storage system. The virtual volumes having a file system and one or more files organized as buffer trees, the buffer trees utilizing indirect blocks to point to the data blocks. The indirect block at the level above the data blocks are grouped into compression groups that point to a set of physical volume block number (pvbn) block pointers.

Abstract: Techniques are provided for multi-tier write allocation. A storage system may store data within a multi-tier storage environment comprising a first storage tier (e.g., storage devices maintained by the storage system), a second storage tier (e.g., a remote object store provided by a third party storage provider), and/or other storage tiers. A determination is made that data (e.g., data of a write request received by the storage system) is to be stored within the second storage tier. The data is stored into a staging area of the first storage tier. A second storage tier location identifier, for referencing the data according to a format utilized by the second storage tier, is assigned to the data and provided to a file system hosting the data. The data is then destaged from the staging area into the second storage tier, such as within an object stored within the remote object store.

Abstract: Techniques are provided for orphan object detection, invalid sequence number detection, and asynchronous object cleanup. A storage system may store data within one or more tiers of storage, such as a storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Orphan objects, within the remote object store, that are no longer used by the storage system may be detected and/or deleted. When an aggregate of volumes is deleted, corresponding objects, within the remote object store, may be identified and/or deleted. Invalid sequence numbers (e.g., lost or corrupt sequence numbers locally maintained in a metafile) assigned to objects within the remote object store may be identified, deleted, and/or fixed.

Abstract: Techniques are provided for remote object store error handling. A storage system may store data within one or more tiers of storage, such as a local storage tier (e.g., solid state storage and disks maintained by the storage system), a remote object store (e.g., storage provided by a third party storage provider), and/or other storage tiers. Because the remote object store may not provide the same data consistency and guarantees that the storage system provides for clients such as through the local storage tier, additional validation is provided by the storage system for the remote object store. For example, when data is put into an object of the remote object store, a verification get operation is performed to read and validate information within a header of the object. Other verifications and checks are performed such as using a locally stored metafile to detect corrupt or lost metadata and/or objects.

Abstract: Techniques are provided for providing a storage abstraction layer for a composite aggregate architecture. A storage abstraction layer is utilized as an indirection layer between a file system and a storage environment. The storage abstraction layer obtains characteristic of a plurality of storage providers that provide access to heterogeneous types of storage of the storage environment (e.g., solid state storage, high availability storage, object storage, hard disk drive storage, etc.). The storage abstraction layer generates storage bins to manage storage of each storage provider. The storage abstraction layer generates a storage aggregate from the heterogeneous types of storage as a single storage container. The storage aggregate is exposed to the file system as the single storage container that abstracts away from the file system the management and physical storage details of data of the storage aggregate.

Abstract: A method, non-transitory computer readable medium, and computing device that receives metadata for a block associated with an object from a source storage node. The metadata comprises a source object identifier and the object is associated with a source volume of a source aggregate owned by the source storage node. A determination is made when another block associated with the object has been previously received. A destination object identifier is obtained based on the source object identifier, when the determining indicates that the other block associated with the object has been previously received. A new aggregate block number is assigned to the block based on the destination object identifier and another portion of the metadata. Ownership of the source volume is transferred upon receipt of an indication of a cutover from the source storage node in order to migrate the source volume to a destination volume of a destination aggregate.

Abstract: Methods and systems for a networked system are provided. One method includes receiving a request by a processor to transfer a data block stored at a first storage tier to the second storage tier; using an object staging data structure to determine that an object is available for transferring the data block from the first storage tier to the second storage tier. The object staging data structure an indicator providing a status for the object and an object length and an offset value of a transfer log indicating where information regarding the data block is stored. The method further includes updating an address of the storage tier where the information regarding data block is stored at the transfer log; increasing the object length and the offset value at the object staging data structure; and creating the object at the second tier.

Abstract: Methods and systems for a networked system are provided. One method includes receiving a request by a processor to transfer a data block stored at a first storage tier to the second storage tier; using an object staging data structure to determine that an object is available for transferring the data block from the first storage tier to the second storage tier. The object staging data structure an indicator providing a status for the object and an object length and an offset value of a transfer log indicating where information regarding the data block is stored. The method further includes updating an address of the storage tier where the information regarding data block is stored at the transfer log; increasing the object length and the offset value at the object staging data structure; and creating the object at the second tier.

Abstract: Methods and systems for a networked system are provided. One method includes receiving a request by a processor to transfer a data block stored at a first storage tier to the second storage tier; using an object staging data structure to determine that an object is available for transferring the data block from the first storage tier to the second storage tier. The object staging data structure an indicator providing a status for the object and an object length and an offset value of a transfer log indicating where information regarding the data block is stored. The method further includes updating an address of the storage tier where the information regarding data block is stored at the transfer log; increasing the object length and the offset value at the object staging data structure; and creating the object at the second tier.

Abstract: A method, non-transitory computer readable medium and storage server computing device that determines when a generated storage operation corresponds with one of a set of predefined storage operations. The storage operation is executed and a result of the execution of the storage operation is withheld from a file system, when the determining indicates that the storage operation corresponds with one of the set of predefined storage operations. A determination is made when a consistency point operation has completed. The result of the execution of the storage operation is presented to the file system, when the determining indicates that a consistency point operation has completed. This technology reduces the number of storage operations that are required to be journaled to maintain consistency of a file system, thereby reducing the runtime resources required to facilitate the journaling and replay resource required to replay the storage operations in a journal following a recovery.

Abstract: Conventional storage filers utilize a data reading process that requires client read request messages to be suspended in the operating system while the data is retrieved from the physical storage system. Then, once the operating system retrieves the data from the physical storage system, the operating system must restart the suspended read message in order to forward the retrieved data to the client. Accordingly, the inventors have developed a system and method that allows the physical storage system's server to send the data directly to the client rather than routing back through the operating system.

Abstract: Methods and systems for a networked storage environment are provided. One method includes scanning a first data structure by a processor executing instructions out of a memory for a storage operating system to determine whether any data chunk of a first object stored at a first storage tier is referenced by the storage operating; when the storage operating system references a certain number of data chunks, the processor using an object staging data structure to identify a second object that is in the process of being built with space for transferring the certain number of data chunks from the first object to the second object; and updating information regarding the second object at a transfer log with location information of the certain number of data chunks at the first storage tier.