Abstract: The technology described herein efficiently determines whether a real inode is shared among views, or owned. In-memory data structures include a view snapshot generation counter that is increased as a snapshot that generates a view is created, and an inode total weight. An in-memory virtual inode cache dataset for a filesystem object associated with the view is instantiated with the value of snapshot generation counter, sharing-related data based on the inode mapping file entry for the object, and an inode access weight. To determine whether the inode is shared (and needs to be split), such as on a write to the object, the in-memory data is evaluated. The real inode is shared if the generation counters are unequal, if the sharing-related data indicates sharing at an intermediate indirect block level, or indicates sharing at the inode level and the inode access weight is less than the inode total weight.

Abstract: A method for storing data, the method comprising receiving, by an offload component in a client application node, an augmented write request originating from an application executing in an application container on the client application node, wherein the augmented write request is associated with data and wherein the offload component is located in a hardware layer of the client application node, and processing, by the offload component, the augmented write request by a file system (FS) client and a memory hypervisor module executing in a modified client FS container on the offload component, wherein processing the request results in at least a portion of the data being written to a location in a storage pool.

Abstract: A method for processing requests, the method comprising: receiving, from a client application node and by a metadata node, a request for a data layout, determining a QoS tag associated with the request, identifying at least one file system block (FSB) in a sparse virtual space, wherein the FSB is mapped to a location in a storage pool that is associated with the QoS tag, generating the data layout based on the at least one identified FSB; and providing the data layout to the client application, wherein the client application node uses the data layout to directly write data to the location in the storage pool, wherein the metadata node does not participate in the writing of the data to the location in the storage pool.

Abstract: Described is hard link handling when a directory snapshot exists that includes the hard link's connected file object. A hard link is created by allocating a virtual inode number for the hard link, with the virtual inode number mapped to a real inode number that identifies a real inode of the file object; the hard link is assigned weight. A total weight associated with the real inode is increased by the hard link weight, and a hard link data store is updated with an entry for the hard link. Upon receiving data write request to the hard link, weight data determines that the file object is shared as a result of the snapshot; the hard link is disassociated from the real inode file, and associated with a new real inode number and new real inode of a new file object. The data is written based on the new real inode.

Abstract: A method for storing data, the method comprising receiving, by an offload component in a client application node, a request originating from an application executing in an application container on the client application node, wherein the request is associated with data and wherein the offload component is located in a hardware layer of the client application node, and processing, by the offload component using an advanced data services pipeline, the request by a file system (FS) client and a memory hypervisor module executing in a modified client FS container on the offload component, wherein processing the request results in at least a portion of the data in a location in a storage pool.

Abstract: A method for processing requests includes receiving a request to write data, in response to the request, identifying a sparse virtual space segment using an available space tracking metadata hierarchy, and initiating writing of the data to a physical segment, wherein the physical segment is associated with the sparse virtual space segment.

Abstract: A method for storing data, comprising: obtaining, from a metadata node and by file system client executing on a client application node, a data layout, generating, by the client application node, an error detection value (EDV) for the data stored on the client application node; generating, by a memory hypervisor module executing on the client application node, at least one input/output (I/O) request specifying a location in a storage pool, wherein the location is determined using the data layout; issuing, by the memory hypervisor module, the at least one I/O request to the storage pool, wherein processing the at least one I/O request results in at least a portion of the data being stored at the location; and after issuing the at least one I/O request to the storage pool, transmitting the EDV to the metadata node, wherein the metadata node stores the EDV.

Abstract: A method for deduplicating data comprising: obtaining, from a metadata node and by file system client executing on a client application node, a data layout; generating, by the client application node, a fingerprint for the data stored on the client application node; generating, by a memory hypervisor module executing on the client application node, at least one input/output (I/O) request specifying a location in a storage pool, wherein the location is determined using the data layout; issuing, by the memory hypervisor module, the at least one I/O request to the storage pool, wherein processing the at least one I/O request results in at least a portion of the data being stored at the location; and after issuing the at least one I/O request to the storage pool, transmitting the fingerprint to the metadata node, wherein the metadata node attempts to deduplicate the data using the fingerprint.

Abstract: Described is a technology that facilitates directory level snapshots at any level of a filesystem's namespace. A directory inode mapping file associates a first directory's virtual inode number(s) with real inode number(s). A snapshot of the first directory creates a second directory with a second directory inode mapping file copied from the first directory inode mapping file, thereby sharing the real inode data structure(s) of the first directory. In the event that one of the directories is to be modified, the directory real inode is split to provide a new directory real inode file for the to-be-modified directory, with the modification after the split.

Abstract: In general, embodiments relate to a method for configuring client application nodes in a distributed system, the method comprising: detecting, by a client application node, a file system, wherein the file system is not mounted on the client application node; in response to the detecting, determining a metadata node on which the file system is mounted; sending a request to the metadata node to obtain a scale out volume record associated with the file system; generating a mapping between a plurality of storage devices and the scale out volume using the scale out volume record received from the metadata node and a topology file; and completing, after the mapping, mounting of the file system, wherein after the mounting is completed an application in an application container executing on the client application node may interact with the file system.

Abstract: A method for storing data, the method comprising receiving, by an offload component in a client application node, an augmented write request originating from an application executing in an application container on the client application node, wherein the augmented write request is associated with data and wherein the offload component is located in a hardware layer of the client application node, and processing, by the offload component, the augmented write request by a file system (FS) client and a memory hypervisor module executing in a modified client FS container on the offload component, wherein processing the request results in at least a portion of the data being written to a location in a storage pool.

Abstract: A method for storing data, the method comprising receiving, by an offload component in a client application node, a request originating from an application executing in an application container on the client application node, wherein the request is associated with data and wherein the offload component is located in a hardware layer of the client application node, and processing, by the offload component using an advanced data services pipeline, the request by a file system (FS) client and a memory hypervisor module executing in a modified client FS container on the offload component, wherein processing the request results in at least a portion of the data in a location in a storage pool.

Abstract: A method for storing data, the method comprising receiving, by an offload component in a client application node, a request originating from an application executing in an application container on the client application node, wherein the request is associated with data and wherein the offload component is located in a hardware layer of the client application node, and processing, by the offload component, the request by a file system (FS) client and a memory hypervisor module executing in a modified client FS container on the offload component, wherein processing the request results in at least a portion of the data in a location in a storage pool.

Abstract: A method for rebuilding data, comprising: obtaining, from a metadata node, a source file data layout for a source file and a target file data layout for a target file, wherein the source file is associated with a degraded mapped RAID group and the target file is associated with a new mapped RAID group; generating, by the client application node, a plurality of input/output (I/O) requests to read a portion of the data associated with the source file using the source file data layout; obtaining, in response to the plurality of I/O requests, the portion of the data associated with the source file; rebuilding a second portion of the data associated with source file using the portion of the data; and initiating, storage of at least the second portion of the data associated with the source file in the storage pool using the target file data layout.

Abstract: A method for storing data, comprising receiving, by a file system (FS) client executing on an offload component, a first request from a translation module, wherein the translation module translated a second request that was to be performed on an emulated block device into the first request, wherein the first request is specified using file semantics, wherein the first request is associated with data, wherein the offload component is located in a hardware layer of a client application node, and wherein the translation module is located on the offload component, and processing the first request by the FS client and a memory hypervisor module, wherein the FS client and the memory hypervisor module are executing in a modified client FS container on the offload component, wherein processing the first request results in at least a portion of the data being stored in a location in a storage pool.

Abstract: Described is hard link handling when a directory snapshot exists that includes the hard link's connected file object. A hard link is created by allocating a virtual inode number for the hard link, with the virtual inode number mapped to a real inode number that identifies a real inode of the file object; the hard link is assigned weight. A total weight associated with the real inode is increased by the hard link weight, and a hard link data store is updated with an entry for the hard link. Upon receiving data write request to the hard link, weight data determines that the file object is shared as a result of the snapshot; the hard link is disassociated from the real inode file, and associated with a new real inode number and new real inode of a new file object. The data is written based on the new real inode.

Abstract: The technology described herein efficiently determines whether a real inode is shared among views, or owned. In-memory data structures include a view snapshot generation counter that is increased as a snapshot that generates a view is created, and an inode total weight. An in-memory virtual inode cache dataset for a filesystem object associated with the view is instantiated with the value of snapshot generation counter, sharing-related data based on the inode mapping file entry for the object, and an inode access weight. To determine whether the inode is shared (and needs to be split), such as on a write to the object, the in-memory data is evaluated. The real inode is shared if the generation counters are unequal, if the sharing-related data indicates sharing at an intermediate indirect block level, or indicates sharing at the inode level and the inode access weight is less than the inode total weight.

Abstract: A method for file system reorganization in the presence of inline compression includes obtaining a virtual block pointer for an original compressed segment to be reorganized, the original compressed segment comprising compressed allocation units of data stored in a storage system, wherein the virtual block pointer comprises an extent list identifying the compressed allocation units in the original compressed segment and a pointer to where the original compressed segment is stored; copying only the referenced compressed allocation units in the original compressed segment to a new compressed segment in a substantially contiguous manner; updating the extent list to identify the referenced compressed allocation units in the new compressed segment, and the pointer to where the new compressed segment is stored; and freeing the original compressed segment.

Abstract: A method for rebuilding data, comprising: obtaining, from a metadata node, a source file data layout for a source file and a target file data layout for a target file, wherein the source file is associated with a degraded mapped RAID group and the target file is associated with a new mapped RAID group; generating, by the client application node, a plurality of input/output (I/O) requests to read a portion of the data associated with the source file using the source file data layout; obtaining, in response to the plurality of I/O requests, the portion of the data associated with the source file; rebuilding a second portion of the data associated with source file using the portion of the data; and initiating, storage of at least the second portion of the data associated with the source file in the storage pool using the target file data layout.

Abstract: A method for storing data, comprising receiving, by a file system (FS) client executing on an offload component, a first request from a translation module, wherein the translation module translated a second request that was to be performed on an emulated block device into the first request, wherein the first request is specified using file semantics, wherein the first request is associated with data, wherein the offload component is located in a hardware layer of a client application node, and wherein the translation module is located on the offload component, and processing the first request by the FS client and a memory hypervisor module, wherein the FS client and the memory hypervisor module are executing in a modified client FS container on the offload component, wherein processing the first request results in at least a portion of the data being stored in a location in a storage pool.