Abstract: A data storage system operates a file system as a thinly provisioned file system having a host-visible virtual space and a smaller allocated space of underlying physical storage from a pool. The file system applies formatting to the allocated space and utilizes formatted allocated space for storing host files. The file system receives a file I/O request that requires increasing the allocated space of the file system, and in response (1) obtains an extent of physical storage from the pool and adds it to the file system as added allocated space, and (2) formats an initial portion of the added allocated space and satisfies the file I/O request using the formatted initial portion. Over a subsequent period the file system incrementally formats additional portions of the added allocated space and satisfies subsequent file I/O requests from the added portions.

Abstract: A method for accessing data where the method includes receiving, by an access client executing in user space, a request to read data that includes a file system identifier and an offset, identifying a storage server associated with the data, generating a command to copy the data to memory, and sending the command to the storage server.

Abstract: A technique for managing metadata in a data storage system includes receiving a set of data to be stored in a file system of the data storage system and calculating values of metadata blocks that the file system will use to organize the set of data in the file system. The technique aggregates values of the metadata blocks and sends them in a single, atomic transaction to a persistent cache. The transaction either succeeds, in which case the persistent cache stores the values of all of the metadata blocks, or it fails, in which case the persistent cache stores none of the values of the metadata blocks. Over time, the persistent cache flushes the values of the metadata blocks to a set of non-volatile storage devices that back the metadata blocks in the data storage system.

Abstract: A technique for supporting snapshots and provisioning in a NAS (network attached storage) cluster includes implementing a file system built upon a virtual disk realized in a virtualization platform, the virtual disk itself built upon a LUN (Logical UNit) in block storage. In response to a request to take a snapshot of the file system or to provision a new file system, the NAS cluster bypasses the virtualization platform and directs a request to a block storage manager, either to take a snapshot of the LUN (in the case of snapshot) or to create a new LUN (in the case of provisioning). The NAS cluster records a relationships among LUNs, file systems, and snaps, but the virtualization platform treats the LUNs as independent objects.

Abstract: A technique for archiving NAS (network attached storage) servers includes replicating multiple locally-backed volumes, which support respective file systems of a NAS server, to respective cloud-backed volumes backed by a cloud-based data store. After replication has updated the cloud-backed volumes with contents from the locally-backed volumes, the technique further includes performing a group snapshot operation on the cloud-backed volumes. The group snapshot operation creates a point-in-time version of the cloud-backed volumes, which provides a replica of the NAS server archived in the cloud.

Abstract: A method for managing data includes obtaining, by a hypervisor on a host computing device, a write request for storing first data, and in response to the write request: identifying a first set of memory segments associated with the first data using a memory mapper, wherein the first set of memory segments is associated with a memory device, making a first determination that the memory device is local to the host computing device, and in response to the first determination: storing the first data in the first set of memory segments.

Abstract: A technique for managing data objects in a data storage cluster realizes an application-accessible data object within a file that belongs to an internal file system. The internal file system runs on a computing node of the cluster and is backed by a LUN realized in block storage. The storage cluster achieves snapshots of the data object at the level of the file system, e.g., by snapshotting the file, such that the data object and its snapshots are all backed by the same LUN in the block storage.

Abstract: A technique for replicating a data object from a source data storage system (source) to a destination data storage system (destination) preserves QoS (Quality of Service) by transmitting QoS information from the source to the destination and applying the QoS information when storing data at the destination that arrive in replication instructions sent from the source. The QoS information indicates desired storage tiering for the data object at the source. In the event of a failure at the source, which results in failover from the source to the destination, the destination is able to provide users with the same quality of service as was desired at the source.

Abstract: A method for managing nodes includes detecting, by a failover node, a failure of a primary node, based on the detecting, updating a sparse virtual space to indicate that a file system previously local to the primary node is now local to the failover node, wherein the sparse virtual space is managed by a file system container executing on the failover node, and initiate execution of an instance of an application on the failover node, wherein a second instance of the application was previously executing on the primary node, and wherein the instance of the application uses the file system.

Abstract: A method for processing requests includes receiving a request to write data, in response to the request, identifying a sparse virtual space segment using an available space tracking metadata hierarchy, and initiating writing of the data to a physical segment, wherein the physical segment is associated with the sparse virtual space segment.

Abstract: A method for managing file systems includes receiving, by a unified namespace manager, a first message that indicates a new file system has been created on a first node, performing an update on a top level file system namespace to include a reference to the new file system to generate an updated top level file system namespace, generating, based on the update, a second message that indicates the top level file system namespace has been updated, and sending, to a second node, the second message.

Abstract: A method for storing transaction records includes receiving, by a transaction log manager, a first commit request for a first transaction record from a first core, copying, based on the first commit request, the first transaction record to a first region of memory, making a first determination that the first region surpasses a space threshold, and copying, based on the first determination, a first plurality of transaction records from the first region to storage, wherein the first plurality of transaction records comprises the first transaction record.

Abstract: A system for managing file systems includes memory. The system also includes a manager. The memory is used for storing in-memory data structures. The manager obtains file system references for a file system from a file system manager. The file system manager utilizes storage to provide the file system references. The manager stores the file system references in an in-memory data structure in the memory. To service file system reference requests, the manager utilizes uses the in-memory data structure to service at least one of the file system reference requests.

Abstract: A method for processing requests includes receiving a request comprising a virtual address, wherein the request is associated with an application executing on an operating system, identifying, based on data specified in the request, a logical volume associated with the data, making a first determination, based on the logical volume, that the logical volume is managed by a remote node, generating a data layout request to receive a data layout of the data from the remote node, receiving the data layout, wherein the data layout specifies a first physical address on the remote node that stores the data, initiating a copy request to copy the data from the first physical address to a second physical address on a local node, generating, based on the copy request, a virtual-to-physical address mapping between the virtual address and the second physical address; and initiating processing the request using the virtual-to-physical address mapping.

Abstract: A method for managing file systems, includes receiving, by a processor coordinator, a first operation request, identifying a file system associated with the first operation request, making a first determination that the file system is local, and in response to the first determination identifying a core thread pool associated with the file system, and directing operation of the first operation request to be executed on a core associated with the core thread pool, wherein the core is associated with a processor.

Abstract: A method for processing data includes receiving, by a space manager, a request comprising a virtual address, wherein the request is associated with an application executing on an operating system and wherein the space manager is executing in a container on the operating system, and identifying a sparse virtual space segment of a sparse virtual space, wherein the sparse virtual space segment is associated with a memory segment and wherein the memory segment is associated with a physical address associating the sparse virtual space segment with the virtual address generating, based on the associating, a virtual-to-physical address mapping between the virtual address and the physical address, and initiating processing the request using the virtual-to-physical address mapping.

Abstract: A method for moving data includes identifying, by a staging manager in a container, a trigger condition associated with data being used by an application external to the container, performing an analysis on the trigger condition, making a first determination, based on the analysis, that the trigger condition is satisfied, and processing, based on the first determination, a data movement action.

Abstract: A technique for operating a file system in a NAS (network attached storage) cluster includes discovering, by a data node running in the NAS cluster, a LUN (Logical UNit) provisioned from block storage. The data node renders the LUN as a block device and creates a local volume, built on the block device. The data node then deploys the file system on the local volume. The file system, the local volume, the block device, and the LUN all have address spaces, and the address space of each corresponds one-to-one with the address space of each of the others. Write requests directed to addresses in the file system propagate to the LUN, where they effect writes at corresponding addresses in the LUN.

Abstract: Improved techniques of managing a data storage system involve selectively inserting block virtualization structures (BVS) in access paths between data blocks of a file system and block pointers pointing to the data blocks. A BVS provides metadata for supporting deduplication of data in that data block. In some arrangements, a file system may support selective insertion of such a BVS between a block pointer and data block according to a specified criterion. For example, such a file system might support insertion of BVS's between block pointers and those data blocks storing cold data for which access latency is not important to overall performance of the data storage system.

Abstract: A technique for replicating file systems to any number of targets includes obtaining a writeable snapshot of a file system on a source, performing cloud tiering of files in the snapshot, replacing the file contents of the snapshot with stubs pointing to the cloud, and generating a namespace backup of the snapshot that includes directories and stubs. The source system ships the namespace backup to the cloud, allowing any authorized target system to restore from the namespace backup to a local file-system replica, such that contents of the cloud-tiered files are made accessible on the target system from the restored namespace via the stubs.