Abstract: The disclosed embodiments disclose techniques for managing a global namespace for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in a cloud storage system; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. Furthermore, a global namespace for the distributed filesystem is also split across these cloud controllers, with each cloud controller “owning” (e.g., managing write accesses for) a distinct portion of the global namespace and maintaining a set of namespace mappings that indicate which portion of the namespace is assigned to each cloud controller. During operation, an initial cloud controller receives a request from a client system to access a target file in the distributed system. This initial cloud controller uses the namespace mappings for the global namespace to determine a preferred cloud controller that will handle the request.

Abstract: The disclosed embodiments disclose techniques for managing metadata and data storage for a cloud controller in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems. More specifically, the cloud controllers cache and ensure data consistency for the data stored in the cloud storage systems, with each cloud controller maintaining (e.g., storing) in a local storage device: (1) one or more metadata regions containing a metadata hierarchy that reflects the current state of the distributed filesystem; and (2) cached data for the distributed filesystem. During operation, the cloud controller receives an incremental metadata snapshot that references new data written to the distributed filesystem. The cloud controller stores updated metadata from this incremental metadata snapshot in one of the metadata regions on the local storage device.

Abstract: The disclosed embodiments disclose techniques for using byte-range locks to manage multiple concurrent accesses to a file in a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in the cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives from a first client a request to access a portion of the file. The cloud controller contacts the owning cloud controller for the portion of the file to request a byte-range lock for that portion of the file. The owning cloud controller returns a byte-range lock to the requesting cloud controller if no other clients of the distributed filesystem are currently locking the requested portion of the file with conflicting accesses.

Abstract: The disclosed embodiments disclose techniques for optimizing caching performance for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and cloud controllers cache portions of the distributed filesystem on hard drives with rotating disk platters. The outside portions of these disk platters have higher I/O bandwidth than the inside portions of the disk platters. During operation, a cloud controller determines that a hot subset of cached data is likely to be accessed frequently by clients, and stores this hot subset to an outside portion of a disk platter. The cloud controller further determines that a cold subset of cache data is less likely to be accessed by clients, and stores the cold subset on an inside portion of a disk platter.

Abstract: The disclosed embodiments provide techniques for performing deduplication for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives an incremental metadata snapshot that references new data that was added to the distributed filesystem by a remote cloud controller. The cloud controller extracts a set of deduplication information from this incremental metadata snapshot. Upon receiving a subsequent client write request (e.g., a file write that includes one or more data blocks), the cloud controller uses the extracted deduplication information to determine that one or more data blocks in the client write request have already been written to the distributed filesystem.

Abstract: The disclosed embodiments disclose techniques for deleting a file from a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers store metadata for the distributed filesystem, and cache and ensure data consistency for the data stored in the cloud storage systems. During operation, a cloud controller receives a request from a client to delete a file from the distributed filesystem. The cloud controller updates a user view of the distributed filesystem to present the appearance of the target file being deleted to the client, and then initiates a background deletion operation to delete the target file without negatively affecting the performance of the other users of the distributed filesystem.

Abstract: The disclosed embodiments provide a system that adjusts the characteristics of a distributed filesystem using a locality policy. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a locality policy that specifies one or more management policies for the cloud controller. The portion of the distributed filesystem's data that is managed, created, and/or cached at the cloud controller is then managed based on this locality policy. Locality policies facilitate customizing and optimizing data management for the distributed filesystem to fit the needs of an organization (e.g., specific sets of users, applications, and/or datasets).

Abstract: The disclosed embodiments disclose techniques for providing distributed change notifications for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in the cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a request from a client to access a file in the distributed filesystem. The cloud controller determines that the file needs strong read-after write consistency and sends a registration notification to the other cloud controllers of the distributed filesystem to request immediate notification of any changes to the file.

Abstract: The disclosed embodiments disclose techniques that facilitate the recovery of a virtual machine using a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem in a local storage pool. During operation, a host server executes program instructions for an application in a virtual machine (VM); data associated with this application and/or this virtual machine is stored in the distributed filesystem. Upon detecting a subsequent failure, the system can recover and resume the execution of the virtual machine and application using the previous application and virtual machine data that was stored in the distributed filesystem.

Abstract: The disclosed embodiments disclose techniques for sending interim notifications to a client of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a client request to access a file. The cloud controller determines that it will need to contact at least one of another peer cloud controller or a cloud storage system to service the request, and sends an interim notification to the client to notify the client that the request is pending.

Abstract: The disclosed embodiments disclose techniques for facilitating access to a remote cloud service via a cloud controller for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in the cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller detects a request from a co-located client to access a network address that is in the same subnet of their local network. The cloud controller determines that the network address is associated with the remote cloud service (which executes in a cloud storage environment) and forwards the request to the remote cloud service.

Abstract: The disclosed embodiments disclose techniques for maintaining global name consistency for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a client request to perform a namespace operation upon a filesystem object. The cloud controller contacts the cloud controller that manages (“owns”) the portion of the global namespace for the distributed filesystem that includes the filesystem object. This second cloud controller ensures the consistency of the filesystem object across the distributed filesystem during the namespace operation.

Abstract: The disclosed embodiments disclose techniques for sending interim notifications to clients of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a client request to perform a namespace operation upon a filesystem object. The cloud controller determines that it will need to contact at least one of another peer cloud controller or a cloud storage system to service the request, and sends an interim notification to the client to notify the client that the request is pending.

Abstract: The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller receives a request from a client for a data block of a file stored in the distributed filesystem. Upon determining that the requested data block is not currently cached in the cloud controller, the cloud controller sends a peer cache request for the requested data block to a peer cloud controller in the distributed filesystem.

Abstract: The disclosed embodiments provide a system that distributes data for a distributed filesystem across multiple cloud storage systems. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. Whenever each cloud controller receives new data from a client, it outputs an incremental metadata snapshot for the new data that is propagated to the other cloud controllers and an incremental data snapshot containing the new data that is sent to a cloud storage system. During operation, a backup cloud controller associated with the distributed filesystem is also configured to receive each (incremental) metadata snapshot, such that, upon determining the failure of a cloud controller, the backup cloud controller can immediately begin receiving data requests from clients associated with the failed cloud controller.

Abstract: The disclosed embodiments provide a system that archives data for a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. During operation, a cloud controller determines that a cloud file in a previously stored data snapshot is no longer being actively referenced in the distributed filesystem. The cloud controller transfers this cloud file from the (first) cloud storage system to an archival cloud storage system, thereby reducing storage costs while preserving the data in the cloud file in case it is ever needed again.

Abstract: The disclosed embodiments provide a system that distributes data for a distributed filesystem across multiple cloud storage systems. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers cache and ensure data consistency for the stored data. Whenever each cloud controller receives new data from a client, it outputs an incremental metadata snapshot for the new data that is propagated to the other cloud controllers and an incremental data snapshot containing the new data that is sent to a cloud storage system. During operation, data stored in the distributed filesystem can be distributed across two or more cloud storage systems to optimize performance and/or cost for the distributed filesystem.

Abstract: The disclosed embodiments provide a system that facilitates pre-fetching data for a distributed filesystem. During operation, a cloud controller (e.g., a computing device that caches data from the distributed filesystem) that maintains a set of metadata for the distributed filesystem receives a request to access a data block for a file. The cloud controller traverses the metadata to identify a metadata entry that is associated with the block, and then uses this metadata entry to download a cloud file containing the data block from a cloud storage system. While performing these operations, the cloud controller additionally determines that an additional cloud file in the cloud storage system includes data that is likely to be accessed in conjunction with the data block, and proceeds to pre-fetch this additional cloud file from the cloud storage system.

Abstract: An apparatus and a method for delegating commands for a network attached storage system is described. A command permission associated with a command to be executed on a Network Attached Storage (NAS) server is delegated to a user of a NAS client having a mounted network filesystem of the NAS server. The command is issued through the filesystem of the NAS client.

Abstract: According to one embodiment of the invention, a method is devised for tailoring downloaded content to the requesting device by receiving attribute parameters associated with a device that initiates a request for downloaded content. The attribute parameters are values that identify specific characteristics of the device. Thereafter, the content is modified according to the attributes parameters, and thereafter, the modified content is transmitted to the device.