Abstract: File names and inodes are indexed directly in a filesystem namespace consisting of a layered set of key value store (KVS) database logical tables. The KVS is used directly for organizing file and directory metadata on a cluster file system instead of traditional Btree structures. Embodiments provide file deletion in the file system namespace, and other features include snapshot deletion and cleaning the namespace. A name hash index is used to speed filename lookup in the file system namespace. Two key types of INODE and DIRENT are used to encode namespace metadata using self-identify variable length records in the namespace. Point-in-time snapshot copies are also formalized using KVS database operations as well operations on sets of snapshots for operations including garbage collection and data replication.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. In an offline condition, the filesystem no longer processes read/write requests due to a system crash and requires a verification and recovery of a namespace of the system. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. In an offline condition, the filesystem no longer processes read/write requests due to a system crash and requires a verification and recovery of a namespace of the system. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for performing load balancing in a filesystem of a distributed network comprising worker nodes accessing shared resources that are controlled by distributed locks to grant and prevent access from clients, by assigning a range of directory entries to each worker node, and providing feedback from each worker node to a load balancer to expose an internal state of acquired resources and other parameters of each worker node. In response to periodic updates of the directory ranges to be collocated, the system maintains directory ranges in each worker so that complete subtrees of the directory tree fit in a single range assigned to a worker. Upon receipt of a client request to access a data asset, the load balancer selects the most optimum worker to satisfy the request based on the worker internal states and other parameters.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: A method of generating a hybrid snapshot includes receiving a request to generate a snapshot of a distributed file system and identifying a first storage resource of the distributed file system and a second storage resource of the distributed file system based on the request. The method further includes generating the snapshot of the distributed file system, the snapshot including a data-full snapshot of the first storage resource and a data-less snapshot of the second storage resource.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for providing adaptive namespace verification with high efficiency. The verification process is used in modern filesystems to check the integrity of key data structures, such for backup systems where providing thorough and efficient mechanisms for verification is critical. Embodiments accelerate the verification process by selectively applying continuous, differential, and integral verification techniques for startup and runtime repair purposes to overcome present problems for system capacities that exceed the petabyte range and impose significant time requirements using existing verification methods. Embodiments work with both single-node and distributed filesystems.

Abstract: Embodiments for enabling granular migration of data with high efficiency. A defined metadata element, a tag, is assigned to each file, and then tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer, and such a group of tags is referred to as a tag set. Embodiments can be used with a defined backup system file migration process, such as present in the Data Domain File System. By using snapshots, incoming new data (ingested file) is allowed to continue while the migration is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method is efficient became it makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Abstract: Embodiments for granular replication of data with high efficiency. A defined metadata element embodied as a tag is assigned to each file. Tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer. Embodiments can be used with a defined backup system file replication process, such as present in the Data Domain File System. By using snapshots, incoming new data is continued to be ingested while the replication is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method efficiently makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Abstract: Embodiments for performing load balancing in a filesystem of a distributed network comprising worker nodes accessing shared resources that are controlled by distributed locks to grant and prevent access from clients, by assigning a range of directory entries to each worker node, and providing feedback from each worker node to a load balancer to expose an internal state of acquired resources and other parameters of each worker node. In response to periodic updates of the directory ranges to be collocated, the system maintains directory ranges in each worker so that complete subtrees of the directory tree fit in a single range assigned to a worker. Upon receipt of a client request to access a data asset, the load balancer selects the most optimum worker to satisfy the request based on the worker internal states and other parameters.

Abstract: Embodiments for granular replication of data with high efficiency. A defined metadata element embodied as a tag is assigned to each file. Tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer. Embodiments can be used with a defined backup system file replication process, such as present in the Data Domain File System. By using snapshots, incoming new data is continued to be ingested while the replication is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method efficiently makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Abstract: Embodiments for enabling granular migration of data with high efficiency. A defined metadata element, a tag, is assigned to each file, and then tag filtering is used to direct the data to the proper location. Files with different tags can be selected for transfer, and such a group of tags is referred to as a tag set. Embodiments can be used with a defined backup system file migration process, such as present in the Data Domain File System. By using snapshots, incoming new data (ingested file) is allowed to continue while the migration is in process and maintaining data consistency at the same time. This is achieved by performing operations on B+ Tree snapshots in conjunction with tag filtering on keys present in the leaf pages of these structures. This method is efficient became it makes a single pass walk of a B+ Tree in contrast with previous methods that look up files one-by-one via their pathname.

Abstract: A method of capturing a fractional snapshot of a distributed file system includes receiving a snapshot request to generate a factional snapshot of a file system. The request includes an indication of a subset of data of the file system. The method further includes identifying, by a processing device, the subset of data of the file system indicated by the snapshot request and generating the fractional snapshot comprising the subset of data of the file system indicated by the snapshot request.

Abstract: A method of generating a hybrid snapshot includes receiving a request to generate a snapshot of a distributed file system and identifying a first storage resource of the distributed file system and a second storage resource of the distributed file system based on the request. The method further includes generating the snapshot of the distributed file system, the snapshot including a data-full snapshot of the first storage resource and a data-less snapshot of the second storage resource.