Abstract: A key value store is maintained at a destination and includes pages of a B+ tree holding filing metadata of first and second files belonging to a logical partition of a filesystem. The pages are associated with KVS keys, each KVS key including a logical partition id, snapshot id, and page number. The snapshot id corresponds to a current snapshot id at a time the pages were written to the KVS. A clone is created at the destination. The parent is assigned a first logical partition id, clone child is assigned a second logical partition id, and current snapshot id is incremented. The destination receives changes to the first and second files. As soon as updates are applied to one of the first or second files, the updated file is scanned for viruses while updates continue to be applied to another of the first or second files.

Abstract: A key value store is maintained in which the key value store includes pages of a B+ Tree structure holding file metadata of a logical partition of a filesystem. The pages are associated with keys through which the pages are retrieved from the key value store. The keys are named. A clone is created of the logical partition, the logical partition now being a parent and the clone now being a child. The naming of the keys is used to allow the pages stored in the key value store to be read by any of the parent or child. The naming of the keys is used to allow any of the parent or child to make updates to the pages, where updated pages are owned by a respective parent or child that made the updates, and pages not updated remain accessible by any of the parent or child.

Abstract: Page keys are formed in a key-value store (KVS) storing metadata of a filesystem. Page keys identify pages of trees within which namespaces of logical partitions are held. Leaf pages include first and second full keys for each file. First full keys index inodes. Second full keys index filenames. Inode keys are formed in the KVS. Inode keys identify inodes of the files. Metadata is embedded into the page and inode keys such that each page key includes an identifier of a logical partition, an identifier of a snapshot taken of a namespace of the logical partition, and a page number to a page tree holding the namespace. Each inode key includes the logical partition identifier, the snapshot identifier, and an inode number of an inode to a file. Upon damage to the namespace, queries are conducted of the KVS and the namespace is recovered based on the query results.

Abstract: Page keys are formed in a key-value store (KVS) storing metadata of a filesystem. Page keys identify pages of trees within which namespaces of logical partitions are held. Leaf pages include first and second full keys for each file. First full keys index inodes. Second full keys index filenames. Inode keys are formed in the KVS. Inode keys identify inodes of the files. Metadata is embedded into the page and inode keys such that each page key includes an identifier of a logical partition, an identifier of a snapshot taken of a namespace of the logical partition, and a page number to a page tree holding the namespace. Each inode key includes the logical partition identifier, the snapshot identifier, and an inode number of an inode to a file. Upon damage to the namespace, queries are conducted of the KVS and the namespace is recovered based on the query results.

Abstract: Described is a system (and method) that maintains deduplication efficiency when storing data within a clustered data storage environment that implements a global namespace. To provide such a capability, the system may obtain granular data source identifying information from a client system that provides data to be backed-up by a backup component. The data source identifying information may take the form of a placement tag that is associated with the received data. The backup component may then provide such placement tags when providing the backup data to the clustered storage system. The placement tags may then be used to intelligently distribute backup files to particular storage nodes of the clustered storage system to improve deduplication efficiency.

Abstract: Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata for the file. If a legacy location in the metadata for the file includes any value which was stored after a most recent replication involving the file, the source node identifies a key for the legacy location in the metadata for the file. The source node adds the grouping tag as a value for the identified key. The source node sends the metadata for the file, including the identified key, to a destination node.

Abstract: Systems and methods for reconstructing a namespace in response to a loss of data are described herein. To reconstruct the namespace, metadata super segments within a set of file containers may be located in response to a determination that original namespace metadata associated with files stored within the containers is unreadable. Content handles linking the super segment to metadata for each of the files may then be reconstructed from the super segments. Both inode numbers and M-tree identifiers are also extracted from the super segments for each of the files referenced by the super segments. Then, a plurality of U-trees is assembled by aggregating and linking the inode numbers of the files referenced within the super segments based on the M-tree identifiers. Finally, the recovered namespace metadata tree data structure for all of the files within the plurality of LP containers is assembled by aggregating and linking the U-trees.

Abstract: Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata. If the legacy location in the metadata includes any value which was stored after the most recent replication involving the file, and the size of the legacy location in the metadata was not changed after the most recent replication involving the file, the system adds the grouping tag to the legacy location in the metadata which includes any value which was stored after the most recent replication involving the file. The system sends the metadata, which includes the grouping tag and any value stored at the legacy location in the metadata, to a destination node.

Abstract: A destination node receives metadata for a file, via replication, from a source node. If a legacy location in the metadata for the file does not include any new value for a legacy location in local metadata for the file, then the destination node stores the metadata to a local data structure. If the legacy location in the metadata includes any new value for the legacy location in the local metadata, and the new value is any new grouping tag, then the destination node extracts the new grouping tag from the legacy location in the metadata. The destination node constructs a key based on the legacy location in the metadata and uses the constructed key to search for an alternative location in the metadata for the file, for the new grouping tag. The destination node stores the new grouping tag in the alternative location in the metadata.

Abstract: A destination node receives metadata for a file, via replication, from a source node. If a legacy location in the metadata for the file does not include any new value for a legacy location in local metadata for the file, then the destination node stores the metadata to a local data structure. If the legacy location in the metadata includes any new value for the legacy location in the local metadata, and the new value is any new grouping tag, then the destination node extracts the new grouping tag from the legacy location in the metadata. The destination node constructs a key based on the legacy location in the metadata and uses the constructed key to search for an alternative location in the metadata for the file, for the new grouping tag. The destination node stores the new grouping tag in the alternative location in the metadata.

Abstract: Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata for the file. If a legacy location in the metadata for the file includes any value which was stored after a most recent replication involving the file, the source node identifies a key for the legacy location in the metadata for the file. The source node adds the grouping tag as a value for the identified key. The source node sends the metadata for the file, including the identified key, to a destination node.

Abstract: Replication of tags in global scale systems is described. A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata. If the legacy location in the metadata includes any value which was stored after the most recent replication involving the file, and the size of the legacy location in the metadata was not changed after the most recent replication involving the file, the system adds the grouping tag to the legacy location in the metadata which includes any value which was stored after the most recent replication involving the file. The system sends the metadata, which includes the grouping tag and any value stored at the legacy location in the metadata, to a destination node.

Abstract: A source node creates a file and a grouping tag for the file. The source node stores the grouping tag in a location in metadata for the file. When preparing for replication, the source node retrieves the grouping tag from the location in the metadata. If the legacy location in the metadata includes any value which was stored after the most recent replication involving the file, and the size of the legacy location in the metadata was changed after the most recent replication involving the file, then the system enables a destination node to store the grouping tag in the legacy location in the metadata by sending records comprising a request to replace a record stored at the legacy location in the metadata with a larger record, and combining the grouping tag with any value which changed the size of the metadata after a most recent replication involving the file.

Abstract: Described is a system (and method) that intelligently distributes data within a clustered storage environment. To provide such a capability, the system may distribute backup files by considering a source of the data to be backed-up. In particular, the system may leverage the ability of front-end components such as a backup application to perform a granular data source identification of data. Such information may be propagated to back-end components such as a storage filesystem in the form of a data source identifier (e.g. placement tag). The data source identifiers may then be accessed by the clustered storage system to intelligently distribute backup files amongst a set of storage nodes forming a cluster. For example, backup files from the same data source may be stored on the same storage node to obtain the same deduplication efficiency as a single storage system.

Abstract: Disclosed herein are techniques for generating fractional checkpoints on a distributed file system by snapshotting subcomponents referred to as “file sets.” The techniques include capturing a present time; calculating from configured parameters a future wakeup time for a subsequent set of checkpoints from the present time; walking a database of meta file set objects to locate at least one meta file set object; calculating a retention period for a snapshot that is being created; and generating a global data-less snapshot for the meta file set object and remote data file set objects associated with the meta file set object, and then repeating the process for subsequent file set objects in the database.

Abstract: Described is a system that provides the ability to distribute backup data according to performance characteristics associated with nodes that form a clustered storage environment. For example, the clustered storage environment may organize nodes into tiers that have certain storage performance-related characteristics such as read/write latency, deduplication processing efficiency, bandwidth, etc. Accordingly, the system may associate the backup data with an identifier for an appropriate storage performance tier. The identifier may then be utilized by the clustered storage environment to redirect storage operations to a node within the appropriate storage performance tier. Thus, the system allows for the distribution of backup data to particular storage performance tiers based on particular storage or workload requirements.

Abstract: In one example, a method includes receiving metadata in the form of a modification to metadata represented by a file system namespace abstraction, wherein the file system namespace abstraction corresponds to less than an entire file system namespace, and the file system namespace abstraction includes one or more pages, and one of the pages corresponds to a particular cached block, updating the file system namespace abstraction based on the received metadata, determining if caching is enabled for the file system namespace abstraction, when caching is enabled for the file system namespace abstraction, caching the updated file system namespace abstraction in SSD storage that includes the cached block, and maintaining a status of the cached block in a Dtable of the SSD storage.

Abstract: Disclosed herein are techniques for generating fractional checkpoints on a distributed file system by snapshotting subcomponents referred to as “file sets.” The techniques include capturing a present time; calculating from configured parameters a future wakeup time for a subsequent set of checkpoints from the present time; walking a database of meta file set objects to locate at least one meta file set object; calculating a retention period for a snapshot that is being created; and generating a global data-less snapshot for the meta file set object and remote data file set objects associated with the meta file set object, and then repeating the process for subsequent file set objects in the database.

Abstract: Described is a system for reclaiming unused storage space when distributing backup data to storage performance-based tiers within a clustered storage environment. The clustered storage environment may organize nodes into storage performance tiers, and each node within a tier may be tuned for a particular type of workload. Accordingly, the system implements a file system that provides the ability to reclaim unused storage space (e.g. perform garbage collection) for specific nodes or tiers independently despite implementing a global namespace. The global namespace may group (or co-locate) aspects of file information with the backup files. For example, file information may be stored on the same node that stores the associated backup files. Accordingly, the system may reclaim unused storage space for each node, or set of nodes forming a storage performance tier, independently and efficiently.

Abstract: In one example, a method includes receiving metadata in the form of a modification to metadata represented by a file system namespace abstraction, wherein the file system namespace abstraction corresponds to less than an entire file system namespace, and the file system namespace abstraction includes one or more pages, and one of the pages corresponds to a particular cached block, updating the file system namespace abstraction based on the received metadata, determining if caching is enabled for the file system namespace abstraction, when caching is enabled for the file system namespace abstraction, caching the updated file system namespace abstraction in SSD storage that includes the cached block, and maintaining a status of the cached block in a Dtable of the SSD storage.