Abstract: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Abstract: Embodiments include a data aware deduplicating object store. The data aware deduplicating data store includes a consistent hashing logic that manages a consistent hashing architecture for the object store. The consistent hashing architecture includes a metadata ring and a bulk ring. The consistent hashing architecture may be a multiple ring architecture comprising a metadata ring and two or more bulk rings. A bulk ring may include a key/value (k/v) data store, where a k/v data store stores a shard of an index and a reference count that facilitates the individual approach to garbage collection or data reclamation. The data aware deduplicating data store also includes a deduplication logic that provides data deduplication for data to be stored in the object store. The deduplication logic performs variable length deduplication and provides a shared nothing approach.

Abstract: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Abstract: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Abstract: Embodiments include a data aware deduplicating object store. The data aware deduplicating data store includes a consistent hashing logic that manages a consistent hashing architecture for the object store. The consistent hashing architecture includes a metadata ring and a bulk ring. The consistent hashing architecture may be a multiple ring architecture comprising a metadata ring and two or more bulk rings. A bulk ring may include a key/value (k/v) data store, where a k/v data store stores a shard of an index and a reference count that facilitates the individual approach to garbage collection or data reclamation. The data aware deduplicating data store also includes a deduplication logic that provides data deduplication for data to be stored in the object store. The deduplication logic performs variable length deduplication and provides a shared nothing approach.

Abstract: Example apparatus and methods identify files that are so small or so large that they compromise the efficient operation of a file system that uses re-assignable one-to-one inodes and inode numbers. Small files are aggregated into collections of files and large files are subdivided into collections of smaller files. Information for locating multiple related files with fewer lookups is generated and stored in a folder. An inode having a new type of inode number is then created. The new type of inode number encodes information for finding the folder. The encoded information may include a folder identifier that acts as a primary key into a database that is configured to locate a member of the aggregated or subdivided files with a single lookup. A filter file system may be updated with the new inode. The new inode number is unique within the filter file system and may not be re-assigned.

Abstract: A method and apparatus for scheduling data movers is described. The method and apparatus comprise monitoring utilization of a resource used by a data mover to process a data flow, determining rank information based on the utilization of the resource and scheduling a task according to the rank information.

Abstract: A method and apparatus for optimizing a de-duplication rate for backup streams is described. In one embodiment, the method for optimizing data de-duplication using an extent mapping of a backup stream includes processing a backup stream to access an extent mapping associated with a plurality of data files, wherein the plurality of the data files are arranged within the backup stream and examining the extent mapping to identify at least one extent group within the backup stream, wherein the plurality of the data files are de-duplicated using at least one location of the at least one extent group.

Abstract: Method and apparatus for allocating resources among backup tasks in a data backup system is described. One aspect of the invention relates to managing backup tasks in a computer network. An estimated resource utilization is established for each of the backup tasks based on a set of backup statistics. A resource reservation is allocated for each of the backup tasks based on the estimated resource utilization thereof. The resource reservation of each of the backup tasks is dynamically changed during performance thereof.

Abstract: Method and apparatus for allocating resources among backup tasks in a data backup system is described. One aspect of the invention relates to managing backup tasks in a computer network. An estimated resource utilization is established for each of the backup tasks based on a set of backup statistics. A resource reservation is allocated for each of the backup tasks based on the estimated resource utilization thereof. The resource reservation of each of the backup tasks is dynamically changed during performance thereof.