Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for a unique directory in the hierarchical file system is received. The sub-file system of the hierarchical file system containing the unique directory is provided while ensuring that each directory resides in only one sub-file system. The system also maintains, in a top-file system, a corresponding directory and a symbolic pointer which points from the corresponding directory to a sub-file system where a given directory resides. Creation of new directories in sub-file systems also includes creation of corresponding directories in the top-file system and symbolic pointers from corresponding directory to new directory.

Abstract: One embodiment provides a method for file system namespace rebuilding. The method includes creating attribute data structures for a top-file system and sub-file system hierarchy system. The attribute data structures including hierarchy relationship information. The attribute data structures are stored in the sub-file systems. The top-file system namespace is rebuilt by extracting the hierarchy relationship information from an extended attribute of the attribute data structures in each stub of each sub-file system to build a table. The top-file system hierarchy is built one level at a time starting with the root directory having a parent of NULL.

Abstract: One embodiment provides a method for re-balancing data and metadata across multiple sub-file systems of a file system. The method includes determining sub-file systems including an amount of data that exceeds a threshold. At least one cell in the sub-file systems is identified as a candidate for re-balancing. A re-balance process is performed on the at least one cell that includes performing a flush operation to flush dirty data from file system buffers, copying an inode table for an independent set of files in the at least one cell to a destination sub-file system, notifying an allocation manager for the destination sub-file system of a new storage pool to manage, and performing an un-quiesce operation on I/O operations to each element in the at least one cell.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a corresponding directory in a top-file system and a number of symbolic pointers in the corresponding directory which each point from the corresponding directory to a sub-file system where any portion of the tracked directory resides. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory does not exist in the second sub-file system, the system creates the second portion of the directory in the second sub-file system and creates a link from the first portion of the directory to the second portion of the directory.

Abstract: One embodiment provides a method for storage management in an aggregated file system. The method includes tracking inode numbers for sub-file systems of the aggregated file system. A request for a unique range of inode numbers in the aggregated file system is received. The unique range of inode numbers is provided to one or more sub-file systems of the aggregated file system while providing that each sub-file system of the aggregated file system has a required range of inode numbers and that no two sub-file systems have overlapping inode numbers.

Abstract: One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for creation of an additional directory in the hierarchical file system is received. In response to the request, the additional directory is created in a sub-file system of the hierarchical file system while providing that each directory resides in only one sub-file system. If the additional directory is a child directory, the additional directory is created in the sub-file system containing the parent directory of the child directory.

Abstract: Embodiments for using a migration data placement tool between storage systems based on data access by a processor. Events from a plurality of heterogeneous storage systems may be indexed into a common repository. A migration data placement tool uses analytics to determine data placement of the data in a target storage system according to the data identified as hot data, warm data, or cold data based on the events indexed in the common repository. The data may be placed according to a migration template during migration to the target storage system.

Abstract: Embodiments for a system using a migration data placement tool between storage systems based on data access by a processor. Data may be cataloged in a migration data placement tool by receiving events from a plurality of heterogeneous storage systems. The events may be migrated into the migration data placement tool using analytics to determine data placement of the data in a target storage system according to the data identified as hot data, warm data, or cold data. The data may be placed according to the migration template during migration to the target storage system.

Abstract: Embodiments relate to enabling sequential access for data scans on files with metadata. More specifically, object files are stored in a data storage. The objects are classified. A metaobject file is created in the data storage. The metaobject file is created by extracting attributes from the file from objects in a first classification. An entry is created in the metaobject file and the extracted attributes are stored in that entry. A file attribute is extracted from a file in the second classification and associated with the created entry in the metaobject file. A link is created between the file in the second classification and the related entry in the metaobject file.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression for efficient random access to the data. A first partition of a first data block of a compression group is compressed. The first compressed partition is stored in a first compression entity. An in-memory table is maintained, which includes updating the in-memory table with data associated with an address of the stored compressed first partition. At such time as it is determined that the first compression entity is full, the in-memory table is compressed and written to the first compression entity. Accordingly, the in-memory table, which stores partition compression data, is store with the compression entity.

Abstract: A method for secure portable operating system interface (POSIX) directory traversing for opening and accessing files by inode number. The method includes receiving, by a process executed by a processor, a request for a ticket for traversing a file system. The process generates a secure key for a unique handle object based on the request for the ticket. An authentication code is generated for the ticket using a numeric file identifier and the secure key. In response to reading a directory with POSIX x and r permissions according to directory permission bits or an access control list (ACL), the ticket is returned including ticket information including the numeric file identifier, generation information and the authentication code. In response to a request to open a directory, the ticket information is validated based on the secure key. A directory is opened for reading using the validated ticket information and the unique handle.

Abstract: A technique for restoring file systems by applying full and incremental backups together while streaming them from multiple devices provides a faster system restore from a full backup where incremental backups must also be applied. A first storage device streams the full backup while at least one second storage device streams at least one incremental backup. As files are detected in the incremental backup, they are written instead of the corresponding file in the full backup. Incremental backups can be pre-merged to reduce their number to one less than a number of storage devices available to stream the backups, so that the full backup can be streamed along with the pre-merged incremental backup(s) to restore the file system.

Abstract: A key-value store for storing and retrieving user files based on key-value pairs, hereafter referred to as KVPs. For each user of the key-value store, each file of a set of files of said each user is stored as one or more pairs of KVPs. Each of said one or more pairs comprises a KVP of a first type and a KVP of a second type. The KVP of the first type comprises at least a part of contents of said each file, whereas the KVP of the second type comprises metadata (and possibly attributes) of said each file. Each KVP of the second type links to one or more KVPs of the first type. Further provided are related method and computer program products.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A first compression is performed on a first set of data to create first compressed data. The first compressed data is stored in one or more blocks of a first compression group. A size of free space of a last block of the first compression group is discovered and calculated. A second compression is performed on a second set of data to create second compressed data. At least a portion of the second compressed data is supplied to the first compression group for padding into the last block in response to determining that the size of the free space is sufficient. An unpadded portion of the second compressed data is stored in one or more blocks of a second compression group.

Abstract: A method for secure portable operating system interface (POSIX) directory traversing for opening and accessing files by inode number. The method includes receiving, by a process executed by a processor, a request for a ticket for traversing a file system. The process generates a secure key for a unique handle object based on the request for the ticket. An authentication code is generated for the ticket using a numeric file identifier and the secure key. In response to reading a directory with POSIX x and r permissions according to directory permission bits or an access control list (ACL), the ticket is returned including ticket information including the numeric file identifier, generation information and the authentication code. In response to a request to open a directory, the ticket information is validated based on the secure key. A directory is opened for reading using the validated ticket information and the unique handle.

Abstract: A method for early compression related processing in a file system with offline compression. The method includes receiving a data file in a buffer. A processor detects that at least a portion of a data block of the data file resides in the buffer. A compressibility indication of the data block is determined based on performing at least one compressibility analysis operation on the data block. The compressibility indication of the data block is stored. A background compression task is performed on the data block based on: determining a compression decision for the data block based on the compressibility indication, and compressing the data block based on the compression decision.

Abstract: In an approach to backup and restore one or more clone file trees, one or more processors generate a clone management table. One or more processors record a clone management information in the clone management table about each clone file. To backup a clone file, one or more processors update the clone management information, and store one or more data blocks that are in the clone file trees. To restore a clone file, one or more processors analyze a clone management information and restore one or more data blocks in the clone file. To restore a clone file parent, one or more processors analyze a clone management information and restore one or more data blocks in the clone file parent. To restore a clone file tree, one or more processors analyze a clone management information and restore one or more data blocks in the clone file tree.

Abstract: In an approach to backup and restore one or more clone file trees, one or more processors generate a clone management table. One or more processors record a clone management information in the clone management table about each clone file. To backup a clone file, one or more processors update the clone management information, and store one or more data blocks that are in the clone file trees. To restore a clone file, one or more processors analyze a clone management information and restore one or more data blocks in the clone file. To restore a clone file parent, one or more processors analyze a clone management information and restore one or more data blocks in the clone file parent. To restore a clone file tree, one or more processors analyze a clone management information and restore one or more data blocks in the clone file tree.

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.