Abstract: Provided are a computer program product, integrated cache manager, and method for maintaining data in a first level memory and buckets representing regions of memory devices to extend data cache. A plurality of buckets represent distinct regions of memory devices. The buckets are associated with different threshold access count ranges. Data having an access count is stored in one of the buckets associated with a threshold access count range including the access count of the data to store. Data evicted from a first level memory is copied to an initial bucket comprising one of the buckets. Data is moved from a source bucket comprising one of the buckets, including the initial bucket, to a target bucket of the buckets having a target threshold access count range including an access count of the data to move.

Abstract: A computer-implemented method according to one embodiment includes monitoring for occurrence of trigger events corresponding to creation of metadata in a computing environment, and detecting occurrence of a trigger event. The method further includes loading a rule that is associated with the detected trigger event in response to detecting occurrence of the trigger event. The rule includes a condition and an action, and the action defines a subset of metadata associated with processing a job in the computing environment. The method further includes collecting the defined subset of metadata in accordance with the action in response to the condition of the rule being met, and storing the collected subset of metadata. A system according to another embodiment includes a processor, and logic integrated with the processor, executable by the processor, or integrated with and executable by the processor. The logic is configured to perform the foregoing method.

Abstract: Embodiments for storage resource utilization analytics using metadata tags by a processor. Storage capacity utilization in the plurality of heterogeneous storage systems may be determined using one or more events indexed into a centralized search index.

Abstract: Provided are a computer program product, integrated cache manager, and method for maintaining data in a first level memory and buckets representing regions of memory devices to extend data cache. A plurality of buckets represent distinct regions of memory devices. The buckets are associated with different threshold access count ranges. Data having an access count is stored in one of the buckets associated with a threshold access count range including the access count of the data to store. Data evicted from a first level memory is copied to an initial bucket comprising one of the buckets. Data is moved from a source bucket comprising one of the buckets, including the initial bucket, to a target bucket of the buckets having a target threshold access count range including an access count of the data to move.

Abstract: One embodiment provides a method for optimizing data read-ahead for machine learning applications including obtaining, by a processor, next file information from a workflow scheduler for next input files for a next processing stage that are to be accessed by a machine learning application. Data for the next processing stage for the machine learning application and at least one system job is prefetched. The next input files are prefetched as the prefetching data reaches an end of current inputs. A schedule is caused to chain the next input files for the next processing stage to current input files of a current processing stage.

Abstract: One embodiment provides a method for storage management in a hierarchical file system that includes creating a structure for the hierarchical file system including a top-file system and sub-file systems that each include files and directories. Directories for the sub-file systems of the hierarchical file system are tracked. A request for creation of a unique directory in the hierarchical file system is received. Upon a request by a particular sub-file system for more inodes without any inodes available, inode numbers are revoked from at least one other sub-file system having unused inodes.

Abstract: In one embodiment, a computer-implemented method for namespace and content manipulation within a storage system includes: defining policy rule(s), each policy rule dictating: how namespaces within the storage system may be transformed; and/or how content stored within the storage system may be transformed; receiving a request for access to a particular data set stored within the storage system; parsing the request to determine one or more transformations requested therein and/or required thereby; determining whether the particular data set is compliant with the one or more transformations; in response to determining the particular data set is compliant with the one or more transformations, providing access to the particular data set; and in response to determining the particular data set is not compliant with the one or more transformations, performing the one or more transformations to generate a transformed data set, and providing access to the transformed data set.

Abstract: Embodiments relate to policy-based, multi-scheme data reduction for a computer memory. An aspect includes receiving a plurality of policy rules by a policy engine of the computer memory, wherein a first policy rule specifies applying a first data reduction scheme to data in the computer memory based on the data matching first characteristics, wherein a second policy rule specifies applying a second data reduction scheme to data in the computer memory based on the data matching second characteristics, wherein the first data reduction scheme is different from the second data reduction scheme. Another aspect includes determining, by the policy engine, that first data in the computer memory matches the first characteristics, and that second data in the computer memory matches the second characteristics. Yet another aspect includes applying the first data reduction scheme to the first data, and applying the second data reduction scheme to the second data.

Abstract: Embodiments for namespace management between storage systems by a processor. Events, containing metadata associated with an operation, from a plurality of heterogeneous storage systems may be received such that the events are indexed into a common repository according to the metadata to maintain a single namespace for data in the plurality of heterogeneous storage systems.

Abstract: One embodiment provides a method for optimizing data read-ahead for machine learning applications including obtaining, by a processor, next file information from a workflow scheduler for next input files for a next processing stage that are to be accessed by a machine learning application. Data for the next processing stage for the machine learning application and at least one system job is prefetched. The next input files are prefetched as the prefetching data reaches an end of current inputs. A schedule is caused to chain the next input files for the next processing stage to current input files of a current processing stage.

Abstract: A computer-implemented method according to one embodiment includes identifying an event associated with a high speed storage tier of a data storage system, determining a policy rule that is triggered in response to the event, and implementing one or more data management actions associated with the high speed storage tier according to the policy rule.

Abstract: One embodiment provides a method for optimizing data read-ahead for workflow and analytics applications including obtaining, by a processor, next file information from a workflow scheduler for next files for a next processing stage that are to be accessed by a process. Data for the next processing stage for at least one application and at least one system job is prefetched. The next files are prefetched as the prefetching data reaches an end of current inputs.

Abstract: In an approach to maintain data integrity in a computer data storage system, a clone management table is generated for a clone file tree that includes a relationship between at least one parent file and a clone file. The parent file is restored by: (i) determining that the parent-child relationship exists between the parent file and the clone file based on an entry in the clone management table, (ii) restoring data blocks in the parent file to the file system based on an analysis of an entry in the clone management table for the parent file and an identification of data blocks included in the clone file, and (iii) generating a new instance of the parent file based on the data blocks included in the file system and the entry in the clone management table for the clone file tree.

Abstract: A computer-implemented method according to one embodiment includes identifying an event within a system; applying a model to the event to create an aggregated event; and storing the aggregated event.

Abstract: Embodiments for triggering custom metadata extraction by a processor. Information may be extracted from an event so as to access data across a plurality of heterogeneous storage systems. Metadata may be extracted from the data that is accessed such that the metadata is assigned as custom metadata and indexed into a centralized search index, wherein the custom metadata is correlated to existing metadata associated with the data in the centralized search index.

Abstract: A computer-implemented method according to one embodiment includes identifying an accessing of a file within an operating system, checking an in-memory cache for path information associated with the file, checking an external cache for the path information associated with the file, conditionally retrieving the path information associated with the file by performing a file system lookup and adding the path information associated with the file to the in-memory cache and the external cache, and returning the path information.

Abstract: A computer-implemented method according to one embodiment includes monitoring for occurrence of trigger events corresponding to creation of metadata in a computing environment, and detecting occurrence of a trigger event. The method further includes loading a rule that is associated with the detected trigger event in response to detecting occurrence of the trigger event. The rule includes a condition and an action, and the action defines a subset of metadata associated with processing a job in the computing environment. The method further includes collecting the defined subset of metadata in accordance with the action in response to the condition of the rule being met, and storing the collected subset of metadata. A system according to another embodiment includes a processor, and logic integrated with the processor, executable by the processor, or integrated with and executable by the processor. The logic is configured to perform the foregoing method.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A first compression is performed on a first set of data to create first compressed data. The first compressed data is stored in one or more blocks of a first compression group. A size of free space of a last block of the first compression group is discovered and calculated. A second compression is performed on a second set of data to create second compressed data. At least a portion of the second compressed data is supplied to the first compression group for padding into the last block in response to determining that the size of the free space is sufficient. An unpadded portion of the second compressed data is stored in one or more blocks of a second compression group.

Abstract: The embodiments described herein relate to managing compressed data to optimize file compression. A compression is performed on a first set of data to create a first set of compressed data partitions in a compression group. A partition table is constructed, and partition entries for the first data block are added to the table in conjunction with the first set. A current size of the compression group is assessed. In response to a compression ratio being greater than a target compression ratio and internal fragmentation of the compression group being smaller than a threshold, the compression group is dynamically completed. The dynamic completion decides a size for the compression group. The partition table is added to the compression group by assigning space within the first compression group for the table. The compression group is written to persistent storage.

Abstract: A computer-implemented method according to one embodiment includes identifying an event associated with data stored in a data storage system, determining a policy rule that is triggered in response to the event, and implementing one or more actions according to the policy rule.