Abstract: Embodiments for storage resource utilization analytics using metadata tags by a processor. Storage capacity utilization in the plurality of heterogeneous storage systems may be determined using one or more events indexed into a centralized search index.

Abstract: One embodiment provides a method for optimizing data read-ahead for machine learning applications including obtaining, by a processor, next file information from a workflow scheduler for next input files for a next processing stage that are to be accessed by a machine learning application. Data for the next processing stage for the machine learning application and at least one system job is prefetched. The next input files are prefetched as the prefetching data reaches an end of current inputs. A schedule is caused to chain the next input files for the next processing stage to current input files of a current processing stage.

Abstract: Embodiments for namespace management between storage systems by a processor. Events, containing metadata associated with an operation, from a plurality of heterogeneous storage systems may be received such that the events are indexed into a common repository according to the metadata to maintain a single namespace for data in the plurality of heterogeneous storage systems.

Abstract: One embodiment provides a method for optimizing data read-ahead for machine learning applications including obtaining, by a processor, next file information from a workflow scheduler for next input files for a next processing stage that are to be accessed by a machine learning application. Data for the next processing stage for the machine learning application and at least one system job is prefetched. The next input files are prefetched as the prefetching data reaches an end of current inputs. A schedule is caused to chain the next input files for the next processing stage to current input files of a current processing stage.

Abstract: One embodiment provides a method for optimizing data read-ahead for workflow and analytics applications including obtaining, by a processor, next file information from a workflow scheduler for next files for a next processing stage that are to be accessed by a process. Data for the next processing stage for at least one application and at least one system job is prefetched. The next files are prefetched as the prefetching data reaches an end of current inputs.

Abstract: A computer-implemented method according to one embodiment includes identifying an event within a system; applying a model to the event to create an aggregated event; and storing the aggregated event.

Abstract: Embodiments for triggering custom metadata extraction by a processor. Information may be extracted from an event so as to access data across a plurality of heterogeneous storage systems. Metadata may be extracted from the data that is accessed such that the metadata is assigned as custom metadata and indexed into a centralized search index, wherein the custom metadata is correlated to existing metadata associated with the data in the centralized search index.

Abstract: One embodiment provides a method for optimizing data read-ahead for workflow and analytics applications including obtaining, by a processor, next file information from a workflow scheduler for next files for a next processing stage that are to be accessed by a process. Data for the next processing stage for at least one application and at least one system job is prefetched. The next files are prefetched as the prefetching data reaches an end of current inputs.

Abstract: A method for collecting digital provenance from heterogeneous sources is disclosed. In one embodiment, such a method includes detecting events generated by multiple heterogeneous sources. The method further filters the events to extract relevant events therefrom. The method further transforms the relevant events into a standard format and saves the relevant events in a queue. The relevant events are then consumed from the queue. Consuming relevant events in the queue includes determining actions to be performed for each relevant event in the queue. These actions may include collecting digital provenance associated with the relevant events. The method then executes the actions. A corresponding system and computer program product are also disclosed.

Abstract: Embodiments for triggering custom metadata extraction by a processor. Information may be extracted from an event so as to access data across a plurality of heterogeneous storage systems. Metadata may be extracted from the data that is accessed such that the metadata is assigned as custom metadata and indexed into a centralized search index, wherein the custom metadata is correlated to existing metadata associated with the data in the centralized search index.

Abstract: Embodiments for namespace management between storage systems by a processor. Events, containing metadata associated with an operation, from a plurality of heterogeneous storage systems may be received such that the events are indexed into a common repository according to the metadata to maintain a single namespace for data in the plurality of heterogeneous storage systems.

Abstract: Embodiments for storage resource utilization analytics using metadata tags by a processor. Storage capacity utilization in the plurality of heterogeneous storage systems may be determined using one or more events indexed into a centralized search index.

Abstract: A system, method and computer readable medium for providing a snapshot of a subset of a file system. A first snapshot of a first set of source files in a file system is generated. The first snapshot includes an inode corresponding to each source file in the first set of files. Stored in each inode is a first identifier associated with the first set of files and a second identifier associated with the time of the first snapshot. Next, a second snapshot of a second set of source files is taken. The second snapshot includes an inode corresponding to each source file in the second set of files. Stored in each inode are a first identifier and a second identifier. Subsequent snapshots are taken every first period and every second period for the first set of files and the second set of files, respectively.

Abstract: Space is allocated on data storage devices in proportion to weights associated with the storage devices. The weights can be dynamically adjusted at any time in order to accommodate changes in the system and to better utilize the storage devices. The technique used to perform the allocating is independent of the weights used by the allocating. Further, the allocation technique can accommodate general purpose data streams having varying lengths and/or varying access patterns, as well as special purpose data streams, such as video streams.

Abstract: A system, method and computer readable medium for providing a snapshot of a subset of a file system. A first snapshot of a first set of source files in a file system is generated. The first snapshot includes an inode corresponding to each source file in the first set of files. Stored in each inode is a first identifier associated with the first set of files and a second identifier associated with the time of the first snapshot. Next, a second snapshot of a second set of source files is taken. The second snapshot includes an inode corresponding to each source file in the second set of files. Stored in each inode are a first identifier and a second identifier. Subsequent snapshots are taken every first period and every second period for the first set of files and the second set of files, respectively.