Abstract: A method is used in analyzing a storage system using a machine learning system. Data gathered from information associated with operations performed in a storage system is analyzed. The storage system is comprised of a plurality of components. A bitmap image is created based on the gathered data, where at least one of the plurality of components is represented in the bitmap image. The machine learning system is trained using the bitmap image, where the bitmap image is organized to depict the plurality of components of the storage system.

Abstract: Techniques for detecting state changes in a system may include receiving a first neural network that is trained to detect when the system transitions into a first resulting state, wherein the system transitions into at least a first intermediate state prior to transitioning into the final resulting state; training the first neural network using a first plurality of inputs denoting the system in the first intermediate state; obtaining a plurality of sets of internal state information of the first neural network, each set of the plurality of sets denoting an internal state of the first neural network at a different point in time after the first neural network has processed at least a portion of the first plurality of inputs; and training a second neural network, using the plurality of sets of internal state information, to detect the first intermediate state.

Abstract: A method system, and computer program product for improving data reduction using aggregate machine learning systems comprising receiving, by an aggregating machine learning system from one or more machine learning systems associated with a set of one or more storage arrays, a first set of output parameters indicative of performance metrics for the set of the one or more storage arrays, aggregating, by the aggregating machine learning system, the first set of output parameters, resulting in a second set of output parameters, and sending, from the aggregating machine learning system, at least one member of the second set of output parameters as an input to at least one of the one or more machine learning systems associated with the set of the one or more storage arrays.

Abstract: Methods and apparatus are provided for filtering dynamically loadable namespaces (DLNs). An exemplary method comprises, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of a storage system; obtaining filtering directives from a user; reducing the DLN using a filtering mechanism on a directory tree associated with the DLN, based on the filtering directives, by removing files in the directory tree of the DLN that do not satisfy requirements of the filtering directives to generate a filtered DLN; and dynamically loading the filtered DLN, including reduced metadata for the filtered DLN relative to the DLN, from the capacity tier into a performance tier of the storage system for processing by the application.

Abstract: Methods and apparatus are provided for reorganizing dynamically loadable namespaces (DLNs). In one exemplary embodiment, a method comprises the steps of, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system; obtaining one or more reordering directives from a user; rearranging one or more files in the DLN into a new directory hierarchy based on the one or more reordering directives to generate a reordered DLN; and dynamically loading the reordered DLN, including the metadata only for the reordered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by the application. The reordered DLN is merged into the DLN following one or more modifications to the reordered DLN.

Abstract: A multi-tier storage system is provided with direct client access to an archive storage tier for input/output operations. An exemplary method comprises communicating over a network with (i) a cluster file system on a first storage tier, and (ii) a second archive storage tier comprising an object store; providing a client of the cluster file system with access to one or more files in the cluster file system on the first storage tier; and executing a translation shim to provide the client of the cluster file system with one or more of read and write access to one or more files on the second archive storage tier. The translation shim converts between the protocols of the cluster file system and the protocols of the second archive storage tier, to allow unmodified applications to optionally access the second archive storage tier using existing cluster file system protocols.

Abstract: A method system, and computer program product for improving data reduction using aggregate machine learning systems comprising receiving, by an aggregating machine learning system from one or more machine learning systems associated with a set of one or more storage arrays, a first set of output parameters indicative of performance metrics for the set of the one or more storage arrays, aggregating, by the aggregating machine learning system, the first set of output parameters, resulting in a second set of output parameters, and sending, from the aggregating machine learning system, at least one member of the second set of output parameters as an input to at least one of the one or more machine learning systems associated with the set of the one or more storage arrays.

Abstract: Techniques for determining data reduction options may include: receiving first data reduction information regarding compression and deduplication of chunks of a data set; determining, in accordance with the first data reduction information for the data set, first settings denoting whether compression and deduplication are enabled or disabled for the data set; receiving, during a first time period when the first settings are effective, writes directed to the data set; receiving second data reduction information regarding compression and deduplication of chunks of the data set modified by writes during the first time period; and determining, in accordance with the second plurality of data reduction statistics for the data set, second settings denoting whether compression and deduplication are enabled or disabled for the data set.

Abstract: An apparatus in one embodiment comprises a multi-tier storage system having at least a front-end storage tier, a back-end storage tier and a data mover module configured to control movement of data between the storage tiers. The multi-tier storage system is configured to receive a plurality of files, such as relatively small size files containing data from respective Internet of Things (IoT) devices. The front-end storage tier comprises a file system and at least one key-value store. The multi-tier storage system is configured to combine multiple ones of the received files written to at least one specified directory of the file system into an object for storage in association with a key in the key-value store. The object comprises multiple sub-keys identifying respective ones of the multiple files within the object. The data mover module may be configured to copy the object from the key-value store to the directory.

Abstract: Techniques for detecting state changes in a system may include receiving a first neural network that is trained to detect when the system transitions into a first resulting state, wherein the system transitions into at least a first intermediate state prior to transitioning into the final resulting state; training the first neural network using a first plurality of inputs denoting the system in the first intermediate state; obtaining a plurality of sets of internal state information of the first neural network, each set of the plurality of sets denoting an internal state of the first neural network at a different point in time after the first neural network has processed at least a portion of the first plurality of inputs; and training a second neural network, using the plurality of sets of internal state information, to detect the first intermediate state.

Abstract: Techniques for determining data reduction options may include: receiving first data reduction information regarding compression and deduplication of chunks of a data set; determining, in accordance with the first data reduction information for the data set, first settings denoting whether compression and deduplication are enabled or disabled for the data set; receiving, during a first time period when the first settings are effective, writes directed to the data set; receiving second data reduction information regarding compression and deduplication of chunks of the data set modified by writes during the first time period; and determining, in accordance with the second plurality of data reduction statistics for the data set, second settings denoting whether compression and deduplication are enabled or disabled for the data set.

Abstract: An apparatus in one embodiment comprises a storage system having at least first and second storage tiers each comprising a plurality of storage devices. The storage system implements a parallel file system comprising at least first and second sets of storage servers. The first set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the first storage tier, and the second set of storage servers is configured to access directories of the parallel file system that are stored in storage devices of the second storage tier. The storage system is further configured to move data between the first and second storage tiers by moving data files between different directories of the respective first and second storage tiers. One or more servers of the first set may be generated as respective new server instances on one or more compute nodes.

Abstract: A System, Computer program product, and computer-executable method of authenticating a user to a remote computing network, the System, Computer program product, and computer-executable including receiving a first portion of authentication data at a server, wherein the first portion of authentication data includes at least one feature of an image and analyzing the received first portion of authentication data by comparing the first portion of authentication data to a second portion of authentication data stored at the server.

Abstract: A computer-executable method, system, and computer program product for managing a cloud computing resource in communication with a data storage system, the computer-executable method, system, and computer program product comprising receiving configuration information at the data storage system related to a host application; determining an optimized configuration for the host application; and creating an implementation of the optimized configuration based on one or more resources enabled to be accessed by the data storage system.

Abstract: A cluster file system comprises a front-end file system, a back-end file system, data mover modules arranged between the front-end and back-end file systems, and a quota manager associated with at least a given one of data mover modules. The data mover modules are configured to control archiving of data between the front-end file system and the back-end file system for respective users based at least in part on respective user quotas established by the quota manager and identifying corresponding portions of the back-end file system available to the users. The front-end file system may comprise archive directories configured for respective ones of the users, with the data mover modules being configured to archive a given file from the front-end file system in the back-end file system responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system.

Abstract: A storage system comprises a shared storage environment that includes a storage array having at least one storage volume shared between first and second host devices. The storage system further comprises a server associated with the storage array, at least first and second clients associated with the respective first and second host devices, and a first block cache arranged between the first client and the storage array. The server is configured to coordinate operations of the first and second clients relating to the storage volume shared between the first and second host devices in a manner that ensures coherency of data stored in the first block cache. The server may comprise a storage block mapping protocol (SBMP) server and the first and second clients may comprise respective SBMP clients. The block cache is illustratively implemented using a VFCache or other type of server flash cache.

Abstract: A method is used in analyzing a storage system using a machine learning system. Data gathered from information associated with operations performed in a storage system is analyzed. The storage system is comprised of a plurality of components. A bitmap image is created based on the gathered data, where at least one of the plurality of components is represented in the bitmap image. The machine learning system is trained using the bitmap image, where the bitmap image is organized to depict the plurality of components of the storage system.

Abstract: A method is used in predicting performance of applications using machine learning systems. A machine learning system is trained on a sample server executing an application. An expected performance of the application is determined using the machine learning system for a server having different characteristics than the sample server by predicting the expected performance of the application on the server without having to actually measure a performance of the application on the server.

Abstract: Example embodiments of the present invention relate to a method and an apparatus for automatic cleanup of disfavored content. The method includes scanning a storage infrastructure for content having an attribute, identifying a disfavored instance of the content having the attribute, and removing the disfavored instance of the content from circulation.

Abstract: Techniques are provided for scheduling computational tasks among multiple classes of storage resources based on a job classification. A job to be executed is classified into one of a plurality of predefined job classes. Each predefined job class is associated with a corresponding one of a plurality of predefined storage classes. The job is then assigned based on the classification to one of the storage resources of the predefined storage class associated with the classified predefined job class. Exemplary predefined storage classes include a performance class, a capacity class, a key-value storage class, and a shingled disk drive class. Exemplary predefined job classes include a CPU Intensive job class, an IO Intensive job class and a Small IO job class. Data required for a job is optionally prefetched before the job is assigned to a storage device. Data objects to be evicted from a storage device are optionally selected based on an anticipated future access.