Abstract: Methods and apparatus are provided for filtering dynamically loadable namespaces (DLNs). An exemplary method comprises, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of a storage system; obtaining filtering directives from a user; reducing the DLN using a filtering mechanism on a directory tree associated with the DLN, based on the filtering directives, by removing files in the directory tree of the DLN that do not satisfy requirements of the filtering directives to generate a filtered DLN; and dynamically loading the filtered DLN, including reduced metadata for the filtered DLN relative to the DLN, from the capacity tier into a performance tier of the storage system for processing by the application.

Abstract: A computer-executable method, system, and computer program product for managing metadata in a distributed data storage system, wherein the distributed data storage system includes a first burst buffer having a key-value store enabled to store metadata, the computer-executable method, system, and computer program product comprising receiving, from a compute node, metadata related to data stored within the distributed data storage system, indexing the metadata at the first burst buffer, and processing the metadata in the first burst buffer.

Abstract: Techniques are provided for pattern-aware prefetching using a parallel log-structured file system. At least a portion of one or more files is accessed by detecting at least one pattern in a non-sequential access of the one or more files; and obtaining at least a portion of the one or more files based on the detected at least one pattern. The obtaining step comprises, for example, a prefetching or pre-allocation of the at least the portion of the one or more files. A prefetch cache can store the portion of the one or more obtained files. The cached portion of the one or more files can be provided from the prefetch cache to an application requesting the at least a portion of the one or more files.

Abstract: Methods and apparatus are provided for reorganizing dynamically loadable namespaces (DLNs). In one exemplary embodiment, a method comprises the steps of, in response to a job submitted by an application, obtaining a DLN portion of a global single namespace of a file system, wherein the DLN is associated with the job and is maintained in a capacity tier of object storage of a storage system; obtaining one or more reordering directives from a user; rearranging one or more files in the DLN into a new directory hierarchy based on the one or more reordering directives to generate a reordered DLN; and dynamically loading the reordered DLN, including the metadata only for the reordered DLN, from the capacity tier of object storage into a performance tier of storage of the storage system for processing by the application. The reordered DLN is merged into the DLN following one or more modifications to the reordered DLN.

Abstract: A multi-tier storage system is provided with direct client access to an archive storage tier for input/output operations. An exemplary method comprises communicating over a network with (i) a cluster file system on a first storage tier, and (ii) a second archive storage tier comprising an object store; providing a client of the cluster file system with access to one or more files in the cluster file system on the first storage tier; and executing a translation shim to provide the client of the cluster file system with one or more of read and write access to one or more files on the second archive storage tier. The translation shim converts between the protocols of the cluster file system and the protocols of the second archive storage tier, to allow unmodified applications to optionally access the second archive storage tier using existing cluster file system protocols.

Abstract: An apparatus in one embodiment comprises a multi-tier storage system having at least a front-end storage tier, a back-end storage tier and a data mover module configured to control movement of data between the storage tiers. The multi-tier storage system is configured to receive a plurality of files, such as relatively small size files containing data from respective Internet of Things (IoT) devices. The front-end storage tier comprises a file system and at least one key-value store. The multi-tier storage system is configured to combine multiple ones of the received files written to at least one specified directory of the file system into an object for storage in association with a key in the key-value store. The object comprises multiple sub-keys identifying respective ones of the multiple files within the object. The data mover module may be configured to copy the object from the key-value store to the directory.

Abstract: Techniques are provided for pattern-aware prefetching using a parallel log-structured file system. At least a portion of one or more files is accessed by detecting at least one pattern in a non-sequential access of the one or more files; and obtaining at least a portion of the one or more files based on the detected at least one pattern. The obtaining step comprises, for example, a prefetching or pre-allocation of the at least a portion of the one or more files. A prefetch cache can store the portion of the one or more obtained files. The cached portion of the one or more files can be provided from the prefetch cache to an application requesting the at least a portion of the one or more files.

Abstract: Example embodiments of the present invention relate to a method, a system, and a computer program product for maintaining dynamically loadable namespaces. The method includes presenting, to an application, a file system having a global single namespace (SN) comprising a plurality of dynamically-loadable namespaces (DLNs) maintained in a capacity tier (CT) of object storage of a storage system. In response to a job submitted by the application, dynamically loading a first dynamically-loadable namespace, including the metadata only for the first dynamically-loadable namespace, associated with the job from the capacity tier of object storage into a fast tier (FT) of storage of the storage system.

Abstract: A computer-executable method, system, and computer program product for managing a Parallel-Logged File System (PLFS) on a data storage system, wherein the data storage system is in communication with an application that updates the data storage system with checkpoint data, the computer-executable method, system, and computer program product comprising receiving a first checkpoint including data from the application; analyzing the data in the first checkpoint to determine whether a pattern exists in the checkpoint data; and upon a positive determination, adding the pattern into PLFS.

Abstract: A System, Computer program product, and computer-executable method of authenticating a user to a remote computing network, the System, Computer program product, and computer-executable including receiving a first portion of authentication data at a server, wherein the first portion of authentication data includes at least one feature of an image and analyzing the received first portion of authentication data by comparing the first portion of authentication data to a second portion of authentication data stored at the server.

Abstract: A computer-executable method, system, and computer program product for managing a cloud computing resource in communication with a data storage system, the computer-executable method, system, and computer program product comprising receiving configuration information at the data storage system related to a host application; determining an optimized configuration for the host application; and creating an implementation of the optimized configuration based on one or more resources enabled to be accessed by the data storage system.

Abstract: Methods and apparatus are provided for simulating asynchronous modifications of shared data objects by a number of distributed processes within a distributed application. An exemplary method comprises obtaining a plurality of simulation parameters that define parameters of a simulated distributed application, parameters of a plurality of distributed compute nodes on which the simulated distributed application executes and a storage system employed by the simulated distributed application; and applying the plurality of simulation parameters to a simulation engine, wherein the simulation engine simulates both synchronous and asynchronous modifications of at least one shared data object by a plurality of distributed processes within the simulated distributed application.

Abstract: A cluster file system comprises a front-end file system, a back-end file system, data mover modules arranged between the front-end and back-end file systems, and a quota manager associated with at least a given one of data mover modules. The data mover modules are configured to control archiving of data between the front-end file system and the back-end file system for respective users based at least in part on respective user quotas established by the quota manager and identifying corresponding portions of the back-end file system available to the users. The front-end file system may comprise archive directories configured for respective ones of the users, with the data mover modules being configured to archive a given file from the front-end file system in the back-end file system responsive to a link to that file being stored in a corresponding one of the archive directories of the front-end file system.

Abstract: A cluster file system comprises a burst buffer appliance coupled to a plurality of object storage servers via a network. The burst buffer appliance comprises a controller configured to implement storage tiering control functionality for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers. The burst buffer appliance is thereby configured to control movement of data between the first and second storage tiers. The object storage servers in the first storage tier may be configured to interface with object storage targets of a first type and the object storage servers in the second storage tier may be configured to interface with object storage targets of a second type different than the first type. For example, the object storage targets of the first type may comprise non-volatile electronic storage devices such as flash storage devices, and the object storage targets of the second type may comprise disk storage devices.

Abstract: Example embodiments of the present invention relate to a method and an apparatus for automatic cleanup of disfavored content. The method includes scanning a storage infrastructure for content having an attribute, identifying a disfavored instance of the content having the attribute, and removing the disfavored instance of the content from circulation.

Abstract: Techniques are provided for scheduling computational tasks among multiple classes of storage resources based on a job classification. A job to be executed is classified into one of a plurality of predefined job classes. Each predefined job class is associated with a corresponding one of a plurality of predefined storage classes. The job is then assigned based on the classification to one of the storage resources of the predefined storage class associated with the classified predefined job class. Exemplary predefined storage classes include a performance class, a capacity class, a key-value storage class, and a shingled disk drive class. Exemplary predefined job classes include a CPU Intensive job class, an IO Intensive job class and a Small IO job class. Data required for a job is optionally prefetched before the job is assigned to a storage device. Data objects to be evicted from a storage device are optionally selected based on an anticipated future access.

Abstract: A system, computer program product, and computer-executable method for managing data from an Internet of Things (IoT) Device, the system, computer program product, and computer-executable method comprising receiving data from the IoT Device, analyzing the data using an analytic filter, and updating a mobile device based on the analyzing.

Abstract: Example embodiments of the present invention relate to a method, a system, and a computer program product for managing metadata. The method includes determining a common characteristic of metadata entries for a plurality of files in a directory of a file system and computing a representative metadata entry for the plurality of metadata entries according to the common characteristic of the metadata entries. The plurality of metadata entries in the file system then may be represented according to the computed representative metadata entry.

Abstract: Parallel storage systems are provided with a burst buffer appliance for storage of a partitioned key-value store across a plurality of storage tiers. The burst buffer appliance is configured to store key-value data on first and second storage tiers comprising respective disjoint subsets of storage. One or more servers in the partitioned key-value store provide (i) an interface between one or more local key-value stores and one or more additional storage architectures, and/or (ii) a key value interface that provides access to at least one hardware key-value store. The local key-values are optionally sorted both locally by the local store and globally by the overall MDHIM framework. The local and global sorting can be leveraged to provide batch input/output (IO) operations that aggregate multiple requests into batch operations on one or more sorted ranges.

Abstract: An apparatus in one embodiment comprises a cloud-based web object indexing platform configured to communicate with multiple web servers over at least one network. The cloud-based web object indexing platform comprises a plurality of indexing applications having respective machine learning engines, with a given one of the indexing applications being configured to receive web objects over the network from one or more of the web servers and to process the received web objects utilizing its corresponding machine learning engine. The given indexing application is further configured to generate semantic information for association with particular ones of the web objects based at least in part on processing results provided by the corresponding machine learning engine. The semantic information is made accessible by the cloud-based web object indexing platform in association with the particular web objects over the network, for example, to a search engine or data analytics engine.