Abstract: A method for distributed file deletion or truncation, performed by a storage system, is provided. The method includes determining, by an authority owning an inode of a file, which authorities own data portions to be deleted, responsive to a request for the file deletion or truncation. The method includes recording, by the authority owning the inode, the file deletion or truncation in a first memory, and deleting, in background by the authorities that own the data portions to be deleted, the data portions in one of a first memory or a second memory. A system and computer readable media are also provided.

Abstract: Generating a transformed dataset for use by a machine learning model in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: storing, within one or more storage systems, a transformed dataset generated by applying one or more transformations to a dataset that are identified based on one or more expected input formats of data received as input data by one or more machine learning models to be executed on one or more servers; and transmitting, from the one or more storage systems to the one or more servers without reapplying the one or more transformations on the dataset, the transformed dataset including data in the one or more expected formats of data to be received as input data by the one or more machine learning models.

Abstract: A system with storage memory and a processing device has a logical deletion to physical erasure time bound. The system dereferences data, responsive to a direction to delete the data. The system monitors physical blocks in storage memory for live data and the dereferenced data. The system cooperates garbage collection with monitoring the physical blocks, so that at least a physical block having the dereferenced data is garbage collected and erased within a logical deletion to physical erasure time bound.

Abstract: Accelerating artificial intelligence workflows, including: receiving, from a computing process of an artificial intelligence workflow, a request for information stored on a data repository; issuing, from a user space of an operating system environment, parallel requests to the data repository using a network protocol that operates serially at the kernel level of the operating system environment; receiving, from the data repository, one or more responses to the parallel requests; and providing, to the computing process of the artificial intelligence workflow and based on the one or more responses to the parallel requests, a response to the request for information.

Abstract: A system for cloud-based file services, comprising: a plurality of single-tenant file system nodes configured to provide file system access to an object store via a plurality of multitenant storage nodes; the plurality of multitenant storage nodes sharing access to the object store; and one or more management nodes configured to provision resources for the plurality of single-tenant file system nodes and the plurality of multitenant storage nodes, including modifying resources within the system.

Abstract: A method for accessing a file in a storage system is provided. The method includes determining, for each file chunk of the file, an authority among differing storage nodes of the storage system and receiving from the authority having ownership of the file chunk, location information for the file chunk. The method includes accessing file chunks of the file as directed by each of the determined authorities.

Abstract: A method of replication in a distributed storage system, performed by the distributed storage system is provided. The method includes managing a first index for data or metadata in a first storage system, the first storage system having a first partitioning scheme. The method includes managing a second index for data or metadata in a second storage system, the second storage system having a second partitioning scheme. The method includes replicating the data or metadata from the first storage system to the second storage system, translating an identifier of the data or metadata from the first storage system, and mapping the replicated data or metadata into the second partitioning scheme, via the translating of the identifier of the data or metadata from the first storage system.

Abstract: Executing a machine learning model in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: receiving, by a graphical processing unit (‘GPU’) server, a dataset transformed by a storage system that is external to the GPU server; and executing, by the GPU server, one or more machine learning algorithms using the transformed dataset as input.

Abstract: A system with storage memory and a processing device has a logical deletion to physical erasure time bound. The system dereferences data, responsive to a direction to delete the data. The system monitors physical blocks in storage memory for live data and the dereferenced data. The system cooperates garbage collection with monitoring the physical blocks, so that at least a physical block having the dereferenced data is garbage collected and erased within a logical deletion to physical erasure time bound.

Abstract: Generating a transformed dataset for use by a machine learning model in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: storing, within one or more storage systems, a transformed dataset generated by applying one or more transformations to a dataset that are identified based on one or more expected input formats of data received as input data by one or more machine learning models to be executed on one or more servers; and transmitting, from the one or more storage systems to the one or more servers without reapplying the one or more transformations on the dataset, the transformed dataset including data in the one or more expected formats of data to be received as input data by the one or more machine learning models.

Abstract: A method of distributed file deletion, performed by a storage system, is provided. The method includes receiving, at the storage system, a request to delete a directory and contents of the directory and adding the directory to a first set, listed in a memory in the storage system. The method includes operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory. The method includes deleting in a distributed manner across the storage system without concern for order, contents of directories, and the directories listed in the second set.

Abstract: A method for accessing a file in a storage system is provided. The method includes determining, for each file chunk of the file, an authority among differing storage nodes of the storage system and receiving from the authority having ownership of the file chunk, location information for the file chunk. The method includes accessing file chunks of the file as directed by each of the determined authorities.

Abstract: A storage system has a first memory, and a second memory that includes storage memory. The storage system has a processing device. The processing device is to select whether to write data to the first memory and write the data from the first memory to the second memory, or to write the data to the second memory bypassing the first memory. The processing device is to write portions of data for storage according to such selection.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the storage nodes. The method includes determining that one of the storage nodes is unreachable and determining to rebuild the user data for the one of the storage nodes that is unreachable. The method includes reading the user data across a remainder of the plurality of storage nodes, using the erasure coding and writing the user data across the remainder of the plurality of storage nodes, using the erasure coding. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: A method of distributed file deletion, performed by a storage system, is provided. The method includes receiving, at the storage system, a request to delete a directory and contents of the directory and adding the directory to a first set, listed in a memory in the storage system. The method includes operating on the first set, by examining each directory in the first set to identify subdirectories, adding each identified subdirectory to the first set as a directory, and adding each examined directory to a second set listed in the memory. The method includes deleting in a distributed manner across the storage system without concern for order, contents of directories, and the directories, listed in the second set.

Abstract: Generating a transformed dataset for use by a machine learning model in an artificial intelligence infrastructure that includes one or more storage systems and one or more graphical processing unit (‘GPU’) servers, including: storing, within one or more storage systems, a transformed dataset generated by applying one or more transformations to a dataset that are identified based on one or more expected input formats of data received as input data by one or more machine learning models to be executed on one or more servers; and transmitting, from the one or more storage systems to the one or more servers without reapplying the one or more transformations on the dataset, the transformed dataset including data in the one or more expected formats of data to be received as input data by the one or more machine learning models.

Abstract: A processor-based method for locating data and metadata closely together in a storage system is provided. The method includes writing a first range of a file and a first metadata relating to attributes of the file into at least one segment controlled by a first authority of the file. The method includes delegating, by the first authority, a second authority for a second range of the file, and writing the second range of the file and second metadata relating to the attributes of the file into at least one segment controlled by the second authority.

Abstract: A storage system has a first memory, and a second memory that includes storage memory. The storage system has a processing device. The processing device is to select whether to write data to the first memory and write the data from the first memory to the second memory, or to write the data to the second memory bypassing the first memory. The processing device is to write portions of data for storage according to such selection.

Abstract: A method for proactively rebuilding user data in a plurality of storage nodes of a storage cluster is provided. The method includes distributing user data and metadata throughout the plurality of storage nodes such that the plurality of storage nodes can read the user data, using erasure coding, despite loss of two of the storage nodes. The method includes determining that one of the storage nodes is unreachable and determining to rebuild the user data for the one of the storage nodes that is unreachable. The method includes reading the user data across a remainder of the plurality of storage nodes, using the erasure coding and writing the user data across the remainder of the plurality of storage nodes, using the erasure coding. A plurality of storage nodes within a single chassis that can proactively rebuild the user data stored within the storage nodes is also provided.

Abstract: A processor-based method for locating data and metadata closely together in a storage system is provided. The method includes writing a first range of a file and a first metadata relating to attributes of the file into at least one segment controlled by a first authority of the file. The method includes delegating, by the first authority, a second authority for a second range of the file, and writing the second range of the file and second metadata relating to the attributes of the file into at least one segment controlled by the second authority.