Abstract: An illustrative storage system provides distributed storage for use by containerized applications running in a container system. In some implementations, this includes converting data blocks of storage resources to distributed persistent storage such as thin-provisioned volumes for use by the containerized applications. In some implementations, the conversion includes using a device mapper to virtualize the data blocks of the storage resources to create a virtual storage pool backing datastore in which the thin-provisioned volumes are created.

Abstract: A method for execution by one or more computing devices of a storage network includes dispersed storage error encoding a data segment of data into a set of encoded data slices in accordance with an information dispersal algorithm width number and a decode threshold number, where the decode threshold number of encoded data slices of the set of encoded data slices are needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in primary storage slots of a set of storage units deployed across multiple sites of the storage network in accordance with a mapping of the primary storage slots to storage units of the set of storage units, where a number of the primary storage slots is equal to or greater than the information dispersal algorithm width number.

Abstract: A method for execution by a computing device of a storage network includes detecting a shutdown associated with a local flash memory of the storage network, where a plurality of data segments are dispersed storage error encoded in accordance with distributed data storage parameters to produce pluralities of sets of encoded data slices, and the pluralities of sets of encoded data slices include encoded data slices stored in the local flash memory. The method further includes determining at least one storage location for transferring storage of a group of the encoded data slices. The method further includes transferring the group of the encoded data slices to the at least one storage location.

Abstract: An illustrative method includes a controller associated with a plurality of clusters receiving, from a user, a disaster recovery policy of a containerized application deployed on a first cluster in the plurality of clusters, determining a cluster profile of each second cluster among one or more second clusters that are distinct from the first cluster in the plurality of clusters, identifying, from the one or more second clusters, a particular cluster based on the disaster recovery policy of the containerized application and the cluster profile of each second cluster, and assigning the particular cluster to be a disaster recovery cluster for the containerized application.

Abstract: A method for execution by one or more computing devices of a massive data ingestion system includes receiving data for storage in the massive data ingestion system. The method further includes storing a data segment of the data in a cache memory of the massive data ingestion system. The method further includes retrieving, based on one or more retrieval parameters, the data segment from the cache memory. The method further includes error encoding, in accordance with error encoding parameters, the data segment to produce a set of encoded data slices, wherein the error encoding parameters include a pillar width number and a decode threshold number, and wherein the decode threshold number is a minimum number of encoded data slices needed to reconstruct the data segment. The method further includes storing the set of encoded data slices in a set of storage units of the massive data ingestion system.

Abstract: A method for execution by a storage network begins by receiving data for storage by the storage network and continues by determining data preparation tasks for the data. The method continues by indexing the data in accordance with the data preparation tasks to generate a data index and processing the data in accordance with the data index to produce indexed data. The method then continues by determining distribution criteria for the data based on the data index and distributing the data and the data index to a set of distributed storage units in accordance with the distribution criteria, Finally, the method establishes criteria for analyzing found data of the data in the storage network.

Abstract: A method for execution by a storage network begins by receiving a data object for storage, segmenting the data according to a data segmenting protocol to generate a set of data segments and executing a deterministic function on the set of data segments to generate scoring information for respective data segments of the set of data segments. The method continues by comparing the scoring information for a respective data segment to scoring information for previously stored data segments in the storage network and based on the comparison, facilitating storage of a first portion of the set of data segments and not storing a second portion of the set of data segments.

Abstract: Methods and apparatus for storing data and metadata in a storage network. In a method, one or more processing modules of a storage network establishes a vault as a virtual memory block that maps to memory of the storage network. The method further includes receiving data for storage in the vault, selecting a first vault parameter set regarding distributed storage of the data, and encoding at least a portion of the data in accordance with the first vault parameter set to produce a set of encoded data slices. The method further includes obtaining metadata regarding the set of encoded data slices, selecting a second vault parameter set regarding distributed storage of the metadata, and encoding the metadata in accordance with the second vault parameter set to produce metadata encoded slices. The method further includes facilitating storage of the set of encoded slices and the metadata encoded slices in the vault.

Abstract: A method for execution by one or more computing devices of a storage network includes determining an encoded data slice of a set of encoded data slices needs rebuilding during an encoded data slice reduction operation. The method further includes determining whether a current number of encoded data slices of the set of encoded data slices that are not in need of rebuilding is less than a reduced rebuild threshold number associated with the encoded data slice reduction operation. When the current number is less than reduced rebuild threshold number, the method further includes rebuilding the encoded data slice.

Abstract: A method for execution in a storage network begins by determining to adjust a version number associated with a data object stored in the storage network, where the data object was dispersed storage error encoded to produce a set of encoded data slices. The method continues by identifying a number of versions associated with the data object and based on the number of versions identified and the adjusted version number associated with the data object, generating metadata associated with the data object. The method then continues by issuing a write request for each encoded data slice of the set of encoded data slices to the set of storage units, where each write request includes the metadata associated with the data object.

Abstract: A processing system of a storage network operates by: selecting a queue memory type of a plurality of memory types to store a data object, based on a size parameter associated with the data object; storing the data object in a queue memory device having the queue memory type, when the queue memory type is selected; selecting a main memory type of a plurality of memory types to store the data object, when the queue memory type is not selected; and storing the data object in a main memory device having the main memory type, when the queue memory type is not selected; wherein the data object is dispersed error encoded and stored as a plurality of encoded data slices.

Abstract: A method includes error encoding data to produce a plurality of data slices. Metadata is determined for a data slice of the plurality of data slices. The metadata is stored in a metadata storage tree. The data slice is stored in a slice storage location indicated by the metadata. Based on determining to access the data slice, the metadata for the data slice is accessed in the metadata storage tree to determine the slice storage location for the data slice, and the data slice is accessed in the slice storage location based on determining the slice storage location for the data slice via accessing the metadata storage tree.

Abstract: A storage network operates by: encoding, via a dispersed error encoding, at least one data object as a write threshold number of encoded data slices; storing the write threshold number of the encoded data slices in a set of storage units of the storage network; determining when, due to a failure, less than a pillar width number of the encoded data slices are retrievable from the set of storage units: retrieving a decode threshold number of the encoded data slices from others of the set of storage units; decoding the decode threshold number of the encoded data slices to reproduce the at least one data object; restoring availability of the pillar width number of the encoded data slices of the common revision by: rebuilding at least a subset of encoded data slices of the set of encoded data slices, based on an encoding of the at least one data object; and storing the at least a subset of encoded data slices in the storage network.

Abstract: A processing system operates by: detecting an access anomaly associated with an access request from a requestor for a set of encoded data slices, the access anomaly having an unfavorable access pattern, wherein the set of encoded data slices is dispersed storage error encoded and stored in at least one storage unit of the storage network; denying the access request in response to detecting the access anomaly; generating, based on the unfavorable access pattern, an anomaly detection indicator identifying the requestor; and sending the anomaly detection indicator to other devices of the storage network.

Abstract: A processing system of a storage network operates by receiving a write request associated with a data object; identifying dispersed storage error encoding parameters; and determining a number of available storage units. When the number of available storage units is determined to exceed a first number, the processing system operates by: determining a first write threshold number in accordance with the dispersed storage error encoding parameters; dispersed error encoding the data object to produce a first write threshold number of encoded data slices; and writing, to available storage units, the first write threshold number of encoded data slices.

Abstract: An illustrative method includes receiving, by a container storage interface (CSI) filter driver executing on a node of a cluster managed by a container orchestrator, a request to deploy a containerized application; selecting, by the CSI filter driver based on one or more attributes of the containerized application, a storage system from a plurality of storage systems attached to a plurality of clusters managed by the container orchestrator; and transmitting, by the CSI filter driver, a command to a CSI driver executing on a node of a cluster to which the storage system is attached, the cluster included in the plurality of clusters, the command configured to direct the CSI driver to provision a volume on the storage system for use with the containerized application.

Abstract: Methods and apparatus for replication of data in storage networks. In an embodiment, a processing module(s) of a computing device identifies a first storage set and a second storage set for replicated storage of data. The processing module(s) maintains a synchronization log for the first storage set and the second storage set. After initiating storage of a data object in the first storage set and the second storage set, the processing module(s) detects an unavailability of one or more storage units of the second storage set. In response to detecting a return to availability of the one or more storage units, the processing module(s) determines, based on the synchronization log, that the second storage set requires at least a portion of the data object to maintain synchronization with the first storage set, and facilitates storage of the at least a portion of the data object in the second storage set.

Abstract: A distributed storage integrity system in a dispersed storage network includes a scanning agent and a control unit. The scanning agent identifies an encoded data slice that requires rebuilding, wherein the encoded data slice is one of a plurality of encoded data slices generated from a data segment using an error encoding dispersal function. The control unit retrieves at least a number T of encoded data slices needed to reconstruct the data segment based on the error encoding dispersal function. The control unit is operable to reconstruct the data segment from at least the number T of the encoded data slices and generate a rebuilt encoded data slice from the reconstructed data segment. The scanning agent is located in a storage unit and the control unit is located in the storage unit or in a storage integrity processing unit, a dispersed storage processing unit or a dispersed storage managing unit.

Abstract: A computing device of a storage network, the computing device including memory, an interface, and at least one processing module operably coupled to the memory and the interface, where the at least one processing module is operable to error encode a data segment in accordance with error encoding parameters to produce a set of encoded data slices. The at least one processing module also sends via the interface, a first subset of encoded data slices of the set of encoded data slices via a first routing path of a plurality of routing paths to a set of storage units, where the first routing path has a performance level greater than a first performance threshold, and sends, via the interface, a second subset of encoded data slices of the set of encoded data slices via a second routing path of the plurality of routing paths to the set of storage units.

Abstract: A device for use in a storage network including storage network memory distributed amongst a plurality of storage units. The device includes a processing module operably couple to a memory that stores operational instructions. The processing module is configured to execute the operational instructions to determine storage parameters associated with encoded data slices generated from data to be stored in the plurality of storage units. The storage parameters include information indicating a read threshold number of encoded data slices required to recover the data. The method further includes facilitating distributed storage of the encoded data slices among the plurality of storage units such that the number of the error-encoded data slices stored in any particular storage unit is chosen so that in the event of an unavailability of any individual storage unit, at least a read threshold number of encoded data slices are still accessible from the remaining storage units.