Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. A computing device receives a data access request for an encoded data slice (EDS) associated with a data object. The computing device compares a slice name of the data access request with slice names stored within RAM. When the data access request slice name compares unfavorably with those stored slice names, the computing device transmits an empty data access response that includes no EDS to the other computing device without needing to access a hard disk drive (HDD) that stores EDSs. Alternatively, the computing device transmits a data access response that includes the EDS.

Abstract: Methods for use in a dispersed storage network (DS) to determine appropriate resources for storing data. An example method, implemented by one or more devices of a dispersed storage network (DSN), includes obtaining storage characteristics relating to data stored in a first pool of storage units associated with a first storage tier, the data stored as a set of encoded data slices. Based on the storage characteristics, the method determines to move the data to a target storage pool of storage units associated with a second storage tier. The method also determines whether to re-encode the data for storage in the target storage pool of storage units. When not re-encoding the data, the method includes retrieving the set of encoded data slices, translating associated slice names into translated slice names, and facilitating storage of the encoded data slices in the target storage pool utilizing the translated slice names.

Abstract: A method of predictive data streaming in a virtual storage system, including: identifying, for a plurality of data objects that have been migrated from a first tier of storage in a virtual storage system to a lower tier of storage, metadata describing one or more access patterns for the plurality of data objects; determining, based at least on the metadata describing the plurality of data objects, one or more data objects that have a greater likelihood of being accessed relative to other data objects among the plurality of data objects; and transferring, from the lower tier of storage, the one or more data objects that have a greater likelihood of being accessed relative to other data objects among the plurality of data objects.

Abstract: A method includes identifying an existing logical storage vault having existing dispersed storage coding properties for vault transformation, where a first set of storage units support the existing logical storage vault, and a data object of first data objects stored within the first set of storage units is stored as a first plurality of sets of encoded data slices in accordance with the existing dispersed storage coding properties. The method includes identifying a new logical storage vault having new dispersed storage coding properties, wherein storage units support the new logical storage vault. The method includes transforming the first data objects from being in accordance with the existing dispersed storage coding properties to being in accordance with the new dispersed storage coding properties to produce transformed first data objects. The method includes storing the transformed first data objects in the new logical storage vault supported by the storage units.

Abstract: A method of data loss prevention during transitions from a replication source, the method including: initiating a transition from using a first data repository as a source for data replication to using a second data repository as the source for data replication; prior to completing the transition and responsive to determining that one or more storage operations directed to the first data repository have not been replicated to the second data repository, delaying transition of the second data repository as the source for data replication; and responsive to completing the transition, promoting the second data repository as the source for data replication such that all storage operations received after completing the transition are directed to the second data repository.

Abstract: A method for execution by a computing device of a dispersed or distributed storage network begins or continues by, for a data access request, accessing a plurality of estimated efficiency models of a plurality of processing units of the storage network, where an estimated efficiency model of the plurality of estimated efficiency models includes a list of estimated efficiency probabilities, and the list of estimated efficiency probabilities corresponds to a list of data access request types for a processing unit of the plurality of processing units. The method continues by selecting one of the processing units from the plurality of processing units based on the plurality of estimated efficiency models to produce a selected processing unit. The method continues by sending the data access request to the selected processing unit for execution.

Abstract: A storage system is provided. The storage system includes a first storage cluster, the first storage cluster having a first plurality of storage nodes coupled together and a second storage cluster, the second storage cluster having a second plurality of storage nodes coupled together. The system includes an interconnect coupling the first storage cluster and the second storage cluster and a first pathway coupling the interconnect to each storage cluster. The system includes a second pathway, the second pathway coupling at least one fabric module within a chassis to each blade within the chassis.

Abstract: A distributed storage network method begins by determining an encoded data slice to rebuild, determining a storage set of distributed storage units associated with the encoded data slice to rebuild, determining an historical performance level of each distributed storage unit of the storage set of distributed storage units, determining an estimated performance level of each distributed storage unit of the storage set of distributed storage units, selecting at least a decode threshold number of distributed storage units of the storage set of distributed storage units to produce selected distributed storage units, determining an encoded data slice partial aggregation scheme for the selected distributed storage units, sending encoded data slice partial request messages to the selected distributed storage units, and receiving encoded data slice partial response messages and extracting encoded data slice partials from the encoded data slice partial response messages to rebuild or verify the encoded data slice.

Abstract: Executing a big data analytics pipeline in a storage system that includes compute resources and shared storage resources, including: receiving, from a data producer, a dataset; storing, within the storage system, the dataset; allocating processing resources to an analytics application; and executing the analytics application on the processing resources, including ingesting the dataset from the storage system.

Abstract: A system and method for effectively scheduling read and write operations among a plurality of solid-state storage devices. A computer system comprises client computers and data storage arrays coupled to one another via a network. A data storage array utilizes solid-state drives and Flash memory cells for data storage. A storage controller within a data storage array comprises an I/O scheduler. The storage controller is configured to receive a read request targeted to the data storage medium, and identify at least a first storage device of the plurality of storage devices which contains data targeted by the read request. In response to either detecting or predicting the first storage device will exhibit variable performance, the controller is configured to generate a reconstruct read request configured to obtain the data from one or more devices of the plurality of storage devices other than the first storage device.

Abstract: A storage controller coupled to a storage array comprising one or more storage devices that performs at least one data reduction operation on decrypted data, encrypts the reduced data using a second encryption key to generate a second encrypted data, and stores the second encrypted data on the storage array.

Abstract: A processor in a storage network operates by: receiving an access request for a data segment, wherein the data segment is encoded utilizing an error correcting information dispersal algorithm as a set of encoded data slices that are stored in a plurality of storage units of the storage network and wherein each encoded data slice of the set of encoded data slices includes a corresponding checksum of a plurality of checksums; retrieving, from the storage network, a subset of encoded data slices that includes a threshold number of encoded data slices of the set of encoded data slices; determining, based on ones of the plurality of checksums corresponding to the subset of encoded data slices, when the subset of encoded data slices includes at least one corrupted encoded data slice; retrieving from at least one of the plurality of storage units an addition number of encoded data slices required to generate a reconstructed data segment based on the subset of encoded data slices; generating the reconstructed data se

Abstract: A method for execution by one or more modules of one or more processors of a storage network includes receiving a data object for storage, segmenting the data object into a plurality of data segments and determining a level of security and a level of performance for the plurality of data segments. The method continues by determining whether one or more data segments of the plurality of data segments is to be transformed using an all-or-nothing transformation and in response to a determination to transform one or more data segments of the plurality of data segments, transforming a data segment of the plurality of data segments to produce a transformed data segment. The method continues by dispersed error encoding the transformed data segment to produce a set of encoded data slices and transmitting the set of encoded data slices to a set of storage units of the storage network.

Abstract: A processing module of a dispersed storage network determines an obfuscation method is determined from a plurality of obfuscation methods for a data segment. The method continues with the processing module obfuscating the data segment according to the obfuscation method to produce an obfuscated data segment. The obfuscated data segment is encrypted and dispersed storage error encoded to produce a set of encoded data slices. The set of encoded data slices is then transmitted for storage in the dispersed storage network.

Abstract: Intelligently moving data between storage systems, including: identifying characteristics associated with a plurality of data communications paths between a source storage system and a plurality of target storage systems; identifying characteristics associated with a plurality of data communications paths between two or more target storage systems; selecting, using the characteristics associated with the plurality of data communications paths between the source storage system and the plurality of target storage systems and the characteristics associated with a plurality of data communications paths between two or more target storage systems, an optimal path between the source storage system and each of the target storage systems; and migrating data stored on the source storage system to each of the target storage systems utilizing the optimal paths.

Abstract: A method for adaptive concurrency for write persistence in a storage system, performed by the storage system, is provided. The method includes selecting a write process from among a plurality of write processes, responsive to receiving a write request for writing data into the storage system, and writing the data into the storage system in accordance with the selected write process. One of the plurality of write processes includes transferring the data into the storage system, locking an inode associated with file information of the data in memory, updating the file information in the inode while the inode is locked, committing the data while the inode is locked, and unlocking the inode.

Abstract: A source storage unit of a storage network operates by: sending a slice to migrate from the source storage unit to a destination storage unit; sending a slice verification request to the destination storage unit, wherein the slice verification request includes one or more of: a slice identifier, the slice, a revision indicator, a verification method indicator, or a nonce; receiving an integrity value from the destination storage unit; determining when the integrity value compares favorably to the slice verification request by determining when a hash of the slice and a nonce matches the integrity value or determining when a decrypted signature of the integrity value matches a hash of the slice and the nonce or the slice and the nonce; when the integrity value compares favorably to the slice verification request, updating a slice identifier assignment associated with a slice identifier corresponding to the slice, wherein updating the slice identifier assignment associates the destination storage unit with the s

Abstract: A plurality of encoded data slices are stored at locations within a memory, and a list of pointers to the encoded data slices is maintained. A request to delete a particular encoded data slice is received. In response to receiving the request, a particular location within the memory used to store the particular encoded data slice is determined based on the list of pointers, and an updated list of pointers is generated indicating that the particular encoded data slice is no longer stored at the particular location within the memory.

Abstract: Placing data within a storage device, including: receiving, by a storage device, information describing an expected longevity of data stored on the storage device; determining, by the storage device, a location for storing the data in dependence upon the expected longevity of the data; adjusting a garbage collection schedule in dependence upon data placement; and providing, to a storage array controller, garbage collection statistics.

Abstract: A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The computing device obtains a data identifier associated with a data object and determines DSN address(es) associated with storage of one or more encoded data slice(s) (EDS(s)). The computing device selects slice names based on the DSN address(es) and issues at least a read threshold number of read slice requests using slice names to at least some storage units (SUs). When an insufficient number of EDSs is received, the computing device issues an alternate read slice request to an alternate SU. When a sufficient number of EDSs is received from the alternate SU and the computing device has received at least the read threshold number of EDSs, the computing device reconstructs the data segment.