Abstract: An illustrative method includes determining an encryption indicator for a first recovery dataset by determining a difference in an amount or percentage of incompressible data associated with the first recovery dataset compared to an amount or percentage of incompressible data associated with a second recovery dataset that temporally precedes the first recovery dataset, the encryption indicator representative of data within or represented by the first recovery dataset that cannot be compressed more than a threshold amount; and performing, based on the encryption indicator for the first recovery dataset, an action with respect to the second recovery dataset, wherein the second recovery dataset is usable to restore data maintained by a storage system to a second state corresponding to a second point in time that temporally precedes a first point in time corresponding to the first recovery dataset.

Abstract: In an embodiment, two or more storage systems are requested to prepare respective local checkpoints for a dataset, wherein each of the two or more storage systems stores portion of the dataset. The two or more storage systems are determined to have established the checkpoint. In response to determining that the local checkpoints have been established, a coordinated checkpoint is completed.

Abstract: A method for execution by a computing device of a storage network includes determining an encoded data slice reduction scheme for a set of encoded data slices stored in a set of storage units of the storage network, where a data segment of data is encoded into the set of encoded data slices in accordance with encoding parameters, and where the encoding parameters include a pillar width number and a decode threshold number. The method further includes maintaining storage of the set of encoded data slices in accordance with the encoded data slice reduction scheme, where the maintaining storage includes keeping a number of encoded data slices of the set of encoded data slices equal to or greater than the decode threshold number and less than the pillar width number.

Abstract: Apparatus for tiered storage of data in a storage network. In an example of operation, a computing device receives a data object for storage and forwards the data object for storage in a first plurality of memory devices of a first memory type. The computing device determines a system level storage efficiency for the data object based, at least in part, on a data attribute associated with the data object. The computing device further selects, based at least in part on the system level storage efficiency preference, a second plurality of memory devices comprised of a second memory type. The computing device determines error encoding parameters based on the second plurality of memory devices, retrieves the data object from the first plurality of memory devices, and encodes the data object with the error encoding parameters to generate a plurality of encoded data slices for storage in the second plurality of memory devices.

Abstract: A plurality of storage nodes in a single chassis is provided. The plurality of storage nodes in the single chassis is configured to communicate together as a storage cluster. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. A plurality of compute nodes is included in the single chassis, each of the plurality of compute nodes is configured to communicate with the plurality of storage nodes. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

Abstract: A redundant array of independent drives (RAID) stripe is formed across a set of storage controllers of a plurality of storage controllers, wherein the RAID stripe comprises two or more of a plurality of modular storage devices of at least one of the set of storage controllers. The RAID stripe is written across the set of storage controllers.

Abstract: A method of performing partial redundant array of independent disks (RAID) stripe parity calculations is disclosed. The method includes receiving a last portion of a RAID stripe among multiple portions of the RAID stripe, all portions for a successful write of the RAID stripe being previously received except for the last portion. The method also includes calculating a parity value based on the last portion of the RAID stripe and a previous parity value without calculating the parity value using a previous portion of the RAID stripe. The method further includes writing of the RAID stripe.

Abstract: Providing for high availability in a data analytics pipeline without replicas, including: creating a data analytics pipeline, wherein each component of the data analytics pipeline is deployed within a container; creating a failover container; detecting that a component within the data analytics pipeline has failed; and responsive to detecting that the component within the data analytics pipeline has failed, deploying the component within the data analytics pipeline that has failed in the failover container.

Abstract: Synchronously replicating a dataset across cloud-based storage systems, including adding a cloud-based storage system to a set of storage systems that the dataset is synchronously replicated across, where access operations are applied to the dataset equivalently through all storage systems in the set, all storage systems in the set store a separate copy of the dataset, and operations to modify the dataset performed and completed through any of the storage systems in the set are reflected in access operations to read the dataset, the cloud-based storage system including one or more cloud computing instances executing a storage controller application, a virtual drive layer that includes one or more cloud computing instances with local storage for storing at least a portion of the dataset as block data, and an object storage layer for storing at least a portion of the dataset as object data.

Abstract: In one implementation, storage system includes embedded storage devices, where each embedded storage device includes a direct-mapped solid state drive (SSD) storage portion and storage system controllers. The storage system controllers may be operatively coupled to the embedded storage devices via a bus. The storage system controllers may receive data to be written to the plurality embedded storage devices, select a plurality of available allocation units from the direct-mapped SSD storage portions of the plurality of embedded storage devices, respectively, and calculate a verification signature corresponding to the data. The storage system controllers may also write the data and the verification signature to a first subset of the plurality of available allocation units, calculate an erasure code corresponding to the data and the verification signature, and write the erasure code to a second subset of allocation units.

Abstract: A storage network processing system includes a processor, a network interface and memory that stores operational instructions. The operation instructions enable the processor to receive a data object for storage and dispersed error encode the data object in accordance with dispersed error encoding parameters to produce a plurality of encoded data slices. The operation instructions further enable the processor to generate to determine a plurality of site slice sets from the plurality of encoded data slices, where each site slice set of the plurality of site slice sets includes a number of unique encoded data slices of the plurality of encoded data slices that is greater than or equal to a site write threshold value. The operation instructions further enable the processor to a designate one of a plurality of storage sites for each of the plurality of site slice sets and transmit each of the plurality of site slice sets to a corresponding designated one of the plurality of storage sites via the network.

Abstract: Nonsequential readahead for deep learning training that includes: receiving an indication of a list of batch storage locations for a batch of data objects; prefetching, for each storage location in the list of batch storage locations, storage content corresponding to the batch of data objects; and storing the storage content corresponding to the batch of data objects within a cache accessible to an artificial intelligence workflow.

Abstract: Locally providing cloud storage array services for a plurality of storage systems within a data center by: receiving a request for storage resources from an operating system level virtualization service; determining, among the plurality of storage systems; an implementation of the request from the operating system level virtualization service; and providing storage resources to the operating system level virtualization service in accordance with the implementation of the request from the operating system level virtualization service.

Abstract: A method includes identifying an independent data object of a plurality of independent data objects for retrieval from dispersed storage network (DSN) memory. The method further includes determining a mapping of the plurality of independent data objects into a data matrix, wherein the mapping is in accordance with the dispersed storage error encoding function. The method further includes identifying, based on the mapping, an encoded data slice of the set of encoded data slices corresponding to the independent data object. The method further includes sending a retrieval request to a storage unit of the DSN memory regarding the encoded data slice. When the encoded data slice is received, the method further includes decoding the encoding data slice in accordance with the dispersed storage error encoding function and the mapping to reproduce the independent data object.

Abstract: Modifying a clone image of a dataset, including: generating, based on metadata describing one or more updates to a dataset, a tracking copy of replica data on a target data repository; generating, after receiving an indication to begin accepting modifications to the tracking copy of the replica data, a cloned image of the dataset that is modifiable without modifying the tracking copy of the replica data; and responsive to a storage operation directed to the target data repository, modifying the cloned image of the dataset without modifying the tracking copy of the replica data.

Abstract: In a storage system that includes a plurality of storage devices configured into one or more write groups, quorum-aware secret sharing may include: encrypting a device key for each storage device using a master secret; generating a plurality of shares from the master secret such that a minimum number of storage devices required from each write group for a quorum to boot the storage system is not less than a minimum number of shares required to reconstruct the master secret; and storing the encrypted device key and a separate share of the plurality of shares in each storage device.

Abstract: A method of using flash storage devices with different sized erase blocks is provided. The method includes allocating a plurality of erase blocks of heterogeneous erase block sizes to a RAID stripe, to form a tile pattern having the heterogeneous erase block sizes in the RAID stripe. The method includes writing the RAID stripe across the flash storage devices in accordance with the allocating, and stopping the writing the RAID stripe, responsive to contents of the RAID stripe reaching a threshold.

Abstract: A storage system that supports independent scaling of compute resources and storage resources, the storage system including: one or more chassis, wherein each chassis includes a plurality of slots, each slot configured to receive a blade; a plurality of compute resources; a plurality of storage resources; a plurality of blades, where each blade includes at least one compute resource or at least one storage resource and each of the storage resources may be directly accessed by each of the compute resources without utilizing an intermediate compute resource; a first power domain configured to deliver power to one or more of the compute resources; and a second power domain configured to deliver power to the storage resources, wherein the first power domain and the second power domain can be independently operated.

Abstract: Systems and method for implementing deduplication process based on performance analyses. The system may include a processing device to determine a first performance metric associated with retrieving a second stored data block that is within a specified range of a duplicate of the first data block and a second performance metric associated with retrieving a hash value corresponding to the second stored data block. The processing device further to retrieve the second stored data block within a specified range of the duplicate of the first data block in response to the first performance metric not exceeding the second performance metric.

Abstract: An illustrative unified data storage method includes providing, by a data storage system, block containers that represent a linear address space of blocks; and using, by the data storage system, the block containers to store content for a plurality of different data storage services. In certain examples, the different data storage services include at least one of a file storage service, an object storage service, or a database service.