Abstract: Techniques are provided for implementing storage block balancing using volume part migration. One method comprises obtaining a capacity utilization of storage blocks each storing a portion of one or more storage volumes; and when the capacity utilization of a given storage block is above an upper target threshold, migrating a volume part of the given storage block to a different storage block having a capacity utilization below a lower target threshold, wherein the different storage block is within a same storage pool as the given storage block, and wherein the volume part comprises a subset of a storage volume.

Abstract: The invention relates to a method, a non-transitory computer-readable storage medium and an apparatus for programming data into flash memory. The method includes: generating a front-end parameter set, a mid-end parameter set and a back-end parameter set for each data-programming transaction; transmitting the front-end parameter set of each data-programming transaction to a routing engine, thereby enabling the routing engine to drive a host interface (I/F) to obtain from the host side; transmitting the mid-end parameter set of each data-programming transaction to an accelerator, thereby enabling the accelerator to drive the RAID engine to encrypt raw data or generate parity-page data according to multiple pages of the raw data; and transmitting the back-end parameter set of each data-programming transaction to the accelerator, thereby enabling the accelerator to drive a data access engine to program source data into a designated physical address of a flash module.

Abstract: Apparatuses, systems, methods, and computer program products are disclosed for persistent memory management. Persistent memory management may include replicating a persistent data structure in volatile memory buffers of at least two non-volatile storage devices. Persistent memory management may include preserving a snapshot copy of data in association with completion of a barrier operation for the data. Persistent memory management may include determining which interface of a plurality of supported interfaces is to be used to flush data from a processor complex.

Abstract: A method for more effectively utilizing computing resources in a data replication environment is disclosed. In one embodiment, such a method detects, at a primary system, activity occurring on the primary system. This activity is recorded in systems logs located at the primary system. The method automatically mirrors the system logs from the primary system to a secondary system that is in a mirroring relationship with the primary system. The system logs are analyzed at the secondary system. In the event abnormal activity is detected in the system logs at the secondary system, the method automatically sends, from the secondary system to the primary system, one or more commands that are designed to address the abnormal activity. A corresponding system and computer program product are also disclosed.

Abstract: Provided are a computer program product, system, and method for managing swappable data structures in a plurality of memory devices based on access counts of the data structures. Data structures indicated as swappable are updated less frequently than most frequently updated data structures. Data structures not indicated as swappable are maintained in a first level memory device and not moved to a second level memory device. The first level memory device has lower latency than the second level memory device. Access counts are maintained for the data structures stored in the first level memory device that are indicated as swappable. Data structures are selected in the first level memory device having lowest access counts. The selected data structures are removed from the first level memory device and retained in the second level memory device.

Abstract: Storage volumes are pre-provisioned in the operating system of a storage system before they are required to be assigned to an emulation on the storage system. Details of the previously created storage volumes are stored in a management database. If a storage volume is required, the process that needs the storage volume obtains details about a pre-provisioned storage volume from the management database, and instructs the storage system operating system to add the storage volume to a storage group for the process. By pre-provisioning storage volumes in the operating system, it is possible to greatly reduce the amount of time it takes to add storage volumes to processes. The number of storage volumes to be pre-created can be determined based on a regression between the previous storage volume usage and time, and a deterministically obtained value of a number of filesystems that will be created in an upcoming time interval.

Abstract: A method comprises receiving an input specifying one or more rules in connection with archiving one or more of a plurality of files from a source storage location to a target storage location. The one or more rules specify one or more constraints for retention of the one or more of the plurality of the files. In the method, the one or more of the plurality of files are retrieved from the source storage location for migration to the target storage location. A request is sent to the target storage location that the target storage location invoke the one or more rules to retain the one or more of the plurality of files in the target storage location. The target storage location comprises a cloud storage platform.

Abstract: The disclosure relates to technology for up-evicting cache lines. An apparatus comprises a hierarchy of caches comprising a first cache having a first cache controller and a second cache having a second cache controller. The first cache controller is configured to store cache lines evicted from a first processor group to the first cache and to down-evict cache lines from the first cache to the second cache. The second cache controller is configured to store cache lines evicted from a second processor group into the second cache, to up-evict a first cache line from the second cache to the first cache in response to an eviction of a second cache line from the second processor group to the second cache, and to provide the up-evicted first cache line from the first cache to the second processor group in response to a request from the second processor group.

Abstract: A method, computer system, and a computer program product for storing a write data in a storage system that operates using a log-structured filing system is provided. The present invention may include compressing a write data. The present invention may also include identifying a region of a storage system based on a first size of the compressed write data. The present invention may further include the storage system operating using a log-structured filing system. The present invention may also include a different region of the storage system having a different compression ratio. The present invention may further include writing the compressed write data in the identified region of the storage system.

Abstract: Described is a system (and method) for providing multiple types of granular delete operations for a cloud-based object storage. The system may include a server that acts as an intermediary between a client device and an object storage that stores client data. The server may allow a client to perform a hard delete (or permanent) delete and a soft delete. The server may perform a specialized processing to synthetically create the soft delete feature using a permanent object delete operation provided by the object storage. The specialized processing may include manipulating retention periods associated with objects depending on whether the object is subject to a hard delete or a soft delete. As a result, the server may provide the ability for clients to perform a soft delete at an object level using existing storage level APIs that do not directly support such a feature.

Abstract: Transforming data that is provided by a first instance of an application that uses application-instance specific data includes determining if a component of the data is an application-instance specific component and, if the component is an application-instance specific component, transforming the component either at a storage system containing the data or as the component is being accessed by a second instance of the application, different from the first instance. Transforming the component at a storage system containing the data may be performed independently of any accesses of the data. Transforming the component at a storage system containing the data may be performed by the storage system. The first instance of the application may run on a first host and the second instance of the application may run on a second host different from the first host. The first and second instances of the application may run on a same host.

Abstract: A method includes sending, from an application layer, a chunk size setting to an erasure coding layer. The method further includes receiving, at the application layer, user data. The method further includes aligning, at the application layer, the user data based on the chunk size setting. The method further includes sending the aligned user data to the erasure coding layer. The method further includes partitioning, at the erasure coding layer, the aligned user data into a first data chunk and a second data chunk. The method further includes generating, at the erasure coding layer, a parity chunk based on the first data chunk and the second data chunk. The method further includes sending, from the erasure coding layer, the first data chunk, the second data chunk, and the parity chunk to a storage system.

Abstract: A system includes a cluster of nodes including a storage domain, a memory, and a processor. The processor is configured to receive a request to determine an amount of allocated blocks associated with a virtual disk comprising a first volume. Each volume that includes metadata associated with allocated blocks is designated into a first set. Each volume within the one or more layers that lacks metadata associated with allocated blocks and includes an allocation table is designated into a second set. Each volume within the one or more layers that is omitted from the first set and second set is designated into a third set. The amount of allocated blocks within the first volume is determined based on inspecting the metadata of each volume of the first set, inspecting each allocation table of each volume of the second set, and inspecting each block of each volume in the third set.

Abstract: Methods, apparatus, and processor-readable storage media for implementing differential snapshots without array support are provided herein. An example computer-implemented method includes determining whether one or more of a plurality of data blocks in a first snapshot of a storage volume have changed relative to a plurality of data blocks in a second snapshot of the storage volume; generating a respective data structure for each of the plurality of data blocks in the first snapshot, wherein the data structure comprises data content of the corresponding data block of the first snapshot only if the corresponding data block has changed relative to the second snapshot; and transmitting the data structures to a remote storage system, wherein the remote storage system generates a copy of the first snapshot based at least in part on the data structures.

Abstract: A method of detecting object corruption of deduplicated cloud objects includes a two-layered approach. In a first layer, only metadata areas from segments of a deduplicated cloud object are processed in order to detect corruptions in the metadata. If no corruption is detected in the metadata, a second layer analysis can be performed. The second layer analysis includes a progressive scanning of data only on objects that were not found to be corrupted in the first layer analysis.

Abstract: A computer-implemented method includes migrating an item to a cache cluster from a home cluster in a storage system in a cutover process and initiating a data validation process with the home cluster. The method includes enabling at least one tunable parameter on the item on the cache cluster in the storage system. The tunable parameter is configured to adjust the data validation process with the home cluster. The tunable parameter includes at least one condition. The method includes enabling at least one flag on the item on the cache cluster in the storage system. The flag is configured to indicate that the at least one condition of the tunable parameter on the item in the storage system is met. In response to the at least one condition of the tunable parameter being met, the method includes ending the data validation process with the home cluster in the storage system.

Abstract: A method includes: identifying, by a first program module, a bucket having an object lifecycle management rule in a dispersed storage network; creating, by a second program module, ranges of object names included in a namespace of the bucket; determining, by a third program module and for respective objects in each of the ranges, whether the respective objects satisfy the object lifecycle management rule of the bucket; and performing, by a fourth program module and based on the determining, an action on one of the respective objects that satisfies the object lifecycle management rule, the action being defined by the object lifecycle management rule.

Abstract: Disaster recovery operations are disclosed. A copy of data may include segments and delta journals or journals. RTO can be reduced by pre-processing the delta journals after the segments and delta journals are uploaded. The RTO is improved during recovery because the need to process the delta journals is eliminated. Applying the delta journals can be performed to both reduce cost and RTO using graphs.

Abstract: Bandwidth consumption and/or an I/O transmission rate on an I/O path between a port of a storage system and a physical host port may be managed, including determining when multiple virtual host ports correspond to (i.e., are mapped to) a same physical host port. This virtual host port mapping information may be used to more accurately determine bandwidth consumption and I/O transmission rates on I/O connections along an I/O path including the physical host port, and to adjust the bandwidth consumption and/or I/O transmission rate on one more of these I/O connections according to bandwidth thresholds and I/O count thresholds defined for the I/O path (e.g., for the Physical host port of the I/O path).

Abstract: A system that implements a scalable data storage service may maintain tables in a data store on behalf of storage service clients. The service may maintain table data in multiple replicas of partitions that are stored on respective computing nodes in the system. In response to detecting an anomaly in the system, detecting a change in data volume on a partition or service request traffic directed to a partition, or receiving a service request from a client to split a partition, the data storage service may create additional copies of a partition replica using a physical copy mechanism. The data storage service may issue a split command defined in an API for the data store to divide the original and additional replicas into multiple replica groups, and to configure each replica group to maintain a respective portion of the table data that was stored in the partition before the split.