Abstract: One or more techniques and/or computing devices are provided for resilient replication of storage operations. For example, a first storage controller may host first storage having a replication relationship with second storage hosted by a second storage controller. To improve resiliency against transient network issues of a network between the storage controllers, the first storage controller may implement a queue and retry mechanism to retry replication operations not acknowledge back by the second storage controller within a threshold time. The second storage controller may maintain a cumulative sequence number of a latest replication operation performed in order, an operation response map of replication operations performed out of order, and an operation finder map identifying currently implemented replication operations, which may be used to process incoming replication operations. Single write semantics, write order consistency, and reduction of write amplification may be provided.

Abstract: A method for searching for free blocks in bitmap data. For searching for free blocks in the bitmap data, distribution information of free blocks can be directly obtained by performing array value solving operation, with the value of the integer data obtained by dividing the bitmap data into blocks being used as an index, on a template array. Also provided are an apparatus for searching for free blocks in the bitmap data, a device, and a readable storage medium.

Abstract: One example method includes, at a replication data source, initiating a replication process that includes transmitting a replication stream to a replication destination vault, and data in the replication stream is transmitted by way of a closed airgap between the replication data source and the replication destination vault, switching, by the replication data source, from a transmit mode to a receive mode, receiving, at the replication data source, a first checksum of a file, and the first checksum and file were created at the replication destination vault, receiving, at the replication data source, the file, calculating, at the replication data source, a second checksum of the file, and when the second checksum matches the first checksum, ending the replication process.

Abstract: Techniques for generating a crash-consistent clone of file(s) stored by a distributed file system are described herein. To generate the crash-consistent clone, a coordinator node may identify multiple nodes (e.g., at least a first node and a second node) of the distributed file system that are storing different portions of data of the file. The coordinator node may then send a first command to the multiple nodes to cause each of the multiple nodes to quiesce the file and clone the different portions of the data to generate the crash-consistent clone of the file. The coordinator node may then receive, from the multiple nodes, a status associated with generating the crash-consistent clone of the file. Based at least in part on the status, the coordinator node may send a second command to the multiple nodes to cause each of the multiple nodes to unquiesce the file.

Abstract: A method and apparatus for creating a system disk snapshot of a virtual machine comprises: receiving an instruction for creating a snapshot of a virtual machine; determining existence of a power-on state; renaming a top file in a system disk in response to the received instruction if the power-on state exists; creating a new top file in response to renaming the top file, directing dependency of the new top file to the renamed top file and opening the new top file; determining whether a preceding snapshot is created successfully; using the renamed top file as a first snapshot file if the preceding snapshot is successful; and uploading the first snapshot file to a file management server. Accordingly, when a system disk snapshot is created in a power-on state, the snapshot may be created without stopping the virtual machine, thereby reducing requirements for data back up, storage space, and network bandwidth.

Abstract: The efficiency of a container location database (“CLDB”) that is used for retrieving location information of data containers stored on nodes in a cluster is improved by decreasing the number of key-value store transactions performed during snapshot creation and deletion, thereby decreasing both disk I/O and computing cycles. The CLDB is made more scalable by improving the deletion and creation time of snapshots, allowing one to create and delete more snapshots during a given interval of time. CLDB resilience is improved by decreasing the time required to resync a CLDB node. CLDB data is reduced to decrease IO operations, improve resync times, and to cache most of the data in memory.

Abstract: A system includes a data management server and a data store. The data store uses an external file system to store data block of a client virtual machine. The client virtual machine uses an internal file system. The data management server comprises a proxy agent and a staging virtual machine. In response to receiving a request to retrieve a file indexed by the client virtual machine, the proxy agent imports data of the internal file system to a staging virtual machine. The proxy agent determines an internal file address that corresponds to the requested file. The staging virtual machine translates the internal file address to an external file address of the external file system. The staging virtual machine retrieves the file stored in the data store based on the external file address. The staging virtual machine provides the retrieved file to the target device.

Abstract: An information processing apparatus includes a storage unit configured to store setting that associates a node number assigned to a first external device with a first command transmittable and receivable to/from the first external device, a determination unit configured to determine, in a case where the first command is not transmittable and receivable to/from a second external device that takes over the node number of the first external device, whether a second command compatible with the first command is transmittable and receivable to/from the second external device, and a control unit configured to perform, in a case where the determination unit determines that the second command is transmittable and receivable to/from the second external device, control to change the first command in the setting to the second command.

Abstract: A sub-archive is initiated. The sub-archive saves changes that have occurred since a previous final archive. Changes to an operational database (i.e., the operational database that is being archived) are allowed during the first sub-archive. A final archive is initiated in series after the sub-archive has completed. The final archive does not allow changes to the operational database when the final archive is active. In one embodiment, the sub-archive may comprise a plurality of sub-archives that depend on an amount of outstanding changes that exist in the operational database.

Abstract: In an embodiment, two or more storage systems are requested to prepare respective local checkpoints for a dataset, wherein each of the two or more storage systems stores portion of the dataset. The two or more storage systems are determined to have established the checkpoint. In response to determining that the local checkpoints have been established, a coordinated checkpoint is completed.

Abstract: Certain embodiments described herein relate to an improved block-level replication system. One or more components in an information management system may receive a request to perform a block-level replication between a source storage device and a destination storage device, and depending on the specific replication mode requested, (i) store block-level changes directly to the destination storage device or (ii) first to a recovery point store and then later to the destination storage device.

Abstract: A method, non-transitory computer readable medium, and device that replicates data and provides instantaneous access to data includes receiving in a destination volume one or more named data extents and one or more references to the one or more named data extents associated with a file in parallel from a source volume. A determination is made to check whether the one or more references arrive before the one or more named data extents arrive. Each of the received one or more references which are determined to have arrived prior to the one or more data extents is allocated as absent by the storage management computing device. An instantaneous access to the file is provided during the allocation.

Abstract: A method of reassembling a local disk manager (LDM) and array group (AGRP) includes starting a physical extent manager (PEM) configured to run on a number of nodes. The PEM on each node is configured to manage an AGRP running on the same node. A number of LDMs are reassembled, and each LDM is configured to manage virtual disks on each of the nodes. Once enough LDMs are reassembled, an AGRP can be reassembled.

Abstract: Fabric modules in a storage system offer differing device IDs from a deterministic sequence to a storage device being added to the storage system. The storage device that is being added accepts a device ID that is higher in the deterministic sequence. The fabric module that offered the device ID same as was accepted by the storage device determines to proceed with initializing the storage device.

Abstract: A smart network interface card in an information handling system monitors a local host memory associated with a computer resource for an update to a memory page in the local host memory. After the update to the memory page, the smart network interface card copies the memory page to its memory. The smart network interface card sets a watchdog timer to detect a failure in an the information handling system that hosts the computer resource and if the failure is detected, then the smart network interface card migrates the computer resource from its to another information handling system.

Abstract: Obtaining a consistent set of snapshots of a group of storage volumes includes obtaining a plurality of snapshots, each of the snapshots being for one of the volumes, determining if there are any specific write operations to at least one volume that occurred after obtaining a first one of the snapshots and before obtaining a last of the snapshots, if there are any specific write operations, discarding the plurality of snapshots, and, if there are no specific write operations, designating the plurality of snapshots as the consistent set of snapshots. Obtaining a consistent set of snapshots may include repeatedly obtaining a plurality of snapshots and determining if there are specific write operations until either a particular condition is met or there are no specific write operations determined. The particular condition may be exceeding a predetermined number of iterations or a predetermined time limit.

Abstract: Disclosed is an operating method of a memory system that includes a plurality of memory blocks, the operating method including a first step of copying, in order to recover sudden power-off of the memory system, data of an open block to a selected block among the plurality of memory blocks while maintaining map data associated with the open block and open block identification information; a second step of erasing the open block; and a third step of copying the data, which is copied to the selected block, to the erased open block.

Abstract: In the face of ransomware attacks, which can be increasingly difficult to effectively prevent, a solution can be considered to be the minimization of the cost and time taken to recover data and, hence business activities. Embodiments perform a restore operation that include automatically identifying the most recent healthy backup, from which data should be restored, and the prioritizing of the order in which data should be restored.

Abstract: An embodiment of the present invention is directed to configuring optimal parallelism on Hadoop processes at specific processing points for improved performance and throughput. An embodiment of the present invention forces Hadoop into high parallelism independent of a HDFS file size. An embodiment of the present invention achieves an order of magnitude improvement in performance and data throughputs to meet the SLAs. An embodiment of the present invention provides a design pattern unique to distributed processing in Hadoop to accomplish the required data throughputs for the CDC process.

Abstract: The invention relates to a method, and an apparatus for accessing to data in response to a power-supply event. The method, performed by a flash controller, includes steps for: reading a plurality of physical pages of data in a current block from a flash module during a sudden power off recovery procedure; determining whether a power-supply event has occurred according to an error correction result corresponding to read physical pages; reconstructing a first flash-to-host mapping (F2H) table to include physical-to-logical mapping (P2L) information from the 0th page to a page before a last valid page in the current block when the power-supply event has occurred; and programming the reconstructed first F2H table into a location of the flash module.

Abstract: A memory system including a first central processing unit, a first memory module connected to the first central processing unit by a first channel, a second memory module connected to the first central processing unit by a second channel, and a third memory module connected to the first central processing unit by a third channel may be provided. Each of the first memory module, the second memory module, and the third memory module may be configured to write the same data in a data area thereof and a mirroring data area thereof in response to an address in a mirroring mode.

Abstract: Technology for enabling a hypervisor to perform copy on write features on encrypted storage of a virtual machine. An example method may involve: receiving, by a guest program from a hypervisor, an indication that identifies a first storage block of a first virtual machine, wherein the first storage block is write protected by the hypervisor; identifying, by the guest program, a second storage block of a second virtual machine; and copying, by the guest program, data of the first storage block to the second storage block, wherein the data of the first storage block and data of the second storage block are encrypted using different cryptographic inputs.

Abstract: Techniques manage application systems in an application environment. The application environment includes a first application system, a second application system and a third application system. First snapshot information of a first group of snapshots of the first application system is obtained, the first application system being in active state. Second snapshot information of a second group of snapshots of the second application system is obtained, the second application system being in standby state. It is determined whether the second application system and the third application system have a common snapshot based on the first snapshot information and the second snapshot information. Data is synchronized to the third application system depending on whether the second application system and the third application system have a common snapshot. Overheads required during data synchronization may be reduced as far as possible, and the efficiency of data synchronization may be improved.

Abstract: A host of a storage system is coupled to multiple SSDs. Each SSD is configured with a migration cache, and each SSD corresponds to one piece of access information. The host obtains migration data information of to-be-migrated data in a source SSD, determines a target SSD, and sends a read instruction carrying information about to-be-migrated data and the target SSD to the source SSD. The source SSD reads a data block according to the read instruction from a flash memory of the source SSD into a migration cache of the target SSD. After a read instruction is completed by the SSD, the host sends a write instruction to the target SSD to instruct the target SSD to write the data block in the cache of the target SSD to a flash memory of the target SSD.

Abstract: A system, according to one embodiment, includes: a processor, as well as logic that is integrated with the processor, executable by the processor, or integrated with and executable by the processor. The logic is configured to: send, by the processor, point in time copies of data to an object storage system. A directive for manipulating the point in time copies of the data are also set to the object storage system by the processor. Moreover, the point in time copies of the data are manipulated by a storlet on the object storage system according to the directive.

Abstract: Some embodiments provide a method for quantifying the workload placed on a database by an application. The method identifies a first group of database queries that the application directed towards the database. The method produces a second group of queries by removing, from the first group of queries, queries that are duplicates based on the semantic structure of the queries. Based on a set of properties of the second group of queries, the method computes a complexity indicator that represents a complexity expression of the second group of queries.

Abstract: Example implementation relates to generating a database for a set of entities associated with sampled data units of a storage system. A first data structure in the database maps, in a bit vector, each entity with a sampled signature of a set of sampled signatures. The set of sampled signatures are associated with the sampled data units. The set of entities associated with the set of sampled signatures are managed using the first data structure.

Abstract: Described herein are techniques for replicating external stages between deployments of e.g., a cloud data lake using a modified storage integration. The modified storage integration may be defined with multiple storage locations that it can point to, as well as a designation of an active storage location. The storage integration may also be defined with base file paths for each storage location as well as a relative file path which together may serve to synchronize data loading operations between deployments when e.g., a fail-over occurs from one deployment to another. The storage integration may be replicated from a first deployment to a second deployment, and when database replication occurs, an external stage may be replicated to the second deployment and bound to the replicated storage integration. Thus, a fail-over to the second deployment may result in a seamless transition of data loading processes to the second deployment.

Abstract: A system for estimating one or more data storage parameters and/or statistics in a data storage system is presented. The data storage system includes a plurality of storage containers. The system includes a snapshot module, a container stats aggregator, a synchronization module, a global stats aggregator, and storage stats estimator.

Abstract: In accordance with embodiments disclosed herein, there are provided methods, systems, mechanisms, techniques, and apparatuses for presentation of direct accessed storage under a logical drive model; for implementing a distributed architecture for cooperative NVM Data protection; data mirroring for consistent SSD latency; for boosting a controller's performance and RAS with DIF support via concurrent RAID processing; for implementing arbitration and resource schemes of a doorbell mechanism, including doorbell arbitration for fairness and prevention of attack congestion; and for implementing multiple interrupt generation using a messaging unit and NTB in a controller through use of an interrupt coalescing scheme.

Abstract: A data replication system is disclosed comprising a change event detection module configured to run at a first data repository and a change event implementation module configured to run at a second data repository. A messaging service configured to transmit received messages in accordance with message subscriptions is also provided. The event detection module is configured to detect changes made to the first data repository and transmit change event messages to the messaging service. The event implementation module is configured to subscribe to change event messages at the messaging service and, in response to receipt of a change event message, to implement the change event at the second data repository.

Abstract: The subject matter of this specification can be implemented in, among other things, a method including creating a new disk file at a reference point-in-time, wherein an original disk file is a backing file of the new disk file, copying the original disk file to a disk snapshot file, in response to the original disk file being copied to the disk snapshot file, merging the original disk file and the new disk file to form a merged file, wherein a virtual machine is to continue to perform disk operations using the merged file, and determining whether the merged file is synchronized with the original disk file and the new disk file by determining whether entries of a bitmap for the merged file match corresponding entries of a bitmap for the new disk file.

Abstract: Managing backup copies in a right to delete personal data environment is described. A system creates a revised backup copy by deleting any data that a deletion request ledger specifies for deletion from a backup copy that was created for a data object. The system associates an updating time with the revised backup copy. The system retrieves the revised backup copy in response to receiving a restoration request corresponding to the revised backup copy. The system enables restoration of the data object by deleting any data i) that the deletion request ledger specifies for deletion from the revised backup copy and ii) that is associated with a deletion request time which is subsequent to the updating time.

Abstract: Systems, methods, and machine-storage medium for classifying snapshot image processing are described. The system receives read requests to read snapshot information. Each read request includes an offset identifying a storage location and a length. The snapshot information includes snapshots including a full snapshot and at least one incremental snapshot. The read requests include a first read request to read data from the snapshot information. The system generates a first plurality of read events including a second plurality of read events that are generated by processing the first read request. The second plurality of read events includes first and a second read events. The system identifies whether utilizing a cache optimizes the job based on the first plurality of read events.

Abstract: Aspects of the disclosure provide for mechanisms for managing containers across operating systems in a computer system. A method of the disclosure includes: running a first container corresponding to a first operating system, wherein the first container is created in view of a container image; receiving a user request to switch from the first operating system to a second operating system; in response to receiving the user request, switching, by a processing device, from the first container to a second container corresponding to the second operating system, wherein the second container is created in view of the container image, wherein switching from the first container to the second container comprises: configuring the second container in view of a plurality of configurations of the first container; and running the configured second container.

Abstract: A system and method for backing up workloads for multiple tenants of a cloud computing system are disclosed. A method of backing up workloads for multiple tenants of a computing system includes triggering an archival process according to an archival policy set by a tenant, and executing the archival process by reading backup data of the tenant stored in a backup storage device of the computer system and transmitting the backup data to an archival store designated in the archival policy, and then deleting or invalidating the backup data stored in the backup storage device.

Abstract: Systems and methods are provided for optimizing data structures to improve the data retrieval through the use of bucketing techniques. A number of objects within an environment is drastically reduced utilizing bucketing techniques. Within the buckets, items are sequentially organized such that location is quicker. Items, or keys, are aligned with the same hash value together in a bucket and a mapping of the hash value to the offset of the first key occurrence in that bucket. This guarantees each lookup operation is only two random read accesses. Systems and methods provided herein control the pressures on a system for garbage collection and minimize memory usage with minimal impacts on performance.

Abstract: Techniques for creating and using snapshots may include: receiving a request to create a new snapshot of a source object; determining whether a first generation identifier associated with the source object matches a second generation identifier associated with a base snapshot of the source object; determining whether the source object has been modified since the base snapshot was created; and responsive to determining the first generation identifier matches the second generation identifier and also determining that the source object has not been modified since the base snapshot was created, associating the new snapshot with the base snapshot thereby indicating that the new snapshot and the base snapshot have matching content and denote a same point in time copy of the source object.

Abstract: Example implementations described herein involve systems and methods which automatically determine volumes to be replicated for disaster recovery based on the execution priority of an application which uses the volumes. Such example implementations can involve systems and methods involving creating a volume in a first storage system for each of one or more containers newly launched on one or more servers managing a container orchestrator; and establishing replication of the volume for the each of the newly launched one or more containers to a second storage system in order from highest container priority to lowest container priority.

Abstract: A reference snapshot of a storage is stored. Data changes that modify the storage are received. The data changes are captured by a write filter of the storage. The received data changes are logged. The data changes occurring after an instance time of the reference snapshot are applied to the reference snapshot to generate a first incremental snapshot corresponding to a first intermediate reference restoration point. The data changes occurring after an instance time of the first incremental snapshot are applied to the first incremental snapshot to generate a second incremental snapshot corresponding to a second intermediate reference restoration point.

Abstract: Snapshot lifecycle management systems and methods are disclosed herein. An example method includes establishing a repository for a user, determining indices for the user, generating a snapshot lifecycle policy for the indices of the cluster. The snapshot lifecycle policy comprises snapshot gathering parameters that dictate when and how often snapshots of indices of the cluster are obtained, as well as retention parameters that control how long the snapshots are stored and when the snapshots are to be deleted. The method includes storing the snapshots for the indices of the cluster in the repository according to the snapshot gathering parameters, and managing retention of the snapshots stored in the repository according to the retention parameters.

Abstract: For restoring data, a system with uninterrupted block-based restore has a hybrid container. The hybrid container has an operational buffer and a virtual container unit. The system receives blocks of data from a target device into the hybrid container and sends the blocks of data from the hybrid container to a destination device for a restore session. The system writes information about the blocks of data into one or more virtual containers. If there is an abort of the restore session, the system freezes state of the hybrid container and the virtual container(s). For a next restore session that resumes from where the aborted restore session stopped, the system resumes receiving blocks of data from the target device into the hybrid container and sends only leftover blocks of data to the destination device in accordance with the information in the virtual container(s).

Abstract: Provided is a method, a computer program product, and a system for providing perfect forward secrecy in virtual machines. The method includes receiving a secure memory allocation function from an application, including a connection secret to be stored in memory. The method further includes allocating memory for the connection secret according to the memory size parameter and storing an entry relating to the connection secret in a secure database. The memory information includes a memory location and a memory size of the memory. The method also includes monitoring an operation state relating to the virtual machine. The method further includes receiving, from the application, a secure deallocation function relating to the connection secret and retrieving the memory information from the secure database. The method also includes deleting the connection from the memory and sanitizing the memory location logged by the memory information.

Abstract: A backup method is provided for a file system for managing hierarchical storage including one or more storage media on a computer system including one or more computers. Each of the one or more storage media includes a data part which is separated from an index and on which updated information is appended and kept. Each of a plurality of pieces of data within the file system is managed to be migrated to at least any of the one or more storage media. The backup method includes backing up metadata of the plurality of pieces of data migrated within the file system; and backing up the index of each of the one or more storage media to which the plurality of pieces of data have been migrated, substantially at the same time point as the metadata.

Abstract: A storage array that uses NVMEoF to interconnect compute nodes with NVME SSDs via a fabric and NVME offload engines implements flow control based on transaction latency. Transaction latency is the elapsed time between the send side completion message and receive side completion message for a single transaction. Counts of total transactions and over-latency-limit transactions are accumulated over a time interval. If the over limit rate exceeds a threshold, then the maximum allowed number of enqueued pending transactions is reduced. The maximum allowed number of enqueued pending transactions is periodically restored to a default value.

Abstract: Techniques are directed to managing Redundant Array of Independent Disks (RAID) storage space. One technique involves providing, by RAID circuitry, a storage space request to mapper circuitry; receiving, by the RAID circuitry, a storage space reply from the mapper circuitry in response to the storage space request; and performing, by the RAID circuitry, a follow-up operation based on the storage space reply from the mapper circuitry. Another technique involves, receiving, by mapper circuitry, a storage space request from RAID circuitry; performing, by the mapper circuitry, a storage space management operation in response to the storage space request; and providing, by the mapper circuitry, a storage space reply to the RAID circuitry, the storage space reply identifying a result of the storage space management operation.

Abstract: Efficiently transferring data between tiers in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; transforming, within storage provided by a first tier of storage of the virtual storage system, the data to generate transformed data; and migrating, from the first tier of storage to a second tier of storage that is more durable than the first tier of storage of the virtual storage system, at least a portion of the transformed data.

Abstract: In one embodiment, a system for managing a virtualization environment includes a set of host machines, each of which includes a hypervisor, virtual machines, and a virtual machine controller, one or more virtual disks comprising a plurality of storage devices, the one or more virtual disks being accessible by the virtual machine controllers, where the virtual machine controllers conduct I/O transactions with the virtual disks, and a deployment system configured to receive a request to deploy a virtualized file server on a plurality of host machines, where the request is associated with deployment parameters, select a deployment image for the host machines based on the deployment parameters, and provide the deployment image to each host machine via the virtual disk. The virtual machine controller located on the host machine stores at least a portion of the deployment image on a storage device associated with the host machine.

Abstract: One example method includes receiving data segments that are not already part of a full disk image of a backup, storing the data segments in storage, determining whether or not an aggregate total of data segments in the storage, that are not already part of a full disk image of a backup, equals or exceeds a threshold, when the aggregate total of data segments in the storage equals or exceeds the threshold, creating a full disk image of a backup that includes the data segments in storage, and storing the created full disk image of the backup to a recovery disk.

Abstract: A memory system may improve the endurance and performance of a plurality of memories included in the memory system mounted on a server system or a data processing system. For example, the memory system may throttle energy of a first memory using a second memory having a different characteristic from the first memory, control accesses to a memory region according to a refresh cycle, and control accesses to memories having different temperatures according to a priority of a request for each of the memories.