Abstract: Method and apparatus for managing data buffers in a data storage device. In some embodiments, a write manager circuit stores user data blocks in a write cache pending transfer to a non-volatile memory (NVM). The write manager circuit sets a write cache bit value in a forward map describing the NVM to a first value upon storage of the user data blocks in the write cache, and subsequently sets the write cache bit value to a second value upon transfer of the user data blocks to the NVM. A read manager circuit accesses the write cache bit value in response to a read command for the user data blocks. The read manager circuit searches the write cache for the user data blocks responsive to the first value, and retrieves the requested user data blocks from the NVM without searching the write cache responsive to the second value.

Abstract: Container images may be generated from a backup system that includes a backup of one or more applications from a computing system of an entity. During a backup process, an application can be identified and its storage location in a secondary storage can be tracked or saved in a backup index. Configuration information and data or files created by user interaction with the application can be backed up and the location of the backed up data or files may be stored in the backup index along with the location of the configuration information. Using the backup index, a container image can be created that includes a selected application, its configuration information, and data, if any, created by the application. The container image can be generated from the backup stored in the secondary storage.

Abstract: The presently disclosed subject matter includes a data storage system and method for the purpose of migrating data in a distributed storage system comprising a plurality of separate storage areas.

Abstract: To replicate a source LUN to a different storage system platform, a first storage system transmits a request to replicate a LUN along with attributes for the LUN to a second storage system. The second storage system maps the attributes to attributes used and understood by the platform of the second storage system. The second storage system then creates a destination LUN based on the mapped attributes. Since the destination LUN is created with similar attributes as the source LUN, the destination LUN can store the replicated data of the source LUN while still being accessed and recognized as a LUN by the second storage system. The second storage system also stores any proprietary attributes received from the first storage system so that the proprietary attributes can be supplied to the first storage system to recover the source LUN after a data loss event.

Abstract: According to one embodiment, a storage system includes a representative node and a plurality of general nodes including a plurality of resources. Each of the general nodes stores resource status information indicating respective statuses of the plurality of resources in a first storage unit thereof. The representative node stores resource status information collected from the plurality of general nodes in a second storage unit thereof, decides whether to acquire the resource status information from the first storage unit of the general node or to acquire the resource status information from the second storage unit based on a received request, and transmits the resource status information acquired from a decided acquisition destination to an issuing source of the request.

Abstract: Provided are a storage method and apparatus for a distributed file system. The method includes: determining access times to a same file in preset time; and migrating the file to a storage device with higher performance than that of a current storage device according to the access times. Through the technical solution, tiered storage is performed on files in the distributed file system according to access enthusiasm of the files, that is, a file with a high access frequency in the preset time is migrated to a storage device with higher performance, and a file fragment storage location is associated with file fragment access enthusiasm through differential storage, so that IOPS pressure between storage devices is balanced and the overall performance is improved.

Abstract: A storage device includes a memory device; and a controller configured to fetch a command from a host, the command indicating a logical address, process the command based on the logical address, and receive, from a first replica storage device, an acknowledgment signal indicating that the command has been processed by the first replica storage device.

Abstract: Systems and Methods for dispersed data structures (DDS) in a distributed storage network are disclosed. A dispersed storage processing unit handling a request to insert a key value pair into a DDS could lookup what the most up to date DDS is, which could be held by a single source with a pointer to the current DDS root. The processing unit could then descend the DDS until it finds the leaf node that owns the requester's key and make a copy of the leaf with the key inserted. The processing unit could then make a copy of the parent of the node, replacing the pointer to the copied node with a pointer to the new copy, repeat this step until the root is reached, and make a copy of the root in a similar fashion but also including a pointer to the original DDS root.

Abstract: A first cloud is provided with a first abstraction layer between a first instance of an application and a first set of different storage types. A second cloud, remote from the first cloud, is provided with a second abstraction layer between a second instance of the application and a second set of different storage types. A replication protection appliance (RPA) is connected between the first and second abstraction layers. The first abstraction layer intercepts a write command from the first instance of the application to storage of a particular type among the first set of different storage types. The first abstraction layer sends a copy of the write command to the RPA. The RPA replicates the write command to the second abstraction layer for the second abstraction layer to send to corresponding storage of the particular type among the second set of different storage types.

Abstract: A data storage service stores a dataset on a set of storage nodes in accordance with a first encoding. A set of shards constituting quorum, and one or more additional shards, are stored on the storage nodes. The data storage system determines to store the dataset according to a second encoding, in which the second encoding has a greater number of shards. The data storage system reconfigures the storage of the dataset in accordance with the second encoding, such that the reconfigured storage forms additional shards for the second encoding by combining portions of shards of the first encoding.

Abstract: Retrieving backup parameters for recovery is described. A clone copy is created of a backup file stored on a disk. The clone copy is stored on an external destination node including an external disk and/or a tape. Backup parameters are stored to a backup catalog. The backup parameters identify a location of the clone copy. A request is received to recover the clone copy. The backup parameters are retrieved from the backup catalog. The clone copy is recovered from the external destination node based on the backup parameters. The clone copy is restored to a database associated with the backup file.

Abstract: It is determined that a first data unit is to be written to a storage device and that the first data unit is associated with a first attribute. In response to determining that the first data unit is associated with the first attribute, a first identifier is selected from a first identifier space and the first identifier is associated with the first data unit. It is determined that a second data unit is to be written to the storage device and that the second data unit is associated with the second attribute. In response to determining that the second data unit is associated with the second attribute, a second identifier is selected from a second identifier space and the second identifier is associated with the second data unit.

Abstract: A data storage system allows a user to search a secondary copy such as a back up, archive, or snapshot and receive an abbreviated version of the secondary copy of the file system as a result of the search. The abbreviated version of the secondary copy of the file system may include metadata such as a listing of secondary copy data that correspond with a search term or other filtering criteria provided by the user. An operating system of the user's computing device may mount the abbreviated version so that it can be displayed within a view of the file system of the computing device. Once a user selects one or more desired data objects listed in the abbreviated version, the selected actual one or more data objects are retrieved from secondary storage.

Abstract: A method for scheduling data services in a heterogeneous storage environment is disclosed. In one embodiment, such a method includes instrumenting, in a heterogeneous storage environment, each of a plurality of storage systems to generate events when operations are performed thereon. The events are detected and indexed in a single namespace. These events may then be analyzed to generate a statistical model of I/O activity occurring in the heterogeneous storage environment over a period of time. From the statistical model, the method determines periods of reduced I/O workload across the heterogeneous storage environment. The method then schedules data services to occur during these periods of reduced I/O workload. A corresponding system and computer program product are also disclosed.

Abstract: Systems and methods for replicating data from a production server to a backup server include recording at least one operation on one or more data items stored in a volume of a production server. The operation may be recorded as at least one journal event in a memory. A determination may then be made regarding whether a system malfunction incident has occurred in the production server and if so, a first set of journal events may be transferred from the memory to an auxiliary storage at a first time instant. At a second time instant, a second set of journal events recorded in the memory between the first and second time instants may be transferred to the auxiliary storage. At one journal event stored in the auxiliary storage unit may then be sent for replication to a backup server.

Abstract: A method and an apparatus for generating a virtual machine snapshot, where the method includes suspending a virtual machine at a first moment according to a received snapshot command, starting to perform a storage operation on a memory page in memory of the virtual machine and a contamination interception operation on the memory page in the memory, storing a device status, which is at the first moment, of the virtual machine to a snapshot file, and restoring the virtual machine from a suspended state to a running state after the device status is stored.

Abstract: The disclosure provides a memory management method, which includes: selecting at least one logical unit mapped to physical units programmed based on a first operating mode; determining a reference count according to a number of the selected logical unit; receiving a first write command; determining whether the reference count is greater than a threshold value; if the reference count is greater than the threshold value, programming first data into a first physical unit based on the first operating mode, and each memory cell in the first physical unit stores a first number of bit data; if the reference count is not greater than the threshold value, programming the first data into a second physical unit based on a second operating mode, and each memory cell in the second physical unit stores a second number of bit data, and the second number is greater than the first number.

Abstract: According to one embodiment, a request is received for restoration, where the request includes an application identifier (ID) identifying a first application and a timestamp specifying a particular time and date (time/date) of past. A restore module executed by a processor identifies a first application bundle based on the timestamp and the application ID. The first application bundle includes an executable image of the first application and metadata describing an operating environment that hosted the first application at the particular time/date. The restore module restores the first application in a first operating environment configured based on the metadata of the first application bundle.

Abstract: Example apparatus and methods combine erasure coding with data deduplication to simultaneously reduce the overall redundancy in data while increasing the redundancy of unique data. In one embodiment, an efficient representation of a data set is produced by deduplication. The efficient representation reduces duplicate data in the data set. Redundancy is then added back into the data set using erasure coding. The redundancy that is added back in adds protection to the unique data associated with the efficient representation. How much redundancy is added back in and what type of redundancy is added back in may be controlled based on an attribute (e.g., value, reference count, symbol size, number of symbols) of the unique data. Decisions concerning how much and what type of redundancy to add back in may be adapted over time based, for example, on observations of the efficiency of the overall system.

Abstract: Described embodiments provide systems and methods for operating a storage system that may determine one or more properties for each of a plurality of input/output (I/O) to a production volume of a storage system and monitor one or more operating conditions of the storage system. Embodiments may further include determining a score for each I/O based upon one or more of: the one or more properties of the I/O and the one or more operating conditions; adapting a replication threshold based upon the one or more operating conditions; comparing the determined score for each I/O to the adapted replication threshold; and based upon the comparison, performing continuous replication or snapshot replication for each I/O.

Abstract: The present disclosure describes systems and methods for detection and mitigation of malicious activity regarding user data by a network backup system. In a first aspect, a backup system receiving and deduplicating backup data from a plurality of computing devices may detect, based on changes in uniqueness or shared rates for files, atypical modifications to common files, and may take steps to mitigate any potential attack by maintaining versions of the common files prior to the modifications or locking backup snapshots. In a second aspect, the backup system may monitor file modification behaviors on a single device, relative to practices of an aggregated plurality of devices. Upon detection of potentially malicious modification activity, a previously backed up or synchronized store of data may be locked and/or duplicated, preventing any of the malicious modifications from being transferred to the backup system.

Abstract: A method of migrating a virtualized computing instance between source and destination virtualized computing systems includes executing a first migration workflow in the source virtualized computing system between a source host computer and a first mobility agent simulating a destination host, executing a second migration workflow in the destination virtualized computing system between a second mobility agent simulating a source host and a destination host computer, sending, as part of the first migration workflow, a configuration of the migrated virtualized computing instance to the destination virtualized computing system, translating, as part of the second migration workflow, infrastructure-dependent information in the configuration of the migrated virtualized computing instance, and transferring, during execution of the first and second migration workflows, migration data including the virtualized computing instance between the source host and the destination host over a network.

Abstract: Embodiments of the application provide a file access method. A computing node receives a file open request that carries a file identifier. The computing node obtains an index node of a file that is identified by the file identifier. The computing node further obtains, based on the index node, a physical address space of a memory area in a file storage area, in which the file is stored. The computing node allocates a virtual address space to the file, and recodes a virtual-physical address mapping relationship by using a memory page table. The virtual-physical address mapping relationship includes a mapping relationship between the virtual address space and the physical address space.

Abstract: A method for copying data from a primary fully provisioned volume to a secondary thin-provisioned volume is disclosed. In one embodiment, such a method includes hosting, on a primary storage system, a fully provisioned volume comprising a plurality of storage elements. A volume table of contents (VTOC) associated with the fully provisioned volume is analyzed to determine which storage elements in the fully provisioned volume have been allocated to data sets. In certain embodiments, the VTOC is copied into memory of a secondary host system and the secondary host system analyzes the copy to determine which storage elements in the fully provisioned volume have been allocated to data sets. The method then copies, from the fully provisioned volume to a secondary thin-provisioned volume, data in only those storage elements that have been allocated to data sets. A corresponding system and computer program product are also disclosed.

Abstract: A storage system performs transformations of data stored as source snapshots to obtain transformed snapshots. The storage system stores relations between source snapshots, for example, parent child relationships. The storage system analyzes relationships between source snapshots to determine relationships between transformed snapshots. The storage system creates a transformed snapshot based on a source snapshot. The storage system traverses the graph representing source snapshots to identify ancestor or descendants of the source snapshots that were previously transformed. The storage system determines the transformed snapshots corresponding to the ancestor and descendant source snapshots and relates them to the newly created transformed snapshot.

Abstract: The embodiments of disclosure disclose a method for differential upgrading of a variable-partition system, which includes the following steps that: partition information is acquired from a differential upgrade package; data backup is performed to data of old partitions; partition re-division is performed on the old partitions to obtain new partitions according to the partition information, and the new partitions are formatted; the backed-up data is recovered into the new partitions; and differential upgrading is performed of the variable-partition system by virtue of the differential upgrade package. The embodiments of the disclosure also disclose a device for differential upgrading of the variable-partition system. The embodiments of the disclosure further disclose a method for making a differential upgrade package with partition information. The embodiments of the disclosure further disclose a device for making a differential upgrade package with partition information.

Abstract: A device is provided that includes an operation unit having a program installed therein and including a processor that is configured to execute a first monitoring process of monitoring an arrival of a first time that is set up in advance, and a setting process of setting up the first time to be monitored in the first monitoring process in response to a request from the program. When the arrival of the first time is detected in the first monitoring process, the setting process causes the program to recover from a transitioned state to which the program transitions when the operation unit is in a power-saving mode.

Abstract: Techniques for enabling fail-safe operations of storage devices are described. A device may be configured to delete an inode in a failsafe manner. A device may be configured to mark directory entries as not in use and ensure this goes to disk before anything else. A device may be configured to create an inode in a failsafe manner. A device may be configured to allocate and initialize a cluster and ensure this is written to disk before anything else. In this manner, a volume may be in a predictable state upon a failure occurring.

Abstract: One embodiment is related to a method for backing up a shared volume using parallel save streams, comprising: taking a snapshot of the shared storage volume that is shared across a plurality of nodes; identifying used blocks on the shared storage volume based on the snapshot; organizing logically the used blocks into extent sets, wherein a number of extent sets is equal to a number of the plurality of nodes; assigning the extent sets to the plurality of nodes, wherein each of the plurality of nodes is assigned one extent set; and streaming, by each of the plurality of nodes, a respective extent set to a backup apparatus, wherein the plurality of nodes stream the extent sets in parallel.

Abstract: A computer program product, system, and method for determining one or more slices of a logical address space assigned to replication processor; determining an elapsed time since a start of a replication cycle; determining an expected number of slices that should have been replicated based on the elapsed time; and replicating one or more slices of the logical address space in response to determining the expected number of slices that should have been replicated is less than an actual number of slices replicated by the replication processor within the replication cycle.

Abstract: Provided are a computer program product, system, and method for merging multiple point-in-time copies into a merged point-in-time copy. A repository maintains a full copy of the source data and point-in-time copies at point-in-times of the source data. Each of the point-in-time copies have change information indicating changed data in the source data that changed between the point-in-time of the point-in-time copy and a subsequent point-in-time and changed point-in-time data comprising data in the source data as of the point-in-time of the point-in-time copy indicated in the change information as changed. At least two selected of the point-in-time copies in the repository are merged into a merged point-in-time copy by: forming merged change information in the merged point-in-time copy indicating changed data indicated in change information for the selected point-in-time copies; and forming merged changed data in the merged point-in-time copy from the changed data in the selected point-in-time copies.

Abstract: Implementations of the disclosure provide for size adjustable volumes for containers. A method of the disclosure includes determining, by a processing device of the PaaS system, a size used space in a storage volume with respect to a container associated with an execution of an application. The size is a summation of current usage of disk space for the storage volume by the application. This size of the used space is compared to a threshold size. The threshold size indicates a determined amount of the storage volume allocated to the container. Responsive to the size meeting the threshold size, an increase in the allocated amount of the storage volume associated with the container is regulated by the processing device.

Abstract: A method of allocating storage in a distributed storage system includes receiving a data structure including a plurality of identifiers of storage devices organized by failure zones. Any device of a first failure zone does not share a point of failure with any device of a second failure zone. The method also includes receiving a storage allocation request, identifying a first storage device using a first pointer referencing an unallocated device of the first failure zone, and modifying the first pointer to reference a next unallocated device of the second failure zone. The method also includes identifying a second storage device using a second pointer referencing an unallocated device of the second failure zone, modifying the second pointer to reference a next unallocated device of the second failure zone, and responding to the storage allocation request by providing identifiers of the first storage device and the second storage device.

Abstract: Systems and methods of data deduplication are disclosed comprising generating a hash value of a data block and comparing the hash value to a table in a first memory that correlates ranges of hash values with buckets of hash values in a second memory different from the first memory. A bucket is identified based on the comparison and the bucket is searched to locate the hash value. If the hash value is not found in the bucket, the hash value is stored in the bucket and the data block is stored in a third memory. The first memory may be volatile memory and the second memory may be non-volatile random access memory, such as an SSD. Rebalancing of buckets and the table, and use of additional metadata to determine where data blocks should be stored, are also disclosed.

Abstract: A data storage subsystem is disclosed that implements a process for storing and/or reconstructing system data, such as a system mapping table. In certain embodiments, table pages are systematically copied, or flushed, to non-volatile memory in a progressive manner, according to a fixed ratio of flushed table pages per table update trigger, thereby facilitating write and/or load efficiency. Full or partial reconstruction of a table may be performed within a bounded number of operations based on the size of the table, the ratio implemented, and/or other characteristics.

Abstract: Apparatuses, systems, and methods for deduplicating data using small data segments are described. Data strings are divided into a plurality of data segments having an original sequence order, and the data segments are rearranged according to an ordered sequence. The original sequence order of each data string is written to memory with a pointer to the ordered sequence of the data segments.

Abstract: A memory system includes a plurality of volatile memory modules to temporarily store data in a distributed manner, a V storing place management unit included in each of the volatile memory modules, a plurality of nonvolatile memory modules to store the data stored in each of the volatile memory modules in a distributed manner, and a NV storing place management unit included in each of the nonvolatile memory modules. Each V storing place management unit and each NV storing place management unit communicate and determine the destination nonvolatile memory module for each volatile memory module. The data is transmitted to the determined destination nonvolatile memory module and stored in the destination nonvolatile memory module.

Abstract: Based at least in part on an optimization goal determined for a storage group comprising a plurality of data stores for which writes requests are coordinated by a log-based transaction manager, one or more lifecycle transitions are identified for data objects stored in a first data store. In accordance with a first lifecycle transition, a data object is to be transferred from the first data store to a second data store based at least in part on a duration for which the data object has been resident at the first data store. A transition agent is assigned by a configuration manager to implement the first lifecycle transition.

Abstract: A fast snap is a consistent point-in-time copy or logical copy of a storage object that can be created more quickly than an ordinary snap. In order to generate a fast snap a storage array creates a copy or logical copy of all of the dirty data associated with a source device by copying or logically copying dirty pages cache into a separate record in volatile memory, instantiates an unpopulated snap device that is backed by persistent storage, and generates a mapping between the record and the unpopulated snap device. The mapping includes the information needed to populate the unpopulated snap device with the dirty pages from the record. Once the fast snap has been created it is possible to service IOs to the source device and the fast snap device because the data can be accessed from the record and the cache.

Abstract: A storage controller includes a processor and a memory to store first management information indicating data writing to each of a plurality of logical blocks, corresponding to a plurality of physical blocks of a storage device. The processor is configured to receive a write request to write data to a first logical block and assign a first physical block to the first logical block. The processor is also configured to record in the first management information that data has been written in the first logical block, identify a second logical block in which data has been written, write the addresses of the plurality of physical blocks as second management information, read an address of a second physical block assigned to the second logical block from the second management information, and release the second physical block.

Abstract: A method and system for grouping and compressing system logs are described herein. A generation module residing and executing on a computer system may be configured to generate system logs containing system data important for support purposes describing the computer system, and system log metadata for describing the system logs. A grouping module residing and executing on a computer system may be configured to group system logs based on the system log metadata. Each group of system logs may comprise two or more system logs having the same or similar system log metadata. A data container is produced for each group of system logs and the data container is compressed. System logs having the same or similar system log metadata are more likely to have similar data, thus resulting in a higher degree of compression when grouped and compressed together.

Abstract: An asset storage server is provided herein that assigns related files to an asset name and assigns permissions to the asset name such that related files with unrelated names can be assigned permissions independent of the file naming convention and without requiring a user to individually set the permissions of each file. The asset storage server may also generate modified versions of original file names and index a distributed object store based on the modified versions such that related files with related names are not listed in the same partition of the distributed object store. Indexing the distributed object store based on the modified versions of the original file names may reduce data retrieval latency.

Abstract: A loading order according to which function modules are to be loaded from a storage unit is determined based on a predicted use situation of a user. The function modules are loaded according to the loading order thus determined. Subsequently, the function modules thus loaded are started up according to a start-up instruction. With such an arrangement, by determining the loading order based on a prediction of the user's desired function, such an arrangement enables a reduction in the length of a loading time that bothers the user.

Abstract: A method and apparatus for multi-level data protection in a storage system. The storage system comprises a first storage device, a second storage device, and a third storage device, the method comprising: establishing a synchronous replication session between the first storage device and the second storage device so as to synchronously replicate data in the first storage device to the second storage device; establishing an asynchronous replication session between the first storage device and the third storage device so as to asynchronously replicate the data in the first storage device to the third storage device; and recovering the asynchronous replication by establishing an asynchronous replication session between the second storage device and the third storage device in response to a failure of the first storage device.

Abstract: Examples disclosed herein provide systems, methods, and software for avoiding data replication using sparse files. In one example, a method of using a sparse file to manage modifications to read-only files includes identifying an open file request on a computing system for a read-only file, and generating the sparse file corresponding to the read-only file. The method further includes identifying a modification to the read-only file and, responsive to identifying the modification, initiate a write to the sparse file based on the modification and identifying the write in a region map.

Abstract: A method, non-transitory computer readable medium, and device that assists with reducing initialization duration and performance impact during configuration of storage drives includes identifying a plurality of new storage drives in a storage system. Next, one or more zeroed out storage drives is identified from the identified plurality of new storage drives based on information present in a data portion of each the identified plurality of new storage drives. A volume group comprising the identified one or more zeroed out drives is created and this created volume group is provided for data operation.

Abstract: One or more techniques and/or systems are provided for storage device health status synchronization. For example, a first health status registry is maintained for a first storage controller at a first storage site. The first storage controller stores storage device health status information of storage devices currently owned by or assigned to the first storage controller within the first health status registry. A second health status registry is maintained for a second storage controller at a second storage site. The second storage controller stores storage device health status information of storage device currently owned by or assigned to the second storage controller within the second health status registry. The first health status registry and the second health status registry are synchronized during operation of the first storage site and the second storage site and/or are merged between the first storage site and the second storage site after disaster recovery.

Abstract: Provided are techniques for delayed consistent point-in-time copy from a secondary device in an asynchronous mirror relationship. A consistent asynchronous mirror copy is performed that includes an asynchronous mirror copy from a primary volume to a secondary volume and a point-in-time copy from the secondary volume to a tertiary volume. An initiation of a point-in-time establish from the secondary volume to an accessible consistent copy volume is received. A point-in-time reservation is created by storing metadata for a point-in-time relationship from the secondary volume to the accessible consistent copy volume. In response to a commit of the point-in-time copy from the secondary volume to the tertiary volume, a point-in-time copy from the secondary volume to the accessible consistent copy volume to store a consistency group is completed.

Abstract: A technique for accessing replicas of file systems generates a snap (point-in-time version) of a replica on a destination site and provides access to the snap for reading and/or writing while replication operations on the replica between a source site and the destination site continue to be performed. As the snap is a separate object from the replica itself, users may access the snap without affecting the replica or disturbing ongoing replication activities. Users can also mount the snap to perform destination-driven backup operations, to test disaster recovery, and/or to perform other activities.

Abstract: A capacity control module which manages a physical storage area provided by each of storage devices in a cluster by dividing the physical storage area into a physical storage area having a predetermined size, in a plurality of storage nodes configuring a storage system, and, a storage control module which receives an I/O request from a higher-level device are provided, two storage control modules, which are arranged in different storage nodes and configure a redundancy configuration, are managed as a storage control module pair, the capacity control module preferentially allocates each of the physical storage areas in the vicinity of an arrangement destination of each of the storage control modules configuring the storage control module pair, to the storage control module pair, and storage data is read from or is written on the physical storage area, according to a command applied from the storage control module.