Abstract: Systems and methods for reconstructing a namespace in response to a loss of data are described herein. To reconstruct the namespace, metadata super segments within a set of file containers may be located in response to a determination that original namespace metadata associated with files stored within the containers is unreadable. Content handles linking the super segment to metadata for each of the files may then be reconstructed from the super segments. Both inode numbers and M-tree identifiers are also extracted from the super segments for each of the files referenced by the super segments. Then, a plurality of U-trees is assembled by aggregating and linking the inode numbers of the files referenced within the super segments based on the M-tree identifiers. Finally, the recovered namespace metadata tree data structure for all of the files within the plurality of LP containers is assembled by aggregating and linking the U-trees.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date and a selection period to select a subset of data objects to be protected during the protection period. A retention lock is applied to files that meet a set selection criteria, as determined by a matching or filter process. The selection criteria can include one or more of time-based selection, filetype selection, or tag-based selection.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date, and applying a retention lock to one or more data objects stored in cloud storage during the protection duration, the retention lock preventing unauthorized deletion, modification or movement of the data. A renew threshold date is defined within the protection duration, and a new filesystem is spawned on this date. All new data is redirected to the new filesystem and locked for the protection duration, and the first file system is destroyed.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date and a selection period to select a subset of data objects to be protected during the protection period. A retention lock is applied to the subset of data objects stored in cloud storage during the protection duration, the retention lock preventing unauthorized deletion, modification or movement of the data. A cloud bucket is created for storing the subset of data objects and a new filesystem is spawned for attachment to the cloud bucket. Upon expiration of the selection period, the retention locks are expired and the new filesystem is destroyed.

Abstract: A method for providing data immutability using a deduplication filesystem for data objects stored in the cloud by defining a protection duration from a first date to a fixed future date, and applying a retention lock to one or more data objects stored in cloud storage during the protection duration, the retention lock preventing deletion, modification or movement of the data objects by an unauthorized entity. The method defines a renew threshold date within the protection duration, and performs a garbage collection (GC) operation at the renew threshold date to permanently delete dead data objects and carry forward live data objects, and extends the retention lock on remaining live data objects and subsequently written data objects upon completion of the GC operation.

Abstract: Data protection aware data migration in distributed systems is disclosed. While actively migrating data from a source to a target, the process of protecting the data is performed with respect to a recovery system. Thus, data is replicated to both the target and the recovery system. During a final phase, synchronization data is sent to both the target and the recovery system. After transmitting the synchronization data, replication or migration then between the source and the target and between the source and the recovery system ceases. The target then synchronized with the recovery system based on the synchronization data and begins operation as a new source. The old source, from which the data was migrated to the new source, may be decommissioned.

Abstract: Embodiments of a system and method for migrating data from a first cloud network provided from a first cloud service provider (CSP) and a second cloud network provided by a second CSP. The method reads and packs only live data containers from the first cloud using a Garbage Collection (GC) process. It performs a collection replication process replicating a local container set (CSET) from the first cloud network to the second cloud network, and replicates data containers from the first cloud network to the second cloud network. It then uses a container manager process to write data containers on the second cloud network.

Abstract: Methods of cloning data backup across namespaces are disclosed. One or more source files are cloned from a first namespace to a second namespace, as one or more destination files. When the cloning of the source file(s) is performed, a data structure including source file information and destination file information is generated. A source synthetic file is cloned from the first namespace to the second namespace, as a destination synthetic file, where the source synthetic file uses the source file(s) as one or more base files on the first namespace. When the cloning of the source synthetic file is performed, the data structure is looked up to obtain the source file information and the destination file information. Based on the source file information and the destination file information, synthetic information of the destination synthetic file is updated to use the destination file(s) as one or more base files on the second namespace.

Abstract: Methods of cloning data backup across namespaces are disclosed. In response to receiving a file replication request, one or more source files are cloned from a first namespace to a second namespace, as one or more destination files. A first attribute indicating that the source file(s) have been cloned to the second namespace as the destination file(s) is created. A source synthetic file is cloned from the first namespace to the second namespace, as a destination synthetic file, where synthetic information of the destination synthetic file points to the source file(s) as one or more base file. A second attribute indicating that the source synthetic file has been cloned to the second namespace as the destination synthetic file is created. Based on the first and second attributes, the synthetic information of the destination synthetic file is updated or propagated to use the destination file(s) as one or more base files on the second namespace.

Abstract: A method of blocking access to files encrypted with a compromised key by mapping keys and ranges of containers encrypted by the keys. Upon notification that a key is compromised, fencing a container range corresponding to data segments encrypted by the compromised key to prevent deduplication operations on the segments. The method makes a point-in-time copy of the filesystem managing the segments, wherein each file of the file system is represented as tree structure having a root level and other levels. The method iteratively inspects in a level-wise manner, each container in each level of the file trees of the files to identify containers having segments encrypted by the compromised key, and marks files corresponding to the identified containers as not readable to block the access to the files encrypted with the compromised key.

Abstract: Embodiments for balancing cloud resource capacity for cross-node movement of files in a scaled out backup system, and for dynamically allocating cloud storage resources in a multi-node network having a file system. A process determines a destination node with dedicated cloud storage capable of storing a file selected for long term retention. It transfers the file to the cloud storage of the destination node while maintaining metadata of the file in the cloud tier local storage of the destination node, such as by using remote procedure calls between the destination and source nodes. It then updates a global namespace of the file system with a handle indicating a current location of the file as the cloud storage of the destination file, thus allowing access to the file through the metadata stored in the local storage of the source node.

Abstract: Embodiments of a system and method for migrating data from a first cloud network provided from a first cloud service provider (CSP) and a second cloud network provided by a second CSP. The method reads and packs only live data containers from the first cloud using a Garbage Collection (GC) process. It performs a collection replication process replicating a local container set (CSET) from the first cloud network to the second cloud network, and replicates data containers from the first cloud network to the second cloud network. It then uses a container manager process to write data containers on the second cloud network.

Abstract: Embodiments for balancing cloud resource capacity for cross-node movement of files in a scaled out backup system, and for dynamically allocating cloud storage resources in a multi-node network having a file system. A process determines a destination node with dedicated cloud storage capable of storing a file selected for long term retention. It transfers the file to the cloud storage of the destination node while maintaining metadata of the file in the cloud tier local storage of the destination node, such as by using remote procedure calls between the destination and source nodes. It then updates a global namespace of the file system with a handle indicating a current location of the file as the cloud storage of the destination file, thus allowing access to the file through the metadata stored in the local storage of the source node.

Abstract: In general, embodiments of the invention relate to methods and systems for replicating data, which is stored in a source system, in a target system. More specifically, embodiments of the invention enable parallel transmission and verification of portions of the data. Once the portions of the data have been verified, embodiments of the invention combine the verified portions of the data to obtain the final combined data. The combined data is then verified.

Abstract: An example method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, identifying, in the stream, random I/O requests and sequential I/O requests, queuing the sequential I/O requests in a priority/processing queue and processing the sequential I/O requests immediately without waiting for loading of any metadata, and processing a first one of the random I/O requests and a second one of the random I/O request. When the first and second random I/O requests each depend on the same range of metadata, the first and second random I/O requests are processed in the order received, and when the first and second random I/O requests each depend on different respective ranges of metadata, the first and second random I/O requests are processed at the same time.

Abstract: An example method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, identifying, in the stream, random I/O requests and sequential I/O requests, queuing the sequential I/O requests in a priority/processing queue and processing the sequential I/O requests immediately without waiting for loading of any metadata, and processing a first one of the random I/O requests and a second one of the random I/O request. When the first and second random I/O requests each depend on the same range of metadata, the first and second random I/O requests are processed in the order received, and when the first and second random I/O requests each depend on different respective ranges of metadata, the first and second random I/O requests are processed at the same time.

Abstract: In one example, a method includes receiving a stream that identifies I/O operations requested to be performed with respect to a data grouping, and identifying, in the stream, random I/O requests and sequential I/O requests. Next, the stream is split into an incoming queue containing the sequential I/O requests and a priority/processing queue containing the random I/O requests. A metadata load queue is then defined by assigning each of the random I/O requests to a respective range of metadata. The metadata of each range is loaded, and the random I/O requests re-queued. Using the loaded metadata, a random I/O operation is performed that corresponds to one of the random I/O requests. Finally, the random I/O operation and a sequential I/O operation are performed concurrently with each other.

Abstract: A replacement memory page is identified by accessing a first list of page records, and if the first list is not empty, identifying a replacement page from a next page record indicator of the first list. A second list of page records is accessed if the first list is empty, and if the second list is not empty, the replacement page is identified from a next page record indicator of the second list. A third list of page records is accessed if the first and second lists are empty, and the replacement page is identified from a next page record indicator of the third list.

Abstract: A replacement memory page is identified by accessing a first list of page records, and if the first list is not empty, identifying a replacement page from a next page record indicator of the first list. A second list of page records is accessed if the first list is empty, and if the second list is not empty, the replacement page is identified from a next page record indicator of the second list. A third list of page records is accessed if the first and second lists are empty, and the replacement page is identified from a next page record indicator of the third list.

Abstract: The invention provides apparatus and methods for registering shared memory objects. An exemplary embodiment of the invention is a method for registering a shared memory object are presented including: locating the shared memory object, the shared memory object having a first size and a first virtual address; finding a shared memory region associated with the shared memory object, the shared memory region having a second size and a second virtual address, the second virtual address returned as a unique identifier; and registering the shared memory object using the unique identifier. In some embodiments, the methods further include: calculating a memory offset for the shared memory region; and registering the shared memory range using the memory offset. In some embodiments, the methods further include: storing the unique identifier, the second size, and the memory offset in a persistent shared memory data structure.