Abstract: Described is a system for leveraging incremental backup metadata of a deduplication storage system to perform an efficient data restore. The system may reference metadata stored as part of incremental backup infrastructure to determine only the changes to the data that occurred subsequent to the restore point. As a result, the system may reduce data read processing by retrieving only the changed data from the deduplication storage system. Moreover, the system may reduce the required bandwidth for the restore by providing only the changed data to a target device of the restore. Data retained by the client system such as the current version of the data may then be overwritten with the reduced set of data to complete the restore. As a result, the system may perform an efficient restore by reducing read accesses and bandwidth requirements, and thus, improve overall restore performance.

Abstract: Detecting a trend in latency for storage underlying a deduplicated file system includes generating a set of data points by recording when input/output (IO) requests were issued to the storage and recording time required to receive success responses from the storage. Least squares regression is performed on the data points to find a best-fit line through the data points. A slope of the best-fit line is calculated. A determination is made as to whether the slope is positive, a positive slope thereby indicating a trend of increasing latency of the storage. When the slope is determined to be positive, clients accessing the deduplicated file system are throttled.

Abstract: Corruption detection in backups is disclosed. Backups that are received into a backup environment are stored in corresponding lineages. A detection engine is configured to perform corruption detection operations on the most recent backups in each of the lineages based on a sample frequency. Corruption detection operations may also be performed randomly and based on unexpected or unusual changes in backup metadata.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date and a selection period to select a subset of data objects to be protected during the protection period. A retention lock is applied to files that meet a set selection criteria, as determined by a matching or filter process. The selection criteria can include one or more of time-based selection, filetype selection, or tag-based selection.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date, and applying a retention lock to one or more data objects stored in cloud storage during the protection duration, the retention lock preventing unauthorized deletion, modification or movement of the data. A renew threshold date is defined within the protection duration, and a new filesystem is spawned on this date. All new data is redirected to the new filesystem and locked for the protection duration, and the first file system is destroyed.

Abstract: Embodiments for providing file immutability for cloud storage data in a deduplicating filesystem and using a new filesystem that is spawned to receive redirected live data, after which the old filesystem is expired. Data objects are stored in the cloud by defining a protection duration from a first date to a fixed future date and a selection period to select a subset of data objects to be protected during the protection period. A retention lock is applied to the subset of data objects stored in cloud storage during the protection duration, the retention lock preventing unauthorized deletion, modification or movement of the data. A cloud bucket is created for storing the subset of data objects and a new filesystem is spawned for attachment to the cloud bucket. Upon expiration of the selection period, the retention locks are expired and the new filesystem is destroyed.

Abstract: A prefetch mechanism stores data in the memory allocated in a remote passive (remote) node in a High Availability active-passive node pair. A read-ahead cache that can host different parts of a file is created on both the active and the passive nodes. The system improves the performance of sequential restores by using memory available on a passive node to supplement the local (active node) memory to hold the prefetched data for the common case of large sequential restores. This extra memory overcomes issues when a current prefetch fails to keep up with the target restore performance. Filesystem redirection is thus accomplished by servicing an I/O request by sending it another node that itself caches the data.

Abstract: Embodiments of a system and method to track the locality of a file being restored at the time of prefetching; and a mechanism to dynamically adjust the prefetching parallelism, per read batch, optimally based on the locality and other heuristics, such as system load. A process tracks locality of data elements in a batched data stream, as corresponds to a number of different container IDs accessed by the data elements. The prefetch nominally works serially on the data elements, however, if the locality exceeds a threshold separating acceptable versus non-acceptable distribution of data accesses, each batch is divided into a number of smaller sub-batches that are then pre-fetched in parallel with one another.

Abstract: A method for providing data immutability using a deduplication filesystem for data objects stored in the cloud by defining a protection duration from a first date to a fixed future date, and applying a retention lock to one or more data objects stored in cloud storage during the protection duration, the retention lock preventing deletion, modification or movement of the data objects by an unauthorized entity. The method defines a renew threshold date within the protection duration, and performs a garbage collection (GC) operation at the renew threshold date to permanently delete dead data objects and carry forward live data objects, and extends the retention lock on remaining live data objects and subsequently written data objects upon completion of the GC operation.

Abstract: Embodiments for processing fastcopy overwrite backup workloads (FCOW) as virtual synthetic backups, by detecting whether or not a backup workload comprises an FCOW workload in which a base file is fastcopied to a target file, wherein portions of the base file are overwritten through overwrites by new data in the target file by a deduplication backup process.

Abstract: A self-healing method of recovering files using a cyber recovery vault for deduplication filesystems. Data to be backed up from a production system to a backup system in a datacenter is also copied to a cyber security vault coupled to the data center through an air gapped connection. A copy process makes a point-in-time (PIT) copy of the portion of data to a namespace in the vault, and retention locks the copied portion in the vault. An automated process detects a missing segment error, such as due to a bad index lookup in the data to be backed up. The self-healing process provides the missing segment from the PIT copy in the cyber security vault. For any missing fingerprints the process adds the data in new containers and the references in the index, providing an indirection that allows for just writing a new container with the missing L0 references and updating the index.

Abstract: Embodiments of file restores in a Data Domain (DD) file system implementing a DD Bandwidth Optimized Open Storage Technology (DDBoost) library that translates application read and write request to DDBoost application program interfaces (APIs). A prefetch queue processor creates an intent to read the file. The application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to open the file to load into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle. An extended DDBoost API expression is defined to pass the current path and destination path to the application.

Abstract: Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. A prefetch queue processor creates an intent to read the file, rather than opening the file upon receiving the request. During this step, the application passes the file handle of the file, and the destination handle where the data must be read into. As the queue is processed, the prefetch for the request (handle/offset/length) is passed to the file server. The filesystem processes the request to equivalently ‘open’ the file, and bring the data into memory. As the read request for the same file reaches the filesystem the file data is read from memory for writing to the destination handle.

Abstract: Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the small files in a prefetch queue such that a combined size of the small files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A queue processor issues a read request for the first file in the queue, scans the prefetch queue to find a read request for a file at the prefetch horizon, and prefetches the file at the prefetch horizon. The prefetch queue essentially constitutes a hint from the client that a read I/O is imminent for purposes of filling the read-ahead cache and preventing a need to issue a blocking I/O operation.

Abstract: Embodiments of small file restore process in deduplication file system wherein restoration requires issuing a read request within an I/O request to the file system. The process places the files in a prefetch queue such that a combined size of the files meets or exceeds a size of the prefetch queue as defined by a prefetch horizon. A file handler disassociates prefetch streams from read streams. The handler prefetches the read operations and stores them in memory. The stream corresponding to a read will only open as the read requests hit the queue processor. As a result, the stream usage is very low, since the I/O, worker threads and the read streams are disassociated from each other.

Abstract: A system creates a snapshot of a part of copy management content which is displayed by a user interface, and which is selected by a user of the system as context for a subsequent message from the user. The system creates a hyperlink in the message to the snapshot which depicts the part of the copy management content. The system stores user entries for a tag that identifies at least one intended recipient of the message, a time when the message is made available for the intended recipient(s), and text for the message. The system sends the message at the time to the intended recipient(s) via an internal communication channel of the system. In response to the recipient of the message selecting the message's hyperlink to the snapshot, the system outputs the context for the message to the recipient's user interface, which enables the recipient to respond to the message.

Abstract: During a current backup of a file system of a client, a determination is that a file in the current backup includes first data regions found in a file from a previous backup, and second data regions that are new. A virtual synthetic file is generated by writing the second data regions, and synthesizing the first data regions by referencing the file from the previous backup. A request received to restore the particular file includes identifications of other files still present at client. A determination is that the particular file corresponds to the virtual synthetic file and that the other files still present at client include the file from the previous backup. The second data regions are read to send to client. A complete version of the particular file is assembled at client using the file from the previous backup, still present at client, and the second data regions.

Abstract: Methods of cloning data backup across namespaces are disclosed. One or more source files are cloned from a first namespace to a second namespace, as one or more destination files. When the cloning of the source file(s) is performed, a data structure including source file information and destination file information is generated. A source synthetic file is cloned from the first namespace to the second namespace, as a destination synthetic file, where the source synthetic file uses the source file(s) as one or more base files on the first namespace. When the cloning of the source synthetic file is performed, the data structure is looked up to obtain the source file information and the destination file information. Based on the source file information and the destination file information, synthetic information of the destination synthetic file is updated to use the destination file(s) as one or more base files on the second namespace.

Abstract: Methods of cloning data backup across namespaces are disclosed. In response to receiving a file replication request, one or more source files are cloned from a first namespace to a second namespace, as one or more destination files. A first attribute indicating that the source file(s) have been cloned to the second namespace as the destination file(s) is created. A source synthetic file is cloned from the first namespace to the second namespace, as a destination synthetic file, where synthetic information of the destination synthetic file points to the source file(s) as one or more base file. A second attribute indicating that the source synthetic file has been cloned to the second namespace as the destination synthetic file is created. Based on the first and second attributes, the synthetic information of the destination synthetic file is updated or propagated to use the destination file(s) as one or more base files on the second namespace.

Abstract: One example method includes creating a namespace of a deduplication filesystem, adding a ‘temporary’ flag to the namespace, adding a ‘delete’ flag to the namespace when the namespace is to be deleted, and when a maximum number of namespaces permitted by the deduplication filesystem has been reached due to the creating of the namespace, deleting another namespace bearing a ‘temporary’ flag and a ‘delete’ flag. In this way, efficient use may be made of the NVRAM, while also maintaining compliance with deduplication filesystem requirements concerning the maximum number of permitted namespaces.