Abstract: A microservice or serverless process consistency check process comprising locating all the necessary metadata and data objects in the cloud by storing the data objects in the cloud and synchronously mirroring the metadata, which is separately stored in local storage, to the cloud. The process generates a list of data objects in the cloud as “Set A” and the list of metadata objects in the same prefix range as the data objects as “Set B.” The consistency check then verifies whether all objects in Set A are referred to by objects in set B. In the case where there are gaps between the sets, non-existent objects are marked as missing, and unreferenced objects are marked as orphan objects. The list of missing and orphan objects is then sent back to the backup server for analysis and further processing.

Abstract: One example method includes receiving clear text data at a storage system, generating, at the storage system, a clear text data encryption key, requesting a key management system to encrypt the clear text data encryption key with a master key to create an encrypted data encryption key, and the requesting is performed by the storage system, receiving, at the storage system, the encrypted data encryption key from the key management system, encrypting, at the storage system, the clear text data with the clear text data encryption key to create encrypted data, and storing, together, the encrypted data and the encrypted data encryption key.

Abstract: Embodiments of an instant recall process and system for long-term data stored on a cloud storage tier. Embodiments include saving a content handle of a file in a cloud storage tier as an extended attribute in a single file system namespace; moving the file from the cloud storage tier to an active storage tier for data processing; recalling the file from the active storage tier to the cloud storage tier upon completion of the data processing; using the content handle from hidden metadata for a working copy of the file; and saving a hash of a segment reference as part of the extended attribute.

Abstract: Embodiments of method and system are directed to a best effort method to reconstruct the corrupt files from available nodes or tiers and heal or repair the corrupt files. The method looks for the existence of fingerprints of the lost segments in available tiers on the nodes or across nodes within a cluster. The method discovers the copies of these missing segments, creates an Lp tree referring these discovered segments and creates a virtual file in the namespace. The file is then moved to the destination tier where the corruption exists, thus healing the corruption.

Abstract: Data protection systems generate backups that may be stored in a namespace that includes an active tier and a cloud tier. Lost data in either tier can be reconstructed from the available tiers. Metadata can also be reconstructed from the data stored in the available tiers.

Abstract: Embodiments of method and system are directed to a best effort method to reconstruct corrupt files from available nodes or tiers and heal or repair the corrupt files within a cluster of nodes. The method looks for the existence of fingerprints of the lost segments in available tiers on the nodes or across nodes within a cluster. The method discovers the copies of these missing segments, creates an Lp tree referring these discovered segments and creates a virtual file in the namespace. The file is then moved to the destination tier where the corruption exists, thus healing the corruption.

Abstract: A microservice or serverless process consistency check process comprising locating all the necessary metadata and data objects in the cloud by storing the data objects in the cloud and synchronously mirroring the metadata, which is separately stored in local storage, to the cloud. The process generates a list of data objects in the cloud as “Set A” and the list of metadata objects in the same prefix range as the data objects as “Set B.” The consistency check then verifies whether all objects in Set A are referred to by objects in set B. In the case where there are gaps between the sets, non-existent objects are marked as missing, and unreferenced objects are marked as orphan objects. The list of missing and orphan objects is then sent back to the backup server for analysis and further processing.

Abstract: One example method includes receiving clear text data at a storage system, generating, at the storage system, a clear text data encryption key, requesting a key management system to encrypt the clear text data encryption key with a master key to create an encrypted data encryption key, and the requesting is performed by the storage system, receiving, at the storage system, the encrypted data encryption key from the key management system, encrypting, at the storage system, the clear text data with the clear text data encryption key to create encrypted data, and storing, together, the encrypted data and the encrypted data encryption key.

Abstract: A microservice or serverless process consistency check process comprising locating all the necessary metadata and data objects in the cloud by storing the data objects in the cloud and synchronously mirroring the metadata, which is separately stored in local storage, to the cloud. The process generates a list of data objects in the cloud as “Set A” and the list of metadata objects in the same prefix range as the data objects as “Set B.” The consistency check then verifies whether all objects in Set A are referred to by objects in set B. In the case where there are gaps between the sets, non-existent objects are marked as missing, and unreferenced objects are marked as orphan objects. The list of missing and orphan objects is then sent back to the backup server for analysis and further processing.

Abstract: System generates data structure based on unique identifiers of objects in object storages and sets indicators in positions that correspond to hashes of unique identifiers of active objects. If a first number of regions of active data objects in first data storage and second number of regions of active data objects in second data storage each fail to satisfy data threshold, then system creates model identifying locations and sizes of regions of active data objects in first data storage and regions of active data objects in second data storage. System resets indicators in positions in data structure which correspond to hashes of unique identifiers of active data objects associated with model and enables remote storage to use model to copy regions of active data objects in first data storage and second data storage to third data storage, and to delete first data storage and second data storage.

Abstract: Embodiments of method and system are directed to a best effort method to reconstruct the corrupt files from available nodes or tiers and heal or repair the corrupt files. The method looks for the existence of fingerprints of the lost segments in available tiers on the nodes or across nodes within a cluster. The method discovers the copies of these missing segments, creates an Lp tree referring these discovered segments and creates a virtual file in the namespace. The file is then moved to the destination tier where the corruption exists, thus healing the corruption.

Abstract: Embodiments of method and system are directed to a best effort method to reconstruct corrupt files from available nodes or tiers and heal or repair the corrupt files within a cluster of nodes. The method looks for the existence of fingerprints of the lost segments in available tiers on the nodes or across nodes within a cluster. The method discovers the copies of these missing segments, creates an Lp tree referring these discovered segments and creates a virtual file in the namespace. The file is then moved to the destination tier where the corruption exists, thus healing the corruption.

Abstract: A microservice or serverless process consistency check process comprising locating all the necessary metadata and data objects in the cloud by storing the data objects in the cloud and synchronously mirroring the metadata, which is separately stored in local storage, to the cloud. The process generates a list of data objects in the cloud as “Set A” and the list of metadata objects in the same prefix range as the data objects as “Set B.” The consistency check then verifies whether all objects in Set A are referred to by objects in set B. In the case where there are gaps between the sets, non-existent objects are marked as missing, and unreferenced objects are marked as orphan objects. The list of missing and orphan objects is then sent back to the backup server for analysis and further processing.

Abstract: Embodiments are described for performing file restores from remote high-latency storage tiers by reading available data from a local low-latency tier in a deduplication appliance. A request to restore a previously segmented and deduplicated file can be received by a storage appliance from an application, each segment having a fingerprint. The name of the file can be looked up in an index on the storage appliance, and a first batch of fingerprints of segments of the file can be retrieved from the index. Each fingerprint can be looked up in metadata in the index to determine whether the segment corresponding to the fingerprint is available locally and therefore need not be retrieved locally. A list of local and remote prefetch segments is generated, and a prefetch request is generated for each list, if non-empty. Use of the prefetch scheme can be dynamically turned on or off.

Abstract: Embodiments of an instant recall process and system for long-term data stored on a cloud storage tier. Embodiments include saving a content handle of a file in a cloud storage tier as an extended attribute in a single file system namespace; moving the file from the cloud storage tier to an active storage tier for data processing; recalling the file from the active storage tier to the cloud storage tier upon completion of the data processing; using the content handle from hidden metadata for a working copy of the file; and saving a hash of a segment reference as part of the extended attribute.

Abstract: Embodiments are described for performing file restores from remote high-latency storage tiers by reading available data from a local low-latency tier in a deduplication appliance. A request to restore a previously segmented and deduplicated file can be received by a storage appliance from an application, each segment having a fingerprint. The name of the file can be looked up in an index on the storage appliance, and a first batch of fingerprints of segments of the file can be retrieved from the index. Each fingerprint can be looked up in metadata in the index to determine whether the segment corresponding to the fingerprint is available locally and therefore need not be retrieved locally. A list of local and remote prefetch segments is generated, and a prefetch request is generated for each list, if non-empty. Use of the prefetch scheme can be dynamically turned on or off.

Abstract: Techniques for resource operation based on usage, sharing, and recommendations with modular authentication are provided. A resource space is associated with a principal. The resource space comprises resources local to a device of the principal and remote from the device. The resources presented in a merged view within a local file system and operating system of the device. In an embodiment, the device negotiates authentication with an intermediary for access to a legacy service where authentication is performed by the intermediary on behalf of the device, and the intermediary establishes an authentication session between the principal and the legacy service.

Abstract: Techniques for resource operation based on usage, sharing, and recommendations with modular authentication are provided. A resource space is associated with a principal. The resource space comprises resources local to a device of the principal and remote from the device. The resources presented in a merged view within a local file system and operating system of the device. In an embodiment, the device negotiates authentication with an intermediary for access to a legacy service where authentication is performed by the intermediary on behalf of the device, and the intermediary establishes an authentication session between the principal and the legacy service.

Abstract: Techniques for the session sensitive data backups and restores are presented. Data having a plurality of hard linked file references are backed up and restored once during a backup operation. Any modifications to the backed up data are noted as session-specific versions and also backed up once. The hard linked file references are maintained in a data structure and managed during backups to define associations to the backed up data and to the session-specific versions of the data. The data structure is also used during restores to re-establish desired hard linked file reference associations to either the backed up data or to a particular session-specific versions of the data.

Abstract: Techniques for the session sensitive data backups and restores are presented. Data having a plurality of hard linked file references are backed up and restored once during a backup operation. Any modifications to the backed up data are noted as session-specific versions and also backed up once. The hard linked file references are maintained in a data structure and managed during backups to define associations to the backed up data and to the session-specific versions of the data. The data structure is also used during restores to re-establish desired hard linked file reference associations to either the backed up data or to a particular session -specific versions of the data.