Abstract: A distributed data storage system comprises features for integration with application orchestrators such as Kubernetes, and includes a proprietary Container Storage Interface (CSI) driver. Features include setting snapshot scheduling and retention policies, and a “container data mover” that replicates data from a source to a distinct destination distributed data storage system. The migration may be configured one-to-one, one-to-many, unidirectional, and/or bi-directional. Metadata-based snapshots and metadata-based changed block tracking identify payload data to move from source to destination within application orchestrator frameworks at both ends. Payload data is migrated from source to destination using different techniques than those used for migrating metadata, e.g., kernel-to-kernel for copying payload data versus ordinary metadata writes. An illustrative barrier logic ensures that the migration follows a controlled progression of operations.

Abstract: Disclosed techniques relate to security of backup data. In some embodiments, a method includes receiving, by data protection service running on a cloud computing system, a first encrypted copy of a backup of a first data store that is associated with a first account of an organization, where the first encrypted copy is encrypted using a first custodian cryptographic key that is shared between the organization and the data protection service that is different than a first production cryptographic key that is private and used by the organization to encrypt a non-backup version of the first data store. The method may include generating a second encrypted copy of the backup, including by encrypting the backup using a storage cryptographic key. The method may include storing the second encrypted copy of the backup in a second data store that is associated with the data protection service.

Abstract: The disclosed data storage management system enables data owners to model the costs and attributes of archiving their data and to readily capture and implement one or more resultant archiving plans. Modeling enables data owners to make informed choices about cost profiles before data is actually archived. Archiving plans devised according to these choices are intended to save on data storage costs and provide a compliance-ready data archive in cloud storage repository(ies). Armed with archiving simulations supplied by the illustrative data storage management system, a data owner may control data placement to predict costs, free up primary storage, and move inactive data to less expensive archive storage. Preferably, the disclosed system is implemented as a software-as-a-service (SaaS) solution, and the accompanying archive storage is implemented as a cloud storage service, but the invention is not limited to SaaS or to cloud-based data archives.

Abstract: A secure data storage management “recovery system” operates in a cloud computing environment that is apart from a source system and source data being protected. A “recovery manager” in the cloud computing environment is responsible for restoring secondary copies that were generated by the source system. The recovery manager gains knowledge (metadata) about the secondary copies by restoring, for its own use, a management database of, and backed up by, the source system. The recovery manager initiates out-of-place restores of the secondary copies to “recovery clients” in the recovery cloud. The recovery system restores, to the recovery cloud, secondary copies from any cloud platform and/or from non-cloud data centers, including secondary copies stored locally, off-cloud by the source system. The recovery manager comprises new features that enforce its isolation from the source system and they act to protect the integrity of the secondary copies generated by the source system.

Abstract: Hypervisor-independent reference copies of virtual machine payload data based on block-level pseudo-mount infrastructure and techniques are generated and stored in an illustrative data storage management system. An illustrative hypervisor-independent reference copy includes one or more virtual-machine payload data files that originated from a first virtual machine. The hypervisor-independent virtual-machine-payload reference copy is governed by a distinct reference copy policy that controls retention, storage, tiering, scheduling, etc. for the reference copy, independently of how the illustrative system treats other virtual machine payload data files originating from the same virtual machine.

Abstract: An illustrative data storage management system uses a control layer that controls information content presented to users and ensures information privacy between diverse tenants and/or resellers who share the system. The system populates a relationship database as transactions roll in (intake processing), and uses information in the relationship database later when processing responses (output processing). The relationship database comprises associations between e.g., a company ID and any number of entities that were created by or on behalf of the company or that are related to the company's service in the system. The control layer parses raw results that are responsive to requests for information and prevents others' information from being included in the responsive message(s). The techniques disclosed herein are not limited to shared systems managed by service providers, and may be implemented in fully owned and operated systems to add security and privacy among diverse users and/or departments.

Abstract: A client machine writes to and reads from a virtual disk on a remote storage platform. Metadata is generated and stored in replicas on different metadata nodes of the storage platform. A modified log-structured merge tree is used to store and compact string-sorted tables of metadata. During file storage and compaction, a consistent file identification scheme is used across all metadata nodes. A fingerprint file is calculated for each SST (metadata) file on disk that includes hash values corresponding to regions of the SST file. To synchronize, the fingerprint files of two SST files are compared, and if any hash values are missing from a fingerprint file, then the key-value-timestamp triples corresponding to these missing hash values are sent to the SST file that is missing them. The SST file is compacted with the missing triples to create a new version of the SST file. The synchronization is bi-directional.

Abstract: The methods and systems described herein provide a solution for protecting configuration data associated with a virtual private cloud. Network configurations, accounts, and/or security data may be collected from the host of the virtual private network using APIs provided by the host. The point-in-time configuration information of a VPC can be used in disaster recovery, migration to a new VPC host, or manage VPC configurations across accounts and regions. During a backup operation of a computing resource within a VPC, one or more processes requests security, network, and/or gateway configuration information from the VPC host. This information may be collected and requested for the entire VPC topology. The VPC configuration information, in the format/output received from the VPC host, is then backed up and stored with the backup of that computing resource in secondary storage.

Abstract: Certain embodiments described herein relate to an improved synthetic full backup image generation system. In some embodiments, one or more components in an information management system can identify a file-server-created backup copy in a particular backup format of a plurality of backup formats, determine structure information associated with the particular backup format, and generate a synthetic full backup copy according to the structure information, where the synthetic full backup copy is also in the particular backup format identical to that of the file-server-created backup copy.

Abstract: Backup data is leveraged to determine whether primary data has been encrypted by malware. The disclosed approach does not rely on recognizing particular malware instances or malware provenance, and thus can be applied to any body of data. Even a novel and previously unknown malware attack can be detected in this way. An illustrative data storage management system analyzes secondary copies it created over time, applies a multi-factor analysis to data recovered from the secondary copies and, based on the analysis, infers whether the primary data from which the secondary copies were created may be encrypted. The present approach uses successive versions of backup copies to find indicia of malware encryption, rather than trying to trace or identify the malware itself. Indicia of entropy correlate highly with encryption, such as encryption performed by malware attacks. Conversely, indicia of similarity correlate highly with lack of encryption of successive versions of documents.

Abstract: An illustrative media agent (MA) in a data storage management system instructs a NAS file server (filer) to restore an MA-created synthesized-copy instead of larger filer-created backup copies. The synthesized-copy is designed only for the particular files to be restored and mimics, and is typically much smaller than, a filer-created backup copy. The synthesized-copy is fed to the filer on restore as a “restore data image.” When receiving a restore request for certain backed-up data files, the MA synthesizes the synthesized-copy on the fly. The MA generates a header mimicking a filer-created backup header; extracts files from filer-created backup copies arranging them within the synthesized-copy as if in filer-created backups; and instructs filer to perform a full-volume restore from the synthesized-copy. The MA serves the synthesized-copy piecemeal as available, rather than waiting to synthesize the entire synthesized-copy. The synthesized-copy is not stored at the MA.

Abstract: Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, a client obtains information for enabling a secondary storage job (e.g., a backup or restore) from a storage manager and stores the information (which may be referred to as job metadata) in a local cache. The client may then reuse the job metadata for multiple storage jobs reducing the frequency of communication with the storage manager. When a configuration of the information management system changes, or the availability of resources changes, the storage manager can push updates to the job metadata to the clients. Further, a client can periodically request updated job metadata from the storage manager ensuring that the client does not rely on out-of-date job metadata.

Abstract: Presented herein are systems and methods for backing up and restoring related backup data stored in different cloud storage tiers in a cloud storage environment. Backup data generated during a backup job is stored to different cloud storage tiers based on the type of data and a storage policy comprising designations of a first cloud storage tier and a second cloud storage tier. Backup metadata and index data is stored in the designated first cloud storage tier and backup payload data is stored in the designated second cloud storage tier. Designations of the storage tiers are based on the attributes associated with each tier of cloud storage. During restore, the backup payload data is recalled from the second storage tier into the first storage tier where it is used to restore to the destination client or customer.

Abstract: An information management system includes one or more client computing devices in communication with a storage manager and a secondary storage computing device. The storage manager manages the primary data of the one or more client computing devices and the secondary storage computing device manages secondary copies of the primary data of the one or more client computing devices. Each client computing device may be configured with a ransomware protection monitoring application that monitors for changes in their primary data. The ransomware protection monitoring application may input the changes detected in the primary data into a machine-learning classifier, where the classifier generates an output indicative of whether a client computing device has been affected by malware and/or ransomware. Using a virtual machine host, a virtual machine copy of an affected client computing device may be instantiated using a secondary copy of primary data of the affected client computing device.

Abstract: Techniques disclosed herein are well suited to restoring deduplicated backup data from cloud-based storage and from multi-node replicated files systems, and they also improve performance in more traditional data storage technologies. Pre-restore steps include analysis of deduplication indexes to identify data segments that are stored consecutively on storage media. Reading data in aggregate runs of consecutively stored data segments reduces interactions with storage media that hosts the deduplicated data and speeds up retrieval. Parallel reads from multiple storage devices in multi-node replicated file systems also speed up retrieval. An illustrative enhanced media agent pre-fetches data (stored in deduplicated form) in anticipation of read requests that are expected in the restore operation.

Abstract: The present inventors devised techniques and technologies that streamline the protection of data stored on shared file storage e.g., on file servers, network-attached storage (NAS), share drives, network shares, etc., in a data storage management system. Some embodiments feature a proxy-less backup configuration in which backup services hosts are assigned to perform a secondary copy job, at job runtime, based on host attributes relative to the data source. Some embodiments are configured to, at job runtime, automatically resolve data paths to the source data, and automatically mount the data paths, rather than requiring data paths to be expressly pre-administered on all possible candidate backup machines. Some embodiments automatically protect data from shared file storage that was not expressly given a backup plan (“undefined” data) by applying default preferences for protecting undefined data.

Abstract: Techniques are disclosed relating to providing fast access to restored data in a cloud storage environment. A computing system stores multiple sets of incremental backup data that reflect changes in data being backed up between backup intervals and metadata that indicates which set of incremental backup data stores a given object. The computing system generates an endpoint for a requesting computing system, where the endpoint supports requests for restored backup data and data responses. In response to a request from the requesting computing system via the endpoint, the computing system queries the metadata based on the request and stores metadata retrieved from the query in a key/value store using object identification information as key data and the retrieved metadata as value data. The computing system provides requested data according to a lazy loading technique, including providing requested data via the endpoint based on the metadata in the key/value store.

Abstract: A data storage environment can include one or more virtual machines instantiated on a host computing device. Based on physical location data of the one or more virtual machines received from the host computing device, a storage manager can control the performance of a secondary copy operation on one or more storage units that store virtual machine data associated with the one or more virtual machines and/or the performance of a secondary copy operation on the one or more virtual machines.

Abstract: The disclosed approach works without the individualized credentials of failed machines when setting up recovery VMs in a cloud computing environment. Each recovery VMs is customized to properly correspond to the system state of its failed counterpart. An illustrative data storage management system recovers backup data and system states collected from the counterpart computing devices, custom-configures recovery VMs in the cloud computing environment, and injects the desired drivers into each recovery VM during an enhanced bare-metal restore process. The enhanced bare-metal restore process works without the failed computer's credentials. The system also restores the backed up data to recovery volumes attached to the recovery VMs. The present approach is both scalable and secure.

Abstract: Backup data is leveraged to determine whether primary data has been encrypted by malware. The disclosed approach does not rely on recognizing particular malware instances or malware provenance, and thus can be applied to any body of data. Even a novel and previously unknown malware attack can be detected in this way. An illustrative data storage management system analyzes secondary copies it created over time, applies a multi-factor analysis to data recovered from the secondary copies and, based on the analysis, infers whether the primary data from which the secondary copies were created may be encrypted. The present approach uses successive versions of backup copies to find indicia of malware encryption, rather than trying to trace or identify the malware itself. Indicia of entropy correlate highly with encryption, such as encryption performed by malware attacks. Conversely, indicia of similarity correlate highly with lack of encryption of successive versions of documents.