Abstract: An illustrative “Live Synchronization” feature in a data storage management system can reduce the downtime that arises in failover situations. The illustrative Live Sync embodiment uses backup data to create and maintain a ready (or “warm”) virtualized computing platform comprising one or more virtual machines (“VMs”) that are configured and ready to be activated and take over data processing from another data processing platform operating in the production environment. The “warm” computing platform awaits activation as a failover solution for the production system(s) and can be co-located at the production data center, or configured at a remote or disaster recovery site, which in some embodiments is configured “in the cloud.” Both local and remote illustrative embodiments are discussed herein. An “incremental forever” approach can be combined with deduplication and synthetic full backups to speed up data transfer and update the disaster recovery sites.

Abstract: An illustrative live synchronization feature uses file system block-level backup copies, snapshot techniques, change tracking, and volume-level granularity to ensure the integrity of destination volumes. Two protection mechanisms guard the destination data and ensure consistency from one live sync restore to the next. First, an inter-job software snapshot captures the destination volume image after each restore. The snapshot is created at the very end of each live sync restore and is reverted at the beginning of the next live sync restore. A second and more granular protection mechanism uses intra-job block monitoring to detect, and later to reverse, changes that the snapshots cannot capture. This second mechanism acts as a mini-block-level restore nested inside another block-level restore. This dual approach ensures that each incremental live sync restore finds the destination volume with a guaranteed pristine image identical to where the preceding live sync restore left it.

Abstract: Certain embodiments disclosed herein reduce or eliminate a communication bottleneck at the storage manager by reducing communication with the storage manager while maintaining functionality of an information management system. In some implementations, a client obtains information for enabling a secondary storage job (e.g., a backup or restore) from a storage manager and stores the information (which may be referred to as job metadata) in a local cache. The client may then reuse the job metadata for multiple storage jobs reducing the frequency of communication with the storage manager. When a configuration of the information management system changes, or the availability of resources changes, the storage manager can push updates to the job metadata to the clients. Further, a client can periodically request updated job metadata from the storage manager ensuring that the client does not rely on out-of-date job metadata.

Abstract: A strongly consistent distributed data storage system comprises an enhanced metadata service that is capable of fully recovering all metadata that goes missing when a metadata-carrying disk, disks, and/or partition fail. An illustrative recovery service runs automatically or on demand to bring the metadata node back into full service. Advantages of the recovery service include guaranteed full recovery of all missing metadata, including metadata still residing in commit logs, without impacting strong consistency guarantees of the metadata. The recovery service is network-traffic efficient. In preferred embodiments, the recovery service avoids metadata service downtime at the metadata node, thereby reducing the impact of metadata disk failure on the availability of the system. The disclosed metadata recovery techniques are said to be “self-healing” as they do not need manual intervention and instead automatically detect failures and automatically recover from the failures in a non-disruptive manner.

Abstract: Systems described herein may dynamically add one or more proxy data protection agents to a cloud data storage system to process a data protection job. Upon completion of the job or at some other appropriate interval, the system can power down and decommission the proxy data protection agents and/or the virtual machines on which the data protection proxies reside according to a cleanup schedule (e.g., at hourly or minute intervals). In order to improve the allocation of computing resources, the system takes into account currently existing proxies or virtual machines when processing a backup request to determine the need for new proxies to service the backup request. In this manner the system can save costs and computing resources through efficient virtual machine deployment and retirement.

Abstract: Systems and processes disclosed herein perform an automatic remediation process. The automatic remediation process may be a log-based remediation process. Systems disclosed herein may obtain log files from an information management system and determine the occurrence of errors at the information management system based on error codes included in the logs. Further, the systems may access a knowledgebase to determine whether solutions for the errors have been previously generated. The solutions may include patches or hotfixes that can be applied to the information management system without removing user-access or stopping execution of the information management system. The systems may automatically update the information management system to address the errors. Alternatively, or in addition, the systems may alert a user, such as an administrator, of the existence of a solution to the error, and whether the solution may be applied without interrupting service or access to the information management system.

Abstract: Certain embodiments described herein relate to an improved virtual machine restoration system. In one embodiment, an information management system receives a request to perform a restore of a virtual machine using virtual machine data stored on a secondary storage device. In response, the information management system boots up the virtual machine after restoring only a portion of the virtual machine data that is needed to boot up the virtual machine, thereby reducing latencies associated with virtual machine boot-up. The information management system continues to retrieve additional portions of the virtual machine data from the secondary storage device as such portions are requested by the virtual machine, thereby reducing or minimizing unnecessary data transfer from the secondary storage device.

Abstract: Exemplary systems and methods for archiving data objects using secondary copies are disclosed. The system creates one or more secondary copies of primary data that contains multiple data objects. The system may maintain a first data structure that tracks the data objects for which the system has created secondary copies and the locations of the secondary copies. To archive data objects in the primary data, the system identifies data objects to be archived, verifies that previously-created secondary copies of the identified data objects exist, and replaces the identified data objects with stubs. The system may maintain a second data structure that both tracks the stubs and refers to the first data structure, thereby creating an association between the stubs and the locations of the secondary copies.

Abstract: According to certain aspects, a system may include a data agent configured to: process a database file residing on a primary storage device(s) to identify a subset of data in the database file for archiving, the database file generated by a database application; and extract the subset of the data from the database file and store the subset of the data in an archive file on the primary storage device(s) as a plurality of blocks having a common size; and at least one secondary storage controller computer configured to, as part of a secondary copy operation in which the archive file is copied to a secondary storage device(s): copy the plurality of blocks to the secondary storage devices to create a secondary copy of the archive file; and create a table that provides a mapping between the copied plurality of blocks and corresponding locations in the secondary storage device(s).

Abstract: Live mounting a virtual machine (VM) causes the VM to run off a backup copy or snapshot previously taken of a “live” production VM. The live-mounted VM is generally intended for temporary use such as to validate the integrity and contents of the backup copy for disaster recovery validation, or to access some contents of the backup copy from the live-mounted VM without restoring all backed up files. These uses contemplate that changes occurring during live mount are not preserved after the live-mounted VM expires or is taken down. Thus, live mounting a VM is not a restore operation and usually does not involve access to every block of data in the backup copy. However, live mounting provides live VM service in the cloud sooner than waiting for all of the backup copy/snapshot to be restored.

Abstract: Storage of data that no longer has a useful life is established by the data objects' user(s) and overcomes predetermined rules required by information management policies. An illustrative data agent may receive instructions from an illustrative storage manager to perform secondary copy operations for certain data objects. The instructions may instruct that expiry-marked data objects are to be treated based on parsing each data object's user-supplied expiry timeframe. The illustrative system (i) will delete expired data objects including expired secondary copies; (ii) will refrain from making secondary copies of expired data objects; and (iii) if an expiry-marked data object is not expired, the illustrative system will make secondary copies according to the instructions, but will track those expiry-marked secondary copies for future deletion after they expire, (iv) and will delete expired secondary copies even if the original expiry-marked data object has been deleted from the system prior to its expiration.

Abstract: An information management cell health monitoring system is provided herein that can monitor one or more information management systems, identify any performance issues that are occurring within an information management system, and automatically, or in response to a user input, transmit an instruction to the information management system to execute a workflow to resolve the performance issue(s). For example, the information management cell health monitoring system receives operational data, secondary copy policies, and/or similar data from an information management cells via a network. The information management cell health monitoring system analyzes the received information to identify any issues. If an issue is detected, the information management cell health monitoring system retrieves workflows and determines whether any of the workflows can be used to resolve the detected issue.

Abstract: Virtual machine (VM) proliferation may be reduced by determining the availability of existing VMs to perform a task. Tasks may be assigned to existing VMs instead of creating a new VM to perform the task. Furthermore, a coordinator may determine a grouping of VMs or VM hosts based on one or more factors associated with the VMs or the VM hosts, such as VM type or geographical location of the VM hosts. The coordinator may also assign one or more Virtual Server Agents (VSAs) to facilitate managing the group of VM hosts. In some embodiments, the coordinators may facilitate load balancing of VSAs during operation, such as during a backup operation, a restore operation, or any other operation between a primary storage system and a secondary storage system.

Abstract: Systems and methods for performing file-level restore operations for block-level data volumes are described. In some embodiments, the systems and methods restore data from a block-level data volume contained in secondary storage by receiving a request to restore one or more files from the block-level data volume, mounting a virtual disk to the block-level data volume, accessing one or more mount paths established by the virtual disk between the data agent and the block-level data volume, and browsing data from one or more files within the block-level data volume via the established one or more mount paths provided by the virtual disk.

Abstract: In order to avoid writing duplicates of blocks of data into a storage platform, any virtual disk within the storage platform may have a de-duplication feature enabled. Or, all virtual disks have this feature enabled. For virtual disks with de-duplication enabled, a unique message digest is calculated for every block of data written to that virtual disk. Upon a write, these message digests are consulted in order to determine if a particular block of data has already been written, if so, it is not written again, and if not, it is written. All de-duplication virtual disks are written to a single system virtual disk within the storage platform. De-duplication occurs over the entire storage platform and over all its virtual disks because all message digests are consulted before a write is performed for any virtual disk. A read for a de-duplication virtual desk reads from the system virtual disk.

Abstract: The data storage system according to certain aspects can implement table level database restore. Table level database restore may refer to restoring a database table and its related data without restoring the entire database. The data storage system may use table metadata index to implement table level restore. A table metadata index may be created for each table, e.g., during a backup of the database. The table metadata index for a table can include any type of information for restoring the table and its related data. Some examples of the type of information included in the table metadata index include the following: container for the table, table backup location, system data, table index, table relationships, etc. Table metadata index can make the restoring of tables fast and efficient by packaging information that can be used to restore a table and its related data in an easily accessible manner.

Abstract: An improved information management system is described herein that provides on-demand or live mount access to virtual machine data in a secondary copy format. For example, instead of restoring all of the virtual machine data in the secondary copy format to a virtual disk that is then mounted to a virtual machine, the improved information management system can, in response to request to access virtual machine data in a secondary copy format, create a virtual disk having a universal network component (UNC) path and create a virtual machine configured to access data via the UNC path. Once created and booted, the administrator or user can attempt to access the desired virtual machine data via the virtual machine.

Abstract: Local management of data stream throttling in data movement operations, such as secondary-copy operations in a storage management system, is disclosed. A local throttling manager may interoperate with co-resident data agents and/or a media agent executing on any given local computing device, whether a client computing device or a secondary storage computing device. The local throttling manager may allocate and manage the available bandwidth for various jobs and their constituent data streams—across the data agents and/or media agent. Bandwidth is allocated and re-allocated to data streams used by ongoing jobs, in response to new jobs starting and old jobs completing, without having to pause and restart ongoing jobs to accommodate bandwidth adjustments. The illustrative embodiment also provides local users with a measure of control over data streams—to suspend, pause, and/or resume them—independently from the centralized storage manager that manages the overall storage system.

Abstract: Described in detail herein are systems and methods for single instancing blocks of data in a data storage system. For example, the data storage system may include multiple computing devices (e.g., client computing devices) that store primary data. The data storage system may also include a secondary storage computing device, a single instance database, and one or more storage devices that store copies of the primary data (e.g., secondary copies, tertiary copies, etc.). The secondary storage computing device receives blocks of data from the computing devices and accesses the single instance database to determine whether the blocks of data are unique (meaning that no instances of the blocks of data are stored on the storage devices). If a block of data is unique, the single instance database stores it on a storage device. If not, the secondary storage computing device can avoid storing the block of data on the storage devices.

Abstract: Certain embodiments described herein relate to an improved block-level replication system. One or more components in an information management system may receive a request to perform a block-level replication between a source storage device and a destination storage device, and depending on the specific replication mode requested, (i) store block-level changes directly to the destination storage device or (ii) first to a recovery point store and then later to the destination storage device.