Abstract: Systems and methods are provided herein for automatically configuring newly installed secondary storage computing devices and managing secondary storage computing devices when one or more become unavailable. For example, a storage manager can then detect the computing resources available to the newly installed secondary storage computing device, assign a role to the newly installed secondary storage computing device based on the detected computing resources, configure the newly installed secondary storage computing device with deduplication and storage policies used by the other secondary storage computing devices, re-partition secondary storage devices to allocate memory for the newly installed secondary storage computing device, and instruct other secondary storage computing devices to replicate their managed data such that the newly installed secondary storage computing device has access to the replicated data.

Abstract: An information management system according to certain aspects may determine whether storage operations will work prior to executing them. The system may check various factors or parameters relating to a storage policy to verify whether the storage policy will work at runtime without actually executing the policy. Some examples of factors can include: availability of primary storage devices, availability of secondary storage devices, license availability for performing that operation, user credentials for connecting to primary and/or second storage devices, available storage capacity, connectivity to storage devices, etc. The system may also check whether a particular system configuration is supported in connection with storage operations. The result of the determination can be provided in the form of a report summarizing any problems found with the storage policy. The report can include recommended courses of action or solutions for resolving any identified issues.

Abstract: A cache storage area stores key data blocks, usually metadata, used in live browse, file indexing, and/or file restore operations, which can avoid retrieving data on demand from backup copies. An illustrative shared data storage resource hosts the cache storage area, which is subdivided into virtual disk-specific caches, each one accompanied by a respective cache-specific index. A media agent acting as cache arbitrator manages access to the various caches. The cache arbitrator grants to other media agents in the system temporary ownership of particular caches, thereby facilitating parallel usage among the caches by different media agents. By enabling parallelism and a shared cache storage environment, these embodiments improve overall operational performance and add resiliency to the system as a whole. The cache arbitrator media agent is specially appointed to the role by a storage manager that generally manages storage operations throughout the illustrative data storage management system.

Abstract: A method and system for reducing storage requirements and speeding up storage operations by reducing the storage of redundant data includes receiving a request that identifies one or more data objects to which to apply a storage operation. For each data object, the storage system determines if the data object contains data that matches another data object to which the storage operation was previously applied. If the data objects do not match, then the storage system performs the storage operation in a usual manner. However, if the data objects do match, then the storage system may avoid performing the storage operation.

Abstract: A system according to certain aspects may include a secondary storage controller computer configured to: in response to a first instruction to obtain a first secondary copy of a first data set from a secondary storage device(s), the first instruction associated with a first restore operation: instantiate a first restore thread on a processor of the secondary storage controller computer; using the first restore thread, retrieve the first secondary copy from the secondary storage device(s); and forward the retrieved first secondary copy to a primary storage subsystem for storage; and in response to a second instruction to obtain a second secondary copy of a second data set from the secondary storage device(s), the second instruction associated with a second restore operation: using the first restore thread, retrieve the second secondary copy from the secondary storage device(s); and forward the retrieved second secondary copy to the primary storage subsystem for storage.

Abstract: An improved information management system that implements a staging area or cache to temporarily store primary data in a native format before the primary data is converted into secondary copies in a secondary format is described herein. For example, the improved information management system can include various media agents that each include one or more high speed drives. When a client computing device provides primary data for conversion into secondary copies, the primary data can initially be stored in the native format in the high speed drive(s). If the client computing device then submits a request for the primary data, the media agent can simply retrieve the primary data from the high speed drive(s) and transmit the primary data to the client computing device. Because the primary data is already in the native format, no conversion operations are performed by the media agent, thereby reducing the restore delay.

Abstract: If a deduplication database becomes corrupted or is lost, the deduplication database may be reconstructed by restoring an earlier backup copy of the deduplication database. The backup copy of the deduplication database, however, may not be synced with the deduplicated data blocks that are presently stored in the secondary storage subsystem. To address the possibility of data loss and improve reconstruction timing, a deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to restore the deduplication database and resync with the secondary storage by using one or more journal files that track the data blocks that have been or may have been deleted from the secondary storage since the last deduplication database backup.

Abstract: A deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to update the deduplication database and remove records corresponding to data blocks that have been or will be erased from the secondary copies, without using or tracking reference counting values. Some embodiments described herein use a secondary table (for tracking archive file contents) and a bitmap to mark which primary records are present in the secondary table. In another embodiment, once the marking phase is completed, the deduplication system uses the marked-up bitmap to identify the corresponding records from the primary table that can be moved to another table for storing “zero-reference” data blocks. In other embodiments, the system will then traverse the “zero-reference” table and remove those primary data blocks from secondary storage devices.

Abstract: A deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to update the deduplication database and remove records corresponding to data blocks that have been or will be erased from the secondary copies, without using or tracking reference counting values. Some embodiments described herein use a secondary table (for tracking archive file contents) and a bitmap to mark which primary records are present in the secondary table. In another embodiment, once the marking phase is completed, the deduplication system uses the marked-up bitmap to identify the corresponding records from the primary table that can be moved to another table for storing “zero-reference” data blocks. In other embodiments, the system will then traverse the “zero-reference” table and remove those primary data blocks from secondary storage devices.

Abstract: If a deduplication database becomes corrupted or is lost, the deduplication database may be reconstructed by restoring an earlier backup copy of the deduplication database. The backup copy of the deduplication database, however, may not be synced with the deduplicated data blocks that are presently stored in the secondary storage subsystem. To address the possibility of data loss and improve reconstruction timing, a deduplicated storage system is provided according to certain embodiments that uses one or more mechanisms to restore the deduplication database and resync with the secondary storage by using one or more journal files that track the data blocks that have been or may have been deleted from the secondary storage since the last deduplication database backup.

Abstract: Techniques for providing shared access to, e.g., a small computer system interface (SCSI) storage device in a computer network include providing an operational mode on SCSI interfaces with a first media agent and a second media agent such that, in response to inquiry messages on the SCSI interfaces, the SCSI storage device appears as a SCSI target device to the first media agent and the second media agent and mapping data operations between the first media agent and the SCSI storage device and the second media agent and the SCSI storage device to logically unique channel numbers for the first media agent and the second media agent to perform data storage operations over their respective SCSI interfaces by concurrently sharing the SCSI storage device.

Abstract: According to certain aspects, an information management cell can include at least one secondary storage computing device configured to conduct primary data generated by at least one client computing device to a secondary storage device(s) as part of secondary copy operations, wherein the secondary storage computing device normally operates to conduct primary data to the secondary storage device(s) for storage as a secondary copy in a first secondary copy file format, at the direction of a main storage manager; and can include a failover storage manager configured to activate in response to loss of connectivity between the cell and the main storage manager, and instruct a secondary copy application to perform a secondary copy operation in which the primary data generated by the at least one client computing device is stored as a secondary copy in a second secondary copy file format different than the first secondary copy file format.

Abstract: Systems and methods are provided herein for automatically configuring newly installed secondary storage computing devices and managing secondary storage computing devices when one or more become unavailable. For example, a storage manager can then detect the computing resources available to the newly installed secondary storage computing device, assign a role to the newly installed secondary storage computing device based on the detected computing resources, configure the newly installed secondary storage computing device with deduplication and storage policies used by the other secondary storage computing devices, re-partition secondary storage devices to allocate memory for the newly installed secondary storage computing device, and instruct other secondary storage computing devices to replicate their managed data such that the newly installed secondary storage computing device has access to the replicated data.

Abstract: Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.

Abstract: A distributed, deduplicated storage system according to certain embodiments is arranged in a parallel configuration including multiple deduplication nodes. Deduplicated data is distributed across the deduplication nodes. The deduplication nodes can be networked together and communicate with one another according using a light-weight, customized communication scheme (e.g., a scheme based on FTP or HTTP). In some cases, deduplication management information including deduplication signatures and/or other metadata is stored separately from the deduplicated data in deduplication management nodes, improving performance and scalability.

Abstract: Disclosed methods and systems leverage resources in a storage management system to restore a selected backup to a production site. The backup is partitioned into blocks with associated signatures. The production site may have blocks that have not changed from when the backup occurred, so those blocks do not need to be restored. Block signatures from the production site are compared with block signatures from the incremental backup to identify blocks that need to be restored. Efficiency may be achieved by synchronizing the replacement blocks from more easily accessible location where available before synchronizing from less accessible locations. In some embodiments, a user may specify the location of the site with the replacement blocks.

Abstract: An information management system according to certain aspects allows users to share a portion of a file (e.g., a document) stored in secondary storage. The user may specify a portion of a secondary storage file to share and send a link to the portion to another user. The other user can access the shared portion from the link, and just the shared portion may be restored from secondary storage. The system according to certain aspects provides a native view of secondary storage data on a client computing device. The index data and/or metadata relating to secondary storage data may be stored in native application format for access via the native source application.

Abstract: The exemplary system and methods provide a solution for reclaiming the space occupied by invalid blocks of data stored on secondary storage devices. The space reclamation techniques may employ a primary table, a deduplication chunk table, or an index data structure located with the secondary copy of data on secondary storage devices. One exemplary method uses information from the deduplication database media agent and a deduplication chunk table. Another exemplary method uses an index (e.g., single instance file index) that is associated with and stored with each chunk of data. Based on the information provided by the deduplication chunk table and the index, exemplary system and methods identify blocks of invalid data and copy over only valid data blocks to a new container file.

Abstract: An information management system can dynamically adjust the order in which compression and deduplication operations are performed based on the type of file for which a secondary copy operation is requested. For example, the information management system can determine whether the file for which a secondary copy operation is requested is a first type of file that is text-based and/or that has data blocks that are highly compressible or a second type of file that may have data blocks that are not highly compressible. If the information management system determines that the file is of the second type, the information management system can perform a deduplication operation prior to performing a compression operation. Furthermore, the information management system can dynamically adjust the size of various deduplication blocks based on the type of file for which a secondary copy operation is requested.

Abstract: Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.