Abstract: A storage system according to certain embodiments includes a client-side repository (CSR). The CSR may communicate with a client at a higher data transfer rate than the rate used for communication between the client and secondary storage. During copy operations, for instance, some or all of the data being backed up or otherwise copied to secondary storage is stored in the CSR. During restore operations, copies of the data stored in the CSR is accessed from the CSR instead of from secondary storage, improving performance. Remaining data blocks not stored in the CSR can be restored from secondary storage.

Abstract: An improved information management system is provided herein for managing secondary storage computing devices when one or more become unavailable. For example, a first secondary storage computing device in a pool of secondary storage computing devices can run a storage manager virtual machine (VM) that manages secondary copy and restore operations. A second secondary storage computing device in the pool can run a virtualization engine that manages the storage manager VM. The virtualization engine can replicate the storage manager VM data to other secondary storage computing devices in the pool. If the first secondary storage computing devices becomes unavailable, the second secondary storage computing device can instruct a third secondary storage computing device in the pool to launch a replicated storage manager VM using the replicated storage manager VM data stored therein. Thus, the third secondary storage computing device can act as the now unavailable first secondary storage computing device.

Abstract: A high availability distributed, deduplicated storage system according to certain embodiments is arranged to include multiple deduplication database media agents. The deduplication database media agents store signatures of data blocks stored in secondary storage. In addition, the deduplication database media agents are configured as failover deduplication database media agents in the event that one of the deduplication database media agents becomes unavailable.

Abstract: The data storage system according to certain aspects can implement partial file restore, where only a portion of the secondary copy of a file is restored. Such portion may be designated by one or more application offsets for the file. The system may provide an in-chunk index that includes mapping information between the application offsets and the secondary copy offsets. Chunks may refer to logical data units in which secondary copies are stored, and the in-chunk index for a chunk may be stored in secondary storage with the chunk. Because the mapping information may not be provided at a fixed interval, the system can search through application offsets in the in-chunk index to locate the secondary copy offset corresponding to the portion application offset(s). In this manner, the system may restore the designated portion of the secondary copy in a fast and efficient manner by using the in-chunk index.

Abstract: The disclosed techniques enable browsing of data items that are stored in a backup storage system in a backup format. A search query from a user is received where the search query is for a data item stored in a data storage system, such as a tape drive, a disk array, or a cloud-based storage system. The system may use an index to look up a location of the data item and data format of the data item. To present the data to the user, the system may perform a restoration operation to convert the data format to a format that can be presented to the user.

Abstract: Systems and methods are provided herein for automatically configuring newly installed secondary storage computing devices and managing secondary storage computing devices when one or more become unavailable. For example, a storage manager can then detect the computing resources available to the newly installed secondary storage computing device, assign a role to the newly installed secondary storage computing device based on the detected computing resources, configure the newly installed secondary storage computing device with deduplication and storage policies used by the other secondary storage computing devices, re-partition secondary storage devices to allocate memory for the newly installed secondary storage computing device, and instruct other secondary storage computing devices to replicate their managed data such that the newly installed secondary storage computing device has access to the replicated data.

Abstract: A method and system for providing information management of data from hosted services receives information management policies for a hosted account of a hosted service, requests data associated with the hosted account from the hosted service, receives data associated with the hosted account from the hosted service, and provides a preview version of the received data to a computing device. In some examples, the system indexes the received data to associate the received data with a user of an information management system, and/or provides index information related to the received data to the computing device.

Abstract: An information management system can manage the removal of data block entries in a deduplicated data store using working copies of the data block entries residing in a local data store of a secondary storage computing device. The system can use the working copies to identify data blocks for removal. Once the deduplication database is updated with the changes to the working copies (e.g., using a transaction based update scheme), the system can query the deduplication database for the database entries identified for removal. Once identified, the system can remove the database entries identified for pruning and/or the corresponding deduplication data blocks from secondary storage.

Abstract: Multiple data paths may be available to a data management system for transferring data between a primary storage device and a secondary storage device. The data management system may be able to gain operational advantages by performing load balancing across the multiple data paths. The system may use application layer characteristics of the data for transferring from a primary storage to a backup storage during data backup operation, and correspondingly from a secondary or backup storage system to a primary storage system during restoration.

Abstract: Disclosed methods and systems leverage resources in a storage management system to restore a selected backup to a production site. The backup is partitioned into blocks with associated signatures. The production site may have blocks that have not changed from when the backup occurred, so those blocks do not need to be restored. Block signatures from the production site are compared with block signatures from the incremental backup to identify blocks that need to be restored. Efficiency may be achieved by synchronizing the replacement blocks from more easily accessible location where available before synchronizing from less accessible locations. In some embodiments, a user may specify the location of the site with the replacement blocks.

Abstract: Described in detail herein are systems and methods for single instancing blocks of data in a data storage system. For example, the data storage system may include multiple computing devices (e.g., client computing devices) that store primary data. The data storage system may also include a secondary storage computing device, a single instance database, and one or more storage devices that store copies of the primary data (e.g., secondary copies, tertiary copies, etc.). The secondary storage computing device receives blocks of data from the computing devices and accesses the single instance database to determine whether the blocks of data are unique (meaning that no instances of the blocks of data are stored on the storage devices). If a block of data is unique, the single instance database stores it on a storage device. If not, the secondary storage computing device can avoid storing the block of data on the storage devices.

Abstract: The disclosed techniques enable browsing of data items that are stored in a backup storage system in a backup format. A search query from a user is received where the search query is for a data item stored in a data storage system, such as a tape drive, a disk array, or a cloud-based storage system. The system may use an index to look up a data format of the data item. To present the data to the user, the system may perform a restoration operation using two or more staging operations to convert the data format to a format that can be presented to the user.

Abstract: The disclosed techniques that can use deduplication information on a source computer platform to improve the process of performing data backups or restoration from/to the computer platform. In one example aspect, a data backup operation can re-use some of the work already done by a source computer's deduplication system. For example, a storage operation could read a deduplication database on the source computer platform to determine the duplicativeness of a given data chunk being transferred to a backup storage system, without having to perform computations such as data chunk hashing and comparison with previously generated hashes. The technique may additionally or alternatively reuse hashes generated by the source computer during deduplication of the data file on the source computer's file system during deduplication at the external backup storage system.

Abstract: Described in detail herein are systems and methods for managing single instancing data. Using a single instance database and other constructs (e.g. sparse files), data density on archival media (e.g. magnetic tape) is improved, and the number of files per storage operation is reduced. According to one aspect of a method for managing single instancing data, for each storage operation, a chunk folder is created on a storage device that stores single instancing data. The chunk folder contains three files: 1) a file that contains data objects that have been single instanced; 2) a file that contains data objects that have not been eligible for single instancing; and 3) a metadata file used to track the location of data objects within the other files. A second storage operation subsequent to a first storage operation contains references to data objects in the chunk folder created by the first storage operation instead of the data objects themselves.

Abstract: Illustrative storage manager and media agent are enhanced to interoperate with deduplication appliances. Advantages are realized when making secondary and tertiary copies and also when restoring from a deduplication appliance. Tiered indexing minimizes how much data is retained and stored at media agents. Tiered indexing enables media agents to efficiently extract needed information from deduplication appliances to make tertiary copies and to restore backed up copies. Interoperability techniques include media agents generating separate data streams to the deduplication appliance. Each data stream carries a different kind of data, e.g., payload data, metadata content, or high-level index information. On initial backup, the media agent instructs the deduplication appliance to deduplicate the payload data stream but not the other data streams, thus intelligently applying resources to data most likely to benefit from deduplication.

Abstract: An information management system according to certain aspects may determine whether storage operations will execute properly prior to executing them in a cloud services environment. The system may check various factors or parameters relating to a storage policy to verify whether the storage policy will work at runtime without actually executing the policy. Some examples of factors can include: availability of primary storage devices, availability of secondary storage devices, license availability for performing that operation, user credentials for connecting to primary and/or second storage devices, available storage capacity, connectivity to storage devices, etc. The system may also check whether a particular system configuration is supported in connection with storage operations. The result of the determination can be provided in the form of a report summarizing any problems found with the storage policy. The report can include recommended courses of action or solutions for resolving any identified issues.

Abstract: Described are systems and methods for storing a variable number of instances of data objects (e.g., 1, 2, 3, or up to N?1 instances, where N is the number of instances of the data object included in primary data) in secondary storage across a data storage network. In some examples, a system for storing a variable number of instances of data objects includes, one or more computing devices storing a set of data objects and multiple storage devices distinct from the one or more computing devices. Each of the multiple storage devices is configured to store at least a single instance of a data object. The system also includes a database configured to store information associated with the data objects. This information includes substantially unique identifiers for the data objects and, for each of the data objects, a number of instances of the data object stored on the multiple storage devices.

Abstract: A scalable approach is disclosed for processing auxiliary-copy jobs in a storage management system by using distributed media agent resources instead of a centralized storage manager. Enhanced media agents coordinate and control auxiliary-copy jobs and tap the storage manager to reserve data streams and provide job-specific metadata on demand. An enhanced storage manager may initially select a media agent as “coordinator” to coordinate auxiliary-copy jobs with any number of other media agents, which act as “controllers.” A coordinator media agent is generally responsible for obtaining data stream reservation information from the storage manager and assigning auxiliary-copy jobs to respective controller media agents, based on the components involved in the respective reserved data streams.

Abstract: A method and system for reducing storage requirements and speeding up storage operations by reducing the storage of redundant data includes receiving a request that identifies one or more data objects to which to apply a storage operation. For each data object, the storage system determines if the data object contains data that matches another data object to which the storage operation was previously applied. If the data objects do not match, then the storage system performs the storage operation in a usual manner. However, if the data objects do match, then the storage system may avoid performing the storage operation.

Abstract: Techniques disclosed herein are well suited to restoring deduplicated backup data from cloud-based storage and from multi-node replicated files systems, and they also improve performance in more traditional data storage technologies. Pre-restore steps include analysis of deduplication indexes to identify data segments that are stored consecutively on storage media. Reading data in aggregate runs of consecutively stored data segments reduces interactions with storage media that hosts the deduplicated data and speeds up retrieval. Parallel reads from multiple storage devices in multi-node replicated file systems also speed up retrieval. An illustrative enhanced media agent pre-fetches data (stored in deduplicated form) in anticipation of read requests that are expected in the restore operation.