Abstract: The technique introduced here includes a system and method for preprocessing a data set to improve deduplication, and more specifically for reducing latency. The technique illustratively utilizes one or more preprocessing steps, including a “skipping” step and a “folding” step, which can be applied to a data set prior to deduplication to reduce the time consumed by deduplication. The folding step is applied to segments of the data set to reduce the length of the segments. The skipping step can be applied to the data set prior to the folding step to remove particular segments of the data set, to further improve deduplication performance in certain circumstances. The overall effect of the skipping and folding steps of this technique is to produce a data set of reduced total length for consideration in identifying duplicate data, which aids in reducing the time required for deduplication.

Abstract: A method and system for generating data fingerprints is used to de-duplicate a data set having a high level of redundancy. A fingerprint generator generates a data fingerprint based on a data window. Each byte of the data set is added to the fingerprint generator and used to detect an anchor within the received data. If no anchor is detected, the system continues receiving bytes until a predefined window size is reached. When the window size is reached, the system records a data fingerprint based on the data window and resets the window size. If an anchor is detected, the system extends the window size such that the window ends a specified length after the location of the anchor. If the extended window is greater than a maximum size, the system ignores the anchor. The generated fingerprints are compared to a fingerprint database. The data set is then de-duplicated by replacing matching data segments with references to corresponding stored data segments.

Abstract: A method and system for emulating tape library commands is disclosed. Tape library commands implemented in response to commands received from a data protection application are emulated in a disk based storage medium so that existing data protection applications may be used to copy data to and from the disk based storage medium.

Abstract: A system for synchronizing a secondary volume with a primary volume in a continuous data protection system includes scanning means for scanning a region of the primary volume; comparing means for comparing the scanned region with a corresponding region of the secondary volume; storing means for storing an identification of the scanned region in a compare delta map when the comparing means returns a discrepancy between the scanned region and the corresponding region; copying means for copying data from the primary volume to the secondary volume, using the compare delta map as a guide to locate the data to copy; and revising means for revising the compare delta map by removing any changes made to the primary volume during a scan interval, which is a period of time required to scan the primary volume.

Abstract: A method and apparatus for managing backup data is disclosed. A data backup system defines a plurality of time windows for creating and maintaining backup data in accordance with a data backup policy. Each of the time windows is assigned a predetermined amount of storage space. When the data backup system creates backup data, the system determines whether a storage space assigned to a time window is large enough to accommodate new backup data. If the storage space is large enough, the new backup data is stored. However, if the storage space is not large enough, the system deletes the oldest backup data until enough storage space is obtained.

Abstract: A system for the storage and retrieval of data files includes a data disk, a configuration disk and a controller. Each data file includes one or more data blocks, each data block comprising a data section. The data disk stores the data blocks and is divided into a plurality of block groups. The configuration disk stores a copy of at least a portion of the corresponding meta-data. The controller controls both the data disk and the configuration disk.

Abstract: A technique for determining an amount of storage space that would be recovered in a storage system by deleting a list of logical containers of data is provided. In one embodiment, data blocks referenced by the list of files are first identified. The data blocks (e.g., deduplicated data blocks) may be referenced by two or more files. A number of references made by the identified list of files to each data block is compared against a number of references made by all files stored in a dataset of the storage system. A total amount of storage space is then calculated based on a sum of storage spaces utilized by each data block for which the two reference numbers match. This total amount of storage space is the storage space that would be reclaimed through deletion of the identified list of files.

Abstract: A method for tracking changes on a primary volume in a continuous data protection system writes data to the primary volume. A copy of the data is written to a secondary volume, the secondary volume including all writes made to the primary volume. Each write made to the secondary volume is mapped in a data structure, such that the data structure tracks changes on the primary volume between two different points in time.

Abstract: A method and system for generating a snapshot in a consistent state is disclosed. The system comprises a host computer, a primary data storage, a data protection unit, and a secondary data storage. The data protection unit monitors a state of an email application which is running on the host computer. The data protection unit generates a snapshot of data stored in a primary data storage when the email application is in a consistent state, and stores the snapshot on a secondary storage. In the event of a system failure, the data is recovered using the last snapshot. A method and system for recovering email data from a snapshot in a consistent state using a client is also disclosed.

Abstract: A method for browsing objects on a protected volume in a continuous data protection system begins by displaying a list of top-level objects to a user who selects a top-level object to browse. A snapshot is selected from the selected top-level object and is decoded. A list of objects in the selected snapshot is displayed and the user selects an object to browse. The selected object is decoded and operations are performed on the selected object. The user can browse the protected volume by snapshots in the top-level object or by a range of time of coverage of the top-level object.

Abstract: A method for restoring a volume in a continuous data protection system begins by selecting a volume to restore. The selected volume is loaded into a workspace. The selected volume is copied from the workspace to a restore location. Operations are simultaneously performed on the selected volume while the selected volume is being copied. The selected volume is accessed on the restore location once the copying is complete.

Abstract: The invention is a method and system for copying backup data to a virtual tape library (VTL), where data written to the VTL can optionally be stored on removable disk media for offsite storage. The invention minimizes dependency on tape media without taxing the resources of any of the servers and without the need for the data protection application (DPA) to perform a second backup pass. Backup data written to the VTL can optionally be copied to the VTL's default storage area and/or to a predetermined set of disks that may be removed and stored offsite.

Abstract: A method and system for generating a snapshot in a consistent state is disclosed. The system comprises a host computer, a primary data storage, a data protection unit, and a secondary data storage. The data protection unit monitors a state of an application which is running on the host computer. The data protection unit generates a snapshot of data stored in a primary data storage when the application is in a consistent state, and stores the snapshot on a secondary storage. In the event of a system failure, the data is recovered using the last snapshot.

Abstract: A method of protecting a data storage system including the steps of: providing a disk system including a plurality of disk modules, each disk module containing a disk controller and a disk driver, the disk driver being in detachable electrical communication with the disk controller; and physically disconnecting the disk controller from the disk driver for any of the plurality of disk modules that are not required by the data storage system for the transfer of data.

Abstract: A method for synchronizing a secondary volume with a primary volume in a continuous data protection system begins by scanning a region of the primary volume, which is then compared with a corresponding region of the secondary volume. An identification of the scanned region is stored in a compare delta map when the comparison results in a discrepancy between the scanned region and the corresponding region. Data is copied from the primary volume to the secondary volume, using the compare delta map as a guide to locate the data to copy. If the data protection system has failed just prior to the restore process being initiated, it is termed a re-baseline, and the entire primary volume is scanned. If the data protection system is active prior to the restore process being initiated, it is termed a re-synchronization, and is optimized by scanning only select regions of the primary volume.

Abstract: A method and system where data is maintained in a continuous data protection system is disclosed. A primary volume may be protected according to an any-point-in-time (APIT) window wherein restores may be performed at any time as desired. The APIT window may be of any time duration as desired. Outside of the APIT window, a retention policy for phasing out data may be established as desired.

Abstract: A method for stream allocation in a data storage system begins by determining whether a stream needs storage space. Storage space for the stream is requested, and a device on which to allocate the storage space is selected. The storage space on the selected device is allocated to the stream.

Abstract: A system for synchronizing a secondary volume with a primary volume in a continuous data protection system includes scanning means for scanning a region of the primary volume; comparing means for comparing the scanned region with a corresponding region of the secondary volume; storing means for storing an identification of the scanned region in a compare delta map when the comparing means returns a discrepancy between the scanned region and the corresponding region; copying means for copying data from the primary volume to the secondary volume, using the compare delta map as a guide to locate the data to copy; and revising means for revising the compare delta map by removing any changes made to the primary volume during a scan interval, which is a period of time required to scan the primary volume.

Abstract: In a continuous data protection system having a primary volume and a secondary volume, a method for data recovery begins by selecting a snapshot of the primary volume to be recovered and a location on which the snapshot is to be loaded. A point in time (PIT) map is created for the selected snapshot, and the selected snapshot is loaded at the selected location. A data block from the PIT map is resolved to determine which block on the secondary volume is presented as being part of the snapshot. The snapshot is accessed via a host computer as if the snapshot was the primary volume at an earlier point in time, corresponding to the time of the selected snapshot.

Abstract: A method and system wherein data is structured so that writes may be continuously duplicated and a protected volume may be restored to a particular point in time as desired. Writes are continuously duplicated on a secondary volume and organized into delta maps. The delta maps indicate which memory blocks were written to during a particular time frame.