Abstract: A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.

Abstract: A method and apparatus for backing up a computer system. A backup is provided of a data stored on a first computer by the first computer generating a dump stream. The dump stream is transmitted to a second computer, the second computer uses the dump stream to maintain a file system copy of the data. One or more snapshots of the file system of the second computer are stored on a backup computer, the snapshots providing a backup of the data stored on the first computer.

Abstract: A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.

Abstract: A method for sharing data blocks in a hierarchical file system in a storage server includes allocating a plurality of data blocks in the file system, and sharing data blocks in the file system, without using a persistent point-in-time image, to avoid duplication of data blocks. A method for identifying data blocks that can be shared includes computing a fingerprint for each of multiple data blocks to be written to a storage facility and storing the fingerprint with information identifying the data block in an entry in a set of metadata. The set of metadata is used to identify data blocks which are duplicates.

Abstract: A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI's image of the state of the file system to a deduplicated state.

Abstract: In a system and method for updating a remote replicated destination file system snapshot with changes in a source file system snapshot, users and processes are redirected to a local exported snapshot of the replicated snapshot on the active file system on the destination before beginning the next update of the active file system's replicated snapshot. In this manner, an unstable replicated snapshot is not accessed. Indirection is introduced into inode lookup at the destination as the destination's active file system is being updated. The indirection can be based upon a snapshot ID that conforms to a latest exported snapshot ID.

Abstract: In a system and method for updating a remote replicated destination file system snapshot with changes in a source file system snapshot, users and processes are redirected to a local exported snapshot of the replicated snapshot on the active file system on the destination before beginning the next update of the active file system's replicated snapshot. In this manner, an unstable replicated snapshot is not accessed. Indirection is introduced into inode lookup at the destination as the destination's active file system is being updated. The indirection can be based upon a snapshot ID that conforms to a latest exported snapshot ID.

Abstract: The present invention provides a system and method for comparing data sets, to ensure that they are accurate reflections of each other, without the need for performing O(N2) operations, in which N is the size of each data set. A hash table is generated for the first data set. For each of the second data set entries, should the entry not exist in the hash table, the entry is second data set unique. Otherwise, the entry is removed from the hash table. At the end of the pass through the second data set entries, only those entries that exist in the hash table are first data set unique. Alternately, two processes operate in parallel so that each selects entries from one of the data sets and determines if the entry exists in the hash table. If the entry does exist, it is removed. Otherwise, the entry is added to the hash table.

Abstract: A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI's image of the state of the file system to a deduplicated state.

Abstract: A system and method for managing data deduplication of a storage system utilizing persistent consistency point images (PCPIs). Once a target PCPI of a data transfer is generated, a backup management module of the storage system alerts a data deduplication module to begin deduplication of the data contained within the target PCPI. Once the deduplication procedure has been completed, the active file system of the storage system has been deduplicated, however, the target PCPI remains un-deduplicated. In response, the backup management module generates and exports a revised target PCPI. The previous target PCPI may then be deleted, thereby transitioning the exported PCPI's image of the state of the file system to a deduplicated state.

Abstract: A technique is disclosed for restoring data of sparse volumes, where one or more block pointers within the file system structure are marked as ABSENT, and fetching the appropriate data from an alternate location on demand. Client data access requests to the local storage system initiate a restoration of the data from a backing store as required. A demand generator can also be used to restore the data as a background process by walking through the sparse volume and restoring the data of absent blocks. A pump module is also disclosed to regulate the access of the demand generator. Once all the data has been restored, the volume contains all data locally, and is no longer a sparse volume.

Abstract: A method for backing up a computer file system is disclosed. A first snapshot of the file system is generated at a first time, the first snapshot having first pointers to data blocks of the file system at the first time. A network adapter receives storage operations to write new data to the file system. A second snapshot of the file system is generated at a second time, the second snapshot having second pointers to data blocks of the file system at the second time, the data blocks at the second time including the new data. The first pointers to the second pointers are compared. In response to the second pointers differing from the first pointers, data of the file system which have been changed between the first snapshot and the second snapshot are copied to a backup copy of the file system.

Abstract: A technique is disclosed for restoring data of sparse volumes, where one or more block pointers within the file system structure are marked as ABSENT, and fetching the appropriate data from an alternate location on demand. Client data access requests to the local storage system initiate a restoration of the data from a backing store as required. A demand generator can also be used to restore the data as a background process by walking through the sparse volume and restoring the data of absent blocks. A pump module is also disclosed to regulate the access of the demand generator. Once all the data has been restored, the volume contains all data locally, and is no longer a sparse volume.

Abstract: A method for transferring data from a source computer to a destination computer, and restarting the source computer transmission after a halt of the transmission, has the following steps. The source computer inserts a checkpoint number into a data stream, the data flowing in the data stream. The checkpoint number is stored at the source computer. The data stream is halted in response to an error in a path of the data stream. The destination computer locates the last checkpoint number successfully received before halting the data stream. The last checkpoint number successfully received before halting the data stream is transferred to the source computer; and the source computer compares it with checkpoint numbers stored in the source computer to determine where in the data stream to resume transfer of the data. Transfer of the data is resumed in response to the last checkpoint number.

Abstract: A system and method provides continuous data protection using checkpoints in a write anywhere file system. During a consistency point of a write anywhere file system, freed blocks are identified and are appended to a delete log for retention. A consistency point log is updated with a new entry associated with the consistency point. If the file system needs to retrieve its state at a particular point in time, the stored blocks of the delete log may be recovered.

Abstract: A system and method for conforming a metadata file associated with qtree of a storage system to a version used by a file system kernel of the storage system, where the version indicates a format of the metadata file. The metadata file stores state information with respect to each qtree, and may be located outside of the qtree structure. Major and minor versions are compared with latest known major and minor versions associated with the file system kernel. In response to such comparisons, the metadata file may be updated to a newer version, rolled back to an older version, and/or the system may enter a corrective state where further operations are performed.

Abstract: A method and apparatus to generate information representing differences between two data sets are described. Information representing differences between a first data set and a second data set is acquired, where the first and second data sets each include multiple nodes. A traversal map that identifies nodes for which a difference has been detected between the first and second data sets is generated, and then during an output phase, a hierarchy of nodes is traversed to generate output indicating user-level differences between the first and second data sets. The traversal map is used to avoid traversing at least some of the nodes in the hierarchy during the output phase. A child map may be generated to represent parent-child relationships between the nodes and used during the output phase to prefetch certain nodes in the hierarchy.

Abstract: In a system and method for updating a remote replicated destination file system snapshot with changes in a source file system snapshot, users and processes are redirected to a local exported snapshot of the replicated snapshot on the active file system on the destination before beginning the next update of the active file system's replicated snapshot. In this manner, an unstable replicated snapshot is not accessed. Indirection is introduced into inode lookup at the destination as the destination's active file system is being updated. The indirection can be based upon a snapshot ID that conforms to a latest exported snapshot ID.

Abstract: A system and method for remote asynchronous replication or mirroring of changes in a source file system snapshot in a destination replica file system using a scan (via a scanner) of the blocks that make up two versions of a snapshot of the source file system, which identifies changed blocks in the respective snapshot files based upon differences in volume block numbers identified in a scan of the logical file block index of each snapshot. Trees of blocks associated with the files are traversed, bypassing unchanged pointers between versions and walking down to identify the changes in the hierarchy of the tree. These changes are transmitted to the destination mirror or replicated snapshot. This technique allows regular files, directories, inodes and any other hierarchical structure to be efficiently scanned to determine differences between versions thereof.

Abstract: A system and method for updating a replicated destination file system snapshot with changes in a source file system snapshot, provides an extensible, file system-independent format to transmit a data stream of change data over the network. The format enabled backward compatibility between different versions of applications running on the source and destination through use of discrete header types which, if not recognized by a version of the application are ignored. In addition, the headers, which can carry data, or act as basic headers for follow on data, include expansion space for additional information.