Abstract: In an implementation, a set of files are stored into a collection of blocks. The collection of blocks are stored within a storage subsystem that includes a premium storage area and a low-end storage area, wherein the premium storage area has a performance metric that is better than a performance metric of the low-end storage area. The collection of blocks define the set of files at multiple different times within the predefined time window, the collection of blocks include a first baseline of blocks which defines the set of files at a first baseline time within a predefined time window, older blocks which were formed before the first baseline time, and newer blocks which were formed after the first baseline time. The first baseline of blocks and the newer blocks are stored in the premium storage area and the older blocks are stored in the low-end storage area.

Abstract: A cluster manager is configured to manage a plurality of copies of a mid-tier database as a mid-tier database cluster. The cluster manager may concurrently manage a backend database system. The cluster manager is configured to monitor for and react to failures of mid-tier database nodes. The cluster manager may react to a mid-tier database failure by, for example, assigning a new active node, creating a new standby node, creating new copies of the mid-tier databases, implementing new replication or backup schemes, reassigning the node's virtual address to another node, or relocating applications that were directly linked to the mid-tier database to another host. Each node or an associated agent may configure the cluster manager to behave in this fashion during initialization, based on common cluster configuration information. Each copy of the mid-tier database may be, for example, a memory resident database. Thus, a node must reload the entire database into memory to recover a copy of the database.

Abstract: Continuous data protection is performed as two parallel processes: creating an initial backup by copying a data as a file/directory from the storage device into the backup storage, and copying the data to be written to the data storage as a part of a file/directory into the incremental backup. Alternatively, it can be performed as one process: copying the data to be written to the data storage as a part of a file/directory on the storage. A write command to a file system driver is intercepted and redirected to the backup, and the data is written to the incremental backup. If the write command is also directed to a not yet backed up data (a file/directory), the identified data is copied from the storage device to intermediate storage. The write command is executed on the file/directory from the storage device, and the file/directory is copied from the intermediate storage.

Abstract: Techniques for establishing a files system that exposes a virtual file system for backup operations are presented. Changes to files of a file system are maintained separately or collected and presented as a VFS. The VFS is then used to perform backup operations. Moreover, the files identified in the VFS can be customized based on search parameters, criterion, and/or criteria supplied by a user or supplied as parameters to backup and VFS mount operations.

Abstract: A method and system for restoring a computing environment from a secondary data store, and facilitating such restoration, is described. In some examples, the system copies the configuration parameters of a computing environment, such as a file system, as well as the data within the file system, and transfers the copies to secondary storage for later restoration. For example, the system may generate an xml file of the configuration parameters of the computing environment when a backup operation of the computing environment is performed, and store the xml file along with copies of the data from the computing environment in secondary storage.

Abstract: This disclosure describes systems and methods for programmatically determining whether to perform full or partial backups. In some implementations, functionality is provided for intelligently escalating to a full backup based on one or more user-supplied escalation parameters. The escalation parameters can include a threshold of data change between backup files, which if exceeded, can result in escalation to a full backup. The escalation parameters can also include time parameters, computing resource consumption parameters, and other features that can govern full backup escalation. Additional features of the systems and methods described herein can include automatic adjustment of escalation parameters, multi-leveled backup set validation, and intelligent backup file cleanup.

Abstract: A method and system of validating a differential update to a computer system. One method includes comparing a list of files in a new computer system image with a list of files in a former computer system image to construct a manifest list of files comprising a subset of the new computer system images and the former computer system images. The method further includes associating information with the manifest list to facilitate determining whether a differential update comprising differences between the new computer system image and the former computer system image is valid. The method also includes comparing a list of files in a computer system to receive the differential update against the manifest list of files and using the associated information to determine if the differential update is valid for the computer system.

Abstract: Various embodiments of a system and method for restoring a file are described herein. A previous version of the file may be split into segments and backed up to a server computer system. In response to a subsequent request to restore the file to the previous version, the current segments of the file may be compared to the backup segments stored on the server computer system. The segments that have changed may be retrieved from the server computer system and used to restore the corresponding segments of the current version of the file to their previous states. Segments that have not changed do not need to be transferred across the network from the server computer system. In further embodiments, one or more of the segments that have changed may be obtained locally from other files, thus further reducing the amount of data transferred across the network.

Abstract: Various embodiments are provided for facilitation of data retention using logical objects. Following an operation creating a redundant copy of the data performed on a scheduled interval, a logical object containing a number of managed file versions, represented by a number of member objects for a recovery point, is created. The logical object is assigned a policy of a data retention policy construct associated with the recovery point. The logical object is adapted for reassignment between policies of the data retention policy construct associated with various recovery points. During the reassignment, the plurality of member objects representing the plurality of managed file versions are logically retained instead of performing a data copy operation to associate the plurality of managed file versions with another recovery point.

Abstract: A record of changes to virtual machine data of a virtual machine and a record of changes to snapshots of the virtual machine that have been received since the virtual machine was last backed up are maintained. In response to it being time to perform an incremental backup of the virtual machine, a portion of the virtual machine data is backed up based on the record of changes to the virtual machine data, and a portion of the snapshots is backed up based on the record of changes to the snapshots. In response to it being time to migrate the virtual machine to a second host device, the virtual machine data, the record of changes to the virtual machine data, the record of changes to the snapshots, and one or more of the snapshots are migrated to the second host device.

Abstract: Embodiments of the present invention provide an approach for protecting and restoring data within a networked (e.g. cloud) storage computing environment through asynchronous replication and remote backup of data and its associated metadata. Under embodiments of the present invention, data backup and recovery functionality provides data backups by detecting incremental updates to the data and its associated metadata at specific points in time determined by policies. The policies are configurable based on user requirements. Multiple copies of the data backups can be made and stored in separate compressed files at backup/disaster recovery locations. The backups of data and its associated metadata, which includes file system configuration information can be used to restore the state of a computer file system to that of a given point-in-time. Accordingly, a data protection approach is disclosed for protecting data at both the file system level and application level.

Abstract: Systems and methods for replicating data from a production server to a backup server include identifying one or more data blocks of a file that were modified after a first time instant and before a second time instant. The file may be associated with a protected directory of the production server. An representative data block (e.g., including a hash value) for at least one of the identified data blocks may be computed using a cryptography algorithm, e.g., MD5 or SHA-1. The computed cryptographic data block representing the identified data block may then be sent for replication to the backup server. An operation performed between the first time instant and the second time instant on one or more files of the protected directory may be recorded, and sent to the backup server.

Abstract: A system and method for creating a backup of a virtual machine running on a host computer is described herein. The system and method operate by creating a copy or “clone” of a virtual machine running on a first host computer on a second host machine connected thereto. After generation of the clone, a backup of the virtual hard disk of the clone can be obtained in a manner that does not consume any resources of the first host machine. The backup of the virtual hard disk of the clone can then be used as the backup of the original virtual machine.

Abstract: Backing up data blocks is disclosed, including: receiving an indication that a first backed up data block is to be processed; determining that the first backed up data block and a second backed up data block are related at an application level and are to remain stored together; and processing together the first and second backed up data blocks based at least in part on the determination that the first and second backed up data blocks are related at an application level and are to remain stored together.

Abstract: An archival storage cluster of preferably symmetric nodes includes a metadata management system that organizes and provides access to given metadata, preferably in the form of metadata objects. Each metadata object may have a unique name, and metadata objects are organized into regions. Preferably, a region is selected by hashing one or more object attributes (e.g., the object's name) and extracting a given number of bits of the resulting hash value. The number of bits may be controlled by a configuration parameter. Each region is stored redundantly. A region comprises a set of region copies. In particular, there is one authoritative copy of the region, and zero or more backup copies. The number of backup copies may be controlled by a configuration parameter. Region copies are distributed across the nodes of the cluster so as to balance the number of authoritative region copies per node, as well as the number of total region copies per node.

Abstract: A technique for implementing policy-aware backup and restore capability in a tiered storage system. If a data set's contents are backed up from the tiered storage system to a backup storage system, metadata for the data set may also be backed up. Prior to the data set being restored from the backup storage system to the tiered storage system, the backed up metadata is restored and processed to determine a tier among the tiered storage pools to which the data set will be restored.

Abstract: A system, method, and medium for performing incremental backups of a Microsoft SQL server database. A snapshot of the database is created, and then a map identifying the changed extents is retrieved from the snapshot. The changed extents are then retrieved from the snapshot and stored in a backup storage device. For a restore operation, a full database backup file is written to a storage device and then the changed extents from a stored incremental backup file may be merged with the full backup file. Next, the database server is notified of the reconstructed file and then the reconstructed file is mounted by the database server as a live instance of the database.

Abstract: Techniques for normalizing a database as part of a database restore. Embodiments may receive a database restore request indicating a previous state to restore a database to. Responsive to the request, embodiments may restore the database to the previous state using backup data associated with the previous state, and normalize the restored database using historical database usage data based on one or more previous database operations.

Abstract: Various methods and systems for performing data deduplication when encrypting data for storage on an untrusted storage system are disclosed. One method involves generating an encryption key for use in encrypting data and generating an identifier for the data. Generation of the encryption key is based upon a hash of the data to be encrypted. Similarly, generation of the identifier is based upon the hash of the data to be generated. For example, the identifier can be generated by hashing the encryption key. The method then involves detecting whether an encrypted copy of the data is already stored by a storage system, based upon the identifier, and selectively encrypting the data, based upon whether the encrypted copy of the data is already stored by the storage system.

Abstract: Aspects of the subject matter described herein relate to data protection of files that may include both content and embedded metadata. In aspects, an indication that a file has changed is received. If the content has changed, the entire file or just the content may be backed up. If the metadata only has changed, the metadata may or may not be backed up, depending on implementation. Instead of parsing the file to determine the exact size and positioning of the metadata within the file, the file may be divided into three portions: a head portion, a middle portion, and a tail portion. Hashes of these portions may then be used to determine whether the metadata and/or content of the file have changed.

Abstract: A computer creates a full backup of a target volume by copying sectors of the target volume. Incremental backups are created by copying modified sectors that have changed since a preceding backup. Public accesses to a restore volume are prevented by modifying a volume boot record (VBR) of the restore volume to prevent file systems from recognizing the restore volume or by filtering public accesses while allowing private block accesses to the restore volume. Sector data are copied from the backup files to the restore volume while the public accesses are being prevented. The restore can begin, using previously created backup files, while the backup is still in process creating new incremental backups for the target volume. The restore volume is finalized after restoring the full or any number of incremental backups by restoring the VBR to indicate the restore volume is valid or stopping the filtering of public accesses.

Abstract: A server apparatus performs data I/O processing on a P-VOL in response to a request from a client apparatus and includes a file system which receives data write and read requests in units of files from the client apparatus. The server apparatus stores, into a D-VOL, differential data generated by updating the P-VOL, and manages a snapshot management table wherein a block address of the D-VOL storing the acquired differential data is registered. When a remaining capacity of a storage area of the D-VOL becomes equal to or less than a predetermined threshold value, the server apparatus enables a block of the D-VOL which had stored therein the differential data on a certain predetermined file, to be used as a storage area for newly acquired differential data.

Abstract: A system for consolidation of backups. Each backup includes data and a map with offsets or addresses that point to data of current and/or previous backups. Backup are selected for consolidation. The offsets in the maps of the selected backups are combined into a single set of offsets; consolidating the selected backups to a new backup that contains the latest data from the selected backups, determined based on the offsets in the single set. The offsets in the single set are updated to point to the data in the new backup. The updated offsets are stored in the new backup as part of a backup map of the new backup. The selected backups comprise one initial full backup and at least one backup. Offsets from the maps of first backup of the selected sequence of backups up to the last created backup are checked.

Abstract: Methods and apparatus, including computer program products, for distributing data in a data management system. The technique includes identifying one or more objects in a central data store for distribution, determining if a routing exists for at least one object of the one or more objects, and distributing the at least one object to one or more target systems specified by the routing. The one or more objects include master data objects for use by all systems in a data management system, and the one or more target systems are part of the data management system.

Abstract: A backup method that uses snapshot delta views to create backups of a data collection is provided. The method comprises taking a first snapshot of the data collection, in response to determining that a previous snapshot used to create a backup does not exist; creating a backup of the data collection using the first snapshot, in response to determining that the previous snapshot does not exist; recording that the first snapshot was used as a source for the creation of a backup; and taking a second snapshot of the data collection, in response to determining that the previous snapshot that was used as a source for a backup does exist.

Abstract: A server auditing process that stores only a single up-to-date data record along with the differences relative to previous changes in the record that allow the user to move “backward in time” to recreate previous values. The auditing feature introduces a baseline database table and a difference database table for each existing database table. The baseline table stores the most recent auditing data and the differences table complements the baseline table by storing pairs in the form of attribute name and previous value. By applying difference rows to baseline data, the user is able to recreate the data as it existed at any given timestamp and at a minimal storage cost. The architecture minimizes the impact of auditing on system performance by employing an asynchronous priority queue so that task processing is deferred until the system has adequate resources and other work is not affected.

Abstract: A system and method for improving the efficiency of the transmission of updated blocks generated by logging all the block allocations and deletes as well as CPs and PCPI creation and deletion in a persistent log. The log is scanned during each update cycle (in which changes are transmitted to a destination mirror) by the storage operating system, and only changed blocks that are referenced by the file system as it existed at the end of the update cycle or referenced by PCPIs that existed at the end of the update cycle are actually sent in the transmission. This reduces the number of changes being transmitted.

Abstract: Techniques for adjusting the frequency of data backups and initiating event-driven backups in a storage system are disclosed herein. In one embodiment, a self-adjusting backup frequency, known as a “Change Rate Objective,” is defined to conduct or delay backups for one or more volumes in the storage system on the basis of an associated policy value. The Change Rate Objective may be tied to one or more business or data activity events, such as the amount and type of data changes since a last backup. The storage system may also be tailored to conduct or delay full or incremental backups on the basis of a Change Rate Objective that measures whether a full or incremental or differential backup is more appropriate. Various data or system failures, or data or business events may also be used to adjust the retention periods of continuous data protection (CDP) data and delay a rollup of CDP data.

Abstract: A computer implemented method includes monitoring blocks of data on a storage device that are changing as the computer operates. On detecting a computer shut down event, a copy of changes to the monitored blocks are saved. Upon startup of the computer, a backup of the changed blocks of data is performed.

Abstract: Methods, apparatus and computer programs are provided for data replication, such as for replicating a database or updating a cache. Source database change transactions are captured and batched together for efficient data transfer. On receipt at a target replica system, the batch of transactions within a message is separated and separate apply transactions corresponding to the source transactions are performed in parallel. A ‘transaction processed’ table is updated to reflect apply processing for each apply transaction. This table provides a confirmation of the status of each transaction that can be used during recovery processing to prevent re-apply of an apply transaction. The table can also be used to determine when it is safe to delete an incoming message from persistent storage on the target system, and to facilitate one-phase commit processing.

Abstract: Method and system for incremental backup of data containers with fixed length records is provided. Initially, a backup application copies all stored data containers to create an initial backup container. A data structure corresponding to the first backup operation is also created. The data structure maintains unique file identifiers for each data container and information regarding when a record was added. The data structure is created and maintained by an application that also creates, updates, and modifies the fixed length records. The application updates the base data structure when records are added to a data container, after the initial backup. For a next backup operation, the initial data structure is compared to an updated data structure that is current at the next backup time. The comparison identifies the new records that were added (or updated) since the initial backup operation. The backup application then incrementally copies the new records.

Abstract: A technology capable of preventing a problem which occurs when log writing of a preceding transaction fails while reducing waiting time of a subsequent transaction is provided. A database control method for releasing locking of data held by a transaction upon reception of a transaction completion instruction from an operation terminal or the like by a database is provided. The database includes an exclusive control module, a log output module, and a log buffer. The log output module stores a transaction completion log in the log buffer upon reception of the transaction completion instruction from the operation terminal. The exclusive control module releases locking held by the transaction upon reception of storage of the transaction completion log in the log buffer.

Abstract: Image-based data restoration restores an entire image of a volume or selective files. A restore engine receives a request to restore data from a backup image that includes a plurality of data blocks and metadata describing attributes of the data blocks. In response to the request, the restore engine determines a restore mode. If it is determined to be a logical restore mode, the restore engine restores a file by retrieving the metadata associated with the file and the data blocks that are pointed to by the metadata. If it is determined to be an image restore mode, the restore engine restores an entire image of the volume by using the data blocks in the backup image without using the metadata.

Abstract: Writes to blocks of a protected storage volume that occur between the beginning of a prior backup interval and the beginning of a current backup interval are tracked using a first snapshot that is maintained through the current backup interval. At the beginning of the current backup interval, a second snapshot is generated. Blocks that are indicated by a dirty block list of the first snapshot as having been changed are read from the snapshot volume for the second snapshot and copied to a backup storage volume. For the next backup interval, the second snapshot or alternatively, a newly generated third snapshot, is used to track writes to blocks of a protected storage volume that occur between the beginning of the current backup interval and the beginning of a next backup interval.

Abstract: A computer-implemented method may, at a first point in time, back up at least a portion of a data-storage entity to a first virtual-disk file. The computer-implemented method may capture, in a second virtual-disk file, at least one change made to data in the data-storage entity after the first point in time. The computer-implemented method may also create a parent-child relationship between the first virtual-disk file and the second virtual-disk file, with first virtual-disk file being a parent of the second virtual-disk file. The computer-implemented method may further copy data stored in the second virtual-disk file to the first virtual-disk file so that the first virtual-disk file includes the at least one change made to data in the data-storage entity after the first point in time. Various other methods, systems, and computer-readable media are also disclosed.

Abstract: Incremental checkpoint, for use in data replication, track the changes made to a file system after a point in time at which the incremental checkpoint is created. Data replication techniques using the incremental checkpoints may include taking a regular checkpoint of the file system and creating the first time full copy on remote node using the regular checkpoint. Changes made to the file system are then tracked in an incremental checkpoint that are stored on the remote node. The processes of taking the incremental checkpoint and storing the incremental checkpoint are iteratively performed. The first time fully copy and the incremental checkpoints may then be used for data replication, backup, continuous data protection (CDP), or the like.

Abstract: Described are techniques and mechanisms for media data storage management. Briefly stated, a media control component monitors the usage patterns of media files on a local device. Any media files whose usage pattern falls below a given threshold is identified as a trim candidate. The media control component trims those trim candidates by eliminating data (content) from the media files. The data is eliminated from the latter portion of the media file so that the trimmed media file will continue to render properly when initially launched. If the trimmed media file is launched, the media control component retrieves the eliminated data from a content server on which is stored a library of media files.

Abstract: Methods and computer readable media for restoring a database. In a method, a backup of a subset of a database is created, where the subset includes a filegroup and database metadata describing the filegroup. The database is then restored using the database subset. In a computer-readable medium, a first data field contains data representing a database subset having at least one of a plurality of filegroups. A second data field contains data representing database metadata that describes the filegroup. In another method, archived database metadata and an archived filegroup that is one of a plurality of filegroups is accessed, and the filegroup is brought online using the database metadata.

Abstract: A file backup apparatus and method estimates whether a file has been changed in a directory based on directory property information, and backs up the directory which includes a changed file according to the estimation result.

Abstract: A method for backing up a file system, including obtaining a first indirect block comprising a first block pointer, obtaining a first birth time from the first block pointer, determining whether the first birth time is subsequent to a time of a last backup, and backing up a first block referenced by the first block pointer, if the first birth time is subsequent to the time of the last backup.

Abstract: Backing up data blocks is disclosed, including: receiving an indication that a first backed up data block is to be processed; determining that the first backed up data block and a second backed up data block are related at an application level and are to remain stored together; and processing together the first and second backed up data blocks based at least in part on the determination that the first and second backed up data blocks are related at an application level and are to remain stored together.

Abstract: A system, method and computer program product for consolidation of backups, wherein each backup includes data and a backup map with offsets that point to data of current and/or previous backups, including selecting a plurality of backups to be consolidated; combining the offsets in the backup maps of the selected backups into a single set of offsets; consolidating the selected backups to a new backup that contains the latest data from the selected backups, the latest data determined based on the offsets in the single set; updating the offsets in the single set to point to the data in the new backup; and storing the updated offsets in the new backup as part of a backup map of the new backup. The selected backups comprise one initial full backup and at least one backup. In the combining step, the addresses from the backup maps of first backup of the selected sequence of backups up to the last created backup is checked. The selected backups are organized into a tree.

Abstract: A method of intelligently copying recovery points from a primary storage device to a secondary storage device is presented. The decision on how to copy the recovery points depends upon the backup copy policy as well as the availability of previous backups on the secondary storage device.

Abstract: A method and apparatus for efficiently logging changes to a file system are disclosed. A file system filter driver intercepts I/O requests directed to a file system, and analyzes the I/O requests to determine whether the requests are write or modify requests directed to a file included in a list of files to be monitored for changes. If the I/O request is directed to such a file, the file system filter driver logs the I/O request to a b-tree structured change log file. Furthermore, in one embodiment, the file system filter driver is configured to log file system changes to multiple log files to coincide with snapshot operations, such that the file system changes to be included in each snapshot are included in separate log files.

Abstract: Continuous data protection is performed as two parallel processes: copying a data block from the storage device into the backup storage device (creating initial backup) and copying the data block to be written to the data storage into the incremental backup. When a write command is directed to a data storage block, it's intercepted and redirected to the backup storage, and data, which is to be written in accord to the write request, is written to the incremental backup on the backup storage. If write command is also directed to a data storage block identified for backup that has not yet been backed up, the identified data storage block is copied from the storage device to the intermediate storage device, the write command is executed on the identified data storage block from the storage device, and the data storage block is copied from the intermediate storage device to the backup storage device. In case of an error accessing a block on the storage device, the block is marked as invalid.

Abstract: The present invention is capable of determining the rights to a file based on providing a descriptor. The descriptor can be calculated using an algorithm, which may be cryptographic and/or non-cryptographic. The descriptor may further be based on the file contents, metadata of the file, other file data, or any combination thereof to uniquely identify the file in a shared file repository. Since the descriptor is generated based on file data it will be the same regardless of which user generates it. Accordingly, only one copy of the file needs to be maintained in the shared file repository, thereby reducing the amount of network bandwidth required to assure the file is backed up and further reducing the amount of storage required to backup the files. This results in a vastly more efficient method of backup in terms of processing time, network bandwidth, and storage requirements.

Abstract: An embodiment of the invention provides a traversal engine that traverses (walks) through metadata containers (e.g., inodes) in a file system and determines the metadata containers that have been modified (added, changed, deleted, or renamed) within a particular time interval. A modified metadata container corresponds to a data container (e.g., file or directory) that has been modified within the particular time interval. The traversal engine records, into a table, information about the modified metadata containers. The traversal engine also records, into the table, parent information corresponding to an immediate parent of a modified data container.

Abstract: Systems and methods for computer system data backup. A computer system includes an application server operating on a first host and a backup application configured to create new target database images for a sequence of backup datasets. The backup datasets include data from source databases associated with the application server. At least one of the source databases is hosted on a remote source host separate from the first host. The computer system includes a client registered with the application server and installed on a backup host. The client stores a seed document in each of the new target database images and modifies each seed document to specify which portions of a backup dataset in the sequence are to be stored in each new target database image. The backup application updates each new target database image based on the contents of a seed document.

Abstract: A method and system for generating and restoring a backup image are disclosed. A server receives a request, which includes a backup path, to generate a backup image. The server generates a backup image offset map that indicates a relative location where a file is stored in the backup image before file data has been written to the backup image. The server stores the backup image offset map in a portion of the backup image on a storage device. The backup image offset map comprises an array having a number of elements equal to a number of inodes on a file system that hosts a backup path, an element of the array representing an inode on the file system, and indicates the relative location where a file that is associated with a number of the inode is stored in the backup image.

Abstract: Creating a synthetic database backup using binary logs. A full backup of a database may be stored, followed by one or more incremental backups. The one or more incremental backups may include first binary log files. A first synthetic differential backup may be created using the first binary log files. Creating the first synthetic differential backup may include translating the first binary log files into a first plurality of SQL statements. The first plurality of SQL statements may be executable to recreate the changes in the database since the full backup. Translating each of the first binary log files into the first plurality of SQL statements may be performed before performing a recovery process to decrease recovery time.