Abstract: One example method is directed to generating a data sampling parameter, and includes identifying an incremental billing rate for data storage, where the data storage includes a data set having a multiple data pieces that each correspond with a respective unique indicator, and the unique indicators reside in a space. The example method further includes identifying a threshold billing increment, and determining an acceptable error in the threshold billing increment. Finally, the method includes determining a sub sample ratio based upon the threshold billing increment and the acceptable error. The sub sample ratio has a variable value ‘X’ which indicates that sampling of one unique indicator in every ‘X’ unique indicators in the space is sufficient to determine the size of the entire data set.

Abstract: A cloud storage system stores data objects from different customers. Each customer has their own encryption key to encrypt the data objects for storage and the key is not shared. To deduplicate the data objects, a set of base fingerprints associated with a set of base data objects is stored. A first fingerprint associated with a data object from a customer is compared with a base fingerprint. The first and base fingerprints are generated by applying an identical fingerprinting algorithm to unencrypted versions of the data objects. If the fingerprints match, the first fingerprint is associated with a base data object corresponding to the matching base fingerprint, and the data object from the customer is not stored. If the fingerprints do not match, the data object from the customer is stored.

Abstract: In one example method, a hash space is identified that includes hashes that each point to a respective piece of data in the customer data set. A first hash is selected using a sub sample ratio, and the first hash is checked to determine if the first hash points to data previously observed in an associated backup stream. If the first hash points to data not previously observed in the associated backup stream, the number of data pieces to which the first hash points are recorded, and if the first hash points to data previously observed in the associated backup stream, a second hash is selected using the sub sample ratio. The method can be repeated until the entire hash space has been sampled. The required storage capacity is calculated using the sub sample ratio and a sum of the recorded number of data pieces.

Abstract: Views of files in an archival data storage system are generated by a backup view generator. A storage application generates and stores archival data in an archive system, the archival data corresponding to client data stored on a server or in memory associated with one or more client nodes. The storage application also generates backup files of the archival data which may be stored in a local memory. A set of metadata attributes are associated with each of the backup files. The backup views are generated by comparing metadata values in a view definition file to the sets of attributes associated with the backup files. Generated backup views can be exported for processing, including searching the backup views or displaying the backup views in a user interface.

Abstract: One example method includes identifying a space that includes pointers that each point to a respective piece of data in the set of data, selecting a first pointer using a sub sample ratio, checking the first pointer to see if the first pointer points to data previously observed in an associated backup stream, if the first pointer points to data not previously observed in the associated backup stream, recording a number of data pieces to which the first pointer points, and if the first pointer points to data previously observed in the associated backup stream, selecting a second pointer using the sub sample ratio. The selecting, checking and recording processes are repeated until the entire space has been sampled, and the required storage capacity is calculated using the sub sample ratio and a sum of the recorded number of data pieces.

Abstract: A backup client de-duplicates backup data sets using a locally stored, memory resonant, root tag vector and hash cache. To create a new backup data set, the client queries a backup server to determine which of the root hashes in the root tag vector are available on the backup server. If one or more are no longer available, the backup server re-uses a root tag vector entry corresponding to one of the no longer available root hashes. If all are available, the client ages out a root hash for re-use based on a combination of age and represented size. Data is de-duplicated by chunking and hashing it and comparing the resulting hashes to hashes in the hash cache. To prevent the hash cache from growing too large, entries in the hash cache are aged out based on a combination of age and size of data represented by the entries.

Abstract: A backup client de-duplicates backup data sets using a locally stored, memory resonant, root tag vector and hash cache. To create a new backup data set, the client queries a backup server to determine which of the root hashes in the root tag vector are available on the backup server. If one or more are no longer available, the backup server re-uses a root tag vector entry corresponding to one of the no longer available root hashes. If all are available, the client ages out a root hash for re-use based on a combination of age and represented size. Data is de-duplicated by chunking and hashing it and comparing the resulting hashes to hashes in the hash cache. To prevent the hash cache from growing too large, entries in the hash cache are aged out based on a combination of age and size of data represented by the entries.

Abstract: Backup data in a single-instance storage device is accessed through a backup server using hashes representative of and pointing to the backup data. To prevent unauthorized access, the server provides each client with encrypted versions of hashes corresponding to data backed up by the client. The hashes can be encrypted using client-specific symmetric encryption keys known to the server. To request data, a client provides the backup server with a corresponding encrypted hash. The backup server decrypts the encrypted hash using the client's encryption key. The original hash is only obtained if the key used for decryption is identical to the key used for encryption. Consequently, if an encrypted hash is stolen or otherwise acquired by a client different from the client that backed up the corresponding data, it cannot be used by the different client to request the corresponding data from the backup server.

Abstract: This disclosure is describes a system and method for organizing and storing backup data by inode number. Data objects on a file system may be streamed to a backup client that identifies the inode numbers for each streamed data object before streaming the objects to storage. The inode numbers are parsed to create one or more inode directories that can be browsed during a recovery process. In this fashion, the file system can be quickly backed up without requiring the backup client to determine the file system's directory hierarchy.

Abstract: A priority restore agent in a data storage system generates a priority restore data set for a client computer system or device by identifying a set of active data sets and/or a set of key data sets within client system data generated by the client computer system. The priority restore agent looks at or processes file system attributes for the client system data and compares these attributes with predefined restore parameters. The restore parameters may indicate that any file that has been accessed, modified, or created within a particular period of time be included in the priority restore data set. The key data sets may be identified in a set of automated restore rules. A data protection application within the data storage system can restore data in the priority restore data set onto the client computer system after a disaster or system crash.

Abstract: New full backups are generated by combining an incremental backup with a previous full backup. A previous full backup is stored in a backup server in a hash file system format. A file server generates an incremental backup of a data set on the file server by identifying and dumping files/directories of the data set that are new/modified into a tar file that is sent to an accelerator. The accelerator parses the incremental backup tar file and converts it to a hash file system format that includes metadata, hash values, and new/modified data atomics. The accelerator merges the incremental backup into the previous full backup to generate a new full backup by altering metadata and hash values of a copy of the previous full backup such that the resulting metadata and hash values describe and point to new/modified directories and files as well as unmodified directories and files.

Abstract: A backup is performed by a client at a time when a backup server is unable to process the backup. The client maintains a cache including a root tag vector and hash entries. The client begins a backup by writing the root tag vector to a journal file and breaking files into pieces. For each piece, the client performs a hash and compares the resulting hash to entries in the cache. If the hash does not match any entries, the client records a request in the journal file to add the corresponding piece of data to an archive. After completing the backup, the journal file can be sent to the server. Before processing the journal file, the server validates the root tag vector. If the root tag vector is valid, the server processes each of the requests to add data. Otherwise, the server discards the journal file.

Abstract: A priority restore agent in a data storage system generates a priority restore data set for a client computer system or device by identifying a set of active data sets and/or a set of key data sets within client system data generated by the client computer system. The priority restore agent looks at or processes file system attributes for the client system data and compares these attributes with predefined restore parameters. The restore parameters may indicate that any file that has been accessed, modified, or created within a particular period of time be included in the priority restore data set. The key data sets may be identified in a set of automated restore rules. A data protection application within the data storage system can restore data in the priority restore data set onto the client computer system after a disaster or system crash.