Abstract: A system for estimating one or more data storage parameters and/or statistics in a data storage system is presented. The data storage system includes a plurality of storage containers. The system includes a snapshot module, a container stats aggregator, a synchronization module, a global stats aggregator, and storage stats estimator.

Abstract: A file system may include an object storage, a merged index, and a distributed database. When a file is stored in the file system, the file may be converted to an object and be stored in the object storage. The deduplication index of the file may be stored in the distributed database. The namespace metadata of the file may be stored in the merged index. The merged index generates namespace entries of the file when the file is created, deleted, and/or modified. A namespace entry may be associated with a specific file and may include a creation version and a deletion version. When a file is deleted or modified, instead of modifying the existing namespace entries, new entries associated with different versions and including different creation or deletion versions are created. The status of a file may be monitored by one or more entries associated with a file.

Abstract: Techniques and mechanisms described herein provide for verification of data across cloud-based and on-premises data storage systems. According to various embodiments, a backup client implemented on a first compute node can store a data file in a backup data repository. A data guardianship can store first data file state information describing the data file in a key-value store accessible via the internet. A data verification instance can analyze the backup data repository to verify that the data file is stored intact in the backup data repository.

Abstract: Disclosed embodiments include a method (system and non-transitory computer-readable medium) for backing up updated portions of a plurality files having hierarchical relationships through object storage. In one or more embodiments, a file is segregated into chunks, and objects corresponding to the chunks are generated for storage at an object storage. For a chunk, an object for storing the chunk and additional objects for storing mapping information are generated. The mapping information may include path information identifying a path of the file in a hierarchical structure, a file version list identifying a version of the file, a chunk list describing an association between the file and the chunks, a chunk version list identifying a version of the chunk, etc. When a portion of the file is updated, objects corresponding to the updated portion of the file can be generated, and stored at the object storage.

Abstract: A data restoration system including a data management server. The data management server receives one or more data restoration requests for restoring a plurality of data blocks. The data management server determines, based on metadata associated with the data blocks, a first subset of warm data blocks corresponding to warm-tier data and a second subset of cold data blocks corresponding to cold-tier data. The data management server retrieves the warm data blocks in the first subset and restores the warm data blocks in the first subset. The data management server groups the cold data blocks based in part on storage times of the cold data blocks to generate a plurality of cold-tier data retrieval requests. The data management server retrieves the cold data blocks by batches, each batch corresponding to one of the cold-tier data retrieval requests. The data management server restores the cold data blocks in the second subset.

Abstract: A system for data backup is provided. The system includes a memory having computer-readable instructions stored therein and a processor configured to execute the computer-readable instructions to receive a request for full data and/or incremental backup for a volume and to perform a full backup of the volume based on a first block size in response to a full data backup request. The processor is configured to generate a digital fingerprint of the full backup and determine if the full backup exists on a backup media based on the generated digital fingerprint and to upload the full backup to the backup media if it is determined that the first backup is unavailable on the backup media and perform an incremental backup of the volume based on a second block size in response to an incremental backup request. The second block size is substantially smaller than the first block size.

Abstract: A system identifies and causes transmission of differential data generated during device migration. An administrative server accesses a first backup snapshot of a retiring client device. The first backup snapshot includes a set of files stored in the retiring client device during a first checkpoint. The administrative server transmits the set of files in the first backup snapshot to a replacement client device. A cloud server stores the first backup snapshot and a second backup snapshot of the retiring client device. The second backup snapshot is created during a second checkpoint occurring after transmission of the set of files. The cloud server receives an indication that a user has logged on to the replacement client device and causes a transmission of differential data to the replacement client device. The differential data includes at least one file in the second backup snapshot that is not included in the first backup snapshot.

Abstract: A system, method, and non-transitory computer-readable medium provide backup and archive services for computing devices. Typically, multiple backup snapshots are archived together in each archive cycle. A catalogue for the current archive is efficiently created by starting with a copy of the previous archive catalogue and updating it based on metadata associated with the backup snapshots.

Abstract: A file system may include an object storage, a merged index, and a distributed database. When a file is stored in the file system, the file may be converted to an object and be stored in the object storage. The deduplication index of the file may be stored in the distributed database. The namespace metadata of the file may be stored in the merged index. The merged index generates namespace entries of the file when the file is created, deleted, and/or modified. A namespace entry may be associated with a specific file and may include a creation version and a deletion version. When a file is deleted or modified, instead of modifying the existing namespace entries, new entries associated with different versions and including different creation or deletion versions are created. The status of a file may be monitored by one or more entries associated with a file.

Abstract: A system and method for an index-based smart scan for a cloud-computing provider network is provided. The system includes a memory having computer-readable instructions stored therein and a snapshot repository configured to store a plurality of snapshots of a plurality of block storage volumes. Each of the plurality of block storage volumes is configured to perform volume based block storage operations for the cloud-computing provider network. The system further includes a processor communicatively coupled to the snapshot repository. The processor is configured to access contents of each of the plurality of snapshots. Each of the plurality of snapshots includes a point-in-time capture of the respective block storage volume. In addition, the processor is configured to perform a full scan of each of the plurality of snapshots to identify one or more files of the respective block storage volume. The processor is further configured to generate a folder index table for each of the files based upon the scan.

Abstract: A data storage system allows data to be encrypted and de-duplicated at the same system. By way of example, a server of the data storage system may request a client device which intends to upload a data block to transmit a first fingerprint of the data block to the server. The first fingerprint may be derived from the plaintext of the data block. The server may apply a one-way function to the first fingerprint to generate an encryption key and transmit the encryption key to the client device. The client device uses the encryption key to encrypt the data block and generates a second fingerprint which is derived from the ciphertext of the data block. The server uses both the first fingerprint and the second fingerprint to verify the data block and the legitimacy of the client attempting to upload the data block.

Abstract: A proactive data recovery system is provided. The system includes a memory having computer-readable instructions stored therein and a processor configured to execute the computer-readable instructions to access a data storage platform and to monitor a plurality of parameters indicative of a requirement of data restore and/or recovery for the data storage platform. The requirement corresponds to a predicted occurrence of a disaster event. The processor is further configured trigger backup of data stored in the data storage platform based upon the plurality of parameters to create a restore package and to initiate the data restore and/or data recovery operation for the data storage platform using the restore package in response to the occurrence of the disaster event.

Abstract: Aspects of the current patent document include systems and methods to perform search in an index system. In one embodiment, an index system may be implemented in an object storage. A distributed database index is used in conjunction with the object storage. In some cases, data stored in the distributed database may be encrypted and moved to object storage. The object storage stores a plurality of blocks containing words. Each block can contain a large number of words, such as one million words.

Abstract: A smart folder scan system and method is provided. The system includes a memory having computer-readable instructions stored therein and a storage module having a plurality of file folders configured to store data. The system further includes a processor communicatively coupled to the storage module. The processor is configured to access the plurality of file folders stored in the storage module. In addition, the processor is configured to scan and identify one or more modified file folders stored in the storage module. Further, the processor is configured to generate a folder activity table for each of the plurality of file folders based upon the scan. The folder activity table comprises of a listing of the file folders and an associated modification time for each of the file folders. The processor is further configured to generate a skip table database based upon the modification time of each of the file folders.

Abstract: A snapshot usage tracking system and method for a versioned storage is provided. The system includes a memory having computer-readable instructions stored therein and a snapshot repository configured to store a plurality of snapshots of a versioned storage. Each of the plurality of snapshots includes one or more data blocks. The system further includes a processor communicatively coupled to the snapshot repository and configured to maintain a set of snapshot counters corresponding to each of previous snapshots created on or before a current snapshot based upon a size of the data blocks that are modified or deleted in the current snapshot.

Abstract: Techniques and mechanisms described herein provide for facilitating communications between one or more client machines and one or more cloud storage providers. According to various embodiments, a virtual machine may communicate with one or more client machines via a standard data storage protocol. The virtual machine may also communicate with one or more cloud storage providers via the internet. The virtual machine may then perform operations such as receiving data from a client machine and storing it to the cloud, retrieving data from the cloud and providing it to the client machine, querying data stored in the cloud, reporting on and verifying data stored in the cloud, and transferring and/or transforming data stored in the cloud.

Abstract: Techniques and mechanisms described herein provide for verifying data across cloud-based and on-premises data storage systems. According to various embodiments, data can be received by a gateway from a client machine and stored in a file data repository accessible via the internet. The stored data can have a common master namespace. Indications of updates to the master namespace can be received. Updated namespace data and metadata can be transmitted to the gateway.

Abstract: Aspects of the current patent document include systems and methods to adaptive bandwidth throttling, for example, for use in data backup systems and data recovery systems. In embodiments, bandwidth estimation can be performed while sending data. In embodiments, the bandwidth estimation is used in data backups to send data to be backed up. In embodiments, a server performs network bandwidth estimation by receiving relatively small data packets and estimating bandwidth until bandwidth reliability conditions are satisfied.

Abstract: Disclosed embodiments include a method (system and non-transitory computer-readable medium) for backing up updated portions of a plurality files having hierarchical relationships through object storage. In one or more embodiments, a file is segregated into chunks, and objects corresponding to the chunks are generated for storage at an object storage. For a chunk, an object for storing the chunk and additional objects for storing mapping information are generated. The mapping information may include path information identifying a path of the file in a hierarchical structure, a file version list identifying a version of the file, a chunk list describing an association between the file and the chunks, a chunk version list identifying a version of the chunk, etc. When a portion of the file is updated, objects corresponding to the updated portion of the file can be generated, and stored at the object storage.

Abstract: The present invention relates generally to backups and more specifically to virtual machine (VM) backups including file exclusion. Aspects of the present invention related to using a specialized buffer to identify a file for exclusion. In embodiments, a file system used by the VM can be used to search for the specialized buffer. In embodiments, when the specialized buffer is located, offsets are noted related to the file associated with the specialized buffer. In embodiments, the offsets are used to zero out blocks associated with the offsets. Thus, the file can be effectively excluded from the backup.