Abstract: Embodiments disclosed herein provide systems, methods, and computer readable media for sub-cluster recovery in a data storage environment having a plurality of storage nodes. In a particular embodiment, the method provides scanning data items in the plurality of nodes. While scanning, the method further provides indexing the data items into an index of a plurality of partition groups. Each partition group includes data items owned by a particular one of the plurality of storage nodes. The method then provides storing the index.

Abstract: Methods and systems for backing up and restoring sets of electronic files using sets of pseudo-virtual disks are described. The sets of electronic files may be sourced from or be stored using one or more different data sources including one or more real machines and/or one or more virtual machines. A first snapshot of the sets of electronic files may be aggregated from the different data sources and stored using a first pseudo-virtual disk. A second snapshot of the sets of electronic files may be aggregated from the different data sources subsequent to the generation of the first pseudo-virtual disk and stored using the first pseudo-virtual disk or a second pseudo-virtual disk different from the first pseudo-virtual disk.

Abstract: The method disclosed includes scanning data items stored in the first plurality of nodes of a first cluster. While scanning, creating a partition group index indexing the data items into a plurality of partition groups. Each partition group corresponds to a node of the first plurality of nodes and comprises a subset of data items stored in the node. Storing the index. Instantiating a second cluster, comprising generating per node data, for each node of a second plurality of nodes, based on mappings between the partition groups and the first plurality of nodes. Identifying the data items included in the partition groups according to the partition group index and loading the data items included in the partition groups onto the second plurality of nodes.

Abstract: The method disclosed is for instantiating a second cluster based on a first cluster. For at least one node of a second plurality of nodes, generating per node data based on mappings between a plurality of partition groups and a first plurality of nodes, the first plurality of nodes corresponding to the first cluster. The method further discloses identifying data items included in the plurality of partition groups based on the mappings between the plurality of partition groups and the first plurality of nodes. The method further discloses each partition group corresponding to a node of the first plurality of nodes and comprising a subset of data items stored in the node. The method further discloses loading the data items included in the plurality of partition groups onto the second plurality of nodes, the second plurality of nodes corresponding to the second cluster.

Abstract: Methods and systems for backing up and restoring different point in time versions of a virtual machine, an application, a database, or an electronic file using independently managed snapshot chains are described. In some cases, different point in time versions of a virtual machine may be captured and stored using one or more snapshot chains. Each snapshot chain may correspond with a base image (e.g., a full image snapshot) and one or more incremental files (e.g., two forward incremental files) that derive from the base image. A snapshot chain may be split into a plurality of shards or sub-chains such that the width of each shard or the maximum data size of the files in each shard is less than a threshold data size (e.g., is less than 1 TB). Data operations (e.g., consolidation and reverse operations) may be performed on the individual shards or sub-chains in parallel.

Abstract: Methods and systems for backing up and restoring different point in time versions of a virtual machine, an application, a database, or an electronic file using independently managed snapshot chains are described. In some cases, different point in time versions of a virtual machine may be captured and stored using one or more snapshot chains. Each snapshot chain may correspond with a base image (e.g., a full image snapshot) and one or more incremental files (e.g., two forward incremental files) that derive from the base image. A snapshot chain may be split into a plurality of shards or sub-chains such that the width of each shard or the maximum data size of the files in each shard is less than a threshold data size (e.g., is less than 1 TB). Data operations (e.g., consolidation and reverse operations) may be performed on the individual shards or sub-chains in parallel.

Abstract: Methods and systems for improving the read and write performance of a distributed file system while limiting memory usage are described. The type of error correcting scheme applied to data, the partitioning of the data into data chunks, and the sizes of data slices within each of the data chunks used for storing electronic files within the distributed file system may be dynamically adjusted over time to optimize for fast IO performance while limiting memory usage (e.g., requiring less than 256 MB of RAM to generate and store code blocks). The file size of an electronic file to be stored, the amount of available memory for generating code blocks, and the amount of available disk space to store the electronic file may be used to set the data sizes of the data slices and the type of erasure code applied to data blocks associated with the data slices.