Abstract: A system, method and apparatus for storing metadata in a metadata store in a robust and efficient manner including receiving a request from a client to perform a data transaction, updating a key-value pair in a metadata store based on the request, entering the data transaction in a transaction log, updating a read cache with the key-value pair, and replicating the last transaction log entry in at least one other storage node in the metadata store.

Abstract: A system, method and apparatus for storing metadata in a metadata store in a robust and efficient manner including receiving a request from a client to perform a data transaction, updating a key-value pair in a metadata store based on the request, entering the data transaction in a transaction log, replicating the last transaction log entry in at least one other storage node in the metadata store and, in response to a threshold event, migrating a first portion of the transaction log from the a data storage medium to a second data storage medium of each of the plurality of data storage nodes.

Abstract: A computer-implemented method for redistributing data between memory clusters in a key value store. The data is redistributed according to a load balancing algorithm. Memory clusters are sorted into lists according to the number of intervals each cluster contains. Migration jobs are created by the load balancing algorithm to push data fragments from a larger cluster to a smaller cluster. Data fragments, or intervals, are selected for migration according to corresponding data fragments on a target cluster or the number of occupying data fragments on a target cluster. The redistribution of data helps avoid fragmentation of key ranges of intervals, decrease the overall number of non-adjacent key ranges, and to conform to the routing table requirements.

Abstract: A system, method and apparatus for storing metadata in a metadata store in a robust and efficient manner including receiving a request from a client to perform a data transaction, updating a key-value pair in a metadata store based on the request, entering the data transaction in a transaction log, replicating the last transaction log entry in at least one other storage node in the metadata store and, in response to a threshold event, migrating a first portion of the transaction log from the a data storage medium to a second data storage medium of each of the plurality of data storage nodes.

Abstract: A system, method and apparatus for storing metadata in a metadata store in a robust and efficient manner including receiving a request from a client to perform a data transaction, updating a key-value pair in a metadata store based on the request, entering the data transaction in a transaction log, updating a read cache with the key-value pair, and replicating the last transaction log entry in at least one other storage node in the metadata store.

Abstract: A computer-implemented method for redistributing data between memory clusters in a key value store. The data is redistributed according to a load balancing algorithm. Memory clusters are sorted into lists according to the number of intervals each cluster contains. Migration jobs are created by the load balancing algorithm to push data fragments from a larger cluster to a smaller cluster. Data fragments, or intervals, are selected for migration according to corresponding data fragments on a target cluster or the number of occupying data fragments on a target cluster. The redistribution of data helps avoid fragmentation of key ranges of intervals, decrease the overall number of non-adjacent key ranges, and to conform to the routing table requirements.

Abstract: System and method for managing metadata. A computer system includes one or more hosts, each host having an associated backup agent. Each backup agent conveys data from its associated host to a storage pool and stores metadata associated with the data in a metabase. In response to detecting that a first metabase meets a first condition, the system identifies a backup agent that has stored metadata in the first metabase and redistributes at least a portion of the metadata to a second metabase. In one embodiment, the first condition comprises detecting the amount of metadata stored in the first metabase reaching a threshold value. In a further embodiment, the metadata comprises entries, each of which corresponds to a data file. Each entry includes a fingerprint that is derived from and identifies the corresponding data file. The fingerprints identifying the data files may be encrypted.

Abstract: A system and method for managing single instance storage. A computer system includes at least two backup servers, each backup server included in a single-instance storage pool. A first backup server conveys a first de-duplicated list identifying data segments from the first storage pool to a second backup server. The first backup server receives from the second backup server a second de-duplicated list identifying a subset of the data segments and conveys the subset of the data segments to the second backup server. In response to receiving the first list from the first backup server, the second backup server de-duplicates the first list against a second storage pool and conveys the second list to the first backup server. In response to receiving the subset of the data segments, the second backup server adds the received data segments to the second storage pool.

Abstract: A system and method for managing single instance storage. A computer system includes at least two backup servers, each backup server included in a single-instance storage pool. A first backup server conveys a first de-duplicated list identifying data segments from the first storage pool to a second backup server. The first backup server receives from the second backup server a second de-duplicated list identifying a subset of the data segments and conveys the subset of the data segments to the second backup server. In response to receiving the first list from the first backup server, the second backup server de-duplicates the first list against a second storage pool and conveys the second list to the first backup server. In response to receiving the subset of the data segments, the second backup server adds the received data segments to the second storage pool.