Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: Instead of adding resources into a folder by specifying a list of identifiers, client applications add resources to a folder by specifying a search query that matches the resources to be added. The virtual folder is, therefore, a collection of search queries rather than a collection of resource identifiers. This allows the data archive systems to easily store folder information in a database. A client application can then search by specifying the folder as its search criteria to list all content associated with the folder. Additionally, to ensure that the list of results associated with a particular query does not change after the query is added into a folder, the data archive system also captures generation information associated with the particular query (i.e., the iteration of the search indexes at the time the query is received) and stores the generation information with the query as part of the folder.

Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: Methods and systems are disclosed that relate to single instance storage of asset parts utilizing a back reference count associated with each asset part. Entries in a journal associated with the asset parts reflect the intention to increment or decrement an asset part's back reference count. The journal may be used to maintain an accurate and reliable reference count for asset parts replicated across multiple disk drives in a distributed storage system, where the replication is structured to reduce the risk of data loss in the event of a disk drive failure. The journal entries may comprise generation numbers used to maintain a back reference count associated with an asset part and to verify a back reference count associated with an asset part.

Abstract: In a data storage system having a plurality of storage nodes storing replicas of stripes, one storage node serves as a primary stripe node for a stripe. Client applications using the data storage system request operations affecting metadata stored in the stripe, and the data storage system creates workflow objects to implement the requested operation. Once the operation is completed, the workflow is acknowledged. A method for establishing a new primary stripe node comprises performing, by a selected storage node, a feasibility determination to determine the feasibility of the selected storage node becoming the new primary stripe node. It is feasible to become a new primary stripe node when doing so does not cause workflows that have been acknowledged, or that will be acknowledged, to result in strays.

Abstract: Instead of adding resources into a folder by specifying a list of identifiers, client applications add resources to a folder by specifying a search query that matches the resources to be added. The virtual folder is, therefore, a collection of search queries rather than a collection of resource identifiers. This allows the data archive systems to easily store folder information in a database. A client application can then search by specifying the folder as its search criteria to list all content associated with the folder. Additionally, to ensure that the list of results associated with a particular query does not change after the query is added into a folder, the data archive system also captures generation information associated with the particular query (i.e., the iteration of the search indexes at the time the query is received) and stores the generation information with the query as part of the folder.

Abstract: Methods and systems are disclosed that relate to single instance storage of asset parts utilizing a back reference count associated with each asset part. Entries in a journal associated with the asset parts reflect the intention to increment or decrement an asset part's back reference count. The journal may be used to maintain an accurate and reliable reference count for asset parts replicated across multiple disk drives in a distributed storage system, where the replication is structured to reduce the risk of data loss in the event of a disk drive failure. The journal entries may comprise generation numbers used to maintain a back reference count associated with an asset part and to verify a back reference count associated with an asset part.

Abstract: Methods and systems are disclosed that relate to encrypting data of a data item for storing in a data storage system comprising a plurality of disks having stripes. A blinding factor is constructed based on a stripe blind that is assigned to a stripe with which the data item is associated and a unique identifier associated with the data item. A first logic operation is performed between the blinding factor and an encryption key to create a blinded encryption key for the data item. The data item is decrypted by identifying the stripe blind with the unique identifier and recreating the data item's blinding factor based on the stripe blind and the unique identifier. A second logic operation, which is selected based on the first logic operation, is performed between the blinding factor and the blinded encryption key to recreate the encryption key.

Abstract: A hash table system having a first hash table and a second hash table is provided. The first hash table may be in-memory and the second hash table may be on-disk. Inserting an entry to the hash table system comprises inserting the entry into the first hash table, and, when the first hash table reaches a threshold load factor, flushing entries into the second hash table. Flushing the first hash table into the second hash table may comprise sequentially flushing the first hash table segments into corresponding second hash table segments. When looking up a key/value pair corresponding to a selected key in the hash table system, the system checks both the first and second hash tables for values corresponding to the selected key. The first and second hash tables may be divided into hash table segments and collision policies may be implemented within the hash table segments.

Abstract: In a data storage system having a plurality of storage nodes storing replicas of stripes, one storage node serves as a primary stripe node for a stripe. Client applications using the data storage system request operations affecting metadata stored in the stripe, and the data storage system creates workflow objects to implement the requested operation. Once the operation is completed, the workflow is acknowledged. A method for establishing a new primary stripe node comprises performing, by a selected storage node, a feasibility determination to determine the feasibility of the selected storage node becoming the new primary stripe node. It is feasible to become a new primary stripe node when doing so does not cause workflows that have been acknowledged, or that will be acknowledged, to result in strays.