Abstract: A method is described. The method includes receiving logs from multiple connector nodes, each of the logs having entries that describe changes made to a file system at its respective one of the connector nodes. The method includes constructing a directed acyclic graph (DAG) from the logs' respective entries, the DAG comprising nodes connected by flows, the nodes describing actions made to directories and files of the file system, wherein, a flow of the flows connects a subset of the nodes that describe actions made to a particular directory or file of the file system. The method includes removing one or more irrelevant nodes from the DAG. The method includes applying actions described by remaining nodes of the DAG to an object storage system having respective objects for directories and files of the file system.

Abstract: A method is described. The method includes storing data extent replicas for each of a plurality of data extents, wherein, data extent replicas for a same one of the data extents are stored in different physical storage devices, and wherein, a same physical storage device is permitted to store respective data extent replicas of different ones of the data extents. The method includes writing to the respective data extent replicas for at least one of the data extents multiple times without calculating one or more parity extents for the plurality of data extents. The method includes upon a writing activity criteria having been met, calculating one or more parity extents for the data extents and storing the parity extents, and, preserving at least one of the data extent replicas for each of the plurality of data extents.

Abstract: A method for implementing a storage system is described. The method includes accepting a filepath from a user that specifies a file. The method includes forming a primary key for a representation of the file. The method includes applying the primary key to a database cloud service to obtain a representation of the file. The representation of the file includes an inode for the file's meta data. The method includes using the inode for the file's meta data to obtain the file's meta data from a high performance object cloud storage service. The file's meta data points to information within the high performance object cloud storage service for accessing the file's stripes. The method includes accessing the information within the high performance object cloud storage service to obtain an object ID for a stripe within the file. The method includes using the object ID to access the stripe from a low performance object cloud storage service.

Abstract: A method is described. The method includes queuing an entry in a queue that identifies an inode that has been changed in an active storage system. The entry further describes a time in the future. The method includes processing the entry upon the time in the future being reached. The processing of the entry includes comparing respective content values maintained for different regions of the inode within the active storage system and a replicate storage system. The method further includes comparing content of those of the regions having different content values as between the active storage system and the replicate storage system. The method further includes updating those regions in the replicate storage system to reflect those regions within the active storage system based on the comparing, and, updating the respective content values of those regions in the replicate storage system.

Abstract: A method is described.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects to implement a quota policy or a part of a relational database. The plurality of objects include a first plurality of immutable objects, a second plurality of immutable objects and a head object. The first plurality of immutable objects contain information kept within the database. The second plurality of immutable objects contain a hierarchy of pages for accessing the first plurality of immutable objects. The plurality of objects also include a mutable object that contains a mapping table that maps identifiers of the pages to identifiers of objects of the second plurality of immutable objects. The information is one of: quota policy information; information within a table of a relational database; a stripe of a thin provisioned file.

Abstract: A method is described. The method includes fragmenting data of an object for storage into an object storage system into multiple data fragments and performing a first error correction encoding process on the data to generate one or more parity fragments for the object. The method also includes sending the multiple data fragments and the one or more parity fragments over a network to different storage servers of the object storage system. The method also includes performing the following at each of the different storage servers: i) incorporating the received one of the multiple data fragments and one or more parity fragments into an extent comprising multiple fragments of other objects; ii) performing a second error correction encoding process on multiple extents including the extent to generate parity information for the multiple extents; and, iii) storing the multiple extents and the parity information.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects containing information. The storage system also includes a cluster of locking nodes to implement a distributed synchronization locking mechanism. The cluster of locking nodes include a proposer node and a plurality of acceptor nodes. The proposer node to broadcast to the acceptor nodes a proposed lock on at least some of the information. The acceptor nodes to vote on the proposed lock based on respective locking information kept in a local store of each acceptor node.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: An apparatus is described including a large scale distributed storage system. The large scale distributed storage system includes an object store. The large scale distributed storage system includes a plurality of connector nodes coupled to the object store. The connector nodes are to provide access to the object store for respective users that are respectively coupled to the connector nodes. The connector nodes are to provide the following services for the respective users: a snapshot of a storage system implemented within said object store; branching from a snapshot taken of said storage system implemented within said object store; locking of information kept within said storage system implemented within said object store.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects to implement a quota policy or a part of a relational database. The plurality of objects include a first plurality of immutable objects, a second plurality of immutable objects and a head object. The first plurality of immutable objects contain information kept within the database. The second plurality of immutable objects contain a hierarchy of pages for accessing the first plurality of immutable objects. The plurality of objects also include a mutable object that contains a mapping table that maps identifiers of the pages to identifiers of objects of the second plurality of immutable objects. The information is one of: quota policy information; information within a table of a relational database; a stripe of a thin provisioned file.

Abstract: A computer implemented method and system for generating secure universal object identifiers on a multipurpose storage system is disclosed. According to one embodiment, a system comprises a client system in communication with a network. An application server is in communication with the network. A storage cluster is in communication with the network. The storage cluster has a plurality of storage nodes. The client system stores a data object via the application server. The application server generates an object identifier assigned to the data object. The application server stores the data object on a storage node of the plurality of storage nodes. The data object is moved to another application server without moving contents of the data object in the storage cluster.

Abstract: A method and system having a probabilistic offload engine for distributed hierarchical object storage devices is disclosed. According to one embodiment, a system comprises a first storage system and a second storage system in communication with the first storage system. The first storage system and the second storage system are key/value based object storage devices that store and serve objects. The first storage system and the second storage system execute a probabilistic algorithm to predict access patterns. The first storage system and the second storage system execute a probabilistic algorithm to predict access patterns and minimize data transfers between the first storage system and the second storage system.

Abstract: A multipurpose storage system based upon a distributed hashing mechanism with transactional support and failover capability is disclosed. According to one embodiment, a system comprises a client system in communication with a network, a secondary storage system in communication with the network, and a supervisor system in communication with the network. The supervisor system assigns a unique identifier to a first node system and places the first node system in communication with the network in a location computed by using hashing. The client system stores a data object on the first node system.