Abstract: A method is described. The method includes storing data extent replicas for each of a plurality of data extents, wherein, data extent replicas for a same one of the data extents are stored in different physical storage devices, and wherein, a same physical storage device is permitted to store respective data extent replicas of different ones of the data extents. The method includes writing to the respective data extent replicas for at least one of the data extents multiple times without calculating one or more parity extents for the plurality of data extents. The method includes upon a writing activity criteria having been met, calculating one or more parity extents for the data extents and storing the parity extents, and, preserving at least one of the data extent replicas for each of the plurality of data extents.

Abstract: A method is described. The method includes executing a PUT operation for an object including attempting to store the object's data and first protection information into S storage devices and attempting to store second protection information into one or more R additional storage devices. The method includes determining that at least one write attempt into the S storage devices was not successful. The method includes repairing the object's stored state by causing the object's data and first protection information to be stored into the S storage devices.

Abstract: A method is described. The method includes processing first meta data for first units of stored content in a first storage system. The processing includes constructing a first tree of first nodes where first lower nodes represent fewer of the first units of stored content and first higher nodes represent more of the first units of stored content. The first nodes include respective first hash values calculated from the first meta data to represent the respective content of the respective first units of stored content that the first nodes respectively represent. The method includes processing second meta data for second units of stored content in a second storage system. The processing includes constructing a second tree of second nodes where second lower nodes represent fewer of the second units of stored content and second higher nodes represent more of the second units of stored content.

Abstract: A method for implementing a storage system is described. The method includes accepting a filepath from a user that specifies a file. The method includes forming a primary key for a representation of the file. The method includes applying the primary key to a database cloud service to obtain a representation of the file. The representation of the file includes an inode for the file's meta data. The method includes using the inode for the file's meta data to obtain the file's meta data from a high performance object cloud storage service. The file's meta data points to information within the high performance object cloud storage service for accessing the file's stripes. The method includes accessing the information within the high performance object cloud storage service to obtain an object ID for a stripe within the file. The method includes using the object ID to access the stripe from a low performance object cloud storage service.

Abstract: An apparatus to implement a sparse file system has been described. The sparse file system includes an execution engine cloud service to receive user requests to the sparse file system and implement a cache for the sparse file system and cache coherence protocol logic for the cache. The sparse file system also includes a database cloud service to store meta data for any of files, stripes and stripe sections of the sparse file system and implement a search function that identifies any of files, stripes and stripe sections of the sparse file system that meet certain meta data search criteria. The sparse file system also includes an object storage cloud service that assigns individual object IDs to individual stripes within the sparse file system.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A method is described. The method includes fragmenting data of an object for storage into an object storage system into multiple data fragments and performing a first error correction encoding process on the data to generate one or more parity fragments for the object. The method also includes sending the multiple data fragments and the one or more parity fragments over a network to different storage servers of the object storage system. The method also includes performing the following at each of the different storage servers: i) incorporating the received one of the multiple data fragments and one or more parity fragments into an extent comprising multiple fragments of other objects; ii) performing a second error correction encoding process on multiple extents including the extent to generate parity information for the multiple extents; and, iii) storing the multiple extents and the parity information.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects to implement a quota policy or a part of a relational database. The plurality of objects include a first plurality of immutable objects, a second plurality of immutable objects and a head object. The first plurality of immutable objects contain information kept within the database. The second plurality of immutable objects contain a hierarchy of pages for accessing the first plurality of immutable objects. The plurality of objects also include a mutable object that contains a mapping table that maps identifiers of the pages to identifiers of objects of the second plurality of immutable objects. The information is one of: quota policy information; information within a table of a relational database; a stripe of a thin provisioned file.

Abstract: A method is described. The method includes fragmenting data of an object for storage into an object storage system into multiple data fragments and performing a first error correction encoding process on the data to generate one or more parity fragments for the object. The method also includes sending the multiple data fragments and the one or more parity fragments over a network to different storage servers of the object storage system. The method also includes performing the following at each of the different storage servers: i) incorporating the received one of the multiple data fragments and one or more parity fragments into an extent comprising multiple fragments of other objects; ii) performing a second error correction encoding process on multiple extents including the extent to generate parity information for the multiple extents; and, iii) storing the multiple extents and the parity information.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects containing information. The storage system also includes a cluster of locking nodes to implement a distributed synchronization locking mechanism. The cluster of locking nodes include a proposer node and a plurality of acceptor nodes. The proposer node to broadcast to the acceptor nodes a proposed lock on at least some of the information. The acceptor nodes to vote on the proposed lock based on respective locking information kept in a local store of each acceptor node.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: An apparatus is described including a large scale distributed storage system. The large scale distributed storage system includes an object store. The large scale distributed storage system includes a plurality of connector nodes coupled to the object store. The connector nodes are to provide access to the object store for respective users that are respectively coupled to the connector nodes. The connector nodes are to provide the following services for the respective users: a snapshot of a storage system implemented within said object store; branching from a snapshot taken of said storage system implemented within said object store; locking of information kept within said storage system implemented within said object store.

Abstract: A method is described. The method includes receiving a request to search meta data for objects stored within a large scale object storage system. The request identifies a looked for value of the meta data. The objects belong to a same bucket used to identify a subset of objects stored by the large scale object storage system. The method includes forwarding the request to a meta data database system that contains pages listing all objects within the bucket and the associated meta data for each of the objects within the bucket. The method includes forwarding the pages from the meta data database system over a network to a high performance computing resource that concurrently processes multiple ones of the pages to identify matching ones of the objects whose meta data matches the looked for value.

Abstract: A method is described. The method includes fragmenting data of an object for storage into an object storage system into multiple data fragments and performing a first error correction encoding process on the data to generate one or more parity fragments for the object. The method also includes sending the multiple data fragments and the one or more parity fragments over a network to different storage servers of the object storage system. The method also includes performing the following at each of the different storage servers: i) incorporating the received one of the multiple data fragments and one or more parity fragments into an extent comprising multiple fragments of other objects; ii) performing a second error correction encoding process on multiple extents including the extent to generate parity information for the multiple extents; and, iii) storing the multiple extents and the parity information.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects to implement a quota policy or a part of a relational database. The plurality of objects include a first plurality of immutable objects, a second plurality of immutable objects and a head object. The first plurality of immutable objects contain information kept within the database. The second plurality of immutable objects contain a hierarchy of pages for accessing the first plurality of immutable objects. The plurality of objects also include a mutable object that contains a mapping table that maps identifiers of the pages to identifiers of objects of the second plurality of immutable objects. The information is one of: quota policy information; information within a table of a relational database; a stripe of a thin provisioned file.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects to implement a quota policy or a part of a relational database. The plurality of objects include a first plurality of immutable objects, a second plurality of immutable objects and a head object. The first plurality of immutable objects contain information kept within the database. The second plurality of immutable objects contain a hierarchy of pages for accessing the first plurality of immutable objects. The plurality of objects also include a mutable object that contains a mapping table that maps identifiers of the pages to identifiers of objects of the second plurality of immutable objects. The information is one of: quota policy information; information within a table of a relational database; a stripe of a thin provisioned file.

Abstract: A method is described that includes providing a snapshot counter for a storage system implemented with multiple distributed consistent database instances. The method further includes recognizing the taking of a snapshot of the storage system by incrementing the snapshot counter from a first snapshot counter value to a second snapshot counter value. The method further includes, in response to a first change for one of the distributed consistent databases subsequent to the taking of the snapshot, performing the following: saving state information of the distributed consistent database as the state information existed prior to the change and associating the first snapshot counter value to the state information. Associating the second snapshot counter's value with the distributed consistent database having the first change. Another method for the taking of a snapshot of a distributed consistent database is also described.

Abstract: A storage system is described. The storage system includes an object store having a plurality of objects containing information. The storage system also includes a cluster of locking nodes to implement a distributed synchronization locking mechanism. The cluster of locking nodes include a proposer node and a plurality of acceptor nodes. The proposer node to broadcast to the acceptor nodes a proposed lock on at least some of the information. The acceptor nodes to vote on the proposed lock based on respective locking information kept in a local store of each acceptor node.