Abstract: Example distributed storage systems, delete managers, and methods provide for managing dependent delete operations among data stores. Dependent data operation entries and corresponding dependency sets may be identified in an operations log. Dependent data operations may be identified in each shard and data operation entries. A delete process for the data objects in the dependency set may be delayed until the delete process for the dependent data object completes.

Abstract: Example distributed storage systems, replication managers, and methods provide barriers for dependent data operations among sharded data stores. Shard operation logs may be associated with different shards and storage nodes. Dependent data operations may be identified in each shard and data operation entries corresponding to dependency sets for those dependent data operations may be determined. Log-based operations may be executed by delaying execution of the dependent data operations in all shards until the dependency set data operations in all shards are completed.

Abstract: Example distributed storage systems, replication state engines, and methods manage replication state for guaranteed replication between data stores. An object data store may store data objects that have been determined for deletion and rendered inaccessible to a client application. A replication state may be queried for deleted data objects and, if the replication state indicates that replication to another object data store is incomplete, physical deletion may be delayed until the replication state indicates that replication is complete and the data object may be physically deleted.

Abstract: Example object storage systems, bookkeeping engines, and methods provide quota-based access control for control entities, such as accounts, users, and buckets. An object data store is configured to enable control entities to access data objects associated with each control entity. Quota thresholds and usage values are determined for control entities and used to determine a quota status. Quota status is used to determine data object access response based on the requesting control entities.

Abstract: Example object storage systems, meta object generators, and methods provide versioned meta objects for internal operational data that may be replicated between data object stores. A meta object may be generated that includes meta object data, such as internal operational data. A meta object identifier may be assigned to the meta object. A first version identifier may be associated with the meta object data and a second version identifier may be associated with a marker for the meta object, where the marker prevents exposure as a user data object.

Abstract: Example distributed storage systems, file system interfaces, and methods provide duplicate request checking. A file interface manages a plurality of file data requests from client systems using a file system protocol. A metadata store acts as a cross-reference between file data references and storage data references. A request data store in the metadata store includes request entries with transaction identifiers for corresponding file data requests. A duplicate checker searches the request data store and excludes incoming file data requests when it finds a matching request entry.

Abstract: Example object storage systems, replication managers, and methods provide meta objects for priority replication of configuration data between data object stores. A meta object may be generated that includes configuration data. Data objects are selected for replication from a first replication queue. Meta objects associated with the data objects in the first replication queue are replicated, independent of the first replication queue, between object data stores.

Abstract: Example distributed storage systems, replication managers, and methods provide barriers for dependent data operations among sharded data stores. Shard operation logs may be associated with different shards and storage nodes. Dependent data operations may be identified in each shard and data operation entries corresponding to dependency sets for those dependent data operations may be determined. Log-based operations may be executed by delaying execution of the dependent data operations in all shards until the dependency set data operations in all shards are completed.

Abstract: Example distributed storage systems, delete managers, and methods provide for managing dependent delete operations among data stores. Dependent data operation entries and corresponding dependency sets may be identified in an operations log. Dependent data operations may be identified in each shard and data operation entries. A delete process for the data objects in the dependency set may be delayed until the delete process for the dependent data object completes.

Abstract: Example distributed storage systems, replication state engines, and methods manage replication state for guaranteed replication between data stores. An object data store may store data objects that have been determined for deletion and rendered inaccessible to a client application. A replication state may be queried for deleted data objects and, if the replication state indicates that replication to another object data store is incomplete, physical deletion may be delayed until the replication state indicates that replication is complete and the data object may be physically deleted.

Abstract: Example distributed storage systems, replication managers, and methods provide replication barriers for dependent data transfers between data stores. An object data store may include a barrier object and be configured to identify dependencies between a dependency set of data objects and the barrier object. When replicating data objects to another data store, the dependency set of data objects may be transferred first, delaying the transfer of the barrier object while the dependency set is being transferred.

Abstract: Example object storage systems, meta object generators, and methods provide versioned meta objects for internal operational data that may be replicated between data object stores. A meta object may be generated that includes meta object data, such as internal operational data. A meta object identifier may be assigned to the meta object. A first version identifier may be associated with the meta object data and a second version identifier may be associated with a marker for the meta object, where the marker prevents exposure as a user data object.

Abstract: Example object storage systems, replication managers, and methods provide meta objects for priority replication of configuration data between data object stores. A meta object may be generated that includes configuration data. Data objects are selected for replication from a first replication queue. Meta objects associated with the data objects in the first replication queue are replicated, independent of the first replication queue, between object data stores.

Abstract: Example object storage systems, bookkeeping engines, and methods provide quota-based access control for control entities, such as accounts, users, and buckets. An object data store is configured to enable control entities to access data objects associated with each control entity. Quota thresholds and usage values are determined for control entities and used to determine a quota status. Quota status is used to determine data object access response based on the requesting control entities.

Abstract: Example distributed storage systems, file system interfaces, and methods provide duplicate request checking. A file interface manages a plurality of file data requests from client systems using a file system protocol. A metadata store acts as a cross-reference between file data references and storage data references. A request data store in the metadata store includes request entries with transaction identifiers for corresponding file data requests. A duplicate checker searches the request data store and excludes incoming file data requests when it finds a matching request entry.

Abstract: A environment and method are provided for increasing the storage capacity of a data storage environment. Additional storage clusters may be added to the storage environment without affecting the performance of each individual storage cluster. When data is written to the storage environment, a selection may be made as to which storage cluster is to store the data. When data is read from the storage environment, it may be determined which storage cluster stores the data and the data may be retrieved from that storage cluster.

Abstract: One embodiment is a system for locating content on a storage system, in which the storage system provides a location hint to the host of where the data is physically stored, which the host can resubmit with future access requests. In another embodiment, an index that maps content addresses to physical storage locations is cached on the storage system. In yet another embodiment, intrinsic locations are used to select a storage location for newly written data based on an address of the data. In a further embodiment, units of data that are stored at approximately the same time having location index entries that are proximate in the index.

Abstract: One embodiment is a system for locating content on a storage system, in which the storage system provides a location hint to the host of where the data is physically stored, which the host can resubmit with future access requests. In another embodiment, an index that maps content addresses to physical storage locations is cached on the storage system. In yet another embodiment, intrinsic locations are used to select a storage location for newly written data based on an address of the data. In a further embodiment, units of data that are stored at approximately the same time having location index entries that are proximate in the index.

Abstract: One embodiment is a system for locating content on a storage system, in which the storage system provides a location hint to the host of where the data is physically stored, which the host can resubmit with future access requests. In another embodiment, an index that maps content addresses to physical storage locations is cached on the storage system. In yet another embodiment, intrinsic locations are used to select a storage location for newly written data based on an address of the data. In a further embodiment, units of data that are stored at approximately the same time having location index entries that are proximate in the index.

Abstract: One embodiment is a system for locating content on a storage system, in which the storage system provides a location hint to the host of where the data is physically stored, which the host can resubmit with future access requests. In another embodiment, an index that maps content addresses to physical storage locations is cached on the storage system. In yet another embodiment, intrinsic locations are used to select a storage location for newly written data based on an address of the data. In a further embodiment, units of data that are stored at approximately the same time having location index entries that are proximate in the index.