Abstract: A method for grouping and storing objects across different storage solutions in storage systems according to analysis of multiple dimensions of information may be implemented as or in a storage management module. The module collects information about objects (e.g., data objects) in a storage system (e.g., a data storage system). The objects may be objects already stored in the storage system or may be new objects to be stored in the storage system. The module analyzes the collected information across multiple dimensions to determine groupings of the objects, and determines a storage solution for each determined grouping. The module may then direct storage of the objects in the groupings according to the determined storage solutions. Upon obtaining new information about object(s) in the storage system, the module may direct movement of the object(s) from one storage solution to another storage solution according to an analysis including the new information.

Abstract: A cost-effective, durable and scalable archival data storage system is provided herein that allow customers to store, retrieve and delete archival data objects, among other operations. For data storage, in an embodiment, the system stores data in a transient data store and provides a data object identifier may be used by subsequent requests. For data retrieval, in an embodiment, the system creates a job corresponding to the data retrieval and provides a job identifier associated with the created job. Once the job is executed, data retrieved is provided in a transient data store to enable customer download. In various embodiments, jobs associated with storage, retrieval and deletion are scheduled and executed using various optimization techniques such as load balancing, batch processed and partitioning. Data is redundantly encoded and stored in self-describing storage entities increasing reliability while reducing storage costs. Data integrity is ensured by integrity checks along data paths.

Abstract: A cost-effective, durable and scalable archival data storage system is provided herein that allow customers to store, retrieve and delete archival data objects, among other operations. For data storage, in an embodiment, the system stores data in a transient data store and provides a data object identifier may be used by subsequent requests. For data retrieval, in an embodiment, the system creates a job corresponding to the data retrieval and provides a job identifier associated with the created job. Once the job is executed, data retrieved is provided in a transient data store to enable customer download. In various embodiments, jobs associated with storage, retrieval and deletion are scheduled and executed using various optimization techniques such as load balancing, batch processed and partitioning. Data is redundantly encoded and stored in self-describing storage entities increasing reliability while reducing storage costs. Data integrity is ensured by integrity checks along data paths.

Abstract: A cost-effective, durable and scalable archival data storage system is provided herein that allow customers to store, retrieve and delete archival data objects, among other operations. For data storage, in an embodiment, the system stores data in a transient data store and provides a data object identifier may be used by subsequent requests. For data retrieval, in an embodiment, the system creates a job corresponding to the data retrieval and provides a job identifier associated with the created job. Once the job is executed, data retrieved is provided in a transient data store to enable customer download. In various embodiments, jobs associated with storage, retrieval and deletion are scheduled and executed using various optimization techniques such as load balancing, batch processed and partitioning. Data is redundantly encoded and stored in self-describing storage entities increasing reliability while reducing storage costs. Data integrity is ensured by integrity checks along data paths.

Abstract: Embodiments may include a consistency measurement component that utilizes memory-efficient sets (e.g., Bloom filters) to generate consistency metrics for read operations performed on different replicated data objects of distributed storage system. Based on the consistency metrics, the consistency measurement component may identify a subset of replicated data objects associated with low levels of consistency. The consistency measurement component may target this subset for consistency improvement by generating instructions to improve the consistency of the subset. In other cases, the consistency measurement component may notify a consistency improvement component about the targeted subset. In response, the consistency improvement component may generate instructions to improve the consistency of the targeted subset.