Abstract: Synchronously replicating a dataset across cloud-based storage systems, including adding a cloud-based storage system to a set of storage systems that the dataset is synchronously replicated across, where access operations are applied to the dataset equivalently through all storage systems in the set, all storage systems in the set store a separate copy of the dataset, and operations to modify the dataset performed and completed through any of the storage systems in the set are reflected in access operations to read the dataset, the cloud-based storage system including one or more cloud computing instances executing a storage controller application, a virtual drive layer that includes one or more cloud computing instances with local storage for storing at least a portion of the dataset as block data, and an object storage layer for storing at least a portion of the dataset as object data.

Abstract: A system and method for efficiently starting up a plurality of solid-state storage devices. A computing system includes one or more storage devices storing data in multiple allocation units (AUs). In a boot region, a data storage controller maintains an AU management set that identifies a state of various AUs in the system. In various embodiments, the management set includes an identification of a subset of free AUs, transitional AU, unincorporated AUs, and speculative AUs. At various times, information corresponding to the AU management set is stored to non-volatile storage. During a boot sequence, the AU management set information is accessed and the identified AUs are searched to identify allocated AUs rather than performing a search of all of the AUs in the system.

Abstract: Determining storage consumption in a storage system, including scanning a group of data units within the storage system storing a plurality of client entities, including: for each data unit in the group of data units, determining whether the data unit is reachable by one of the plurality of client entities; and for each data unit reachable by one of the plurality of client entities, determining a category for the data unit; calculating storage consumption for a client based on the category of each data unit reachable by one of the plurality of client entities; and reporting the calculated storage consumption.

Abstract: Servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.

Abstract: Servicing I/O operations in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; storing, within staging memory provided by one or more virtual drives of the virtual storage system, the data; and migrating, from the staging memory to more durable data storage provided by a cloud services provider, at least a portion of data stored within the staging memory.

Abstract: Servicing I/O operations in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; storing, within staging memory provided by one or more virtual drives of the virtual storage system, both the data and an erasure code based on the data; and migrating, from the staging memory to more durable data storage provided by a cloud services provider, at least a portion of data stored within the staging memory without migrating the erasure code based on the data.

Abstract: A storage system for synchronously replicating a dataset across hardware-based storage systems and cloud-based storage systems, the storage system including specifying a set of managed objects, a set of management operations, and a set of access operations to correspond to the dataset, where management operations can modify or query managed objects equivalently through the hardware-based storage systems and the cloud-based storage systems; access operations to read or modify the dataset operate equivalently through the hardware-based storage systems and the cloud-based storage systems; each hardware-based storage system and each cloud-based storage system stores a separate copy of the dataset as a proper subset of the datasets stored and advertised for use; and operations to modify managed objects or the dataset performed and completed through the hardware-based storage systems and the cloud-based storage systems are reflected in subsequent management objects to query the dataset or subsequent access operatio

Abstract: Servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.

Abstract: A cloud-based storage system contained in a cloud computing environment, the cloud-based storage system including: determining, at the cloud-based storage system and in response to a request for data, that the data previously stored within one or more virtual instances of a virtual instance layer is no longer stored within the one or more virtual instances; generating, within the virtual instance layer, a quantity of virtual instances to receive data restored from a cloud-based storage layer of the cloud-based storage system; and restoring, into the quantity of virtual instances at the virtual instance layer, data from the cloud-based storage layer of the cloud-based storage system.

Abstract: Servicing I/O operations in a cloud-based storage system, including: receiving, by the cloud-based storage system, a request to write data to the cloud-based storage system; storing, in solid-state storage of the cloud-based storage system, the data; storing, in object storage of the cloud-based storage system, the data; detecting that at least some portion of the solid-state storage of the cloud-based storage system has become unavailable; identifying data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; retrieving, from object storage of the cloud-based storage system, the data that was stored in the portion of the solid-state storage of the cloud-based storage system that has become unavailable; and storing, in solid-state storage of the cloud-based storage system, the retrieved data.

Abstract: Coalescing write operations in a cloud-based storage system including receiving, from a storage controller application of the cloud-based storage system, a first plurality of write operations, wherein each of the first plurality of write operations comprises a respective write to a storage volume; coalescing the first plurality of write operations into a plurality of coalesced write operations, wherein each of the coalesced write operations are configured to effect two or more of the first plurality of write operations; and performing the plurality of coalesced write operations on the storage volume.

Abstract: Symmetric storage using a cloud-based storage system, including: receiving, at a cloud-based storage system among storage systems synchronously replicating a dataset, an I/O operation directed to the dataset; determining, in dependence upon the I/O operation, a metadata update describing a mapping of segments of content to an address within a storage object, wherein the storage object includes the dataset; and synchronizing metadata on another storage system of the storage systems by sending the metadata update from the cloud-based storage system to the other storage system to update a metadata representation on the second storage system in accordance with the metadata update.

Abstract: Cloning storage systems in a cloud computing environment, including: receiving a request to create a cloud-based storage system; retrieving, from cloud-based object storage, one or more objects to include in the cloud-based storage system; and creating the cloud-based storage system, including storing, in block storage of the cloud-based storage system, data contained in the one or more objects retrieved from the cloud-based object storage.

Abstract: Determining active membership among a set of storage systems, including: determining, by a cloud-based storage system among the set of storage systems, that a membership event corresponds to a change in membership to the set of storage systems synchronously replicating the dataset; applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems.

Abstract: Servicing I/O operations in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; storing, within staging memory provided by one or more virtual drives of the virtual storage system, the data; and migrating, from the staging memory to more durable data storage provided by a cloud services provider, at least a portion of data stored within the staging memory.

Abstract: Efficiently transferring data between tiers in a virtual storage system, including: receiving, by the virtual storage system, a request to write data to the virtual storage system; transforming, within storage provided by a first tier of storage of the virtual storage system, the data to generate transformed data; and migrating, from the first tier of storage to a second tier of storage that is more durable than the first tier of storage of the virtual storage system, at least a portion of the transformed data.

Abstract: Data recovery in a virtual storage system, including: detecting, within storage provided by a first tier of storage of the virtual storage system, data loss within a dataset, wherein recovery data for the dataset is stored in a second tier of storage; determining a recovery point for the dataset up to which a consistent version of the dataset is recoverable from the recovery data stored in the second tier of storage; and restoring, within the storage provided by the first tier of storage of the virtual storage system, the consistent version of the dataset.

Abstract: A system and method for efficiently distributing data among multiple storage devices. A data storage array receives read and write requests from multiple client computers. The data storage array includes multiple storage devices, each with multiple allocation units (AUs). A storage controller within the data storage array determines a RAID layout for use in storing data. In response to determining a failure of a first AU, the storage controller begins reconstructing in a second AU the data stored in the first AU. For read and write requests targeting data in the first AU, the request is serviced by the first AU responsive to determining no error occurs when accessing the first AU.

Abstract: In one implementation, a method includes receiving code associated with two or more cores of a storage array controller. The method further includes determining, by the storage array controller, that the code is executable and read-only. The method further includes loading, based on the determination, the code into two or more memory pages corresponding to the two or more cores, wherein each of the two or more memory pages is local to each of the two or more cores, respectively.

Abstract: Determining active membership among a set of storage systems synchronously replicating a dataset, where determining active membership includes: determining that a membership event corresponds to a change in membership to the set of storage systems synchronously replicating the dataset; applying, in dependence upon the membership event, one or more membership protocols to determine a new set of storage systems to synchronously replicate the dataset; and for one or more I/O operations directed to the dataset, applying the one or more I/O operations to the dataset synchronously replicated by the new set of storage systems.