Abstract: Methods for RAID stripe update parity calculation are disclosed. The methods include: receiving a first portion of data of a RAID stripe for writing to a first memory location of a plurality of solid state drives. A first parity value for the first portion of data of the RAID stripe is calculated. The methods include receiving a second portion of data of the RAID stripe for writing to a second memory location that is different from the first memory location. A second parity value is calculated based upon the second portion of the data and the first parity value Based on successfully writing the second portion of data of the RAID stripe, the first parity value is replaced with the second parity value.

Abstract: Content masking within a storage system includes: responsive to receiving a first request to access a portion of a stored snapshot, creating a transformed snapshot portion by applying a transformation specified in an access policy to one or more data objects contained within the portion of the stored snapshot; and presenting the transformed snapshot portion.

Abstract: A system and method for efficiently starting up a plurality of solid-state storage devices. A computing system includes one or more storage devices storing data in multiple allocation units (AUs). In a boot region, a data storage controller maintains an AU management set that identifies a state of various AUs in the system. In various embodiments, the management set includes an identification of a subset of free AUs, transitional AU, unincorporated AUs, and speculative AUs. At various times, information corresponding to the AU management set is stored to non-volatile storage. During a boot sequence, the AU management set information is accessed and the identified AUs are searched to identify allocated AUs rather than performing a search of all of the AUs in the system.

Abstract: Some embodiments provide a method for providing insight into applicability of policies that authorize access to at least one service through application programming interface (API) calls by multiple users. The method receives at least one authorization policy that defines access to the service by the users, where the policy includes two or more access rules. The method identifies a subset of unnecessary access rules in the received policy, based on a set of contextual data that is associated with the users, and filters the received policy by removing the identified subset of unnecessary access rules. The method receives a query regarding access to the service from a particular set of one or more users, and uses the filtered policy to provide a response to the query that describes access to the service for the particular user set.

Abstract: A system and method for efficiently starting up a plurality of solid-state storage devices. A computing system includes one or more storage devices storing data in multiple allocation units (AUs). In a boot region, a data storage controller maintains an AU management set that identifies a state of various AUs in the system. In various embodiments, the management set includes an identification of a subset of free AUs, transitional AU, unincorporated AUs, and speculative AUs. At various times, information corresponding to the AU management set is stored to non-volatile storage. During a boot sequence, the AU management set information is accessed and the identified AUs are searched to identify allocated AUs rather than performing a search of all of the AUs in the system.

Abstract: Some embodiments provide a method for gaining insight into authorization policy enforcement for application programming interface (API) calls to at least one service that includes multiple resources. The method generates a permissions graph including nodes that represent the resources and multiple users, based on two or more received authorization policies that restrict access to the service for the users. The method receives a selection of a node that corresponds to a user, and in response to the received selection, modifies the graph to display connections between the node corresponding to the user and one or more nodes associated with resources of the service that the user is authorized to access based on the authorization policies.

Abstract: In one implementation, a method includes identifying a first content-dependent feature associated with a data sector. The method further includes determining a baseline data sector associated with the data sector. The method further includes determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector. The method further includes providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.

Abstract: A method and apparatus for incremental RAID stripe update parity calculations. The method includes: receiving, at a first set of solid state drives, a last portion of a redundant array of independent disks (RAID) stripe among multiple portions of the RAID stripe, wherein the RAID stripe includes multiple shards, and wherein each previous portion of the RAID stripe is written to the first set of solid state drives; calculating a current parity value based on the last portion of the RAID stripe and a previous parity value updated after receiving each previous portion of the RAID stripe; and responsive to receiving all portions of a shard of the RAID stripe, copying the shard of the RAID stripe from the first set of solid state drives to a second set of solid state drives.

Abstract: A system and method for efficiently distributing data among multiple storage devices. A data storage array receives read and write requests from multiple client computers. The data storage array includes multiple storage devices, each with multiple allocation units (AUs). A storage controller within the data storage array determines a RAID layout for use in storing data. In response to determining a failure of a first AU, the storage controller begins reconstructing in a second AU the data stored in the first AU. For read and write requests targeting data in the first AU, the request is serviced by the first AU responsive to determining no error occurs when accessing the first AU.

Abstract: In one implementation, a method includes identifying, by a storage system controller associated with a plurality of storage devices, a first content-dependent feature associated with a data sector and determining, by the storage system controller, a baseline data sector associated with the data sector. The method further includes determining, by the storage system controller, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector and providing the content-dependent delta and an indicator to the baseline data sector for storage on the plurality of storage devices.

Abstract: A system and method for efficiently storing data in a storage system. A data storage subsystem includes multiple data storage locations on multiple storage devices in addition to at least one mapping table. A data storage controller determines whether data to store in the storage subsystem has one or more patterns of data intermingled with non-pattern data within an allocated block. Rather than store the one or more pattern on the storage devices, the controller stores information in a header on the storage devices. The information includes at least an offset for the first instance of a pattern, a pattern length, and an identification of the pattern. The data may be reconstructed for a corresponding read request from the information stored in the header.

Abstract: In one implementation, a method includes detecting, by a storage system controller associated with a plurality of storage devices, a first anomaly corresponding to a first data shard of a redundant array of independent disks (RAID) group and determining, by the storage system controller, that the first anomaly corresponding to the first data shard is not to be corrected within the RAID group. The method further includes removing the first data shard from the RAID group and recalculating, by the storage system controller, one or more error recovery codes corresponding to the RAID group from a plurality of remaining data shards of the RAID group, without the first data shard.

Abstract: Utilizing a non-repeating identifier to encrypt data, including: receiving a request to write data to a storage device; selecting a segment-offset pair where the data will be stored, where the selected segment-offset pair is unique to every other segment-offset pair utilized during the lifetime of the storage device; and encrypting the data in dependence upon an identifier of the segment-offset pair.

Abstract: On-demand content filtering of snapshots within a storage system, including: associating an access policy with a snapshot, the access policy specifying a transformation to apply to a predefined data object; receiving a first request to access a portion of the snapshot; and responsive to receiving the first request: creating a transformed snapshot portion by applying the transformation specified in the access policy to one or more data objects contained within the portion of the snapshot; and presenting the transformed snapshot portion.

Abstract: Protecting an encryption key for data stored in a storage system that includes a plurality of storage devices, including: reading, from at least a majority of the storage devices, a portion of an apartment key; reconstructing the apartment key using the portions of the apartment key read by the majority of the storage devices; unlocking the main portion of each of the storage devices utilizing the apartment key; reading, from the main portion of one of the storage devices, a portion of a third-party resource access key; requesting, from the third-party resource utilizing the third-party resource access key, an encryption key; receiving, from the third-party resource, the encryption key; and decrypting the data stored on the storage devices utilizing the encryption key.

Abstract: Incremental RAID stripe update parity calculation includes receiving a first portion of data of a RAID stripe for writing to a first memory location of a plurality of solid state drives. A first parity value is calculated for the first portion of data of the RAID stripe. A second portion of data of the RAID stripe is received for writing to a second memory location that is different from the first memory location A second parity value is calculated in dependence upon the second portion of the data of the RAID stripe and upon the first parity value Responsive to successfully writing the second portion of data of the RAID stripe, the first parity value is replaced with the second parity value.

Abstract: A system and method for efficiently starting up a plurality of solid-state storage devices. A computing system includes one or more storage devices storing data in multiple allocation units (AUs). In a boot region, a data storage controller maintains an AU management set that identifies a state of various AUs in the system. In various embodiments, the management set includes an identification of a subset of free AUs, transitional AU, unincorporated AUs, and speculative AUs. At various times, information corresponding to the AU management set is stored to non-volatile storage. During a boot sequence, the AU management set information is accessed and the identified AUs are searched to identify allocated AUs rather than performing a search of all of the AUs in the system.

Abstract: A system and method for efficiently maintaining metadata stored among a plurality of solid-state storage devices. A data storage subsystem supports multiple mapping tables. Records within a mapping table are arranged in multiple levels. Each level stores at least pairs of a key value and a physical pointer value. The levels are sorted by time. New records are inserted in a created new highest (youngest) level. No edits are performed in-place. A data storage controller determines both a cost of searching a given table exceeds a threshold and an amount of memory used to flatten levels exceeds a threshold. In response, the controller incrementally flattens selected levels within the table based on key ranges. After flattening the records in the selected levels within the key range, the records may be removed from the selected levels. The process repeats with another different key range.

Abstract: A system and method for efficiently distributing data among multiple storage devices. A data storage array receives read and write requests from multiple client computers. The data storage array includes multiple storage devices, each with multiple allocation units (AUs). A storage controller within the data storage array determines a RAID layout for use in storing data. In response to determining a failure of a first AU, the storage controller begins reconstructing in a second AU the data stored in the first AU. For read and write requests targeting data in the first AU, the request is serviced by the first AU responsive to determining no error occurs when accessing the first AU.

Abstract: Incremental RAID stripe update parity calculation includes: receiving a first portion of data of a RAID stripe for writing to a first memory location of a plurality of solid state drives; calculating a first parity value for the first portion of data of the RAID stripe; receiving a second portion of data of the RAID stripe for writing to a second memory location that is different from the first memory location; calculating a second parity value in dependence upon the second portion of the data of the RAID stripe and upon the first parity value; and responsive to successfully writing the second portion of data of the RAID stripe, replacing the first parity value with the second parity value.