Abstract: Systems and methods for upgrading container images within orchestration systems. In various embodiments, a request to run a custom container image can be received. Subsequently, a determination can be made as to whether the current version of the requested custom container image is built upon a most recent version of a corresponding base container image. Where it is determined that the current version of the requested custom container image is not built upon the most recent version of the corresponding base container image, an upgraded version of the custom container image can be generated. Advantageously, a non-privileged user requesting a custom container image can, leveraging the private access token and/or administrative privileges of the system, be able to upgrade the requested custom container image. Custom container images can be automatically upgraded, and relied upon to be up-to-date when used, without burdening the user.

Abstract: Protecting sensitive data in snapshots, including: creating a transformed snapshot portion by applying a transformation specified in an access policy to one or more data objects contained within the portion of the stored snapshot, wherein the stored snapshot is a copy of data in a storage system at a particular point in time prior to a request to access the snapshot; and providing access to the transformed snapshot portion.

Abstract: Utilizing a non-repeating identifier to encrypt data, including: receiving a request to write data to a storage device; selecting a segment-offset pair where the data will be stored, where the selected segment-offset pair is unique to every other segment-offset pair utilized during the lifetime of the storage device; and encrypting the data in dependence upon an identifier of the segment-offset pair.

Abstract: Performing partial redundant array of independent disks (RAID) stripe parity calculations, including: receiving a last portion of a RAID stripe among multiple portions of the RAID stripe, all portions for a successful write of the RAID stripe being previously received except for the last portion; calculating a parity value based on the last portion of the RAID stripe and a previous parity value without calculating the parity value using a previous portion of the RAID stripe; and writing of the RAID stripe.

Abstract: A method of performing partial redundant array of independent disks (RAID) stripe parity calculations is disclosed. The method includes receiving a last portion of a RAID stripe among multiple portions of the RAID stripe, all portions for a successful write of the RAID stripe being previously received except for the last portion. The method also includes calculating a parity value based on the last portion of the RAID stripe and a previous parity value without calculating the parity value using a previous portion of the RAID stripe. The method further includes writing of the RAID stripe.

Abstract: A system and method for efficiently maintaining metadata stored among a plurality of solid-state storage devices. A data storage subsystem supports multiple mapping tables. Records within a mapping table are arranged in multiple levels. Each level stores at least pairs of a key value and a physical pointer value. The levels are sorted by time. New records are inserted in a created new highest (youngest) level. No edits are performed in-place. A data storage controller determines both a cost of searching a given table exceeds a threshold and an amount of memory used to flatten levels exceeds a threshold. In response, the controller incrementally flattens selected levels within the table based on key ranges. After flattening the records in the selected levels within the key range, the records may be removed from the selected levels. The process repeats with another different key range.

Abstract: Migrating similar data to a single data reduction pool, including: determining that storage space consumption may be reduced by migrating similar data between a first storage system and a second storage system; and initiating a migration of the similar data, including selecting a migration direction from one of either migrating the first data from the first storage system to the second storage system and migrating the second data from the second storage system to the first storage system.

Abstract: In one implementation, a method includes identifying a first content-dependent feature associated with a data sector. The method further includes determining a baseline data sector associated with the data sector. The method further includes determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector. The method further includes providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.

Abstract: Some embodiments provide a method gaining insight into applicability of policies that authorize access to at least one service through application programming interface (API) calls by a plurality of users. The method receives an authentication policy that defines multiple users of a system providing the service, and also receives an authorization policy that defines access to the service by the users. The method generates an authorization policy for defining access to the service by authenticated users by combining the first and second policies. The method receives a query regarding access to the service from a particular set of one or more users, and uses the third policy to provide a response to the query that describes access to the service for the particular user set.

Abstract: Storage efficiency driven migration includes: determining a level of similarity between first data stored on a first storage system and second data stored on a second storage system; determining, in dependence upon the level of similarity, that an expected amount of storage space reduction from migrating similar data exceeds a threshold level; and responsive to determining that the expected amount of storage space reduction exceeds the threshold level, initiating a migration of one or more portions of the first data from the first storage system to the second storage system.

Abstract: A system and method for efficiently storing data in a storage system. A data storage subsystem includes multiple data storage locations on multiple storage devices in addition to at least one mapping table. A data storage controller determines whether data to store in the storage subsystem has one or more patterns of data intermingled with non-pattern data within an allocated block. Rather than store the one or more pattern on the storage devices, the controller stores information in a header on the storage devices. The information includes at least an offset for the first instance of a pattern, a pattern length, and an identification of the pattern. The data may be reconstructed for a corresponding read request from the information stored in the header.

Abstract: Protecting an encryption key for data stored in a storage system that includes a plurality of storage devices, including: reading, from at least a majority of the storage devices, a portion of an apartment key; reconstructing the apartment key using the portions of the apartment key read by the majority of the storage devices; unlocking the main portion of each of the storage devices utilizing the apartment key; reading, from the main portion of one of the storage devices, a portion of a third-party resource access key; requesting, from the third-party resource utilizing the third-party resource access key, an encryption key; receiving, from the third-party resource, the encryption key; and decrypting the data stored on the storage devices utilizing the encryption key.

Abstract: In one implementation, a method includes identifying a first content-dependent feature associated with a data sector. The method further includes determining a baseline data sector associated with the data sector. The method further includes determining, by a processing device, a content-dependent delta between the first content-dependent feature and a second content-dependent feature of the baseline data sector. The method further includes providing the content-dependent delta and an indicator to the baseline data sector for storage on a plurality of storage devices.

Abstract: Performing partial redundant array of independent disks (RAID) stripe parity calculations, including: receiving a last portion of a RAID stripe among multiple portions of the RAID stripe, all portions for a successful write of the RAID stripe being previously received except for the last portion; calculating a parity value based on the last portion of the RAID stripe and a previous parity value without calculating the parity value using a previous portion of the RAID stripe; and writing of the RAID stripe.

Abstract: Some embodiments provide a method for gaining insight into applicability of policies that authorize access to at least one service through application programming interface (API) calls by a plurality of users. The method receives at least one authorization policy that defines access to the service by the users, where the service includes multiple resources. The method identifies combinations of users and resources referenced by the policy, and for each identified combination of user and resource, executes the policy in order to define access to the identified resource by the identified user. The method receives a query regarding access to a particular resource from a particular set of one or more users, and uses the executed policy to provide a response to the query that describes access to the particular resource for the particular user set.

Abstract: Some embodiments provide a method gaining insight into applicability of policies that authorize access to at least one service through application programming interface (API) calls by a plurality of users. The method receives an authentication policy that defines multiple users of a system providing the service, and also receives an authorization policy that defines access to the service by the users. The method generates an authorization policy for defining access to the service by authenticated users by combining the first and second policies. The method receives a query regarding access to the service from a particular set of one or more users, and uses the third policy to provide a response to the query that describes access to the service for the particular user set.

Abstract: Some embodiments provide a method for gaining insight into applicability of policies that authorize access to at least one service through application programming interface (API) calls by multiple users. The method receives at least one authorization policy that defines access to the service by the users, where the service includes multiple resources. Based on an analysis of the received policy, the method identifies a set of two or more access rules, each access rule associating at least one user to at least one resource. The method receives a query regarding access to a particular resource from a particular set of one or more users, and uses the identified access rules to provide a response to the query that describes access to the particular resource for the particular user set.

Abstract: Protecting an encryption key for data stored in a storage system that includes a plurality of storage devices, including: reading, from at least a majority of the storage devices, a portion of an apartment key; reconstructing the apartment key using the portions of the apartment key read by the majority of the storage devices; unlocking the main portion of each of the storage devices utilizing the apartment key; reading, from the main portion of one of the storage devices, a portion of a third-party resource access key; requesting, from the third-party resource utilizing the third-party resource access key, an encryption key; receiving, from the third-party resource, the encryption key; and decrypting the data stored on the storage devices utilizing the encryption key.

Abstract: A system and method for efficiently maintaining metadata stored among a plurality of solid-state storage devices. A data storage subsystem supports multiple mapping tables. Records within a mapping table are arranged in multiple levels. Each level stores at least pairs of a key value and a physical pointer value. The levels are sorted by time. New records are inserted in a created new highest (youngest) level. No edits are performed in-place. A data storage controller determines both a cost of searching a given table exceeds a threshold and an amount of memory used to flatten levels exceeds a threshold. In response, the controller incrementally flattens selected levels within the table based on key ranges. After flattening the records in the selected levels within the key range, the records may be removed from the selected levels. The process repeats with another different key range.

Abstract: Protecting sensitive data in snapshots, including: creating a transformed snapshot portion by applying a transformation specified in an access policy to one or more data objects contained within the portion of the stored snapshot, wherein the stored snapshot is a copy of data in a storage system at a particular point in time prior to a request to access the snapshot; and providing access to the transformed snapshot portion.