Abstract: A method is used in managing storage operations in storage systems. Based on a set of criteria, an amount of storage resources required to perform a storage operation is determined. The storage operation is directed to fault tolerant storage devices. The amount of storage resources is allocated prior to start performing the storage operation. The storage operation is performed by using the allocated storage resources.

Abstract: A computer-implemented method for remote direct memory access (RDMA) by a distributed storage node includes receiving a request for an input/output (I/O) process associated with data. In response to the request, a memory segment shared between the operating system and a user process is allocated using an operating system driver of the node. The user process includes an I/O stack for processing the request. The shared memory segment includes a context memory portion storing context information associated with the I/O stack, a header memory portion storing header information for the I/O process, and a data memory portion for storing the data. The shared memory segment is registered for RDMA access with a target storage node. An RDMA transfer is performed between the shared memory segment and the target node to complete the I/O process. The shared memory segment is deallocated upon completion of the RDMA transfer.

Abstract: Techniques for creating snapshots of data storage objects that can perform certain operations (e.g., flushing dirty data, setting up extent pointers, allocating block storage space, etc.) during background (or deferred) processing. The disclosed techniques employ one or more extent copy trackers that can be created during processing of a transaction, while I/O request from host computers are suspended. The extent copy trackers are configured to perform some or all of the certain operations in the background, after the transaction has been committed and/or the processing of the transaction has been completed. By performing such operations during background processing, a processing time required to complete the snapshot transaction is reduced, thereby reducing latency in the resumption of the I/O requests from the host computers.

Abstract: A computer-implemented method for remote direct memory access (RDMA) by a distributed storage node includes receiving a request for an input/output (I/O) process associated with data. In response to the request, a memory segment shared between the operating system and a user process is allocated using an operating system driver of the node. The user process includes an I/O stack for processing the request. The shared memory segment includes a context memory portion storing context information associated with the I/O stack, a header memory portion storing header information for the I/O process, and a data memory portion for storing the data. The shared memory segment is registered for RDMA access with a target storage node. An RDMA transfer is performed between the shared memory segment and the target node to complete the I/O process. The shared memory segment is deallocated upon completion of the RDMA transfer.

Abstract: A data storage system includes interface circuitry receiving data storage requests and sending corresponding responses to data storage clients, and data storage devices providing non-volatile storing and retrieval of client data in response to the data storage requests. Processing circuitry executes software-implemented datapath modules that realize datapath functions to process the data storage requests and responses, as well as service infrastructure modules to realize a service infrastructure to provide for regularized communications among the datapath modules, including Notify, Monitor, and StartIO communications.

Abstract: A technique for managing RAID storage in a data storage system provides a mapping subsystem and a RAID subsystem and employs the mapping subsystem to direct repair operations on damaged RAID stripes. The mapping subsystem stores metadata that provides information about data stored in the RAID subsystem and locations of that data on RAID stripes. In response to detection of a damaged RAID stripe, the mapping subsystem selectively determines, based on the metadata, whether to perform repair operations or to avoid repair operations. As many repair operations can safely be avoided, the disclosed technique has the effect of reducing unnecessary processing in the data storage system. When the RAID subsystem includes flash drives, the technique also reduces write amplification, thus preserving flash drives which might otherwise be subject to premature wear.

Abstract: A technique for managing RAID (Redundant Array of Independent Disks) storage includes maintaining active-stripe metadata that stores, for each of multiple stripes that have been written by a RAID system, an entry that identifies the stripe and identifies a respective configuration plan in effect a last time the stripe was written. In response to a disk drive failure, the technique further includes generating a set of new plans that specify disk drive elements to be used in place of damaged ones and performing a selective repair operation. The selective repair operation iterates over a set of entries in the active-stripe metadata, performs a comparison between the identified plan for each stripe and a current plan implemented by the RAID system for writing to that stripe, and repairs the stripe when the identified plan and the current plan are different.

Abstract: Techniques for managing restriping of data across drives of a data storage system in a mapped-RAID environment. In response to addition of a drive to existing drives of the data storage system, all drive extents of the data storage system are mapped out in an effort to consume as much free space as possible across the drives. Having mapped out the drive extents, how many free extents remain on each drive is determined. If a quantity of free extents on a drive exceeds a predetermined threshold number, then that drive is logically marked as requiring restriping. Restriping of data is then performed for all logically marked drives. Once the restriping of the data is completed, a determination is made as to whether any further restriping of data across the drives of the data storage system is required, taking into account a relative wear and/or utilization of the respective physical drives.

Abstract: A data storage system includes interface circuitry receiving data storage requests and sending corresponding responses to data storage clients, and data storage devices providing non-volatile storing and retrieval of client data in response to the data storage requests. Processing circuitry executes software-implemented datapath modules that realize datapath functions to process the data storage requests and responses, as well as service infrastructure modules to realize a service infrastructure to provide for regularized communications among the datapath modules, including Notify, Monitor, and StartIO communications.

Abstract: A method, computer program product, and computing system for receiving an IO request for a RAID array. An actual queue depth of the RAID array is compared to a maximum queue depth for the RAID array. An IO queue weight is determined for the IO request. The IO queue weight is revised based upon a degrade mode condition for the RAID array, thus defining a revised IO queue weight. The actual queue depth is incremented based upon the revised IO queue weight.

Abstract: A mapping apparatus comprises a mapper that translates from an input key to an output key in one or more storage devices. A pre-mapper for processing update operations comprises a plurality of mapping tablets and an in-memory look-up filter to identify a given mapping table storing a given input key. The mapping tablets comprise at least one dynamic in-memory ingress tablet and a plurality of persisted frozen tablets. For a given update operation, a first entry is added to one dynamic in-memory ingress tablet comprising the input key for the given update operation and a corresponding output key where data for the given update operation is stored; and a second entry is added to the look-up filter comprising the input key of the first entry and an identifier of the dynamic in-memory ingress tablet storing the first entry for the given update operation. The dynamic in-memory ingress tablet is persisted as a persisted frozen tablet.

Abstract: A technique for maintaining RAID (redundant array of independent disks) configuration metadata across multiple SPs (storage processors) includes receiving a change request by a controller within a first SP, writing, by the first SP, a RAID configuration change described by the change request to a persistent intent log, and informing a second SP that the intent log has been written. The second SP, upon being informed of the write to the intent log, reads the RAID configuration change from the intent log and writes the RAID configuration change to a persistent configuration database. In this manner, the first SP and the second SP both receive the RAID configuration change and thus are both equipped to service reads and writes directed to a affected RAID storage.

Abstract: A technique for managing metadata in a data storage system designates block pointers as either sources or copies, where sources contribute to reference counts of pointed-to structures but copies do not. The technique maintains parent-child relationships between parent BPSs (block pointer sets) and child BPSs, where each BPS includes an array of block pointers. Each child BPS is created as a copy of a parent BPS and has block pointers initially designated as copies. The technique performs a metadata-merge operation to merge the block pointers of the parent BPS into those of a child BPS by promoting attributes of block pointers in the child BPS from copy to source, avoiding any need to perform reference count updates on structures pointed to by promoted block pointers.

Abstract: A method is used in managing mapping metadata in storage systems. A request is received to access data organized on a storage device of a storage system. Checksum information is used for validating the data. The checksum information is stored on the storage system separate from a location at which the data has been stored.

Abstract: Techniques are disclosed for drive zeroing that create and maintain a memory resident zeroing map for each drive in a data storage system. The disclosed techniques create, for each drive in an array of non-volatile data storage devices, a zeroing map. The zeroing map for a drive includes multiple slots, each of which corresponds to a data storage area of the drive. Each slot in the zeroing map for a drive stores a zeroing status of the corresponding data storage area. The value of a slot in the zeroing map may indicate that the zeroing status of the corresponding data storage area is i) zeroed, indicating that the corresponding data storage area has previously been zeroed, or ii) unknown, indicating that the zeroing status of the corresponding data is unknown, i.e. the corresponding data storage area may or may not have previously been zeroed.

Abstract: Techniques are disclosed for using virtual drives to provide RAID groups in a data storage system. A pool of virtual drives is generated by i) dividing each one of a set of physical drives into multiple regions of physically contiguous non-volatile data storage, and ii) allocating individual regions of physically contiguous non-volatile data storage to corresponding virtual drives contained in a pool of virtual drives. RAID groups are then formed, where each one of the RAID groups is made up of a set of virtual drives assigned to the RAID group from the pool of virtual drives. One or more storage objects is then mapped to each RAID group, and host I/O operations received by the data storage system and directed to the storage object mapped to the RAID group are serviced using the set of virtual drives assigned to the RAID group.

Abstract: A technique processes input/output (I/O) requests from a set of host computers. The technique utilizes data storage equipment which includes a set of physical storage drives and storage processing circuitry which accesses the set of physical storage drives. The technique involves storing, by the storage processing circuitry, a set of storage drive objects in main memory. The set of storage drive objects includes a set of zeroed chunk tables. Each storage drive object (i) represents a physical storage drive and (ii) includes a zeroed chunk table which identifies storage chunks of that physical storage drive that have been initialized to zero. The technique further involves receiving, by the storage processing circuitry, I/O requests from the set of host computers, and performing, by the storage processing circuitry, I/O tasks in response to the I/O requests. The I/O tasks include zero on demand operations based on the set of zeroed chunk tables.

Abstract: A method is used in managing storage device mappings in storage systems. A set of mappings is created in a distributed storage system. The distributed storage system stores data in a redundant manner at more than one location on a set of storage devices by using a set of stripes. Each stripe of the set of stripes indicates a data slice stored on a storage device. Each mapping of the set of mappings identifies information regarding location of data on a storage device for a stripe. The mapping is stored on the set of storage devices.

Abstract: A technique is directed to providing resiliency to a redundant array of independent disk (RAID) group which includes multiple storage devices. The technique involves operating the RAID group in a normal state in which each storage device is (i) initially online to perform write and read operations and (ii) configured to go offline in response to a respective media error count for that storage device reaching an initial take-offline threshold. The technique further involves receiving a notification that a storage device of the RAID group has encountered a particular error situation. The technique further involves transitioning, in response to the notification, the RAID group to a high resiliency state in which each storage device that is operable is (i) still online to perform write and read operations and (ii) configured to stay online even when the respective media error count for that storage device reaches the initial take-offline threshold.

Abstract: The techniques presented herein provide for verifying the integrity of an encryption key log file generated on a data storage system. Encryption key activity events associated with a storage system's back-end storage drives are identified. A unique signature is generated for each encryption key activity event. Each encryption key activity event and its corresponding signature are stored in an audit log file. An audit log hash file is generated using the contents of the audit log file. At an external location, the audit log file and the audit log hash file are retrieved from the storage system. The integrity of the retrieved audit log file is verified by generating a local audit log hash file and comparing the local audit log hash file to the retrieved audit log hash file and determining if the local audit log hash file matches the retrieved audit log hash file.