Abstract: A method, computer program product, and computer system for controlling, by a computing device, access to a non-volatile memory using a non-volatile lock as a reader of the non-volatile memory. Metadata (MD) non-volatile memory commits may be throttled until capacity of the non-volatile memory is at a threshold capacity.

Abstract: Processing can be performed to persistently record, in a log, a write I/O that writes first data to a target logical address. The processing can include: allocating storage for a first page buffer (PB) located at offsets in a PB pool of non-volatile storage of the log; enqueuing a request to an aggregation queue to persistently store the first data to the first PB of the log, wherein the request identifies the offsets of the PB pool of non-volatile storage which correspond to the first PB; and integrating the request into the aggregation queue. Integrating can include: determining whether a contiguous segment of the offsets of the request is adjacent to a second contiguous segment of the aggregation queue; and responsive to determining the contiguous segment is adjacent to the second contiguous segment, merging the first and second contiguous segments and generating an aggregated continuous segment.

Abstract: In at least one embodiment, processing can include: receiving a request for a transaction of MD (metadata) updates including a first MD update of a first MD update type of a first set and including a second MD update of a second MD update type of a second set; storing, in a first volatile MD log, the first MD update; storing, in a second volatile MD log, the second MD update; storing, in a first non-volatile MD log, the first MD update; and storing, in a second non-volatile MD log, the second MD update, wherein each MD update of the first volatile MD log and the first non-volatile MD log has a corresponding MD update type of the first set, and each MD update of the second volatile MD log and the second non-volatile MD log has a corresponding MD update type of the second set.

Abstract: At least one processing device comprises a processor and a memory coupled to the processor. The at least one processing device is configured to associate different classes of service with respective threads of one or more applications executing on at least one of a plurality of processing cores of a storage system, to configure different sets of prioritized thread queues for respective ones of the different classes of service, to enqueue particular ones of the threads associated with particular ones of the classes of service in corresponding ones of the prioritized thread queues, and to implement different dequeuing policies for selecting particular ones of the enqueued threads from the different sets of prioritized thread queues based at least in part on the different classes of service. The at least one processing device illustratively comprises at least a subset of the plurality of processing cores of the storage system.

Abstract: A technique is directed to processing metadata changes. The technique involves designating a first tablet to ingest metadata changes and a second tablet to destage previously ingested metadata changes, the first tablet being partitioned into a first reserved space and a first regular space that form a first memory pool, and the second tablet being partitioned into a second reserved space and a second regular space that form a second memory pool. The technique further involves, while the first tablet is designated to ingest metadata changes and the second tablet is designated to destage previously ingested metadata changes, ingesting metadata changes into the first tablet and destaging the previously ingested metadata changes from the second tablet. The technique further involves, when the first tablet becomes full, performing a switch operation that designates the second tablet to ingest metadata changes and the first tablet to destage previously ingested metadata changes.

Abstract: A method, computer program product, and computing system for generating a page buffer pool within a data journal of a storage node. A plurality of TO operations may be processed on a storage array using the storage node. A plurality of pages may be persisted in a plurality of page buffers within the page buffer pool based upon, at least in part, the processing of the plurality of TO operations. For each page buffer in the page buffer pool, a hash of the page buffer may be generated, thus defining a page buffer hash. The page buffer hash may be stored in a page descriptor associated with the page buffer.

Abstract: Techniques for address translation can include: performing first processing that maintains a first storage tier including first metadata pages of a hierarchical structure, and performing second processing that maintains a second storage tier including second metadata pages of the hierarchical structure. The first processing can include storing updated versions of the first metadata pages in place at static physical addresses of the first storage tier. The second processing can include storing updated versions of the second metadata pages at new physical storage locations. Prior to updating the second metadata pages, prior versions of the second metadata pages can be stored at other physical storage locations of the second storage tier, where the other physical storage location are different from the new physical storage locations. The first storage tier can be a non-parity RAID configuration such as RAID-1. The second storage tier can be a parity configuration such as RAID-5.

Abstract: Processing can be performed to persistently record, in a log, a write I/O that writes first data to a target logical address. The processing can include: allocating storage for a first page buffer (PB) located at offsets in a PB pool of non-volatile storage of the log; enqueuing a request to an aggregation queue to persistently store the first data to the first PB of the log, wherein the request identifies the offsets of the PB pool of non-volatile storage which correspond to the first PB; and integrating the request into the aggregation queue. Integrating can include: determining whether a contiguous segment of the offsets of the request is adjacent to a second contiguous segment of the aggregation queue; and responsive to determining the contiguous segment is adjacent to the second contiguous segment, merging the first and second contiguous segments and generating an aggregated continuous segment.

Abstract: At least one processing device comprises a processor and a memory coupled to the processor. The at least one processing device is configured to establish one or more groups of synchronization objects in a storage system based at least in part on object type, and for each of the one or more groups, to insert entries into a corresponding object type queue for respective objects of the group, to execute a monitor thread for the group, the monitor thread being configured to scan the entries of the corresponding object type queue, and responsive to at least one of the scanned entries meeting one or more designated conditions, to take at least one automated action for its associated object. The synchronization objects illustratively comprise respective locks, or other objects. The at least one processing device illustratively comprises at least a subset of a plurality of processing cores of the storage system.

Abstract: Data replication techniques can include receiving, at a source system, a write directed to a source logical device configured for asynchronous remote replication to a destination system; performing processing that flushes a transaction log entry for the write; and performing replication processing that uses a replication queue including a replication queue entry corresponding to the write that stores the first content to a logical address. The processing can create a replication log entry in a replication log for the write responsive to determining that the write is directed to the source logical device configured for asynchronous remote replication and that the first content has not been replicated. Responsive to the first content not being in cache, the first content can be retrieved using the reference to a storage location storing the first content. The reference can be obtained from the replication log entry or the replication queue entry.

Abstract: A shared portion of processor cores in a data storage system are allocated for sharing between a storage system application and a containerized service also executing in the data storage system. A non-shared portion of the processor cores are allocated for exclusive use by the storage system application. The storage system application preferentially uses processor cores in the shared portion of the processor cores to execute background tasks, and preferentially uses processor cores in the non-shared portion to perform host I/O request processing. The storage system application may from time to time voluntarily yield one or more of the processor cores in the shared portion of the processor cores for execution of the containerized service.

Abstract: Processing a read request to read metadata from an entry of a metadata page may include: determining whether the metadata page is cached; responsive to determining the metadata page is cached, obtaining the first metadata from the cached metadata page; responsive to determining the metadata page is not cached, determining whether the requested metadata is in a metadata log of metadata changes stored in a volatile memory; and responsive to determining the metadata is the metadata log of metadata changes stored in the volatile memory, obtaining the requested metadata from the metadata log. Processing a write request that overwrites an existing value of a metadata page with an updated value may include: recording a metadata change in the metadata log that indicates to update the metadata page with the updated value; and performing additional processing during destaging that uses the existing value prior to overwriting it with the updated value.

Abstract: Techniques for efficiently flushing a user data log may postpone or delay establishing chains of metadata pages used as mapping information to map logical addresses to storage locations of content stored at the logical addresses. Processing can include: receiving a write operation that writes data to a logical address; storing an entry for the write operation in the user data log; and flushing the entry from the user data log. Flushing can include storing a metadata log entry in a metadata log, wherein the metadata log entry represents a binding of the logical address to a data block including the data stored at the logical address; and destaging the metadata log entry. Destaging can include updating mapping information used to map the logical address to the data block. The mapping information can include a metadata page in accordance with the metadata log entry.

Abstract: A request to perform a first operation in a system that stores deduplicated data can be received. The system can include a data block stored at multiple logical address each referencing the data block. A reference count can be associated with the data block and can denote a number of logical addresses referencing the data block. Processing can be performed to service the request and perform the first operation, wherein the processing can include: acquiring a non-exclusive lock for a page that includes the reference count of the data block; storing, in a metadata log while holding the non-exclusive lock on the page, an entry to decrement the reference count of the data block; and releasing the non-exclusive lock on the page.

Abstract: A protocol for processing write operations can include recording each write operation in a log using a PDESC (page descriptor)-PB (page block) pair. The log entry for the write operation can be included in a container of logged writes. In a dual node system, the protocol when processing the write operation, that writes first data, can include incrementing a corresponding one of two counters of the container, where the corresponding counter is associated with one of the system's nodes which received the write operation and and caches the first data. Each container can be associated with an logical block address (LBA) range of a logical device, where logged writes that write to target addresses in the particular LBA range are included in the container. Nodes can independently determine flush ownership using the container's counters and can flush containers based on the flush ownership.

Abstract: A system can use non-volatile solid state drives (SSDs) to provide storage. The SSDs can implement internal log structured systems (LSSs). A reversible write operation can be serviced by an SSD to write first data to an SSD logical address. The reversible write operation can update the SSD logical address to store the first data rather than old data stored at the SSD logical address prior to servicing the reversible write operation. The old data can be stored at an SSD physical address and mapping information indicates the SSD logical address is mapped to the SSD physical address. Servicing the reversible write operation can include: retaining the mapping information and the old data; and adding second mapping information that maps the SSD logical address to a second SSD physical address storing the first data. A subsequent read operation can read the old data using the retained mapping information.

Abstract: A protocol for processing write operations can include recording each write operation in a log using a PDESC (page descriptor)-PB (page block) pair. The log entry for the write operation can be included in a container of logged writes. In a dual node system, the protocol when processing the write operation, that writes first data, can include incrementing a corresponding one of two counters of the container, where the corresponding counter is associated with one of the system's nodes which received the write operation and and caches the first data. Each container can be associated with an logical block address (LBA) range of a logical device, where logged writes that write to target addresses in the particular LBA range are included in the container. Nodes can independently determine flush ownership using the container's counters and can flush containers based on the flush ownership.

Abstract: Nodes in a storage system can autonomously ingest I/O requests and flush data to storage. First and second nodes determine a sequence separator, the sequence separator corresponding to an entry in a page descriptor ring that separates two flushing work sets (FWS). The first node receives an input/output (I/O) request and allocates a sequence identification (ID) number to the I/O request. The first node determines a FWS for the I/O request based on the sequence separator and the sequence ID number, and commits the I/O request using the sequence ID number. The I/O request and the sequence ID number are sent to the second node.

Abstract: A method of performing write operations that have been received by a data storage apparatus is provided. The method includes (a) storing page descriptors for received write operations within temporary storage, each page descriptor indicating respective data to be written; (b) upon storing each page descriptor, organizing that page descriptor into a shared working-set structure; and (c) operating a plurality of flushers to persist the data indicated by respective page descriptors to long-term persistent storage based on organization of the page descriptors in the shared working-set structure, each flusher accessing page descriptors via the shared working-set structure. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A method, computer program product, and computing system for defining a queue. The queue may be based on a linked list and may be a first-in, first-out (FIFO) queue that may be configured to be use used with multiple producers and a single consumer. The queue may include a plurality of queue elements. A tail element and a head element may be defined from the plurality of elements within the queue. The tail element may point to a last element of the plurality of elements and the head element may point to a first element of a plurality of elements. An element may be dequeued from the tail element, which may include determining if the tail element is in a null state. An element may be enqueued to the head element, which may include adding a new element to the queue.