Abstract: Data replication techniques can include receiving, at a source system, a write directed to a source logical device configured for asynchronous remote replication to a destination system; performing processing that flushes a transaction log entry for the write; and performing replication processing that uses a replication queue including a replication queue entry corresponding to the write that stores the first content to a logical address. The processing can create a replication log entry in a replication log for the write responsive to determining that the write is directed to the source logical device configured for asynchronous remote replication and that the first content has not been replicated. Responsive to the first content not being in cache, the first content can be retrieved using the reference to a storage location storing the first content. The reference can be obtained from the replication log entry or the replication queue entry.

Abstract: A shared portion of processor cores in a data storage system are allocated for sharing between a storage system application and a containerized service also executing in the data storage system. A non-shared portion of the processor cores are allocated for exclusive use by the storage system application. The storage system application preferentially uses processor cores in the shared portion of the processor cores to execute background tasks, and preferentially uses processor cores in the non-shared portion to perform host I/O request processing. The storage system application may from time to time voluntarily yield one or more of the processor cores in the shared portion of the processor cores for execution of the containerized service.

Abstract: Processing a read request to read metadata from an entry of a metadata page may include: determining whether the metadata page is cached; responsive to determining the metadata page is cached, obtaining the first metadata from the cached metadata page; responsive to determining the metadata page is not cached, determining whether the requested metadata is in a metadata log of metadata changes stored in a volatile memory; and responsive to determining the metadata is the metadata log of metadata changes stored in the volatile memory, obtaining the requested metadata from the metadata log. Processing a write request that overwrites an existing value of a metadata page with an updated value may include: recording a metadata change in the metadata log that indicates to update the metadata page with the updated value; and performing additional processing during destaging that uses the existing value prior to overwriting it with the updated value.

Abstract: Techniques for efficiently flushing a user data log may postpone or delay establishing chains of metadata pages used as mapping information to map logical addresses to storage locations of content stored at the logical addresses. Processing can include: receiving a write operation that writes data to a logical address; storing an entry for the write operation in the user data log; and flushing the entry from the user data log. Flushing can include storing a metadata log entry in a metadata log, wherein the metadata log entry represents a binding of the logical address to a data block including the data stored at the logical address; and destaging the metadata log entry. Destaging can include updating mapping information used to map the logical address to the data block. The mapping information can include a metadata page in accordance with the metadata log entry.

Abstract: A request to perform a first operation in a system that stores deduplicated data can be received. The system can include a data block stored at multiple logical address each referencing the data block. A reference count can be associated with the data block and can denote a number of logical addresses referencing the data block. Processing can be performed to service the request and perform the first operation, wherein the processing can include: acquiring a non-exclusive lock for a page that includes the reference count of the data block; storing, in a metadata log while holding the non-exclusive lock on the page, an entry to decrement the reference count of the data block; and releasing the non-exclusive lock on the page.

Abstract: A protocol for processing write operations can include recording each write operation in a log using a PDESC (page descriptor)-PB (page block) pair. The log entry for the write operation can be included in a container of logged writes. In a dual node system, the protocol when processing the write operation, that writes first data, can include incrementing a corresponding one of two counters of the container, where the corresponding counter is associated with one of the system's nodes which received the write operation and and caches the first data. Each container can be associated with an logical block address (LBA) range of a logical device, where logged writes that write to target addresses in the particular LBA range are included in the container. Nodes can independently determine flush ownership using the container's counters and can flush containers based on the flush ownership.

Abstract: A system can use non-volatile solid state drives (SSDs) to provide storage. The SSDs can implement internal log structured systems (LSSs). A reversible write operation can be serviced by an SSD to write first data to an SSD logical address. The reversible write operation can update the SSD logical address to store the first data rather than old data stored at the SSD logical address prior to servicing the reversible write operation. The old data can be stored at an SSD physical address and mapping information indicates the SSD logical address is mapped to the SSD physical address. Servicing the reversible write operation can include: retaining the mapping information and the old data; and adding second mapping information that maps the SSD logical address to a second SSD physical address storing the first data. A subsequent read operation can read the old data using the retained mapping information.

Abstract: A protocol for processing write operations can include recording each write operation in a log using a PDESC (page descriptor)-PB (page block) pair. The log entry for the write operation can be included in a container of logged writes. In a dual node system, the protocol when processing the write operation, that writes first data, can include incrementing a corresponding one of two counters of the container, where the corresponding counter is associated with one of the system's nodes which received the write operation and and caches the first data. Each container can be associated with an logical block address (LBA) range of a logical device, where logged writes that write to target addresses in the particular LBA range are included in the container. Nodes can independently determine flush ownership using the container's counters and can flush containers based on the flush ownership.

Abstract: Nodes in a storage system can autonomously ingest I/O requests and flush data to storage. First and second nodes determine a sequence separator, the sequence separator corresponding to an entry in a page descriptor ring that separates two flushing work sets (FWS). The first node receives an input/output (I/O) request and allocates a sequence identification (ID) number to the I/O request. The first node determines a FWS for the I/O request based on the sequence separator and the sequence ID number, and commits the I/O request using the sequence ID number. The I/O request and the sequence ID number are sent to the second node.

Abstract: A method of performing write operations that have been received by a data storage apparatus is provided. The method includes (a) storing page descriptors for received write operations within temporary storage, each page descriptor indicating respective data to be written; (b) upon storing each page descriptor, organizing that page descriptor into a shared working-set structure; and (c) operating a plurality of flushers to persist the data indicated by respective page descriptors to long-term persistent storage based on organization of the page descriptors in the shared working-set structure, each flusher accessing page descriptors via the shared working-set structure. An apparatus, system, and computer program product for performing a similar method are also provided.

Abstract: A method, computer program product, and computing system for defining a queue. The queue may be based on a linked list and may be a first-in, first-out (FIFO) queue that may be configured to be use used with multiple producers and a single consumer. The queue may include a plurality of queue elements. A tail element and a head element may be defined from the plurality of elements within the queue. The tail element may point to a last element of the plurality of elements and the head element may point to a first element of a plurality of elements. An element may be dequeued from the tail element, which may include determining if the tail element is in a null state. An element may be enqueued to the head element, which may include adding a new element to the queue.

Abstract: A technique protects user data within a storage system. The technique involves, in response to a request signal that directs the storage system to store the user data, generating first parity and second parity based on the user data. The technique further involves storing the first parity and the user data in a data uber of the storage system, the data uber providing redundant array of independent disks (RAID) protection. The technique further involves storing the second parity in designated free storage of the storage system, the designated free storage being separate from the data uber.

Abstract: A technique garbage collects user data within a storage system. The technique involves, in response to a request signal that directs the storage system to garbage collect user data, coalescing first user data and second user data to form new user data. The technique further involves generating first parity and second parity based on the new user data. The technique further involves storing the first parity and the new user data in a data uber of the storage system, the data uber providing redundant array of independent disks (RAID) protection. The technique further involves storing the second parity in designated free storage of the storage system, the designated free storage being separate from the data uber.

Abstract: In at least one embodiment, processing can include receiving a metadata (MD) update for a MD page, wherein the MD update has an associated type; determining whether the type is a custom MD type; responsive to determining the type is a custom MD type, performing first processing including: determining a custom bucket handle for the MD update based at least in part on the type and a logical index (LI) uniquely identifying the MD page; and applying the MD update to a custom bucket associated with the custom bucket handle; and responsive to determining the type is a custom MD type, performing second processing including: determining a regular bucket handle for the MD update based at least in part on the LI of the MD page; and applying the MD update to a regular bucket associated with the regular bucket handle.

Abstract: A method, computer program product, and computing system for processing, using a storage node, one or more updates to one or more metadata pages of a multi-node storage system. The one or more updates may be stored in one or more data containers in a cache memory system of the storage node, thus defining an active working set of data containers. Flushing ownership for each data container of the active working set may be assigned to one of the storage nodes based upon an assigned flushing ownership for each data container of a frozen working set and a number of updates within the frozen working set processed by each storage node, thus defining an assigned flushing storage node for each data container of the active working set. The one or more updates may be flushed, using the assigned flushing storage node, to a storage array.

Abstract: Techniques for reconstructing or building metadata pages in storage nodes that have a delta-log based architecture. The techniques include walking “up” an ancestor chain of a metadata page, detecting the most recent delta update for a metadata entry of the metadata page, writing the most recent delta update to a location of the metadata entry in the metadata page, setting a bitmap entry corresponding to the location of the metadata entry in the metadata page, detecting a less recent delta update for the metadata entry of the metadata page, and, having previously set the bitmap entry corresponding to the location of the metadata entry in the metadata page, avoiding writing the less recent delta update to the location of the metadata entry in the metadata page. In this way, the need to save in memory the entire ancestor chain of the metadata page can be eliminated.

Abstract: Data replication techniques can include receiving, at a source system, a write directed to a source logical device configured for asynchronous remote replication to a destination system; performing processing that flushes a transaction log entry for the write; and performing replication processing that uses a replication queue including a replication queue entry corresponding to the write that stores the first content to a logical address. The processing can create a replication log entry in a replication log for the write responsive to determining that the write is directed to the source logical device configured for asynchronous remote replication and that the first content has not been replicated. Responsive to the first content not being in cache, the first content can be retrieved using the reference to a storage location storing the first content. The reference can be obtained from the replication log entry or the replication queue entry.

Abstract: A system can use non-volatile solid state drives (SSDs) to provide storage. The SSDs can implement internal log structured systems (LSSs). A reversible write operation can be serviced by an SSD to write first data to an SSD logical address. The reversible write operation can update the SSD logical address to store the first data rather than old data stored at the SSD logical address prior to servicing the reversible write operation. The old data can be stored at an SSD physical address and mapping information indicates the SSD logical address is mapped to the SSD physical address. Servicing the reversible write operation can include: retaining the mapping information and the old data; and adding second mapping information that maps the SSD logical address to a second SSD physical address storing the first data. A subsequent read operation can read the old data using the retained mapping information.

Abstract: A method, computer program product, and computing system for executing a plurality of IO traces on a storage system. At least one vertical flow and at least one horizontal flow associated with the at least one vertical flow may be defined for the plurality of IO traces. A hierarchical representation of the plurality of IO traces may be generated with the at least one vertical flow and the at least one horizontal flow associated with the at least one vertical flow defined for the plurality of IO traces.

Abstract: A method, computer program product, and computing system for defining a normal IO write mode for writing data to a storage system, the normal IO writing mode including: writing the data to a cache memory system, writing the data to a journal, in response to writing the data to the journal, sending an acknowledgment signal to a host device, and writing the data from the cache memory system to a storage array. A request may be received to enter a testing IO write mode. In response to receiving the request, the data may be written to the cache memory system. The writing of the data to the journal may be bypassed. The acknowledgment signal may be sent to the host device in response to writing the data to the cache memory system. The data may be written from the cache memory system to the storage array.