Abstract: Techniques are provided for lock reservations for shared storage. A reserve command to reserve a storage structure is received by a driver from a node. The reserve command is formatted according to a storage protocol. The driver translates the reserve command into a lease acquire command formatted according to an object store protocol and targeting an object stored within an object store and corresponding to the storage structure. A lease identifier derived from a node identifier of the node is inserted into the lease acquire command. The lease acquire command is routed to the object store for obtaining a lease on the object for granting the node exclusive write access to the object.
Type:
Grant
Filed:
June 30, 2023
Date of Patent:
January 21, 2025
Assignee:
NetApp, Inc.
Inventors:
Christopher Busick, Stephen Winslow Schmitt
Abstract: Methods and systems for a networked storage system are provided. One method includes assigning a quality of service (QOS) parameter for a storage volume of a networked storage environment having a first storage node and a second storage node, where the QOS parameter is defined by a throughput value that defines a maximum data transfer rate and a number of input/output (I/O) operations executed within a time period (IOPS); distributing the QOS parameter between the first storage node and the second storage node; determining that throughput credit is available for processing an I/O request for using the storage volume; determining that IOPS credit is available for processing the request by the first storage node; and processing the I/O request when both the throughput credit and the IOPS credit is available.
Type:
Grant
Filed:
January 27, 2023
Date of Patent:
January 21, 2025
Assignee:
NetApp, Inc.
Inventors:
Dean Alan Kalman, John Alexander Scott, Sudha Sundaram
Abstract: Systems and methods are described for performing persistent inflight tracking of operations (Ops) within a cross-site storage solution. According to one embodiment, a method comprises maintaining state information regarding a data synchronous replication status for a first storage object of a primary storage cluster and a second storage object of a secondary storage cluster. The method includes performing persistent inflight tracking of I/O operations with a first Op log of the primary storage cluster and a second Op log of the secondary storage cluster, establishing and comparing Op ranges for the first and second Op logs, and determining a relation between the Op range of the first Op log and the Op range of the second Op log to prevent divergence of Ops in the first and second Op logs and to support parallel split of the Ops.
Abstract: Systems and methods for reducing read application in a virtual storage system are provided. According to one embodiment, read amplification is reduced when AZCS compression is being utilized by avoiding restarting of a read process via a slow path via a RAID layer of the virtual storage system when a data block associated with a read request and obtained via a first fast path read has been found not to be compressed. Instead, a second fast path read may be performed to obtain the corresponding checksum. Alternatively, or additionally, heuristics may be used to predict the odds of the data block being compressed. For example, when information encoded within a PVBN of the data block that identifies the PVBN as being within a compressed AZCS zone has shown to be sufficiently/insufficiently predictive of the data block being compressed, then a flag may be set to enable/disable fast path reads.
Abstract: Multi-site distributed storage systems and computer-implemented methods are described for providing common snapshot retention and automatic fanout reconfiguration for an asynchronous leg after a failure event that causes a failover from a primary storage site to a secondary storage site. A computer-implemented method comprises providing an asynchronous replication relationship with an asynchronous update schedule from one or more storage objects of the first storage node to one or more replicated storage objects of the third storage node, creating a snapshot copy of the one or more storage objects of the first storage node, transferring the snapshot copy to the third storage node based on an asynchronous mirror policy, and intercepting the snapshot create operation on the primary storage site and transferring the snapshot copy to the second storage node to provide a common snapshot between the second storage node and the third storage node.
Abstract: A data management system can include a set of storage media configured to implement a storage space and a set of controllers. The set of controllers can be configured to write to the storage space and to implement a set of nodes. The set of controllers can include a first controller that implements a first node and includes a first persistent memory, a second controller that implements a second node and includes a second persistent memory and a third controller that implements a third node and includes a third persistent memory. The third node can be configured to write third node journal data to the first persistent memory. The first node can be configured to generate first node journal data based on a first request received from a backend, write the first node journal data to the first persistent memory, and replicate the journal data to the second persistent memory.
Abstract: A distributed storage management system comprising nodes that form a cluster, a distributed block layer that spans the nodes in the cluster, and file system instances deployed on the nodes. Each file system instance comprises a data management subsystem and a storage management subsystem disaggregated from the data management subsystem. The storage management subsystem comprises a node block store that forms a portion of the distributed block layer and a storage manager that manages a key-value store and virtualized storage supporting the node block store. A file system volume hosted by the data management subsystem maps to a logical block device hosted by the virtualized storage in the storage management subsystem. The key-value store includes, for a data block of the logical block device, a key that comprises a block identifier for the logical block device and a value that comprises the data block.
Type:
Application
Filed:
July 22, 2024
Publication date:
January 9, 2025
Applicant:
NetApp, Inc.
Inventors:
Ravikanth Dronamraju, Ananthan Subramanian, Daniel McCarthy, Christopher Cason, Arindam Banerjee
Abstract: A system, method, and machine-readable storage medium for determining an amount of unique data in a distributed storage system are provided. In some embodiments, a combined efficiency set for a first data set stored in the distributed storage system, such as at a volume, may be generated. The first data set may include a first subset of data and a second subset of data in the distributed storage system. Additionally, a set of efficiency sets for the first subset of data may be generated. A set difference based on the combined efficiency set and the set of efficiency sets may be computed. An amount of memory used for storing unique data of the second subset of data may be estimated based on the set difference. The unique data may be present in the second subset of data but absent from the first subset of data.
Abstract: Methods and systems for a networked storage system is provided. One method includes transforming by a processor, performance parameters associated with storage volumes of a storage system for representing each storage volume as a data point in a parametric space; generating by the processor, a plurality of bins in the parametric space using the transformed performance parameters; adjusting by the processor, bin boundaries for the plurality of bins for defining a plurality of service levels for the storage system based on the performance parameters; and using the defined plurality of service levels for operating the storage system.
Abstract: Systems and methods for multiple device consumption of shared namespaces of ephemeral storage devices by a consumer of a virtual storage system are provided. In an example, multiple namespaces of respective ephemeral storage devices are shared among multiple of consumers of a virtual storage system by creating multiple partitions within each of the namespaces for use by respective consumers of the multiple consumers. Corresponding partitions of respective shared namespace may then be treated as a stripe set to facilitate multiple device consumption for a subsystem (e.g., operation log journaling) of the virtual storage system by striping data associated with input/output (I/O) requests of a consumer (e.g., a journaling driver) across one or more stripe units of one or more stripes within the stripe set.
Type:
Grant
Filed:
April 28, 2023
Date of Patent:
January 7, 2025
Assignee:
NetApp, Inc.
Inventors:
Joseph Brown, Jr., Javier Tsuyoshi Takimoto, Sangramsinh Pandurang Pawar, Michael Scott Ryan
Abstract: Techniques are provided for metadata management for enabling automated switchover in accordance with a configuration of storage solution that expresses a preference for either maintaining availability (e.g., a non-zero RPO mode) of the storage solution or avoiding data loss (e.g., a zero RPO mode). In one example, responsive to detecting a switchover trigger event, a node of a local cluster of a cross-site storage solution determines whether performance of an automated switchover from a failed cluster to a surviving cluster of the cross-site storage solution is enabled. Responsive to an affirmative determination, the node selectively proceeds with the automated switchover based on the configuration.
Type:
Application
Filed:
July 8, 2024
Publication date:
January 2, 2025
Applicant:
NetApp, Inc.
Inventors:
Sasidharan Krishnan, Kalaivani Arumugham, Preksha Bansal, Vijay Kumar Chakravarthy Ekkaladevi, Ryan Edward Bartlett
Abstract: Techniques are provided for on-demand creation and/or utilization of containers and/or serverless threads for hosting data connector components. The data connector components can be used to perform integrity checking, anomaly detection, and file system metadata analysis associated with objects stored within an object store. The data connector components may be configured to execute machine learning functionality to perform operations and tasks. The data connector components can perform full scans or incremental scans. The data connector components may be stateless, and thus may be offlined, upgraded, onlined, and/or have tasks transferred between data connector components. Results of operations performed by the data connector components upon base objects may be stored within sibling objects.
Abstract: Systems and methods for scaling application and/or storage system functions of a distributed storage system based on a heterogeneous resource pool are provided. According to one embodiment, the distributed storage system has a composable, service-based architecture that provides scalability, resiliency, and load balancing. The distributed storage system includes a cluster of nodes each potentially having differing capabilities in terms of processing, memory, and/or storage. The distributed storage system takes advantage of different types of nodes by selectively instating appropriate services (e.g., file and volume services and/or block and storage management services) on the nodes based on their respective capabilities. Furthermore, disaggregation of these services, facilitated by interposing a frictionless layer (e.g.
Type:
Application
Filed:
August 30, 2024
Publication date:
December 26, 2024
Applicant:
NetApp, Inc.
Inventors:
Arindam Banerjee, Daniel McCarthy, Christopher Cason, Ananthan Subramanian
Abstract: Techniques are provided for on-demand creation and/or utilization of containers and/or serverless threads for hosting data connector components. The data connector components can be used to perform integrity checking, anomaly detection, and file system metadata analysis associated with objects stored within an object store. The data connector components may be configured to execute machine learning functionality to perform operations and tasks. The data connector components can perform full scans or incremental scans. The data connector components may be stateless, and thus may be offlined, upgraded, onlined, and/or have tasks transferred between data connector components. Results of operations performed by the data connector components upon base objects may be stored within sibling objects.
Type:
Grant
Filed:
April 28, 2021
Date of Patent:
December 24, 2024
Assignee:
NetApp, Inc.
Inventors:
Sharankumar Yelheri, Atul Ramesh Pandit, Tijin George
Abstract: Techniques are provided for coordinating snapshot operations across multiple file systems. A notification may be received that a snapshot of data stored across a persistent memory file system and a storage file system is to be generated. Forwarding, of modify operations from a persistent memory tier to a file system tier for execution through the storage file system, may be enabled. Framing may be initiated to notify the storage file system of blocks within the persistent memory file system that comprise more up-to-date data than corresponding blocks within the storage file system. In response to the framing completing, a consistency point operation is performed to create the snapshot and to create a snapshot image as part of the snapshot.
Type:
Grant
Filed:
July 31, 2023
Date of Patent:
December 17, 2024
Assignee:
NetApp, Inc.
Inventors:
Ananthan Subramanian, Matthew Fontaine Curtis-Maury, Ram Kesavan, Rupa Natarajan, Vinay Devadas
Abstract: Techniques are provided for microservices management and orchestration. A chart package is selectively retrieved from a chart repository based upon the chart package corresponding to a set of services to host within a cluster and dependencies amongst the set of services. A set of container images may be retrieved from a container repository based upon the set of container images corresponding to the set of services. A cluster may be created within a computing environment. The set of services may be deployed as resources of the computing environment within the cluster and the dependencies may be configured using the chart package and the set of container images.
Abstract: Systems and methods for performing single I/O writes are provided. According to one embodiment, responsive to receipt of a write operation from a client by a file system layer of a node of a distributed storage system and a data payload of the operation having been determined to meet a compressibility threshold, an intermediate storage layer of the node logically interposed between the file system layer and a block storage media is caused to perform a single input/output (I/O) write operation that persists the compressed data payload and corresponding metadata to support asynchronous journaling of the write operation. The single I/O write operation coupled with the use of a new pool file that maintains a list of available blocks for single I/O write operations and a modified node crash recovery approach allows the write operation to be acknowledged to the client while the journaling is performed asynchronously.
Type:
Grant
Filed:
November 29, 2023
Date of Patent:
December 17, 2024
Assignee:
NetApp, Inc.
Inventors:
Mrinal K. Bhattacharjee, Boopathy Krishnamoorthy, Vinay Kumar B C, Shivali Gupta, Saurabh Gupta
Abstract: Techniques are provided for processing read operations while splitting a clone volume from a parent volume whose data is stored within objects of an object store. A transfer map is created to track mappings of child object identifiers to parent object identifiers of the parent objects to copy as child objects having the child object identifiers. The transfer map for the object store is traversed to copy the parent objects as the child objects for the split clone operation. The child objects are verified as being successfully created with valid data. In response to determining that the parent object has been copied as the child object, a context check is performed using the reverse map to verify a block within the child object. In response a successful context check, the read operation is processed using the block of the child object.
Type:
Grant
Filed:
April 28, 2023
Date of Patent:
December 17, 2024
Assignee:
NetApp, Inc.
Inventors:
Cheryl Marie Thompson, Garima Choudhary, Rajesh Sudarsan
Abstract: Techniques are provided for on-demand creation and/or utilization of containers and/or serverless threads for hosting data connector components. The data connector components can be used to perform integrity checking, anomaly detection, and file system metadata analysis associated with objects stored within an object store. The data connector components may be configured to execute machine learning functionality to perform operations and tasks. The data connector components can perform full scans or incremental scans. The data connector components may be stateless, and thus may be offlined, upgraded, onlined, and/or have tasks transferred between data connector components. Results of operations performed by the data connector components upon base objects may be stored within sibling objects.
Type:
Grant
Filed:
April 28, 2021
Date of Patent:
December 17, 2024
Assignee:
NetApp, Inc.
Inventors:
Sharankumar Yelheri, Atul Ramesh Pandit, Tijin George
Abstract: Data traffic management in a computing environment utilizing direct memory access functionality is disclosed. A management agent is configured to operate within a storage node. The management agent includes a storage interface to communicate with a first set of storage devices, a management memory interface to communicate with a first management memory, and an interconnect (IC) interface to communicate with a remote peer node. The management agent controls data traffic between the storage node and the peer node to provide at least mirroring of the first management memory to the peer node and mirroring of a second management memory on the peer node to the storage node. The management agent further controls the data traffic using a traffic control approach selected based on at least a performance evaluation of an IC fabric accessible via the IC interface.