Abstract: Methods and systems for dynamic hashing in cache sub-systems are provided. The method includes analyzing a plurality of input/output (I/O) requests for determining a pattern indicating if the I/O requests are random or sequential; and using the pattern for dynamically changing a first input to a second input for computing a hash index value by a hashing function that is used to index into a hashing data structure to look up a cache block to cache an I/O request to read or write data, where for random I/O requests, a segment size is the first input to a hashing function to compute a first hash index value and for sequential I/O requests, a stripe size is used as the second input for computing a second hash index value.

Abstract: Systems and techniques for cache management are disclosed that provide improved cache performance by prioritizing particular storage stripes for cache flush operations. The systems and techniques may also leverage features of the storage devices to provide atomicity without the overhead of inter-controller mirroring. In some embodiments, the systems and techniques include a storage controller that stores data in a cache. The data is associated with one or more sectors of a storage stripe that is defined over plurality of storage devices. The storage controller identifies a locality of dirty sectors of the one or more sectors, classifies the storage stripe into a category based on the locality, provides a category ordering of the category relative to at least one other category, and flushes the storage stripe from the cache to the plurality of storage devices according to the category ordering.

Abstract: A method, a computing device, and a non-transitory machine-readable medium for changing ownership of a storage volume from a first controller to a second controller without flushing data, is provided. In the system, the first controller is associated with a first DRAM cache comprising a primary partition that stores data associated with the first controller and a mirror partition that stores data associated with the second controller. The second controller in the system is associated with a second DRAM cache comprising a primary partition that stores data associated with the second controller and the mirror partition associated with the first controller. Further, the mirror partition in the second DRAM cache stores a copy of a data in the primary partition of the first DRAM cache and the mirror partition in the first DRAM cache stores a copy of a data in the primary partition of the second DRAM cache.

Abstract: Method and systems for a storage system are provided. Simulated cache blocks of a cache system are tracked using cache metadata while performing a workload having a plurality of storage operations. The cache metadata is segmented, each segment corresponding to a cache size. Predictive statistics are determined for each cache size using a corresponding segment of the cache metadata. The predictive statistics are used to determine an amount of data that is written for each cache size within certain duration. The process then determines if each cache size provides an endurance level after executing a certain number of write operations, where the endurance level indicates a desired life-cycle for each cache size.

Abstract: A method, a computing device, and a non-transitory machine-readable medium for changing ownership of a storage volume from a first controller to a second controller without flushing data, is provided. In the system, the first controller is associated with a first DRAM cache comprising a primary partition that stores data associated with the first controller and a mirror partition that stores data associated with the second controller. The second controller in the system is associated with a second DRAM cache comprising a primary partition that stores data associated with the second controller and the mirror partition associated with the first controller. Further, the mirror partition in the second DRAM cache stores a copy of a data in the primary partition of the first DRAM cache and the mirror partition in the first DRAM cache stores a copy of a data in the primary partition of the second DRAM cache.

Abstract: Methods and systems for managing caching mechanisms in storage systems are provided where a global cache management function manages multiple independent cache pools and a global cache pool. As an example, the method includes: splitting a cache storage into a plurality of independently operating cache pools, each cache pool comprising storage space for storing a plurality of cache blocks for storing data related to an input/output (“I/O”) request and metadata associated with each cache pool; receiving the I/O request for writing a data; operating a hash function on the I/O request to assign the I/O request to one of the plurality of cache pools; and writing the data of the I/O request to one or more of the cache blocks associated with the assigned cache pool. In an aspect, this allows efficient I/O processing across multiple processors simultaneously.

Abstract: Selective multiprocessing in a non-preemptive task scheduling environment is provided. Tasks of an application are grouped based on similar functionality and/or access to common code or data structures. The grouped tasks constitute a task core group, and each task core group may be mapped to a core in a multi-core processing system. A mutual exclusion approach reduces overhead imposed on the storage controller and eliminates the risk of concurrent access. A core guard routine is used when a particular application task in a first task core group requires access to a section of code or data structure associated with a different task core group. The application task is temporarily assigned to the second task core group. The application task executes the portion of code seeking access to the section of code or data structure. Once complete, the application task is reassigned back to its original task core group.

Abstract: A system and method for maintaining operation of a storage array with one or more failed storage devices and for quickly recovering when failing devices are replaced are provided. In some embodiments, the method includes receiving a data transaction directed to a volume and determining that a storage device associated with the volume is inoperable. In response to determining that the storage device is inoperable, a data extent is recorded in a change log in a storage controller cache. The data extent is associated with the data transaction and allocated to the storage device that is inoperable. The data transaction is performed using at least one other storage device associated with the volume, and data allocated to the storage device is subsequently reconstructed using the recorded data extent.

Abstract: A system and method for maintaining operation of a storage array with one or more failed storage devices and for quickly recovering when failing devices are replaced are provided. In some embodiments, the method includes receiving a data transaction directed to a volume and determining that a storage device associated with the volume is inoperable. In response to determining that the storage device is inoperable, a data extent is recorded in a change log in a storage controller cache. The data extent is associated with the data transaction and allocated to the storage device that is inoperable. The data transaction is performed using at least one other storage device associated with the volume, and data allocated to the storage device is subsequently reconstructed using the recorded data extent.

Abstract: Methods and systems for dynamic hashing in cache sub-systems are provided. The method includes analyzing a plurality of input/output (I/O) requests for determining a pattern indicating if the I/O requests are random or sequential; and using the pattern for dynamically changing a first input to a second input for computing a hash index value by a hashing function that is used to index into a hashing data structure to look up a cache block to cache an I/O request to read or write data, where for random I/O requests, a segment size is the first input to a hashing function to compute a first hash index value and for sequential I/O requests, a stripe size is used as the second input for computing a second hash index value.

Abstract: Methods and systems for managing caching mechanisms in storage systems are provided where a global cache management function manages multiple independent cache pools and a global cache pool. As an example, the method includes: splitting a cache storage into a plurality of independently operating cache pools, each cache pool comprising storage space for storing a plurality of cache blocks for storing data related to an input/output (“I/O”) request and metadata associated with each cache pool; receiving the I/O request for writing a data; operating a hash function on the I/O request to assign the I/O request to one of the plurality of cache pools; and writing the data of the I/O request to one or more of the cache blocks associated with the assigned cache pool. In an aspect, this allows efficient I/O processing across multiple processors simultaneously.

Abstract: Method and systems for a storage system are provided. Simulated cache blocks of a cache system are tracked using cache metadata while performing a workload having a plurality of storage operations. The cache metadata is segmented, each segment corresponding to a cache size. Predictive statistics are determined for each cache size using a corresponding segment of the cache metadata. The predictive statistics are used to determine an amount of data that is written for each cache size within certain duration. The process then determines if each cache size provides an endurance level after executing a certain number of write operations, where the endurance level indicates a desired life-cycle for each cache size.

Abstract: Technology is disclosed for generating predictive cache statistics for various cache sizes. In some embodiments, a storage controller includes a cache tracking mechanism for concurrently generating the predictive cache statistics for various cache sizes for a cache system. The cache tracking mechanism can track simulated cache blocks of a cache system using segmented cache metadata while performing an exemplary workload including various read and write requests (client-initiated I/O operations) received from client systems (or clients). The segmented cache metadata corresponds to one or more of the various cache sizes for the cache system.

Abstract: A method for managing data movement in a multi-level cache system includes selecting at least one outgoing data block from a primary cache when an unallocated space of the primary cache has reached a minimum threshold, initiating a de-stage process for de-staging the outgoing data block, and terminating the de-stage process when the unallocated space has reached an upper threshold. The de-stage process includes storing the outgoing data block in a secondary cache when a cache hit has occurred before. The cache hit may be an actual cache hit or a “ghost” cache hit wherein only metadata is stored in the secondary cache. A method for de-staging an outgoing data block from a higher level cache which includes storing an outgoing data block in a lower level cache when a cache hit rate satisfies a predetermined condition and storing metadata in the lower level cache is also disclosed.

Abstract: Methods and structure for automated determination and reconfiguration of the size of a cache memory in a storage system. Features and aspects hereof generate historical information regarding frequency of hits on cache lines in the cache memory. The history maintained is then analyzed to determine a desired cache memory size. The historical information regarding cache memory usage may be communicated to a user who may then direct the storage system to reconfigure its cache memory to a desired cache memory size. In other embodiments, the storage system may automatically determine the desired cache memory size and reconfigure its cache memory. The method may be performed automatically periodically, and/or in response to a user's request, and/or in response to detecting thrashing caused by least recently used (LRU) cache replacement algorithms in the storage system.

Abstract: A method for reading data from a data storage system is provided. The method comprises requesting a virtual data volume to access data from one or more data blocks in the data storage system; requesting a virtual protection information volume to access protection information associated with the data blocks; validating the data using the protection information; and providing the data to the host interface, in response to successful validation of the data. A method for writing data to a data storage system is also provided. The method comprises receiving data to be written to one or more data blocks in the data storage system, wherein the data is stored in a cache; generating protection information to be stored on a virtual protection information volume; requesting a virtual data volume to update the data blocks with the data; and requesting the virtual protection information volume to store the protection information.

Abstract: Methods and structure for automated determination and reconfiguration of the size of a cache memory in a storage system. Features and aspects hereof generate historical information regarding frequency of hits on cache lines in the cache memory. The history maintained is then analyzed to determine a desired cache memory size. The historical information regarding cache memory usage may be communicated to a user who may then direct the storage system to reconfigure its cache memory to a desired cache memory size. In other embodiments, the storage system may automatically determine the desired cache memory size and reconfigure its cache memory. The method may be performed automatically periodically, and/or in response to a user's request, and/or in response to detecting thrashing caused by least recently used (LRU) cache replacement algorithms in the storage system.

Abstract: A method for validating data in a data storage system comprising associating a first data chunk with first check data and storing the first data chunk and the first check data on a first storage device. Additional associated data chunks of the first data and associated additional check data are stored on at least one of the first storage device or one or more additional storage devices. At least a portion of the first check data and at least a portion of the additional check data are stored to a second storage device, which is distinct from the first storage device and the additional storage devices. I/O access to the second storage device is minimized by retaining at least a portion of the first check data and at least a portion of the additional check data in a readily accessible storage medium, during servicing of a first I/O request.

Abstract: A method for managing data movement in a multi-level cache system having a primary cache and a secondary cache. The method includes determining whether an unallocated space of the primary cache has reached a minimum threshold; selecting at least one outgoing data block from the primary cache when the primary cache reached the minimum threshold; initiating a de-stage process for de-staging the outgoing data block from the primary cache; and terminating the de-stage process when the unallocated space of the primary cache has reached an upper threshold. The de-stage process further includes determining whether a cache hit has occurred in the secondary cache before; storing the outgoing data block in the secondary cache when the cache hit has occurred in the secondary cache before; generating and storing metadata regarding the outgoing data block; and deleting the outgoing data block from the primary cache.

Abstract: Methods and associated structures for utilizing write-back cache management modes for local cache memory of disk drives coupled to a storage controller while maintaining data integrity of the data transferred to the local cache memories of affected disk drives. In one aspect hereof, a state machine model of managing cache blocks in a storage controller cache memory maintains blocks in the storage controller's cache memory in a new state until verification is sensed that the blocks have been successfully stored on the persistent storage media of the affected disk drives. Responsive to failure or other reset of the disk drive, the written cache blocks may be re-written from the copy maintained in the cache memory of the storage controller. In another aspect, an alternate controller's cache memory may also be used to mirror the cache blocks from the primary storage controller's cache memory as additional data integrity assurance.