Abstract: A system and method for application aware de-duplication of data blocks in a virtualized storage array is disclosed. In one embodiment, in a method of de-duplication of data, a master list of metadata is created based on a number of occurrences of data blocks within a storage array. A first sublist of metadata is created from the master list of metadata. The first sublist of metadata is provided to a first component of a networked storage system. It is determined whether the data block being written has a corresponding entry in the master list of metadata based on a determination that a data block being written does not have any corresponding entry in the first sublist of metadata. The data block being written is replaced with a pointer based on a determination that the data block being written has a corresponding entry in the master list of metadata.

Abstract: A request is received to clone a source data object. A source block range of the source data object in a source logical storage unit is determined. An empty data object in the destination logical storage unit is created. A destination block range of the empty data object in the destination logical storage unit is determined. The source block range is mapped to the destination block range. The source data object is cloned based on the mapping.

Abstract: Systems and methods for improving caching mechanisms in a storage system are disclosed. The method includes storing data associated with a write input/output (I/O) request at a cache; determining an amount of dirty data stored in the cache, where the dirty data is data in the cache that has not yet been written to a persistent storage location managed by a storage system; determining if the amount of dirty data exceeds a threshold value; determining a cache flush rate based on the amount of dirty data stored at the cache, when the amount of dirty data exceeds the threshold value; and writing data from the cache at the determined cache flush rate to the persistent storage location.

Abstract: Technology is disclosed for providing access to operation performance of various storage mediums in a distributed storage system, which store and host data associated with a database (“the technology”). The technology can identify the storage mediums storing data associated with a particular database, gather data associated with performance of each of the identified storage mediums storing data associated with the particular database, and generate performance statistics associated with each of the identified storage mediums utilizing the gathered data and provide the generated statistics as a performance overview of storage associated with the particular database.

Abstract: Technology is disclosed for partitioning a namespace of a large scale data storage service. In various embodiments, the technology receives a file path; employs a hashing function to produce a hash value for the received file path; and identifies a name node handling the produced hash value, wherein the name node is configured to communicate with one or more data nodes for handling file requests on the file path.

Abstract: In one embodiment, a node of a cluster having a plurality of nodes, executes a storage input/output (I/O) stack having a redundant array of independent disks (RAID) layer. The RAID layer organizes solid state drives (SSDs) within one or more storage arrays as a plurality of RAID groups associated with one or more extent stores. The RAID groups are formed from slices of storage spaces of the SSDs instead of entire storage spaces of the SSDs. This provides for RAID groups to co-exist on a same set of the SSDs.

Abstract: A hardware and/or software facility to enable emulated storage devices to share data stored on physical storage resources of a storage system. The facility may be implemented on a virtual tape library (VTL) system configured to back up data sets that have a high level of redundancy on multiple virtual tapes. The facility organizes all or a portion of the physical storage resources according to a common store data layout. By enabling emulated storage devices to share data stored on physical storage resources, the facility enables deduplication across the emulated storage devices irrespective of the emulated storage device to which the data is or was originally written, thereby eliminating duplicate data on the physical storage resources and improving the storage consumption of the emulated storage devices on the physical storage resources.

Abstract: Examples described herein include a data migration system for migrating data between different data storage environments. The data migration system creates a first volume on a first storage system, and creates a logical unit within the first volume. The data migration system then creates a virtual volume on a virtual storage system associated with a second storage system, wherein data stored in the virtual volume is backed by the logical unit on the first storage system. The data migration system then replicates, on the virtual volume, a set of data stored on the second storage system. Upon replicating the set of data onto the virtual volume the virtual storage system may automatically create a copy of the corresponding data in the logical unit of the first volume.

Abstract: Systems and methods are disclosed for scheduling a plurality of tasks for execution on one or more processors. An example method includes obtaining a counter value of a counter. The method also includes for each work queue of a plurality of work queues, identifying an execution period of the respective work queue and comparing a counter value to an execution period of the respective work queue. Each work queue includes a set of tasks and is defined by an execution period at which to run the respective set of queued tasks. The method further includes selecting, based on the comparing, a subset of the plurality of work queues. The method also includes scheduling a set of tasks of slower frequency queued in a selected work queue for execution on one or more processors before a set of tasks queued in a non-selected work queue. The work items may be scheduled in O(1) because the design inherently prioritizes the tasks based on the urgency of their completion, and may do so by resetting a work queue pointer.

Abstract: Partially overwriting a compression group without decompressing compressed data can consumption of resources for the decompression. A storage server partially overwrites the compression group when a file block identifier of a client's write request resolves to the compression group. The compression group remains compressed while the partial overwriting is performed.

Abstract: A clustered network storage system includes a policy based storage efficiency management framework to allow users to more efficiently manage storage efficiency attributes on a plurality of storage objects in the clustered network storage system. The storage efficiency management framework includes a policy database for storing a plurality of polices, each of which includes a plurality of attributes. Each storage object in the clustered network storage system is assigned a policy. Storage efficiency operations can then be initiated by a storage efficiency engine according to the policies assigned to the storage objects by sending a single trigger to each node in the clustered network storage system and the storage efficiency engine determining on which storage objects to perform the storage efficiency operations.

Abstract: A storage system provides highly flexible data layouts that can be tailored based on reliability considerations. The system allocates reliability values to logical containers at an upper logical level of the system based, for example, on objectives established by reliability SLOs. Based on the reliability value, the system identifies a specific parity group from a lower physical storage level of the system for storing data corresponding to the logical container. After selecting a parity group, the system allocates the data to physical storage blocks within the parity group. In embodiments, the system attaches the reliability value information to the parity group and the physical storage units storing the data. In this manner, the underlying physical layer has a semantic understanding of reliability considerations related to the data stored at the logical level.

Abstract: Data is replicated on a backup node, where the granularity of the replication can be less than a full volume. A data consistency group comprising a subset of data for a volume is defined for a primary node. A set of differences for the data consistency group is sent to a backup node. The backup node creates change logs in response to receiving the set of differences. In response to receiving a request to access a file having data in the data consistency group, the backup node creates a clone of the file. The backup node determines whether an update to a data block of the file exists in the change logs. In response to determining that the update to the data block exists in the change logs, the backup node updates a copy of the data block for the cloned file with data in the change logs.

Abstract: A method, non-transitory computer readable medium, and device that prefetchs includes identifying a candidate data block from one of one or more immediate successor data blocks. The identified candidate data block has a historical access probability value from an initial accessed data block which is higher than a historical access probability value for each of the other immediate successor data blocks and is above a prefetch threshold value. The identifying is repeated until a next identified candidate data block has the historical access probability value below the prefetch threshold value. In the repeating, the identifying next immediate successor data blocks is from the previously identified candidate data block and the historical access probability value for each of the next immediate successor data blocks is determined from the originally accessed data block. The identified candidate data block with the historical access probability value above the prefetch threshold value is fetched.

Abstract: A method performed by a monitoring tool in a computer system, the method including: identifying a software update applied to a plurality of computer system assets, retrieving a first status of a first one of the assets from a first time period before the software update was applied to the first asset, retrieving a second status of the first asset from a second time period after the software update was applied to the first asset, determining a first effectiveness of the software update for the first asset based on comparing the first status and the second status, and displaying within a Graphical User Interface (GUI) an identification of the software update, an identification of the first asset, and a recommendation for the software update based on the first effectiveness.

Abstract: A system and method of cache monitoring in storage systems includes storing storage blocks in a cache memory. Each of the storage blocks is associated with status indicators. As requests are received at the cache memory, the requests are processed and the status indicators associated with the storage blocks are updated in response to the processing of the requests. One or more storage blocks are selected for eviction when a storage block limit is reached. As ones of the selected one or more storage blocks are evicted from the cache memory, the block counters are updated based on the status indicators associated with the evicted storage blocks. Each of the block counters is associated with a corresponding combination of the status indicators. Caching statistics are periodically updated based on the block counters.

Abstract: In one embodiment, a node coupled to one or more storage devices executes a storage input/output (I/O) stack having a volume layer, a persistence layer and an administration layer that interact to create a copy of a parent volume associated with a storage container on the one or more storage devices. A copy create start message is received at the persistence layer from the administration layer. The persistence layer ensures that dirty data for the parent volume is incorporated into the copy of the parent volume. New data for the parent volume received at the persistence layer during creation of the copy of the parent volume is prevented from incorporation into the copy of the parent volume. A reply to the copy create start message is sent from the persistence layer to the administration layer to initiate the creation of the copy of the parent volume at the volume layer.

Abstract: A shared storage architecture is described for coordinating management of a shared storage between nodes of a network storage system. In various embodiments, the shared storage is partitioned into and different partitions are assigned to different nodes of the network storage system. The shared storage architecture provides techniques for asserting reservations on the shared storage, managing state of the shared storage, and implementing various configurations of the network storage system using the shared storage.

Abstract: Methods and systems for securing information are provided. The method includes generating a hash key by an input/output (I/O) processing module interfacing with a processor executable application to encrypt a block of data of a data container to secure and store the data container; generating cipher text for the block of data encrypted with the hash key; using an encryption key to encrypt the hash key for the block of data; providing the cipher text and the encrypted hash key by the I/O processing module to a storage system for storage; where the I/O processing module segregates the encrypted hash key from the cipher text and maintains the encrypted hash key as part of metadata for the cipher text; and storing the cipher text with the encrypted hash key as the metadata for the cipher text for the block of data.

Abstract: Methods and systems for managing resources in a networked storage environment are provided. One method includes using a queuing model for a resource that processes a plurality of requests at a networked storage environment for predicting a relationship between latency and utilization of the resource. The queueing model uses inter-arrival time and service time to determine latency, where inter-arrival time is a duration that tracks when requests arrive at the resource and the service time tracks a duration for servicing the requests by the resource. The method further includes identifying optimum utilization of the resource using the predicted relationship between latency and utilization, where the optimum utilization is an indicator of resource utilization beyond which throughput gains for a workload is smaller than increase in latency; and determining available performance capacity for the resource using the optimum utilization and actual utilization of the resource.