Abstract: A system and method of transposed storage in RAID arrays includes a storage manager with an interface for receiving storage requests associated with multi-segment blocks stored in a storage array, a controller for processing the received storage requests and controlling the storage manager, and a write request handler. The storage array includes a plurality of storage devices for which consecutive logical addresses are assigned to different ones of the storage devices. The write request handler is configured to process block write requests and send segment write requests to the storage array based on the write requests so that each segment of a given multi-segment block is written to a single respective one of the storage devices in the storage array. In some embodiments, the storage manager further includes a read request handler configured to determine logical addresses of requested segments and send segment read requests to the storage array.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data based on the collected QOS data is generated. The process then determines a deviation of each potential bully storage volume of a resource used by any victim storage volume, where the deviation of each bully storage volume is based on a number of current I/O requests (IOPS) that are processed by each potential bully storage volume, a forecasted value of TOPS and a predicted upper threshold TOPS value for each potential bully storage volume; and filters the potential bully storage volumes based on an impact of each potential bully storage volume.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes are provided. QOS data is collected for the plurality of storage volumes and includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. The process determines an average of N collected QOS data points at any given time; and iteratively analyzes each QOS data point to detect if a step-up or a step-down function has occurred, where a step-up function represents an unpredictable increase in value of a data point and a step-down function is an unpredictable decrease in value of the data point. A subset of the N QOS data points based on when the step-up function or step-down function occurs is selected for analysis and an expected range for future QOS data based on the subset of the N QOS data points is generated.

Abstract: Technology is disclosed for subpartitioning a namespace region. In various embodiments, the technology creates at least two subpartitions from a partitioned namespace, wherein the partitioned namespace corresponds to at least two different name nodes of the large scale data storage service; and stores data corresponding to each subpartition as a separate file, e.g., so that it can be easily mounted by an operating system executed by a different computing device.

Abstract: The present invention provides a system and method for a shared write address protocol (SWAP) that is implemented over a remote direct memory address (RDMA) connection. Each party to a connection establishes a flow control block that is accessible to its partner via a RDMA READ operation. The novel protocol operates so that each module needs to have one outstanding RDMA READ operation at a time, i.e., to obtain the current flow control information from its partner. In operation, if data to be transmitted is less than or equal to a buffer size, an INLINE message data structure of the SWAP protocol is utilized to send the data to be target. However, if the data is greater than the buffer size, a second determination is made as to whether sufficient space exists in the message pool for the data. If insufficient space exists, the sender will wait until sufficient space exists before utilizing a novel WRITE operation of the SWAP protocol to transmit the data.

Abstract: Methods and systems for monitoring quality of service (QOS) data for a plurality of storage volumes from a storage operating system of a storage system are provided. A performance manager collects the QOS data from the storage operating system and the QOS data includes a response time in which each of the plurality of storage volumes respond to an input/output (I/O) request. An expected range for future QOS data is generated based on the collected QOS data. The QOS data is monitored for each storage volume for determining whether a current QOS data for each storage volume is within the expected range.

Abstract: Methods and systems for managing resources in a storage system are provided. The methods include tracking performance of a plurality of resources used for reading and writing information at storage devices in a networked storage system, each resource represented by a logical object in a hierarchical structure and performance data associated with each logical object is maintained by a processor executing a management application out of a memory device; identifying a root object associated with a resource having a performance issue as indicated by a threshold violation for the resource; selecting a related object associated with a resource similar to the resource of the root object by the management application for comparing performance data of the root object with the related object; and using the comparison to verify that the root object is a root cause of the performance issue.

Abstract: In an aspect of the subject matter, a “full” amount of the flash cache (e.g., storage cells) is initially utilized to store data i.e., substantially all of the storage space of the flash cache may be designated to store user data, with the remaining storage space designated to store ECC information (e.g., parity bits) associated with a predefined ECC algorithm utilized to encode the user data. When a bit errors associated with the user data reaches a predefined threshold value, the storage space of the flash cache may transition to store less user data so as to accommodate the space needed to store ECC information associated with a stronger ECC algorithm. The storage space of the flash cache designated to store user data is reduced, while the storage space designated to store ECC information is increased to accommodate the stronger ECC algorithm.

Abstract: A system and method for specifying the placement of and managing the placement of objects on a distributed networked grid. Some embodiments of the invention comprise a user interface for specifying rules for the placement of objects on the grid and an execution module that places objects on the grid based on the rules specified. Rules may comprise a matching criteria specification, which determines whether a rule applies to a particular object, and a placement specification, which directs the locations at which an object will be placed under the rule.

Abstract: Techniques for notifying clients whose locks did not persist about a loss of lock state after a failover event are disclosed herein. According to one embodiment, a storage node of a high availability cluster constructs a lock data structure based on a mirrored lock data structure for a lock on a data container. The storage node receives a data operation request permitted by the lock from a client to operate on the data container. The storage node instructs to terminate a connection with the client if the lock data structure for the lock on the data container has been recently reconstructed. Accordingly, the client revokes the connection and refrains from committing the locally cached operations to the data container stored in the storage node. The client has an option to reconnect with the cluster and regain the data container privilege controlled by the lock.

Abstract: Individual wire defects in a data transfer/communication system that employs differential signaling can be detected during connectivity verification of a link prior to link training by individually disconnecting wires by wire polarity and testing the link. For example, the positive transmit wire of a lane may be verified by disconnecting the negative transmit wire of the lane and performing link connectivity verification. If the link passes connectivity verification, then the positive transmit wire of the lane is functioning normally. Connectivity of the negative transmit wire of the lane may then be verified by disconnecting the positive transmit wire of the lane and determining if the passes the connectivity verification.

Abstract: Systems and methods herein obtain snapshots of one or more applications, while having minimal impact on performance of the applications. Embodiments may involve obtaining snapshots of modified application data within a node without disturbing the operations of the application. Systems may be configured to leverage the fast characteristics of local storage devices while utilizing the advantages provided by the remote storage of data. One or more applications may be preconfigured to record data modifications in one or more associated application state storages located on local memory. Further, one or more systems may operate in the background to identify modifications made to the associated application state storages and to determine an appropriate moment to take one or more snapshots of the stored data and transfer the one or more snapshots to one or more remote locations to be stored. Several factors may be used to determine when an appropriate moment has occurred.

Abstract: Methods and systems for collecting and processing quality of service (QOS) data are provided. A collection module receives the QOS data from a storage operating system for a plurality of storage volumes at time t2, when a process for collecting the QOS data began at time t1 such that t2>t1. The collection module estimates a QOS data value for time t1 and provides the estimated QOS data value to a performance manager that uses the estimated QOS data value for monitoring QOS for the plurality of storage volumes using a plurality of resources for processing input/output (I/O) requests.

Abstract: Systems and methods disclosed herein provide intelligent filtering of system log messages having low utility value. In providing the filtering, the systems and methods determine the utility value of a system log message and delete the message from the system log if the message is determined to be of low utility value. As such, embodiments herein provide an system log filter, which reduces the amount of data stored in the system log based on the utility value of the message.

Abstract: A novel RDMA connection failover technique that minimizes disruption to upper subsystem modules (executed on a computer node), which create requests for data transfer. A new failover virtual layer performs failover of an RDMA connection in error so that the upper subsystem that created a request does not have knowledge of an error (which is recoverable in software and hardware), or of a failure on the RDMA connection due to the error. Since the upper subsystem does not have knowledge of a failure on the RDMA connection or of a performed failover of the RDMA connection, the upper subsystem continues providing requests to the failover virtual layer without interruption, thereby minimizing downtime of the data transfer activity.

Abstract: A method includes binding, using a plurality of processors, a process to a wildcard address and a port on each of a plurality of nodes. The process receives, on a redirector node, a first request for a first address of a first volume located on the cluster from a first client. The first request is sent to the port and a first address associated with a first virtual local area network (VLAN) that is not the wildcard address. The process determines the first address from the first request and a name of the first VLAN based on the first address. The process determines a first node that contains information regarding the first volume and an address of the first node that is part of the first VLAN. The process determines that a volume identifier associated with the first volume of the first request is present on a volume list.

Abstract: Methods and systems for managing resources in a storage system are provided. As an example, one method includes determining a maximum node utilization of a first node operating in a networked storage environment for storing information at a plurality of storage devices, where the maximum node utilization is based on a first factor that tracks utilization of a processor of the first node; comparing the maximum node utilization to a first threshold value; and triggering an event when the maximum node utilization has reached the first threshold value.

Abstract: Methods and systems for managing resources in a storage system are provided. The methods include tracking performance of a plurality of resources used for reading and writing information at storage devices in a networked storage system, each resource represented by a logical object in a hierarchical structure and performance data associated with each logical object is maintained by a processor executing a management application out of a memory device; identifying a root object associated with a resource having a performance issue as indicated by a threshold violation for the resource; selecting a related object associated with a resource similar to the resource of the root object by the management application for comparing performance data of the root object with the related object; and using the comparison to verify that the root object is a root cause of the performance issue.

Abstract: In one embodiment, a layered file system includes a volume layer and an extent store layer configured to provide sequential log-structured layout of data and metadata on solid state drives (SSDs) of one or more storage arrays. The data is organized as variable-length extents of one or more logical units (LUNs). The metadata includes volume metadata mappings from offset ranges of a LUN to extent keys and extent metadata mappings of the extent keys to storage locations of the extents on the SSDs. The extent store layer maintaining the extent metadata mappings determines whether an extent is stored on a storage array, and, in response to determination that the extent is stored on the storage array, returns an extent key for the stored extent to the volume layer to enable global inline de-duplication that obviates writing a duplicate copy of the extent on the storage array.

Abstract: A method and system for dynamic rule-based automatic crash dump analysis are described. In an example, a dynamic rule-based crash dump analysis system retrieves debug symbol data, rules, and commands from a server over a network. The actions are executed based on the retrieved rules in order to automatically analyze a crash dump using a debugger and the debug symbol data. During the process of analyzing the crash dump, the system parses output from the debugger for further rule processing and creates a human-readable analysis file from the parsed output.