Abstract: In one embodiment, at least one characteristic of an ecosystem is monitored. The ecosystem includes i) a plurality of compute resources, and ii) a number of storage applications, wherein the number of storage applications are configured to use a plurality of storage access protocols. Which of the storage access protocols are implemented by which of the compute resources are tracked; and in response to the monitored characteristic(s) of the ecosystem being found unacceptable, an attempt is made to improve the monitored characteristic(s) by repurposing one or more of the compute resources to implement, or not implement, one of the storage access protocols.

Abstract: Embodiments include methods, apparatus, and systems for managing resources in a storage system. One embodiment includes a method that discovers storage resources for arrays in a cluster storage system. A table defines how the resources are divided among the arrays and is provided to the arrays so each array can identify which resources to manage.

Abstract: Embodiments include methods, apparatus, and systems for choosing nodes in a multi-network. One embodiment includes a method that receives demand characteristics for nodes to form a cluster in multi-network and collapses groups of nodes in the multi-network to form a node group. The method then selects nodes that satisfy the demand characteristics in order to form the cluster.

Abstract: Embodiments of the present invention provide methods and systems for leadership allocation in a distributed computer system. In certain embodiments of the present invention, a leader-election-service process runs within each node of a distributed computer system, together cooperatively providing a distributed-leader-election service. The distributed-leader-election service employs a distributed consensus service to manage distributed state information related to roles and leadership allocation within a distributed computer system. Client processes within each node interface with the leader-election-service process of the node in order to assume leadership of particular roles within the distributed computer system. Leadership-allocation management is thus centralized, within each node. In alternative embodiments, the distributed-leader-election service may be implemented as a collection of library routines that run in the context of client processes.

Abstract: In one embodiment, at least one characteristic of an ecosystem is monitored. The ecosystem includes i) a plurality of compute resources, and ii) a number of storage applications, wherein the number of storage applications are configured to use a plurality of storage access protocols. Which of the storage access protocols are implemented by which of the compute resources are tracked; and in response to the monitored characteristic(s) of the ecosystem being found unacceptable, an attempt is made to improve the monitored characteristic(s) by repurposing one or more of the compute resources to implement, or not implement, one of the storage access protocols.

Abstract: Systems and methods for managing a fault tolerant system are disclosed. In one implementation a system for managing a fault tolerant system comprises a configuration manager that receives configuration events from the fault tolerant system, a fault normalizer that receives fault events from the fault tolerant system; and a fault tolerance logic engine that constructs a model of the fault tolerant system based on inputs from the configuration manager and generates reporting events in response to inputs from the fault normalizer.

Abstract: Various method and system embodiments of the present invention are directed to data and data-state-describing data structures employed in storage components of a distributed data-storage system. In one embodiment of the present invention, a hierarchical data structure stores the data state of a component data-storage system of a distributed data-storage system. In another embodiment of the present invention, a data-block address, stored in a computer-readable memory within a component data-storage system of a distributed data-storage system, includes a block identifier and additional data fields that serve to uniquely specify the addressed data block when multiple copies of the data block are stored in the component data-storage system under different redundancy schemes.

Abstract: A data storage system includes file system processes, array controller processes, and one or more data storage devices accessed through the array controller processes. Certain file system access operations are diverted from the file system processes so that the array controller processes can directly perform the requested access operation. The array controller processes then communicate with the file system processes so that the file system processes may update the file system data management arrangement as appropriate. Thus, the hierarchical file system management is maintained while the storage media may be accessed through one of multiple routes in response to a file system access operation request. The particular access route for a given file in the system is determined by a route determinant characteristic of the file. The access route for a file may be switched when it is determined that the route determinant characteristic for the file is inconsistent with the then specified route for that file.

Abstract: A data storage system (201) includes file system processes (308), array controller processes (300), and one or more data storage devices (203) accessed through the array controller processes. Certain file system access operations are diverted from the file system processes (308) so that the array controller processes (300) can directly perform the requested access operation. The array controller processes (300) then communicate with the file system processes (308) so that the file system processes may update the file system data management arrangement as appropriate. In this way, the hierarchical file system management is maintained while the storage media may be accessed directly through the array controller processes (300) in response to a file system access operation request.

Abstract: A data storage device may be constructed with a disk array; an array controller for controlling the array; and a queue for queuing commands from a host system to the disk array. Programming installed on the array controller adjusts a logical size of the queue to optimize performance.

Abstract: A data storage device may be constructed with a disk array; an array controller for controlling the array; and a queue for queuing commands from a host system to the disk array. Programming installed on the array controller adjusts a logical size of the queue to optimize performance.

Abstract: An automated process of assigning storage resources to logical units (“LU's”) is informed of contention avoidance, in order to yield reduced contention. LU's are defined and assigned to logical groups. Then a process having automatic aspects assigns physical storage resources to the LU's responsive to the LU logical groupings. In another aspect, the assignment of physical storage resources is responsive to the logical groups in a manner such that the LU's in one logical group cannot contend for physical storage resources with the LU's in another group, but the LU's in a single group may contend with one another. In lieu of LU's being manually assigned to logical groups, certain contention avoidance relations among the LU's are defined and then an automated process assigns the LU's to logical groups responsive to the contention avoidance relations.

Abstract: A data storage system includes file system processes, array controller processes, and one or more data storage devices accessed through the array controller processes. Certain file system access operations are diverted from the file system processes so that the array controller processes can directly perform the requested access operation. The array controller processes then communicate with the file system processes so that the file system processes may update the file system data management arrangement as appropriate. Thus, the hierarchical file system management is maintained while the storage media may be accessed through one of multiple routes in response to a file system access operation request. The particular access route for a given file in the system is determined by a route determinant characteristic of the file. The access route for a file may be switched when it is determined that the route determinant characteristic for the file is inconsistent with the then specified route for that file.

Abstract: A storage system maintains a log image consisting of log records kept in a page log and log records kept in a distributed log kept in staging buffers of multiple storage media. The distributed log contains incremental log records that are occasionally forced to the storage media prior to filling an entire page of log records. The storage system has a log recovery manager that recovers the log image following a failure. The log recovery manager first reads the log records from the page log to reproduce a majority of the log image. The log recovery manager then scans the distributed log to locate any more recent log records that may exist. Once a more recent log record is found, the log recovery manager adds it to the recaptured log image and proceeds to find even more recent log records. To speed the recovery, the log recovery manager intelligently begins the search for the next incremental log record at a location in the storage system that the next log record is likely to reside.

Abstract: A compression system stores meta-data in the compressed record to allow better access and manage merging data. Markers are added to the compression stream to indicate various things. Each compressed record has a marker to indicate the start of the compressed data. These markers have sector number as well as the relocation block numbers embedded in their data. A second marker is used to indicate free space. When compressed data is stored on the disk drive, free space is reserved so that future compression of the same, or modified, data has the ability to expand slightly without causing the data to be written to a different location. Also the compressed data can shrink and the remaining space can be filled in with this free space marker. A third type of marker is the format pattern marker. Compression algorithms generally compress the format pattern very tightly. However, the expectation is that the host will write useful data to the storage device.

Abstract: An automated process of assigning storage resources to logical units (“LU's”) is informed of contention avoidance, in order to yield reduced contention. LU's are defined and assigned to logical groups. Then a process having automatic aspects assigns physical storage resources to the LU's responsive to the LU logical groupings. In another aspect, the assignment of physical storage resources is responsive to the logical groups in a manner such that the LU's in one logical group cannot contend for physical storage resources with the LU's in another group, but the LU's in a single group may contend with one another. In lieu of LU's being manually assigned to logical groups, certain contention avoidance relations among the LU's are defined and then an automated process assigns the LU's to logical groups responsive to the contention avoidance relations.

Abstract: A data storage system (201) includes file system processes (308), array controller processes (300), and one or more data storage devices (203) accessed through the array controller processes. Certain file system access operations are diverted from the file system processes (308) so that the array controller processes (300) can directly perform the requested access operation. The array controller processes (300) then communicate with the file system processes (308) so that the file system processes may update the file system data management arrangement as appropriate. In this way, the hierarchical file system management is maintained while the storage media may be accessed directly through the array controller processes (300) in response to a file system access operation request.

Abstract: A data computing system has a host computing unit connected to a peripheral data storage system. The data storage system has a storage medium and a storage space manager to manage space on the storage medium. When the host computing unit deallocates a designated storage space by deleting a file or moving data, the host generates a predefined data pattern and writes that data pattern to the storage space being deallocated. The peripheral data storage system detects the predefined pattern and, in response, deallocates the designated storage space to which the data pattern is being written. As a result, the data storage system now reflects the deallocation that has already been realized by the host software. When the host initiates a read operation for a designated storage space, the data storage system determines whether the storage space is presently unallocated. If it is, the data storage system generates the predefined data pattern and sends that pattern back to the host.

Abstract: A transaction log stored in a first memory is posted to one of two separate log areas, selectively, on a storage system having a plurality of storage media such as disk drives. Posting occurs to a "disk log" area when a page-full status of the first memory transaction log is detected. Alternatively, when a posting request occurs before a page-full status of the transaction log has been detected, then posting occurs immediately to a least busy disk of a "staging log" area. Posting occurs to the least busy disk of the "staging log" area by circumventing normal storage system data management and redundancy techniques such that data in the staging log area is not kept redundantly on the storage media. Rather redundancy is maintained by the fact that the transaction log remains with the first memory in addition to being copied to the staging log area. The staging log area includes reserved space on each of the plurality of storage media and is divided into logically separate portions on each of the storage media.

Abstract: A data storage system has an administrator tool that presents an existing LUN arrangement composed of one or more types of LUNs and an available capacity for the existing LUN arrangement. The administrator tool provides a graphical user interface (UI) that allows the administrator to propose different configurations with one or more additional hypothetical LUNs, without in fact creating them. The UI provides a set of controls representative of the different LUN types. The administrator can graphically manipulate the controls to vary characteristics of the hypothetical LUNs. As the administrator manipulates the controls, the system dynamically computes available capacity assuming a LUN arrangement that includes both the existing LUNs and the hypothetical LUNs. The graphical UI reports the changing available capacity as a response to the administrator's manipulation so that the administrator can gain an appreciation of how the hypothetical LUNs might affect available capacity.