Multipath redundant storage system architecture and method
Disclosed is a storage system and method that provides multi-path bus and component interconnection and isolation in a data storage system. A plurality of data storage devices in a removable assembly are connected to a fabric that is configurable to connect some or all of the data storage devices to a disc controller and configurable to isolate one or more data storage devices from the disc controller. Multiple controllers, fabrics, and interconnecting buses may be employed to provide redundancy in the event of a connector, bus, or controller failure. Computer program code operating in a host, interface controller, and/or disc controller configures the fabric to isolate failed devices and may be employed to optimize data transfer rates. Data storage devices may be multi-ported. The fabric may comprise any device or devices capable of configurably interconnecting data storage devices to one or more controllers and may comprise multiplexers, cross point switches, port bypass controllers. Fabrics may also provide translation or conversion of one bus or interface format to another format.
a. Field of the Invention
The present invention pertains generally to data storage systems and more specifically to a system and method of interconnection of storage components in fault tolerant data storage systems.
b. Description of the Background
Data storage systems may comprise one or more disc drives connected to one or more disc controllers that are connected to a host or network interface. Each component of the storage system, such as disc drives, controllers, connectors, and wiring are a potential point of failure in the system. Some systems, such as personal computers, for example, may lose access to data in the event of a failure of a controller, bus, or connector. Access to data may require that a failed component be repaired or replaced or that a disc drive be installed in another system to access data. Failure of a disc drive usually results in loss of stored data. Larger storage systems may employ redundancy methods such as RAID to distribute data across a plurality of drives such that data is not lost in the event of a drive failure. In a RAID system, data from the failed drive may be copied from a mirror drive, or the data may be reconstructed from data and parity information on functioning drives. After the failure of a disc or disc controller, the system may often operate in a reduced performance condition until failed components are replaced or repaired. Failure of a bus may require removal of drives and installation of the drives in another fixture or system in order to access data.
The level of fault tolerance, storage capacity, operating life, and data availability are key contributors to the value of a storage system. Fault tolerance may be expressed in terms of the number of failures (both sequential and simultaneous) of discs, controllers, and buses that may be incurred while still maintaining data integrity and data access. Storage capacity reflects the number of disc drives, capacity of each drive, and data encoding methods used. As the number of drives increases, the number of interconnections and likelihood of failure increases. Storage system operating life is reflected in the longevity of components and level of fault tolerance of the system. Spare disc drives may be employed to store copied or reconstructed data to extend operation of the system after the failure of a disc drive. Data availability may be expressed in terms of data transfer rates, fault tolerance, and system performance following failure of one or more components.
The commercial viability of a storage system reflects the architectural decisions and component selections made by the designer to provide a desired level of fault tolerance, storage capacity, operating life, and data availability. Components with very long MTBF (mean time between failure) ratings may adversely affect system cost.
SUMMARY OF THE INVENTIONEmbodiments of the present invention furnishes redundant storage system architectures and isolation methods that provide fault tolerance in data storage systems and that can be employed to eliminate single points of failure.
Embodiments of the present invention therefore can comprise a data storage system comprising: a multiple disc assembly containing a plurality of data storage devices disposed within having at least one connector that provides a plurality of signals and that has at least one independent signal for each data storage device of the plurality of data storage devices; a multiple disc assembly receptacle adapted to receive the assembly having a fixture connector that engages the at least one connector; at least one disc controller; and at least one fabric that is configurable such that the fabric can selectively connect the at least one independent signal for each data storage device of the plurality of data storage devices to the disc controller when in a first configuration and can selectively disconnect the at least one independent signal for each data storage device when the fabric is in another configuration.
Embodiments of the present invention can further comprise a multiple disc assembly comprising: a plurality of data storage devices disposed in the assembly; a connector that communicates signals. from the assembly to a fixture adapted to receive the assembly; and a fabric disposed in the assembly in communication with the connector that is configurable to selectively connect and disconnect at least one data storage device of the plurality of data storage devices to at least one signal of the connector.
Embodiments of the present invention can further comprise a removable data storage assembly comprising: a plurality of data storage devices arranged as pairs disposed in the assembly, the assembly having at least two pairs of data storage devices; and a connector that provides external communication for at least one independent signal for each pair of data storage device of the plurality of data storage devices.
Embodiments of the present invention can further comprise a data storage system comprising: a multiple disc assembly containing a plurality of dual ported data storage devices and having at least one connector that communicates at least two independent signals to a fixture and having a first fabric configurable to connect a first port of each data storage device of the plurality of data storage devices to a first signal of the at least two independent signals and having a second fabric configurable to connect a second port of each data storage device of the plurality of data storage devices to a second signal of the at least two independent signals; a multiple disc assembly receptacle adapted to receive the assembly having a fixture connector that engages the at least one connector; and at least one disc controller that can access at least one data storage device of the plurality of data storage devices through the fixture connector.
Embodiments of the present invention can further comprise a method of configuring a data storage system having a multiple disc assembly containing a plurality of data storage devices installed in a multiple disc assembly receptacle and at least one fabric connected to the assembly, said method comprising: detecting an error in said data storage system; identifying one data storage device of the plurality of data storage devices contained in the assembly as being inoperative; and configuring the at least one fabric to isolate the at least one data storage device.
Embodiments of the present invention can additionally comprise a data storage system comprising: a multiple disc assembly containing a plurality of data storage devices and having a connector that provides at least one separate signal line for each pair of data storage device of the plurality of data storage devices; a fixture connected to a host system having a disc controller and fabric disposed within, the fixture having a multiple disc assembly receptacle adapted to receive the assembly and communicate signals therewith; and computer program operable to detect an error in the storage system and to identify an inoperative data storage device in the assembly and to configure the fabric to isolate the inoperative data storage device.
Embodiments of the present invention can further yet comprise a data storage system comprising: a multiple disc assembly containing a plurality of data storage devices and at least one fabric that can be configured to connect and disconnect each data storage device of the plurality of data storage devices to at least one signal of a connector that communicates signals external to the assembly; a fixture having a disc controller disposed within and having a multiple disc assembly receptacle adapted to receive the assembly and communicate therewith; and computer program code that detects an error in the storage system and identifies an inoperative data storage device in the assembly and that configures the at least one fabric to isolate the inoperative data storage device.
BRIEF DESCRIPTION OF THE DRAWINGSIn the drawings,
Embodiments of the present invention provide redundant components and data paths, and isolation of points of failure within a storage subsystem such that data access may be maintained following failure of a bus or component. Failures may most frequently occur in connectors and components with moving parts, such as disc drives, for example. In general, electronic components, such as integrated circuits, may exhibit a lower rate of failure than connectors or disc drives.
Embodiments of the present invention are applicable to differing storage architectures including systems that employ arrays of single or multiple discs installed in cabinet fixtures and systems that employ removably installable multiple disc assemblies. A multiple disc assembly is defined as a removably installable unit of a predefined size, shape and connector configuration that can contain differing internal data storage devices, components and configurations. In one embodiment, a multiple disc assembly may comprise a first number of 3½-inch discs while another embodiment may comprise a different number of 2½-inch discs. Various multiple disc assembly embodiments may be installed into a single fixture design. This allows a single fixture (cabinet, shelf, etc.) design to be used to produce systems of varying storage capacity, data rate, and processing power. Multiple disc assembly embodiments may vary in complexity, ranging from units that contain only discs and connectors to units that comprise discs, one or more fabrics, one or more disc controllers, and one or more interface controllers. Multiple disc assembly embodiments may employ interfaces such as fibre channel, for example, that allow devices ranging from simple storage devices, to intelligent disc and interface controllers to be used while employing the same connectors. Computer program code operating in a host or other system reflects the complexity of the multiple disc assembly. Multiple disc assemblies may simplify storage system assembly and upgrade, and may reduce the likelihood of radio frequency emissions. A multiple disc assembly receptacle is defined as a receptacle in a shelf, rack, enclosure, or other fixture into which individual multiple disc assemblies that can vary in internal architecture can be removably installed. Embodiments of the present invention may be employed to create storage systems wherein a multiple disc assembly may be considered a “maintenance-free” storage appliance. Multiple disc assembly embodiments may provide one or more spare drives, multiple buses and spare controller capacity such that it may operate for extended periods without user intervention, even after failure of a bus, controller, and/or one or more disc drives. Embodiments of the preset invention may provide levels of fault tolerance sufficient to provide high performance operation after component failures.
The architecture of
Loop bypass methods may be employed to isolate one or more drives. More than one drive may be connected to each port of a port bypass controller.
Loop bypass architectures may employ a plurality of drives connected to each port wherein each drive is dual ported.
Two or more dual ported disc drives may be connected to each port of a port bypass controller.
In addition to buses, connectors, disc drives, fabrics and controllers, isolation and redundancy methods may further applied to power distribution in a storage system such that the system has no single point of failure that might render the system inoperative.
The foregoing figures have included switches, switching devices, port bypass switches, and fabrics to provide a configurable connection between data storage devices and disc controllers. The term fabric shall refer to any type of device that can provide a configurable connection between data storage devices and disc controllers and shall include fibre channel fabrics, switches, cross-point switches, multiplexers, port bypass controllers and other devices. A fabric may replace the depicted switches, switching devices, or port bypass controllers in the figures.
Embodiments of the present invention can be advantageously employed with a multiple disc assembly (MDA) that comprises a plurality of storage devices and that is inserted into or removed from a cabinet or other fixture as a single unit. The MDA may contain storage devices, may contain storage devices and fabrics, may contain storage devices, fabrics and disc controllers, or may contain data storage devices, fabrics, disc controllers and interface controllers. In other words, embodiments of the present invention as exemplified by the figures may be partitioned between components that are disposed in an MDA and components that are disposed in a cabinet, shelf or other fixture. Such partitioning may reflect MDA size, number of connectors, interface types, drive strength of bus signals, and other factors. In some embodiments, an MDA may employ transversely mounted storage devices where the devices are mounted with the longest axis of the body of at least one storage device orthogonal to the direction of insertion of the MDA into a cabinet, shelf or other fixture. These embodiments allow connectors of storage devices, such as disc drives, for example, to directly engage connectors disposed on a backplane, eliminating intermediate connectors, cables and the like and the additional possible points of failure introduced by intermediate connections.
Computer program code operating in a host system and/or one or more interface controllers, and/or one or more disc controllers is employed to configure fabrics of the present invention. Fabrics may be controlled by computer program code operating in one or more host computers. Such program code may include performance monitoring and load balancing functions. Configuration of fabrics may be performed as a result of a detected failure, or in response to other conditions including load, data type, data size, data storage format, desired response time, etc. as may reflect services provided such as transaction processing, or video streaming, for example. One or more disc controllers may control fabrics. Computer program code operating in a disc controller may configure fabrics in response to a failure or other condition. Configuration of fabrics may be shared between one or more host computers and one or more disc controllers. As previously noted, switch control may employ one or more control buses, such as I2C, may employ one or more disc buses, or both. Fabrics may be mapped as a device on one or more disc array buses and control signals for one or more fabrics may be conveyed across the disc array bus or buses. Some of the figures depict a separate switch control block. In some embodiments the switch control block may be a part of the fabric.
Embodiments of the present invention can be employed to provide maintenance free multiple disc storage assemblies that can be installed and removed in fixtures such as storage cabinets, bays, shelves, and the like. The multiple interface controllers, disc controllers, buses and fabrics allow continued operation following failure of a disc, disc controller, interface controller, connector, or bus. Systems with a large number of drives may employ a third bus as illustrated in
The foregoing description has employed various descriptions employing disc drives and disc controllers to illustrate embodiments of the present invention. Embodiments of the present invention are not limited to a specific number of data storage devices and are not limited to the type of data storage device, including storage media type and bus type. Disc controller shall refer to any type of controller employed to access data from storage devices. Disc controllers may also provide fault tolerant data formatting functions such as RAID, ECC, or other formats. Data storage devices may comprise any type of data storage device including electrical, magnetic, optical, or chemical data storage devices including but not limited to hard disc drives, optical drives, RAM drives including solid state memory devices, and the like and may include combinations thereof and further may include combinations of volatile and non-volatile data storage devices. The fabric or fabrics interconnecting one or more disc controllers and one or more storage devices may be any device or devices that allows configurable connections between disc controllers and storage devices and may include interface type and data format translation. For example, a fabric may convert serial attached SCSI storage device data and interface signals into fibre channel signals that are communicated to a controller. Interface controllers may provide interface type and data format conversion and may also execute computer program code to configure one or more fabrics.
The foregoing description of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed, and other modifications and variations may be possible in light of the above teachings. The embodiment was chosen and described in order to best explain the principles of the invention and its practical application to thereby enable others skilled in the art to best utilize the invention in various embodiments and various modifications as are suited to the particular use contemplated. It is intended that the appended claims be construed to include other alternative embodiments of the invention except insofar as limited by the prior art.
Claims
1. A data storage system comprising:
- a multiple disc assembly containing a plurality of data storage devices disposed within having at least one connector that provides a plurality of signals and that has at least one independent signal for each data storage device of said plurality of data storage devices;
- a multiple disc assembly receptacle adapted to receive said assembly having a fixture connector that engages said at least one connector;
- at least one disc controller; and
- at least one fabric that is configurable such that said fabric can selectively connect said at least one independent signal for each data storage device of said plurality of data storage devices to said disc controller when in a first configuration and can selectively disconnect said at least one independent signal for each data storage device when said fabric is in another configuration.
2. The system of claim 1 wherein said at least one fabric comprises a port bypass controller.
3. The system of claim 1 wherein said at least one fabric comprises a cross point switch.
4. The system of claim 1 wherein said at least one fabric is configurable by a host system.
5. The system of claim 1 further comprising at least one interface controller that conveys signals between said at least one disc controller and an external interface and that is operable to configure said at least one fabric.
6. The data storage system of claim 1 wherein said at least one connector has at least two independent signals for each data storage device of said plurality of data storage devices.
7. The system of claim 6 further comprising:
- a second fabric; and
- a second disc controller wherein said at least one fabric is configurable to connect a first signal of said at least two independent signals for each data storage device of said plurality of data storage devices to said at least one disc controller and said second fabric is configurable to connect a second signal of said at least two independent signals for each data storage device of said plurality of data storage devices to said second disc controller.
8. The system of claim 7 comprising at least one interface controller that conveys signals between said at least one disc controller and an external interface and that is operable to configure said at least one fabric and said second fabric.
9. The system of claim 8 comprising a second interface controller that conveys signals between said at least one disc controller and said second disc controller and an external interface, and that is operable to configure said at least one fabric and said second fabric.
10. A multiple disc assembly comprising:
- a plurality of data storage devices disposed in said assembly;
- a connector that communicates signals from said assembly to a fixture adapted to receive said assembly; and
- a fabric disposed in said assembly in communication with said connector that is configurable to selectively connect and disconnect at least one data storage device of said plurality of data storage devices to at least one signal of said connector.
11. A removable data storage assembly comprising:
- a plurality of data storage devices arranged as pairs disposed in said assembly, said assembly having at least two pairs of data storage devices; and
- a connector that provides external communication for at least one independent signal for each pair of data storage device of said plurality of data storage devices.
12. A data storage system comprising:
- a multiple disc assembly containing a plurality of data storage devices and having at least one connector that communicates at least one signal to a fixture and having a fabric configurable to connect each data storage devices of said plurality of data storage devices to said at least one signal and configurable to isolate at least one data storage device of said plurality of data storage devices from said at least one signal while at least one other data storage device remains connected to said signal;
- a multiple disc assembly receptacle adapted to receive said assembly and having a fixture connector that engages said at least one connector; and
- at least one disc controller that can access at least one data storage device of said plurality of data storage devices through said fixture connector.
13. The data storage system of claim 12 wherein said plurality of data storage devices are arranged in pairs with each pair having a connection to said fabric and said fabric being configurable to connect each pair of data storage devices to said at least one signal.
14. A data storage system comprising:
- a multiple disc assembly containing a plurality of dual ported data storage devices and having at least one connector that communicates at least two independent signals to a fixture and having a first fabric configurable to connect a first port of each data storage device of said plurality of data storage devices to a first signal of said at least two independent signals and having a second fabric configurable to connect a second port of each data storage device of said plurality of data storage devices to a second signal of said at least two independent signals;
- a multiple disc assembly receptacle adapted to receive said assembly having a fixture connector that engages said at least one connector; and
- at least one disc controller that can access at least one data storage device of said plurality of data storage devices through said fixture connector.
15. The data storage system of claim 14 wherein said plurality of data storage devices are arranged in pairs with each pair of data storage devices having a first port connected to said first fabric and each pair of data storage devices having a second port connected to said second fabric, said first fabric configurable to connect and disconnect each pair of data storage devices to said first signal and said second fabric configurable to connect and disconnect each pair of data storage devices to said second signal.
16. The data storage system of claim 14 further comprising:
- a second disc controller having two ports with a first port of said two ports connected to said first signal and having a second port of said two ports connected to said second signal.
17. A data storage system comprising:
- a multiple disc assembly containing a plurality of data storage devices and at least one fabric and at least one disc controller disposed within and having at least one connector that communicates at least one signal to a fixture, said fabric configurable to connect each data storage devices of said plurality of data storage devices to said disc controller, said disc controller connected to said at least one signal; and
- a multiple disc assembly receptacle adapted to receive said assembly and having a fixture connector that engages said at least one connector that provides communication of signals with said at least one disc controller.
18. The data storage system of claim 17 wherein said plurality of data storage devices are arranged in a plurality of groups of at least two data storage devices each and said at least one fabric is configurable to connect and disconnect each group of said plurality of groups to said at least one disc controller.
19. A data storage system comprising:
- a multiple disc assembly containing a plurality of dual ported data storage devices, a first disc controller, a second disc controller, a first fabric and a second fabric disposed within and having at least one connector that communicates at least two signals to a fixture, said plurality of data storage devices each having a first port connected to said first fabric and having a second port connected to said second fabric, said first disc controller and said second disc controller being dual ported and each having a first port connected to said first fabric and having a second port connected to said second fabric, said first disc controller connected to a first signal of said at least two signals and said second disc controller connected to a second signal of said at least two signals; and
- a multiple disc assembly receptacle adapted to receive said assembly and having a fixture connector that engages said at least one connector.
20. The data storage system of claim 19 wherein said plurality of data storage devices are arranged in a plurality of groups of at least two data storage devices and said at least one first fabric is configurable to connect and disconnect each group of said plurality of groups to said first disc controller.
21. The data storage system of claim 19 further comprising at least two voltage regulators wherein a first voltage regulator of said at least two voltage regulators provides power to said first fabric and a second voltage regulator of said at least two voltage regulators provides power to said second fabric.
22. The data storage system of claim 19 further comprising two interface controllers interposed between said connector, and said first disc controller and said second disc controller wherein a first interface controller of said two interface controllers is connected to said first disc controller using a first bus and is connected to said second disc controller using a second bus and a second interface controller is connected to said first disc controller using said fist bus and is connected to said second disc controller using said second bus and wherein said first interface controller and said second interface controller are connected to said first signal and to said second signal.
23. A method of configuring a data storage system having a multiple disc assembly containing a plurality of data storage devices installed in a multiple disc assembly receptacle and at least one fabric connected to said assembly, said method comprising:
- detecting an error in said data storage system;
- identifying one data storage device of said plurality of data storage devices contained in said assembly as being inoperative; and
- configuring said at least one fabric to isolate said at least one data storage device.
24. The method claim 23 wherein said step of configuring said at least one fabric further comprises configuring a port bypass controller.
25. The method claim 23 wherein said step of configuring said at least one fabric further comprises configuring a cross point switch.
26. The method claim 23 wherein said step of configuring said at least one fabric further comprises configuring a multiplexer.
27. The method claim 23 further comprising removing power from said at least one data storage device.
28. A data storage system comprising:
- a multiple disc assembly containing a plurality of data storage devices and having a connector that provides at least one separate signal line for each pair of data storage device of said plurality of data storage devices;
- a fixture connected to a host system having a disc controller and fabric disposed within, said fixture having a multiple disc assembly receptacle adapted to receive said assembly and communicate signals therewith; and
- computer program operable to detect an error in said storage system and to identify an inoperative data storage device in said assembly and to configure said fabric to isolate said inoperative data storage device.
29. A data storage system comprising:
- a multiple disc assembly containing a plurality of data storage devices and at least one fabric that can be configured to connect and disconnect each data storage device of said plurality of data storage devices to at least one signal of a connector that communicates signals external to said assembly;
- a fixture having a disc controller disposed within and having a multiple disc assembly receptacle adapted to receive said assembly and communicate therewith; and
- computer program code that detects an error in said storage system and identifies an inoperative data storage device in said assembly and that configures said at least one fabric to isolate said inoperative data storage device.
Type: Application
Filed: Apr 2, 2004
Publication Date: Oct 13, 2005
Inventors: David DeCenzo (Pueblo, CO), William Pagano (Colorado Springs, CO), Stephen Sicola (Monument, CO)
Application Number: 10/817,565