DATA PROCESSING SYSTEM WITH BACKPLANE AND PROCESSOR BOOKS CONFIGURABLE TO SUPPPRT BOTH TECHNICAL AND COMMERCIAL WORKLOADS
A processor book designed to support both commercial workloads and technical workloads based on a dynamic or static mechanism of reconfiguring the external wiring interconnect. The processor book is configured as a building block for commercial workload processing systems with external connector buses (ECBs). The processor book is also provided with routing logic to enable to ECBs to be utilized for either book-to-book routing or routing within the same processor book. A table specific wiring scheme is provided for coupling the ECBs running off the chips of one MCM to the chips of the second MCM on the processor book so that the chips of the first MCM are connected directly to the chips of a second MCM that is logically furthest away and vice versa. Once the wiring of the ECBs are completed according to the wiring scheme, the operational and functional characteristics reflect those of a processor book configured for technical workloads.
Latest IBM Patents:
- INTERACTIVE DATASET EXPLORATION AND PREPROCESSING
- NETWORK SECURITY ASSESSMENT BASED UPON IDENTIFICATION OF AN ADVERSARY
- NON-LINEAR APPROXIMATION ROBUST TO INPUT RANGE OF HOMOMORPHIC ENCRYPTION ANALYTICS
- Back-side memory element with local memory select transistor
- Injection molded solder head with improved sealing performance
The present application is a continuation of U.S. patent application Ser. No. 10/425,421, filed on Apr. 28, 2003, titled, “Data Processing System with Backplane and Processor Books Configurable to Support Both Technical and Commercial Workloads,” the contents of which is incorporated herein by reference in its entirety.
CROSS REFERENCE TO RELATED APPLICATIONSThe present application shares specification text and figures with the following co-pending application, which were filed concurrently with the present application: application Ser. No. 10/425,420 (Attorney Docket Number AUS920030001US1) “Processor Book for Building Large Scalable Processor Systems.” The content of the co-pending application is incorporated herein by reference.
BACKGROUND OF THE INVENTION1. Technical Field
The present invention relates generally to data processing systems and in particular to multiprocessor data processing systems. Still more particularly, the present invention relates to a method and system for enabling post-manufacture configuration of multiprocessor data processing system via interconnect method and logic.
2. Description of the Related Art
The evolution of data processing systems for use in commercial applications has occurred at a very rapid pace. This development began with the design and utilization of single processor systems and has evolved to design and utilization of more complex multiple processor systems (MPs). Most of the development has been driven by the increasing need in the industry for greater processing power and faster data operations.
Technical and Commercial servers are two examples of systems that have benefited from the additional processing power and faster overall data operations. Notably, in order to provide the faster overall data operations, quicker access to data is required, and the systems are typically designed with distributed memory systems, with each processor having direct access to an affiliated memory block.
One extension of the increasing the number of processors within the system is the creation of a multi-chip module (or MCM), which provides higher overall frequency. The In particular, the MCM configuration provides increased performance for commercial workloads. Multi-chip modules (MCMs) containing multiple chips each having two or more individual processors has replaced traditional single chip modules (SCMs), which include a single processor. In an MCM, two or more processor chips each comprising multiple processors are interconnected with buses having a particular bandwidth. Thus, for example, a four-processor multi-chip module (MCM) may be designed by interconnecting 4 single-processor chips with 16-byte buses.
Each chip 101 contains internal MCM routing logic 107 that manages the inter-chip data transfers on the various buses. MCM routing logic 107 controls both routing to components within MCM 100 and routing to components connected externally to MCM 100. MCM routing logic 107 reads the destination address contained within the data component being routed and selects the appropriate bus on which to route the data component. For example, communication (collectively described herein as data communication, although instructions may also be routed between processor chips) from a processor on chip S to a processor of either of the adjacent processor chips, T or V, are sent by MCM routing logic 107 of chip S on the MCM buses 103 directly coupling the two chips; However, when communication is desired from a processors on chip S to one on chip U (i.e., the processor chip that is logically farthest away and not directly coupled to S), MCM routing logic 107 sends the communication to the processor on chip U via a hop across one of the two adjacent processor chips, T or V. Routing at each stage of the hop is controlled by MCM routing logic 107 on the particular chip. Each communication path between non-adjacent processors has a higher latency because of the extra hop that is required.
Each chip within MCM 100 connects to other external components including memory (not shown) and I/O devices (not shown) via additional buses (e.g., buses labeled 211, 113 and 115 connected directly to each die. The number of additional buses available for connecting external components (i.e., components other than the other processors) is a function of the size of the chip. Typically, only a fixed number of buses can be connected to each die, and thus the connectivity of each chip is limited by the fixed number of buses. Thus, although the 4-chip MCM has been efficiently designed, the 8-processor or 8-chip system with switch interconnect does not scale in performance or costs.
Recent trends in the development of data processing systems to handle large scale or complex tasks include the implementation of both large scale commercial and technical multiprocessor systems (MPs), which each provide respective operating requirements and functional characteristics.
The primary differences between the system configuration and the required operating parameters of a technical SMP and those of a commercial SMP include the following:
-
- (1) technical SMPs typically have fewer processors than “comparable” commercial SMPs (e.g., scalable commercial SMP may be a 32 or 64 way, while a comparable scalable technical SMP may be a 8 or 16 way; and
- (2) technical SMPs typically have very high memory bandwidth and low memory latency requirements, while commercial SMPs have lower bandwidth requirements (mostly due to usage of sophisticated caching mechanisms).
These significant differences in the processing requirements of technical SMPs from those of commercial SMPs have led to a different design and manufacturing process for systems being utilized for technical versus commercial workloads.
Typically, these SMPs comprise two or more processors manufactured on processor chips that are interconnected via a bus or switch to each other. These chips are also connected to other components such as memory and input/output (I/O) via respective buses.
Systems designed for commercial workloads are generally not optimized at handling a technical workloads. Unlike commercial workloads, technical workloads utilize significantly less processor resources, but require much greater efficiency with respect to memory bandwidth and latency. Technical processing systems (i.e., SMPs designed to handle technical workloads) thus are typically configured differently from commercial ones.
Related patent application, Attorney Docket No. AUS920030001US1 provides a processor book that enables the development of large scale commercial systems. Specifically, that patent teaches the creation of processor books and utilization of the processor books as building blocks for a large scale commercial system. Owners of such a large scale system who have purchased these processor books may also wish to run some technical applications.
From the above it is clear that a processor book designed to handle technical workloads must be configured in a manner that enables fast and efficient MCM chip to MCM chip and processor-to-memory communication. The actual speed or and efficiency of the MCM chip-to-MCM chip and processor chip-to-memory operations (and vice versa) is primarily dependent on the size of the bus interconnecting the components, the distance between the components (i.e., length of the connecting bus), and the number of hops required to go from the first component to the next.
The latter factor is especially relevant when the destination chip is several hops away from the source chip. Each processor chip operates as a buffer at each hop and holds the communicated data until the data buses are available before forwarding the communication data to the next hop or the destination. Communication latency may thus be extremely long in the commercial SMP configuration depicted in
Because of the differences in operation requirements between technical and commercial SMPs, both SMP types are manufactured with different functional design (connectivity of processors, size of buses, etc.). Absent the present invention, much of current art degrades commercial or technical performance in order to support both workloads. Also, the higher requirements for technical workload systems results in higher manufacturing costs for a processor book configured for technical workloads than a processor book configured for commercial workloads as illustrated and described above.
The present invention thus recognizes that there would be significant performance gain and cost savings opportunity if a processor book designed for a commercial workload could be easily re-configured to also support a technical workload. A processing system designed for commercial workloads that could be wired post-manufacture for technical workloads without significant additional logic would be a welcomed improvement. These and other benefits are provided by the invention described herein.
SUMMARY OF THE INVENTIONDescribed is a method and system that enables a commercial workload processor book to be configured post-manufacture to support a technical workload based on a wiring of specific external interconnects according to a pre-established wiring configuration. The processor book is configured as a 8-processor system with two 4-chip MCMs and is utilized as a building block for large scale commercial workload processing systems. The routing logic of the processor book includes additional routing logic to control inter-chip book-to-book routing via the specific external interconnects.
The processor book comprises two multi-chip modules (MCMs) each having a plurality of external interconnects coupled to individual chips within the MCMs. A table is provided with specific wiring scheme indicating a connection sequence to follow when coupling the specific external interconnects connected to the chips of one MCM to the external interconnects of the chips of the second MCM. The chips of the first MCM are connected directly to the chips of a second MCM that is logically furthest away (in terms of number of hops for data transmission within the processor book) and vice versa. Once the wiring of the specific external interconnects are completed according to the wiring scheme, the operational and functional requirements with respect to bandwidth and percentage of memory usage, etc., reflect those of a processor book configured for technical workloads.
In one embodiment, the external buses are the MCM external connector buses (ECBs) typically utilized within a commercial workload processing system to connect a first processor book to another processor book in an industry standard rack. In another embodiment, the Input/Output (I/O) buses of the processor books are utilized to wire the processor chips. The ECBs are then available for coupling to other processor books. According to this latter implementation, a technical workload processing system may be placed within a larger commercial processing system comprising multiple processor books in which certain bandwidth intensive operations are processed at the technical workload processing system while other non-bandwidth intensive operations may occur at the other processor books of the larger commercial processing system.
In addition to the logic, the processor book also comprises a commercial-to-technical selection mechanism that is utilized by a user to select when the processor book is to be utilized as a technical workload processing system or commercial workload processing system. The mechanism may be a latch that is set during power-on of the MCM. In response to the configuration mechanism being set to indicate utilization of the processor book for technical workloads (e.g., the latch being set to high), the routing logic for a technical workload processor book is activated, and the routing logic for a commercial workload processor book is deactivated.
The above as well as additional objectives, features, and advantages of the present invention will become apparent in the following detailed written description.
The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as a preferred mode of use, further objectives, and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawings, wherein:
The present invention provides an innovative processor book that can be configured for either commercial or technical workloads after manufacture. Prior to the present invention, a processor book was designed for either technical workloads (hereinafter referred to as a technical processor book) or commercial workloads (hereinafter referred to as a commercial processor book), with no opportunity to reconfigure the processor book after manufacture. Thus, a processor book designed to handle technical workloads could not be re-configured to handle commercial workloads and vice-versa.
The invention recognizes the differences in operational functionality and physical connectivity and routing between technical and commercial processor books. In particular, technical processor books have very high memory bandwidth and low memory latency requirements, while commercial processor books have lower memory bandwidth and latency requirements (due to large on-board caches). Consequently, one reason why a commercial processor book cannot be utilized to support a technical workload is the additional hops required when communicating between processor chips of different MCMs, resulting in aggregate less memory bandwidth. As utilized herein, communication refers collectively to the transmission of data and/or instructions on a bus interconnecting any two components in the processor book described in the illustrative embodiments. Therefore, according to the illustrative embodiment, the invention provides optimal connectivity among processor chips of different processor chip groups (e.g., MCMs) within a commercial processor book to create a processor chip-to-processor chip non-blocking interconnect topology that provides the operating features required for technical processor books.
The invention is described below with specific reference to an 8-way SMP book comprised of two interconnected 4-way MCMs (i.e., two MCMs including four chips having a single processor per die). Those skilled in the art appreciate that the features described herein and specific references to a 8-way SMP book are meant solely for illustrative purpose and should not be construed as limiting on the invention, which may equally apply to more complex systems (e.g., 16-way, 32-way, etc.) or less complex systems.
As shown in
Each chip of processor book 200 also comprises MCM routing logic 207, which also manages the routing of communication between the first MCM and the second MCM. MCM routing logic 207 controls the routing that occurs on all of the external buses of the MCMs including the MCM-to-MCM bus 211 and MCM ECB 209. As shown, a pair of MCM-to-MCM buses 211 run to and from each processor chip of the first MCM from and to the corresponding processor chip of the second MCM (e.g. S0-S1, T0-T1, etc.).
During processor book operation, communication from a first MCM to the second MCM always requires at least one transfer over an 8-byte bus. For example, a communication from S0 to S1 is routed directly on MCM bus 211. Notably, a communication from S0 to U1 requires two intermediate hops (i.e., S0-T0-U0) along the MCM 16-byte bus before being transmitted across the processor book to U1 on the 8-byte MCM bus. Alternatively, the same communication may be routed via the path S0-S1-T1-U1. Determination of the exact route to take is made by MCM routing logic 207, based on current usage on the various paths, etc. Irrespective of which path is taken, the communication takes two hops before arriving at the destination.
Multiple 8-way processing systems designed according to the configuration shown in
Plug-in connector 325 includes buses, which are the terminating wires of MCM ECBs 209 of processor book 200. Thus, according to the 8-processor configuration of processor book, plug-in connector 325 includes a separate connector bus for each of the 8 output ECBs and each of the 8 input ECBs. Manufacture of system rack 300 is completed separately from that of processor books 200 and thus different manufacturing techniques and/or designs may be utilized to enable the connectivity of processor book 200 to system rack 300 and ultimately to each other processor book.
The passive backplane of system rack 300 includes wiring that is meshed into the base material and inter-connects each backplane connector 321 on system rack 300 similarly to the connectivity illustrated in
When a user of the processor book requires the utilization of the processor book for technical workloads, the user is able to switch the wiring of the ECBs to provide the functionality of a technical processor book. The invention recognizes the need for use of the processor book as a technical workload processor book and provides below a series of additional functional features to enable the processor book 200 developed for plugging into the system rack 200 to be also utilized as a technical workload processor book.
Hardware Wiring DesignTurning now to
-
- A. ECB from 1st MCM into 2nd MCM:
- i. S0 to U1; T0 to V1; U0 to S1; V0 to T1
- B. ECB from 2nd MCM into 1st MCM:
- i. S1 to U0; T1 to V0; U1 to S0; V1 to T0
- A. ECB from 1st MCM into 2nd MCM:
For simplicity, the external buses will be referred to hereinafter as ECBs 209 and the illustrated configuration described with the external buses as ECBs wired to each other by interconnects. However, in another embodiment the invention may be implemented with I/O buses (not shown) being wired in the above manner (S0 to U1, etc.), and the routing logic on the chips is then coded to recognize the wired I/O buses to support a technical processor book. With such an implementation, features of the invention may be extended such that the processor book is connected on the system rack 300 to other processor books via the ECBs to provide a larger commercial system that supports commercial workloads, but which forwards all processing that requires technical processing (i.e., high memory bandwidth and faster processing requirements) to the attached technical processor book.
The illustrated wiring of ECBs 209 of the processor chips provide an 8-byte direct communication path, eliminating the buffering and additional latency required to transfer an operation across one or more hops to the furthest processor chip on the other MCM. Additionally, the illustrated wiring of ECBs 209 provides an additional (alternate) path from each chip to any of the other chips to which a communication is to be routed. For example, communication form S0 to T1 can be routed as S0-T0-T1, S0-S1-T1, or S0-U1-T1. The last routing path is made possible by the wiring of ECBs 209 as illustrated, and the addition of the last routing path significantly increases the available bus bandwidth for inter-chip communication.
The number sequence followed for interconnecting a particular ECB 209 of first MCM 413B to ECB 209 of second MCM 413B of technical processor book 400 is clearly numerated in the illustration of
If a particular processor book being connect to system rack 300 is desired for use with a technical workload, certain changes in the connectivity of the particular processor book is required to support the higher bandwidth requirements without memory affinity. Of course, these changes occur post-manufacture and thus requires a delicate allocation of connections between MCM ECBs at plug-in connector 325. In order to accomplish this switch to a technical workload processor book post-manufacture and in an efficient manner, the invention provides several different embodiments that complete change of the “commercially designed” processor book, which was manufactured to be plugged into the system rack 300 and utilized primarily for commercial workloads, into a processor book that supports technical workloads.
The first method involves the selection of specific routing logic to control MUX selectors that are built into the plug-in connector of the processor book, which is wired for bother commercial and technical implementations. According to this implementation, the plug-in connector 325 of the processor book 200 is designed with MUX selector functionality and includes a set of internal re-routed wires that can be selected via the MUX when a technical book is desired.
In a second method, the backplane connector 321 is manufactured with the wiring and additional logic that enables either manual or dynamic selection of the technical wiring configuration within the backplane connector. This is accomplished manually via a switch or dynamically via the hypervisor or OS.
A third method is provided in which an add-on connector is designed, which internally re-routes the bus connections of the plug-in connector 325, similarly to the first method described above. As illustrated by
Another implementation provides a smart connector having logic for dynamic or manual selection.
During operation, a software signal may be generated that sets the internal routing MUXes 427A to select the MCM ECB inputs over the external ECB inputs as their outputs. Simultaneously, the external routing MUXes 427B are set to select the external ECBs inputs as their outputs. In this way, the re-routing of MCM ECBs according to the table is maintained while the eternal ECBs are routed to the next processor book on system rack 300. Connectivity among the other processor books on system rack 300 is maintained while the processor book operates as a technical processor book.
Returning now to the illustrative embodiment of
The above implementation on the system rack 300 provides a commercial rack with a node dedicated to technical workload (i.e., high memory bandwidth operations). Because this may be completed at the customer level, the customer does not have to purchase a separate technical workload processor book in addition to the commercial workload processor books and rack already available. Thus for example, a customer with a 32 way commercial workload processing system can reconfigure the system into a 24 way commercial workload system and an 8-way technical workload system with very little effort, except perhaps the purchase of an add-on connector.
Logic ConsiderationsControl of the routing features of the technical processor book 400 is provided by built-in MCM routing logic 407 of each processor chip 401, 403, which includes a configuration selection mechanism 410.
According to the illustrative embodiment, the programmer of the processor book selects which of the two controlling logic is activated by setting the selection mechanism 410 to a state (i.e., setting 1 or 0) corresponding to the desired configuration. Thus, selection mechanism 410 may be set to a first state (e.g., 0) that activates the routing logic utilized for a commercial processor book, or alternatively, selection mechanism 410 may be set to a second state (e.g., 1) that activates technical routing logic 408a utilized for a technical processor book. In another embodiment, selection mechanism 410 is a latch that is set to a high or low value, each value corresponding respectively to a selection of technical routing logic 408a or commercial routing logic 408b.
In one embodiment, in which selection of a particular state is not always required, the invention contemplates selection mechanism 410 having a default value that provides a commercial or technical workload configuration. Whenever, the processor chip is powered off, selection mechanism 410 returns to the default value, which may then be changed, if necessary, during power-on-reset by hardware or software programming.
Notably, selection between the routing logic 408a or 408b may also be an automatic feature in some implementations. Accordingly, additional logic checks the ECB wiring when processor book 200 is powered-on to determine whether the ECB wiring conforms to the technical processor book or the commercial processor book. The associated logic for routing communications is then activated and utilized to control the communication between chips during operation. Alternatively, selection of the routing logic using mode bit 410 may be performed automatically by boot code following power-on.
Of course, the operation of processor book 200 as a technical processor book requires ECBs 209 to first be wired according to the configuration provided by
The re-configuration of the processor book from a commercial processor book to one that efficiently handles technical workloads is thus completed by wiring together the ECBs according to the above wiring scheme of
Several differences in operation are observed when the same processor book is configured as a commercial processor book versus when configured as a technical processor book. Most notably, with the new “re-wired” configuration, operations transmitted between any two chips requires at most one intermediate hop. Thus, the additional bandwidth previously utilized when completing multiple hops is available to convey the transactions of the other processors because in the re-configured processor book, only one bus is required to transmit a transaction to the processor chip logically furthest away. The other bus is no longer utilized and is thus available to convey to other bus transactions occurring within the processor book.
The invention substantially reduces transmission latencies caused by the buffering of a communication at multiple processors during transit. An additional benefit provided by the ECB wiring scheme is the provision of multiple transmission paths, which can be utilized to transmit two chip-to-chip transactions simultaneously from a single processor chip.
Additionally, with the re-configured/re-wired processor book an 8-way SMP is provided across all of the memory without requiring or exhibiting any memory affinity. The increased bandwidth for data transmission enables each memory subsystem to run at substantially 100% of capacity since required data transfer does not have to wait on other processes before gaining access to the data buses. Thus, higher memory bandwidth and lower memory latency are achieved from the 8-way processor book originally designed for commercial workloads so that the processor book is optimized to support a technical workload.
Although the invention has been described with reference to specific embodiments, this description should not be construed in a limiting sense. Various modifications of the disclosed embodiments, as well as alternative embodiments of the invention, will become apparent to persons skilled in the art upon reference to the description of the invention. For example, although each chip is illustrated and described as having a single ECB output and a single ECB input, other bus counts fall within the scope of the invention (e.g., a separate ECB bus for each processor). Also, although described as an 8-way processor book, the invention may be implemented with different size processor books. For example, a 16-way processor book comprising dual processor chips in the same MCM-to-MCM configuration may be utilized. Further, although specific bit sizes are provided for the width of the various buses, the actual widths of the buses, number of buses, and connectivity of the buses may vary by design. It is therefore contemplated that such modifications can be made without departing from the spirit or scope of the present invention as defined in the appended claims.
Claims
1. A data processing system comprising:
- a processor book configured on a motherboard, wherein said processor book includes: a first processor chip module including a first plurality of processor chips interconnected by a first set of intra-module buses that are internal to said first processor chip module, said first plurality of processor chips including at least processor chips S0 and T0;
- a second processor chip module including a second plurality of processor chips interconnected by a second set of intra-module buses that are internal to said second processor chip module, said second plurality of processor chips including processor chips S1 and T1;
- a third set of buses external to said first processor chip module and said second processor chip module and which interconnect each of processor chips S0 and T0 a respective one of processor chips S1, and T1; and
- a fourth set of buses extending externally from the outer physical perimeter edge of the motherboard of said processor book, said fourth set of buses including a plurality of external routing buses each connected to a respective processor chip in said processor book, wherein said external routing buses provide a connection point for components external to the processor book;
- means for selectively connecting an external routing bus of one processor chip of said first processor chip module to an external routing bus of a processor chip of said second processor chip module, wherein said means for selectively connecting connects the external routing bus of the one processor chip of the first processor chip module to the external routing bus of the processor chip of the second processor chip module when the processor book is to be utilized to process technical workloads, said means for connecting directly connects each chip of the first processor chip module to a chip of the second processor chip module that is a largest number of processor hops away from the chip of the first processor chip module; and
- distributed routing logic located on each processor chip of both the first processor chip module and the second processor chip module, said distributed routing logic comprising: first routing protocol for routing data among processor chips of both the first processor chip module and the second processor chip module without utilizing any of said external routing buses; second routing protocol for routing data among said processor chips of both the first processor chip module and the second processor chip module of said processor book utilizing said external routing bus, when said external routing bus is selectively wired to another processor chip in said processor book to produce a technical workbook configuration; and selection mechanism for selecting between said first routing protocol and said second routing protocol for said processor book.
2. The data processing system of claim 1, wherein one bus of said external routing buses is an external connector bus (ECB).
3. The data processing system of claim 1, wherein one bus of said external routing bus is an input/output (I/O) bus.
4. The data processing system of claim 3, wherein said first and second modules are interconnected to provide a first processor book, and said first processor book comprises a plurality of external connector buses (ECB) each providing an external wire connection to and from each chip, wherein further said data processing system includes:
- a second processor book also having ECBs and connected to said first processor book via said ECBs, wherein said second processor book comprises I/O buses and said I/O buses of said second processor book are utilized for standard I/O operations for both said first and second processor books; and
- means for controlling data transmission between said first and second modules within both said first processor book and said second processor book to enable the data processing system to support commercial workloads at said second processor book while supporting technical workloads at said first processor book.
5. The data processing system of claim 1, wherein:
- said first and said second modules both include identical configurations of first, second, third and fourth processor chips coupled sequentially by said first set of buses; and
- said connecting directly connects:
- first chip of first module to third chip of second module;
- second chip of first module to fourth chip of second module;
- third chip of first module to first chip of second module;
- fourth chip of first module to second chip of second module;
- first chip of second module to third chip of first module;
- second chip of second module to fourth chip of first module;
- third chip of second module to first chip of first module;
- fourth chip of second module to second chip of first module;
6. The data processing system of claim 1, further comprising:
- distributed routing logic for controlling efficient data transmission by selecting among multiple paths including a direct path along one bus of the external routing buses wired from a source processor chip to a destination processor chip, wherein said distributed routing logic is configured as individual blocks of logic on each of said processor chips of said processor book, and each block of logic controls routing of transmission from its respective processor chip.
7. The data processing system of claim 6, further comprising:
- a default logic that routes transmissions from one processor chip to a next processor chip of said processor book without utilizing said external routing bus;
- a specific routing logic that accounts for an additional path provided by a wiring of one bus of said external routing buses during route selection;
- means for determining when said processor chips are correctly wired via said external routing buses to provide an additional path for data transmission; and
- means, responsive to said determination, for automatically initiating the specific routing logic.
8. The data processing system of claim 7, further comprising:
- means, when the external routing buses of said processor chips are not wired according to a pre-set technical workload wiring scheme, for automatically initiating said default logic, wherein transmissions are routed without utilizing said additional path provided via the external routing buses.
9. The data processing system of claim 7, wherein said fourth set of buses includes input/output (I/O) buses, said data processing system further comprising:
- an output mechanism;
- means, when the external routing buses of said processor chips are not wired according to a pre-set wiring scheme, for generating a message indicating an error in wiring; and
- means for forwarding said message to the output mechanism.
10. The data processing system of claim 6, wherein said distributed routing logic further comprises:
- a first routing logic applicable for commercial workloads that utilizes only said third set of buses to route cross-transmissions between a processor chip of said first module and a corresponding processor chip of said second module;
- a second routing logic applicable to handling operation of the system for technical workloads by utilizing the additional direct routing path provided by the wiring of the external routing bus;
- a selection mechanism having a first selection indicating processor book operation for commercial workloads and a second selection indicating processor book operation for technical workloads; and
- means, responsive to said selection mechanism providing said first selection, for initiating the first routing logic, and when said selection mechanism provides said second selection, initiating the second routing logic.
11. The data processing system of claim 10, wherein said selection mechanism is a mode bit that has a setable value of 1 or 0 respectively indicating a particular one of processor book operation for commercial workload and processor book operation for technical workload.
12. The data processing system of claim 10, wherein said selection mechanism is a latch that is setable to a high or low value respectively indicating a particular one of processor book operation for commercial workload and processor book operation for technical workload.
13. The data processing system of claim 10, wherein said selection mechanism includes the operating system and hypervisor that dynamically triggers one of said first or second selections.
14. The data processing system of claim 10, further comprising means for setting a value of said selection mechanism during Power-On Reset.
1615. The data processing system of claim 1, wherein each of said external routing buses is directly wired to a destination processor chip that is a largest number of processor hops away from the source processor chip from which said external routing bus originates, such that a direct routing path is provided for transmissions from said source processor chip to said destination processor chip, and wherein further said direct routing path provides an alternate routing path and thus increased bandwidth to standard routing paths already available.
16. A data processing system comprising:
- a system rack including a backplane with a plurality of receptors for receiving a plug-in head of processor books that are configured with MCMs having ECBs terminating within the plug-in head, wherein each receptor of said plurality of receptor are wired sequentially to each other;
- a first processor book configured on a motherboard and having said plug-in head coupled to a first one of said plurality of receptors, said first processor book comprising: a first processor chip module including a first plurality of processor chips interconnected by a first set of intra-module buses that are internal to said first processor chip module, said first plurality of processor chips including at least processor chips S0 and T0; a second processor chip module including a second plurality of processor chips interconnected by a second set of intra-module buses that are internal to said second processor chip module, said second plurality of processor chips including processor chips S1 and T1; a third set of buses external to said first processor chip module and said second processor chip module and which interconnect each of processor chips S0 and T0 a respective one of processor chips S1, and T1; a fourth set of buses extending externally from the outer physical perimeter edge of the motherboard of said processor book, said fourth set of buses including a plurality of external routing buses each connected to a respective processor chip in said processor book, wherein said external routing buses provide a connection point for components external to the processor book; and means for connecting the fourth set of buses to from a first processor chip of said processor book to another processor chip of said processor book, wherein when the processor book is being utilized to process technical workload, said means is selectively utilized to connect each processor chip of both the first processor chip module and the second processor chip module to another processor chip on the processor book that is a largest number of processor hops away.
17. The data processing system of claim 16, said first processor book further comprising:
- a distributed memory with individual memory components externally coupled to each of said processor chips of said first and said second processor chip modules; and
- wherein said first, second, third, and fourth set of buses enable full access to each of said individual memory components by each processor within said processor chips without memory affinity.
18. The data processing system of claim 16, further comprising:
- a second processor book coupled to a second one of said plurality of receptors, said second processor book similarly configured to said first processor book and capable of interconnecting with said first processor book; and
- means for interconnecting said fourth set of buses of said first processor book to similar busses of said second processor book via the backplane to enable expansion of the processing power of the data processing system for use as a commercial workload system.
19. The data processing system of claim 18, further comprising:
- wiring means for completing a connection from one receptor to another when said receptor does not contain a processor book coupled thereto so that a complete connection path is always provided within said system rack.
20. The data processing system of claim 18, further comprising:
- wiring means for completing a connection from one receptor to another when said receptor contains a processor book wired as a technical workload processor book, so that a complete connection path is provided within said system rack supporting both said technical workload processor book and any other processor book coupled thereto, wherein said wiring means directly connects each of said external routing buses to a destination processor chip that is a largest number of processor hops away from the source processor chip from which said external routing bus originates.
21. The data processing system of claim 16, wherein further, said fourth set of buses extend from said first processor chip into said plug-in head and terminate as bus connectors within said plug-in head.
22. The data processing system of claim 18, wherein:
- said plug-in head further comprises: a first wired routing path that routes communication received at the plug-in head in and out of said first processor book to a component of said system rack via said receptor; a second wired routing path that routes communication received at the plug-in head from the first processor book back to a selected chip of said first processor book based on a preset wiring scheme designed for technical workload processor books, wherein said wiring means directly connects each of said external routing buses to a destination processor chip that is a largest number of processor hops away from the source processor chip from which said external routing bus originates; and said backplane comprises selection logic for selecting between said first wired routing path and said second wired routing path and routing communication from processor chips of the processor book connected to the plug-in head according to the selected path.
23. The data processing system of claim 22, wherein said logic includes a software-enabled selection mechanism, dynamically settable by software.
24. The data processing system of claim 22, wherein said logic includes a physical selection mechanism settable by physical user selection.
25. A data processing system comprising:
- a system rack including a backplane with a plurality of receptors for receiving a plug-in head of processor books that are configured with MCMs having ECBs terminating within the plug-in head, wherein each receptor of said plurality of receptor are wired sequentially to each other;
- a first processor book configured on a motherboard and having said plug-in head coupled to a first one of said plurality of receptors, said first processor book comprising: a first processor chip module including a first plurality of processor chips interconnected by a first set of intra-module buses that are internal to said first processor chip module, said first plurality of processor chips including at least processor chips S0 and T0; a second processor chip module including a second plurality of processor chips interconnected by a second set of intra-module buses that are internal to said second processor chip module, said second plurality of processor chips including processor chips S1 and T1; a third set of buses external to said first processor chip module and said second processor chip module and which interconnect each of processor chips S0 and T0 a respective one of processor chips S1, and T1; and a fourth set of buses extending externally from the outer physical perimeter edge of the motherboard of said processor book, said fourth set of buses including a plurality of external routing buses each connected to a respective processor chip in said processor book, wherein said external routing buses provide a connection point for components external to the processor book; and an add-on connector coupled to said first receptor, said add-on connector being a separate component from the first receptor and comprising a receptor end for receiving said plug-in head of said first processor book and a plug-in end for coupling said add-on connector to said first receptor, wherein said first processor book is coupled to the first receptor via the add-on connector to enable the first processor book to be utilized as a technical workbook, wherein said add-on connector further comprises: a first wired routing path that routes communication received at the receptor end from the first processor book out to said system rack via said plug-in end coupled to said receptor; a second wired routing path that routes communication received at the receptor end from the first processor book back to a selected chip of said first processor book based on a preset wiring scheme designed for technical workload processor books, wherein each of said external routing buses is directly wired to a destination processor chip that is a largest number of processor hops away from a source processor chip from which said external routing bus originates, such that a direct routing path is provided for transmissions from said source processor chip to said destination processor chip; and logic for selecting between said first wired routing path and said second wired routing path and routing communication according to the selected wired routing path.
26. The data processing system of claim 25, wherein said logic includes a MUX with a selection mechanism dynamically settable by software.
27. The data processing system of claim 25, wherein said logic includes a MUX with a physical selection mechanism settable by user selection.
28. The data processing system of claim 16, wherein further each receptor comprises:
- logic for selecting between a first routing path and a second routing path for said ECBs of said first processor book and routing communication according to the selected path, wherein said first routing path routes communication received at the receptor from the first processor book out to said system rack, and said second routing path re-routes communication received at the receptor from the first processor book back to a selected chip of said first processor book based on a preset wiring scheme designed for technical workload processor books, wherein said wiring means directly connects each of said external routing buses to a destination processor chip that is a largest number of processor hops away from the source processor chip from which said external routing bus originates.
29. The data processing system of claim 28, wherein said logic includes a MUX with a selection mechanism settable by user selection.
Type: Application
Filed: May 9, 2008
Publication Date: Aug 28, 2008
Applicant: INTERNATIONAL BUSINESS MACHINES CORPORATION (ARMONK, NY)
Inventors: Ravi Kumar Arimilli (Austin, TX), Vicente Enrique Chung (Austin, TX), Jody Bern Joyner (Austin, TX), Jerry Don Lewis (Round Rock, TX)
Application Number: 12/118,199
International Classification: G06F 15/76 (20060101);