MULTI-PIPE APPARATUS FOR A TEST AND MEASUREMENT INSTRUMENT

Abstract

Apparatuses for a test and measurement instrument provide an instrument capable of handling acquisition, transfer, analysis, and display of large quantities of waveform data and complex waveforms. The apparatus includes multiple processors with each processor being connected to its own memory controller, wherein each memory controller is connected to its own memory. Each processor is connected to its own respective bridge, and each respective bridge is connected to its own respective system bus. Each respective system bus is connected to its own respective acquisition module having its own acquisition hardware. Each piece of acquisition hardware is a direct memory access machine that can transfer data to any portion of the memory. Each of multiple signal sources is connected to its own signal bus interface.

Description

CLAIM OF PRIORITY

The subject application claims priority from U.S. Patent Application Ser. No. 60/913,525, entitled, APPARATUS AND METHODS FOR A TEST AND MEASUREMENT INSTRUMENT EMPLOYING A MULTI-CORE HOST PROCESSOR (Sedeh, et al.), filed 23 Apr. 2007, and assigned to the same assignee as the subject invention.

CROSS-REFERENCE TO RELATED CASES

The subject application is related to the following U.S. patent applications, bearing attorney docket numbers 8361-US0, 8287-US1, 8287-US3, and 8287-US4, all claiming priority from U.S. Patent Application Ser. No. 60/913,525, entitled, APPARATUS AND METHODS FOR A TEST AND MEASUREMENT INSTRUMENT EMPLOYING A MULTI-CORE HOST PROCESSOR (Sedeh, et al.), filed 23 Apr. 2007, and all assigned to the same assignee as the subject invention.

FIELD OF THE INVENTION

The present invention relates to an apparatus for a test and measurement instrument for use in connection with analyzing waveforms. The apparatus for a test and measurement instrument have particular utility in connection with providing a scalable test and measurement instrument capable of handling the acquisition, transfer, analysis, and display of large quantities of waveform data as well as complex waveforms.

BACKGROUND OF THE INVENTION

Apparatuses for a test and measurement instrument are desirable for providing a scalable test and measurement instrument capable of handling the acquisition, transfer, analysis, and display of large quantities of waveform data as well as complex waveforms. Demand for new oscilloscope application features is growing, especially the ability to process ever-greater quantities of waveform data, because signals are becoming increasingly complex. Analyzing complex waveforms generates more intermediate data, which in turn requires more system memory access instances.

Most software applications have enjoyed regular performance gains for several decades, even without significant modifications, merely because of increases in computer hardware performance. Central Processing Unit (CPU) manufacturers and, to a lesser degree, memory manufacturers have reliably increased processing speeds and lowered memory access times. However, performance gains through increasing CPU clock speeds are seriously inhibited by heat generation, electron leakage, and other physical limitations, while system memory speeds have historically doubled only every 10 years.

Since major processor manufacturers and architectures can no longer easily boost straight-line instruction throughput, performance gains in test and measurement instruments, such as oscilloscopes, will have to be accomplished in fundamentally different ways. Because CPU manufacturers have adopted dual core and multicore processors to increase performance, oscilloscope applications will have to enable concurrent processing in order to exploit the CPU performance gains that are becoming available. What is therefore needed is a practical apparatus that provides a scalable test and measurement instrument capable of handling large quantities of waveform data as well as complex waveforms.

The use of oscilloscopes is known in the prior art. For example, oscilloscopes currently manufactured by Tektronix, Inc. of Beaverton, Oreg. ship with a single core 3.42 GHz Pentium® processor from Intel. These prior art oscilloscopes cannot have their performance boosted through use of a faster single CPU because CPUs with higher clock speeds do not presently exist. Furthermore, mere replacement of the single core CPU with a dual core or multicore CPU offers minimal benefit because many of the important operations of an oscilloscope application are not CPU constrained. In an instrument that moves and processes a large quantity of data, system memory access times and/or system bus performance often are the instrument's performance bottleneck.

Existing high-end oscilloscopes, such as those currently manufactured by Tektronix, Inc., already incorporate a sizable system memory (2 GB of system RAM is typical). Because of increasing quantities of data to be processed and stored, next-generation oscilloscope architectures will undoubtedly require additional memory. Since increases in main memory speeds are realized infrequently, the time required to access system memory is likely to continue to dominate many applications' performance. Therefore, the addition of a multicore processor to existing oscilloscope architectures provides minimal benefit because system memory cannot provide data as fast as the processors can process it.

Furthermore, the data acquisition process is an inherently sequential four-step process presenting additional challenges to the adoption of multicore CPU technology in oscilloscope applications. FIG. 1 depicts a single core processor prior art oscilloscope architecture that acquires and combines waveform data from four channels 120-126 into a single data record in the system memory 114. Conventionally, waveforms are stored in the local memory 130 of the acquisition hardware 118 in a first step and subsequently transferred serially to the system memory 114 via a Peripheral Component Interconnect (PCI) or Peripheral Component Interconnect Express (PCIe) system bus 116 and bridge 112 in a second step. The CPU 110 then analyzes the waveform data in a third step and causes the results to be shown on a display screen 128 in a fourth and final step. The acquisition hardware 118 may be embodied in a peripheral device attached to the system bus 116 that is operable by the operating system.

This four-step process is not easily amenable to parallelization. These four subtasks cannot be run at the same time on four CPU cores with this prior art architecture because each must be completed before the next can begin. Nor can these four subtasks be pipelined either. In this context, a pipeline is a set of data processing elements connected in series so that the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in a time-sliced fashion. However, because three of the steps require access to the system memory to run and store intermediate data generated as data moves through the pipeline, parallel processing is impossible. Therefore, the inherently sequential nature of the data acquisition process prevents taking full advantage of multicore processor technology.

The system memory also creates a bottleneck because it is used for waveform storage data and shared by several clients, including Analysis, General Purpose Interface Bus, Display, Acquisition, Math, Save/Recall, and Applications. Because these clients must access the data serially from the shared system memory, it is impossible to create parallelism among the clients and run them at the same time. The architecture's data transfer rate and system bandwidth also pose limiting factors, which are likely to worsen. Next-generation real-time data acquisition hardware will have very large record lengths per channel. Existing oscilloscope architectures cannot transfer, analyze, and display that much data in real-time.

An initial prior art attempt to address some of these problems was the TDS-7000-series oscilloscope manufactured by Tektronix, Inc. whose architecture is depicted in FIG. 2. This architecture employed a dual core processor. Although each processor could access the other's memory, this was accomplished using the Direct Memory Access (DMA) process over a PCI bus, a relatively slow computer bus. An inability to transfer data sufficiently rapidly to continuously occupy both processors left the oscilloscope unable to take full advantage of the presence of two processors.

FIG. 3 shows a prior art oscilloscope system architecture employing a quad core CPU 300 developed by the inventors of the current invention. A quad core CPU 310, 328, 330, and 332 is the dominant high-performance computer architecture in industry, known as Symmetric Multiprocessor (SMP) architecture. While the SMP architecture performs adequately in many respects, it unfortunately exhibits architectural limitations. In an SMP-based system, all processors access a shared pool of memory 314 over a central memory bus. While this limited the effectiveness of the dual core system depicted in FIG. 2, an even greater problem with memory access occurs when quad core or higher multicore CPUs are utilized. Because the processors are often fighting each other for access to the single memory bus, a serious bottleneck develops. This occurs because the time to move data back and forth between the processors 310, 328, 330, and 332 and the system memory 314 increases. This major bottleneck is especially severe in an instrument like a high-end oscilloscope. High-end oscilloscopes require the movement of large amounts of data and utilize processor-intensive applications that create considerable traffic between the processors 310, 328, 330, and 332 and the system memory 314. Data sets in modern high-end oscilloscopes can be so large that they are not entirely cacheable, resulting in many system memory access instances. This problem with memory access times is aggravated by use of the same system bus and memory bus for Input/Output (I/O) and DMA transfer of waveform data from the acquisition hardware's 318 local memory 334.

Another architectural problem with SMP architecture is that the memory system does not scale up with increasing numbers of processor cores. Memory access occurs via a single memory controller 522 (shown in FIG. 5) for the entire system, no matter how many processor cores are present. This serious problem prevents taking full advantage of multicore CPUs because they cannot obtain enough data in a timely fashion to always remain busy because memory is a shared resource. Thus, performance of applications with large memory requirements remains largely constrained by memory access times.

Preliminary performance testing on dual core and quad core high performance oscilloscopes using the architectures depicted in depicted in FIGS. 2 and 3 showed no significant performance gains over single core instruments. The lack of performance gain was not surprising because the prior art data acquisition process is sequential in nature. All processor cores must share the system memory, and applications tend to be highly memory intensive. Because the memory system cannot provide data as fast as the application needs it to keep all of the processor cores busy simultaneously, very little parallel processing can occur, making the additional processor cores only marginally utilized.

Therefore, a need exists for a new and improved apparatus for a test and measurement instrument that can be used for providing a scalable test and measurement instrument capable of handling the acquisition, transfer, analysis, and display of large quantities of waveform data as well as complex waveforms. In this regard, the various embodiments of the present invention substantially fulfill at least some of these needs. In this respect, the apparatus for a test and measurement instrument according to the present invention substantially departs from the conventional concepts and designs of the prior art, and in doing so provides an apparatus primarily developed for the purpose of providing a scalable test and measurement instrument capable of handling the acquisition, transfer, analysis, and display of large quantities of waveform data as well as complex waveforms.

SUMMARY OF THE INVENTION

The present invention provides an improved apparatus for a test and measurement instrument, and overcomes the above-mentioned disadvantages and drawbacks of the prior art. As such, the general purpose of the present invention, which will be described subsequently in greater detail, is to provide an improved apparatus for a test and measurement instrument that has all the advantages of the prior art mentioned above.

To attain this, the preferred embodiment of the present invention essentially comprises multiple processors each processor being connected to its own memory controller. Each memory controller is connected to its own memory. There are multiple bridges with each processor being connected to its own bridge. There are multiple system buses with each bridge being connected to its own system bus. There are multiple acquisition modules having signal bus interfaces with each system bus being connected to its own acquisition module and having its own acquisition hardware. Each piece of acquisition hardware is a direct memory access machine that can transfer data to any portion of the memory. There are multiple signal sources with each signal source being connected to its own signal bus interface. There are, of course, additional features of the invention that will be described hereinafter and which will form the subject matter of the claims attached.

There has thus been outlined, rather broadly, the more important features of the invention in order that the detailed description thereof that follows may be better understood and in order that the present contribution to the art may be better appreciated.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram view of a prior art single CPU oscilloscope architecture;

FIG. 2 is a block diagram view of a prior art dual CPU oscilloscope architecture;

FIG. 3 is a block diagram view of a quad core CPU oscilloscope architecture;

FIG. 4 is a block diagram view of the current embodiment of the apparatus for a test and measurement instrument constructed in accordance with the principles of the present invention;

FIG. 5 is a block diagram view of a multicore processor suitable for use with the present invention;

FIG. 6 is a block diagram view of the current embodiment of the apparatus for a test and measurement instrument of the present invention;

FIG. 7A is a block diagram view of the data flow in the prior art single CPU oscilloscope architecture; and

FIG. 7B is a block diagram view of the data flow of an embodiment of the apparatus for a test and measurement instrument of the present invention; and

FIG. 8 is a block diagram view of an embodiment of the apparatus for a test and measurement instrument of the present invention.

The same reference numerals refer to the same parts throughout the various figures.

DESCRIPTION OF THE CURRENT EMBODIMENT

A preferred embodiment of the apparatus for a test and measurement instrument of the present invention is shown and generally designated by the reference numeral 10.

The principles of the present invention are applicable to a variety of computer hardware and software configurations. The term “computer hardware” or “hardware,” as used herein, refers to any machine or apparatus that is capable of accepting, performing logic operations on, storing, or displaying data, and includes without limitation processors and memory; the term “computer software” or “software,” refers to any set of instructions operable to cause computer hardware to perform an operation. A “computer,” as that term is used herein, includes without limitation any useful combination of hardware and software, and a “computer program” or “program” includes without limitation any software operable to cause computer hardware to accept, perform logic operations on, store, or display data. A computer program may, and often is, comprised of a plurality of smaller programming units, including without limitation subroutines, modules, functions, methods, and procedures. Thus, the functions of the present invention may be distributed among a plurality of computers and computer programs. The invention is described best, though, as a single computer program that configures and enables one or more general-purpose computers to implement the novel aspects of the invention.

FIGS. 4 and 6 illustrate improved apparatus for a test and measurement instrument 10 of the present invention. More particularly, an architecture for the apparatus for a test and measurement instrument 10 is depicted with every oscilloscope channel 22, 34, 46, and 58 coupled to its own single or multicore CPU 12, 24, 36, and 48, creating acquisition pipes 60, 62, 64, and 66. For example, oscilloscope channel 22 is connected to acquisition pipe 60 by acquisition module 20. Acquisition module 20 collects data from oscilloscope channel 22 via a signal source bus 68 and includes a demux ring 76 that separates the collected data into separate files, each one containing at least one element of the original file. System bus 18 connects acquisition module 20 to bridge 16. Bridge 16 integrates the data from the system bus to the single or multicore CPU 12. Bridge 16 is connected to single or multicore CPU 12, which is in turn connected to system memory 14. System memory 14 stores incoming data from system bus 18 as well as intermediate and final calculations generated by a single or multicore CPU 12. Each acquisition pipe has its own system bus 18, 30, 42, 54, system memory 14, 26, 38, 50, single or multicore CPU 12, 24, 36, and 48, acquisition module 20, 32, 44, and 56 with local memory 84, 86, 88, and 90, signal source bus 68, 70, 72, and 74, and demux ring 76, 78, 80, and 82. Therefore, all of the acquisition pipes can operate simultaneously and converge in the display subsystem 68, with each bridge being directly connected to the display. This enables the collected data to be observed much faster than is the case with prior art oscilloscope architectures. The oscilloscope's operating system pulls all of the individual system memories into one global address space, distributes threads across the CPUs, and maps a thread's memory requests to local system memory or remote system memory. Local system memory refers to the memory directly connected to a CPU, which can be accessed the fastest. Remote system memory refers to the memory directly connected to the other CPUs. While it can be accessed by the first CPU, it takes longer. The global address space combines all of the individual system memories into a virtual single system memory accessible by the operating system. Threads enable a program to split itself into multiple simultaneously executing tasks. Multiple threads can be executed in parallel on many computer systems, such as those employed by the present invention.

In this architecture, the memory bottleneck of conventional SMP architectures is removed because each channel has its own system memory and CPU, so data remains in proximity to the CPU that needs it. However, as shown in FIG. 6, each CPU can access each of the other CPUs' memory using high-speed interconnects 76, enabling measurements that incorporate data from multiple channels. The high-speed interconnects 76 can be arranged in the square pattern shown, which enables each processor to communicate with its two neighbors directly, but communication with the processors at opposite corners requires the use of one of the neighboring processors as an intermediary. Alternatively, there can be additional high-speed interconnects 76 connecting the CPUs in an X pattern in the middle to enable every CPU to have a direct connection to every other CPU, eliminating the need for using the neighboring CPUs as intermediaries when communicating with the processors at opposite corners. This design is presently more expensive, but it delivers improved performance. In addition, memory bandwidth scales linearly with the number of acquisition pipes. Compared to a conventional four-channel oscilloscope, a four-channel oscilloscope with the improved architecture of the current invention increases the system memory bandwidth by a factor of four. All four acquisition pipes have their own memory 14, 26, 38, and 50, memory controllers 78, system request interfaces 80, and crossbar switches 82, which enables all four acquisition pipes to receive data in parallel.

This architecture also enables system I/O bandwidth to scale linearly with the number of acquisition pipes. A four-channel oscilloscope with this architecture has a system data transfer rate that is four times that of a conventional four-channel of oscilloscope because data can be transferred at the same time from all four channels using all four acquisition pipes simultaneously. The oscilloscope's processing capability also scales upward as the number of acquisition pipes increases because the number of CPUs increases.

FIG. 5 illustrates the architecture of a prior art multicore processor 500 suitable for use with the present invention. For example, the AMD64 Opteron™ dual core processor, manufactured by AMD Corporation of Sunnyvale, Calif., has a Non-Uniform Memory Access (NUMA) architecture 500 especially suitable for use with the present invention. The design is called non-uniform because memory access times vary depending upon the memory's location. This is because a CPU can access its own local memory 530 faster than it can access another CPU's memory. This design feature gives processor cores 510 and 512 access to their own local memory 530 via memory controller 522. When additional multicore processors 500 are present with their own local memory, the high-speed interconnects 524, 526, and 528 can be used to access their local memory. This architecture enables each processor to access other processors' memory quickly and easily. The high-speed interconnects 524, 526, and 528 also enable communication with the data source channels, while the memory controller 522 also provides access to the system bus. The system request interface 518 and crossbar switch 520 control the physical connections between the CPU cores 510 and 512, the memory controller 522, and the high-speed interconnects 524, 526, and 528.

FIGS. 7A and 7B illustrate the differences between data flows in the prior art oscilloscope architecture 100 and that of the present invention 10. More particularly, while data collected by the acquisition hardware 118 can be processed only serially by a single CPU 110 in the prior art oscilloscope architecture 100, the present invention 10 enables parallel processing of data from one channel by breaking it into parts and supplying them to multiple CPUs 12-48 for analysis. Similarly, pipelining the display process between all of the available CPUs greatly increases display performance. Because each CPU 12-48 has its own local memory 14, 26, 38, 50, such pipelining is feasible and eliminates the need for hardware acceleration for display purposes.

FIG. 8 illustrates how the internal high-speed interconnects 68-74 of a multicore Non-Uniform Memory Access processor, such as the one depicted in FIG. 6, can be used as a faster system bus than the prior art depicted in FIG. 2 that uses a PCI system bus 216. The high-speed interconnects 68-74 enable much faster data transfer than does the PCI system bus 216, both because the high-speed interconnects 68-74 are inherently faster and because they are a dedicated data transfer resource instead of being shared.

While current embodiments of the apparatus for a test and measurement instrument have been described in detail, it should be apparent that modifications and variations thereto are possible, all of which fall within the true spirit and scope of the invention. With respect to the above description then, it is to be realized that the optimum dimensional relationships for the parts of the invention, to include variations in size, materials, shape, form, function and manner of operation, assembly and use, are deemed readily apparent and obvious to one skilled in the art, and all equivalent relationships to those illustrated in the drawings and described in the specification are intended to be encompassed by the present invention. For example, any suitable specialized processor such as Graphics Processing Units (GPUs), Digital Signal Processors (DSPs), and Field Programmable Gate-arrays (FPGAs) may be used instead of the general-purpose single or multicore CPUs described. And although providing a scalable test and measurement instrument capable of handling the acquisition, transfer, analysis, and display of large quantities of waveform data as well as complex waveforms has been described, it should be appreciated that the apparatus for a test and measurement instrument herein described are also suitable for use as a logic analyzer, signal source instrument, real-time spectrum analyzer, or any other analytical instrument requiring multiple channels for data collection. Furthermore, any other suitable type of memory in addition to dynamic random access memory (DRAM) could be utilized.

Therefore, the foregoing is considered as illustrative only of the principles of the invention. Further, since numerous modifications and changes will readily occur to those skilled in the art, it is not desired to limit the invention to the exact construction and operation shown and described, and accordingly, all suitable modifications and equivalents may be resorted to, falling within the scope of the invention.

Claims

1. An apparatus for a test and measurement instrument, the instrument comprising:

a plurality of processors;

a plurality of memory controllers, wherein each processor is connected to its own memory controller;

memory, wherein each memory controller is connected to its own memory;

a plurality of bridges, wherein each processor is connected to its own bridge;

a plurality of system buses, wherein each bridge is connected to its own system bus;

a plurality of acquisition modules having signal bus interfaces and acquisition memory, wherein each system bus is connected to its own acquisition module and has its own acquisition hardware, and wherein each piece of acquisition hardware comprises a direct memory access machine that can transfer data to any portion of the memory; and

a plurality of signal sources, wherein each signal source is connected to its own signal bus interface.

2. The apparatus for a test and measurement instrument as defined in claim 1, further comprising a plurality of high-speed interconnects, wherein the high-speed interconnects connect the processors to one another.

3. The apparatus for a test and measurement instrument as defined in claim 2, wherein the high-speed interconnects are used as the system buses.

4. The apparatus for a test and measurement instrument as defined in claim 1, wherein at least one of the plurality of processors is a specialized processor selected from the group comprising graphics processing units, digital signal processors, and field-programmable gate arrays.

5. The apparatus for a test and measurement instrument as defined in claim 1, wherein each processor is connected to its own memory element.

6. The apparatus for a test and measurement instrument as defined in claim 5, wherein the memory elements are interconnected.

7. The apparatus for a test and measurement instrument as defined in claim 5, wherein each memory element is interconnected to at least another of the memory elements by way of the processor to which it is connected.

8. The apparatus for a test and measurement instrument as defined in claim 1, wherein each processor is a multicore processor.

9. The apparatus for a test and measurement instrument as defined in claim 1, including a display connected to each of the processors for displaying images based on signals acquired by the instrument.

10. The apparatus for a test and measurement instrument as defined in claim 1, wherein the display is connected to each of the processors by way of the bridges.

11. An apparatus for a test and measurement instrument, the instrument comprising:

a plurality of memory controllers;

memory, wherein each memory controller is connected to its own memory;

a plurality of processors, wherein each processor is connected to its own memory controller and each processor has a connected acquisition pipe;

each acquisition pipe including a bridge connected to the processor;

each acquisition pipe including a system bus connected to the bridge;

each acquisition pipe including a signal bus interface connected to the signal bus, wherein each system bus is connected to its own acquisition module and has its own acquisition hardware, and wherein each piece of acquisition hardware comprises a direct memory access machine that can transfer data to any portion of the memory; and

a plurality of signal sources, wherein each signal source is connected to its own signal bus interface.

12. The apparatus for a test and measurement instrument as defined in claim 11, further comprising a plurality of high-speed interconnects, wherein the high-speed interconnects connect the processors to one another.

13. The apparatus for a test and measurement instrument as defined in claim 12, wherein the high-speed interconnects are used as the system buses.

14. The apparatus for a test and measurement instrument as defined in claim 11, wherein at least one of the plurality of processors is a specialized processor selected from the group comprising graphics processing units, digital signal processors, and field-programmable gate arrays.

15. The apparatus for a test and measurement instrument as defined in claim 11, wherein each processor is connected to its own memory element.

16. The apparatus for a test and measurement instrument as defined in claim 15, wherein the memory elements are interconnected.

17. The apparatus for a test and measurement instrument as defined in claim 15, wherein each memory element is interconnected to at least another of the memory elements by way of the processor to which it is connected.

18. The apparatus for a test and measurement instrument as defined in claim 11, wherein each processor is a multicore processor.

19. The apparatus for a test and measurement instrument as defined in claim 11, including a display connected to each of the processors for displaying images based on signals acquired by the instrument.

20. The apparatus for a test and measurement instrument as defined in claim 11, wherein the display is connected to each of the processors by way of the bridges.