Abstract: Detecting harmful or illegal intrusions into a computer network or into restricted portions of a computer network uses a features generator or builder to generate a feature reflecting changes in user and user group behavior over time. User and user group historical means and standard deviations are used to generate a feature that is not dependent on rigid or static rule sets. These statistical and historical values are calculated by accessing user activity data listing activities performed by users on the computer system. Historical information is then calculated based on the activities performed by users on the computer system. The feature is calculated using the historical information based on the user or group of users activities. The feature is then utilized by a model to obtain a value or score which indicates the likelihood of an intrusion into the computer network. The historical values are adjusted according to shifts in normal behavior of users of the computer system.

Abstract: Data capture and analysis for debugging embedded systems is disclosed. On a target, there is at least one data collector, each of which publishes predetermined data of the target, and a collection manager for managing the data collectors. On a host, there is at least one viewer, each of subscribes to the predetermined data of a data collector, for processing thereof, and a viewer manager for managing the viewers. Data collectors and viewers can be added, such that an extensible data capture and analysis embedded system architecture is provided.

Abstract: Target control abstraction for debugging embedded systems is disclosed. In an abstracted hardware model, there is a non-hardware-specific debugging interface, for communication between a debugger for the embedded system and the hardware of the embedded system. In an abstracted hardware-via-software model, there is a proxy to receive commands intended for the hardware of the embedded system from the debugger, and a component within a kernel of the embedded system to receive the commands and convey them to the hardware.

Abstract: A method and apparatus for efficiently generating engine error codes is disclosed. An error processing system according to the present invention includes an error table describing error states for use in generating error codes based upon the an error even, an engine for providing an error event and an error processor, coupled to the engine and the error table, the error processor including a general run time error generator for generating a list of errors according to the error table. Accordingly, modifications may be made to the error table for changes to the product error specification and for new products that do not necessitate a re-coding of the error processing system.

Abstract: An integrated circuit verification method and system are disclosed. The method includes generating a test description comprising a set of test cases. The functional coverage achieved by the test description is then determined. The functional coverage achieved is then compared against previously achieved functional coverage and the test description is modified prior to simulation if the test description achieves no incremental functional coverage. In one embodiment, generating the test description comprises generating a test specification and providing the test specification to a test generator suitable for generating the test description. In one embodiment, the test description comprises a generic test description and the generic test description is formatted according to a project specification and simulation environment requirements. If the coverage achieved by the test description satisfies the test specification.

Abstract: Apparatus and method are described for a testing system that generates tests that share similar failure causing characteristics that can be discovered by a genetic generator quickly without going through the chance or long process of traditional random and exhaustive test generations. The testing system generates random test cases that are applied to a system under test (SUT). These tests are validated to determine whether they cause errors within the SUT. All testing programs that cause an error during the execution of the SUT are collected and are used in the genetic algorithm technique to create other testing programs that share similar characteristics.

Abstract: Method for diagnosing data packet transfer faults in a system under test (SUT) are provided. A representative method includes: identifying at least some portions of the data transmission paths of the SUT capable of introducing errors in data packet transfer; providing constraints defining data packet transfer relationships of at least some of the portions of the data transmission paths; and diagnosing the SUT with respect to the constraints. Systems, computer-readable media and other methods also are provided.

Abstract: A method of, and apparatus for, processing the output of a design tool for an integrated circuit, the output relating to a circuit under design. A part of the circuit to be investigated is selected. Information relating to each signal in the selected part of the signal is then selected, and an output containing the selected information for the signals in the selected part of the circuit is generated.

Abstract: A method for efficiently changing the embedded processor type in verification of system-on-chip (SOC) integrated circuit designs containing embedded processors. The verification software is used to generate and apply test cases to stimulate components of a SOC design (“cores”) in simulation; the results are observed and used to de-bug the design. Typically, the embedded processor type changes as SOC designs change. However, changing the processor type may cause errors in verification due to the presence of processor-specific code distributed throughout the verification software. Thus, changing the processor type can entail a substantial re-write of the verification software. In the method according to the present invention, in verification software for verifying a SOC design including an embedded processor, processor-specific code is localized in a processor driver.

Abstract: A method and apparatus are disclosed for verifying the functional design of a system's response to propagation delays from the inputs of source synchronous links during testing. The system emulates propagation delays by receiving data slice from a source, applying a random or known delay to the data slice, and sending the delayed data slice to the chip under test. In one embodiment, multiple data slices having varying delay values may be used to test combinations of delays. A programmable delay.element is used to emulate the propagation delays. This is may be implemented at the hardware description level by receiving the data slice onto multiple data buses, applying a different delay to the data slice on each data bus, and sending the delayed data slices as inputs into a multiplexor. The multiplexor may have a selector input that determines which amount of delay to test. Alternatively, the delay may be emulated using a higher level programming language and creating a multidimensional array.

Abstract: A storage system is described that includes a controller and a disk array. The disk array includes at least a first and a second storage area. The first storage area is associated with a first mean time to failure (MTTF) and the second storage area is associated with a second MTTF. The controller operates to test the first storage area at a first frequency and the second storage area at a second frequency. The first frequency and the second frequency are each based upon the first and second MTTF so as to optimize the overall reliability of the storage system.

Abstract: A fault verification apparatus performs a logic simulation of a circuit having a normal delay and a logic simulation of a circuit in which delay is intentionally changed for a node and compares the simulation results at a specific time and checks whether or not a test pattern can detect a fault due to a delay abnormality. The apparatus performs the logic simulation by applying the test pattern to the normal circuit and a variety of fault types and compares the expected values obtained from the results of the respective logic simulations and verifies whether or not the test pattern can detect the delay fault by whether or not the expected values are different from each other at a specific comparison point.

Abstract: A system for integrated testing of one or more multi-service communications switches including a script execution environment for executing a test script. The script execution environment includes a hardware definition file storing a number of logical hardware device names corresponding to hardware devices used by the test script, including a system under test (SUT) and a number of hardware testing devices. The system further employs a network definition file including a plurality of logical connection names corresponding to connections between the hardware devices in the test environment. These connections in the network definition file are specified in terms of the logical hardware device names defined in the hardware definition file.

Abstract: A method and apparatus for verification of designed logic in a processing unit (100) that performs multiprocessing functions is presented. Once the multiprocessing unit to be exercised as the device under test (350) is defined, an irritator (330) is constructed to provide the stimuli to the interface of the device under test (350) such that the device under test (350) is exercised. The irritator (330) receives instructions (314) as its input, where relevant instructions are converted to transactional stimulus that is applied to the device under test (350). The irritator (330) also monitors the interface of the device under test (350) to detect when multiprocessing operations executed by other processing units (340) included in the system (5) require a response. The response is produced via signal stimulus on the interface of the device under test (350) to which the irritator (330) is coupled.

Abstract: A testcase management system comprises a test suite including a plurality of testcases, wherein these testcases are cooperable with a test harness for executing a test run in which at least one of the testcases is executed on a system to be tested. A method and apparatus are provided for generating a list (overall list) indicative of the testcases that are to be executed during the test run. The overall list is generated as follows: A list is generated of all testcases in the test suite. Some of these testcases will have been fully tested in all environments and be verified as approved. They are manually added to an approved list. Others are disapproved and are manually added to a disapproved list. A list is automatically generated comprising those testcases which are neither approved nor disapproved. They are not-tested. Those testcases on the disapproved and not-tested lists are excluded from the overall list.

Abstract: An improved process for executing a dump is provided. The iteration loops are made “smart” by allowing them to determine how big the arrays are on the fly and adjust their behavior accordingly. The process uses a function to calculate the amount of memory to allocate for the dump list based on the dump mode and array sizes. Thus, if the static arrays are modified to add or delete constants or the diagnostic code is in an abbreviated dump mode, the amount of memory to be allocated will be calculated accurately.

Abstract: A method of verifying a digital circuit in which state transition information is extracted from the output of a non-formal first verification technique. A formal verification tool is then applied to the extracted state transition information to extend the verification coverage of the digital circuit beyond the coverage that is achieved using the first verification technique. In one embodiment, the method includes the initial step of applying a first verification technique such as a simulation technique to a model of the digital circuit. In the preferred embodiment, the application of the formal verification tool comprises applying a model checker to the extracted state transition data to achieve a formal verification of the state machine represented by the state transition diagram. In one embodiment, the extracted state transition information includes a set of data points each representing a present state, a present input, and a next state.

Abstract: A visual Web site analysis program, implemented as a collection of software components, provides a variety of features for facilitating the analysis, management and load-testing of Web sites. A mapping component scans a Web site over a network connection and builds a site map which graphically depicts the URLs and links of the site. Site maps are generated using a unique layout and display methodology which allows the user to visualize the overall architecture of the Web site. Various map navigation and URL filtering features are provided to facilitate the task of identifying and repairing common Web site problems, such as links to missing URLs. A dynamic page scan feature enables the user to include dynamically-generated Web pages within the site map by capturing the output of a standard Web browser when a form is submitted by the user, and then automatically resubmitting this output during subsequent mappings of the site.

Abstract: Test tool logic and testing methods are provided for facilitating testing a duplexed computer function, such as a duplexed coupling facility. The test tool allows a testcase written for a first environment to be automatically driven in a second environment, thereby facilitating testing of a function of the second environment. Other aspects include logic for intercepting a system event by a test tool to facilitate testing of system-managed event processing, and for adjusting a display characteristic of one or more messages to be displayed by the test tool based on message type. Further, logic for propagating an environmental error indication and for facilitating processing a wait state are also provided, as are several new test tool verbs and macros.

Abstract: A system and method for communicating with an integrated circuit is provided that allows an integrated circuit to communicate debugging information and system bus transaction information with an external system. The system may include an interface protocol that provides flow control between the integrated circuit and the external system. The system may include a high-speed link and/or a JTAG link for communicating information. A link may be automatically selected by a debug circuit, or selected by an on-chip device or external system. The high-speed link enables real-time collection of trace information. Links may be memory-mapped, such that on-chip devices and other devices attached to the system bus may access the external system. The high-speed link may also operate at a rate which is integrally coupled with a rate of the processor or system bus. Further, the high-speed link may be adapted to change speeds in response to a change in operating speed of the system bus or processor.

Abstract: A system and method for importing and exporting test executive values from or to a database. A test executive may provide the user various places, referred to as variables and properties, in which data values can be stored. These data values may affect execution of a test executive sequence. Variables may be global to a sequence file or local to a particular sequence. Each step in a sequence can have properties. The user may include a Property Loader step in a sequence, which is operable to dynamically, i.e., at run time, load property and/or variable values from a database. The Property Loader step may be placed in a setup group of the sequence, and the step(s) whose properties are configured may be placed in a main group of the sequence, so that when the sequence is executed the steps in the main group are configured with appropriate property values before running. The test executive may also enable a user to interactively, e.g.

Abstract: Hardware Verification Languages (HVLs) permit the convenient modeling of the environment for a device under test (DUT). HVLs permit the DUT to be tested by stimulating certain inputs of the DUT and monitoring the resulting states of the DUT. The present invention relates to an HVL, referred to as Vera, for the verification of any form of digital circuit design. Vera is preferably used for testing a DUT modeled in a high-level hardware description language (HLHDL) such as Verilog HDL. More specifically, the present invention relates to an HVL capability, know as an “expect,” for monitoring the values at certain nodes of the DUT at certain times and for determining whether those values are in accordance with the DUT's expected performance.

Abstract: An improved method and system for detecting differences between first and second test executive sequence files in a computer system. Each of the test executive sequence files may comprise a plurality of interrelated objects. The objects may be compared and differences between the objects may be displayed. The objects may comprise one or more of: a sequence; a global variable; and/or a data type. A sequence may comprise: a step, a parameter, and/or a local variable. A step of a sequence may comprise a tree structure of step properties. Each step property may comprise one or more of: a property value, property flags, and/or a property comment. An object may comprise a hierarchy of objects (e.g., a parent object and a child sub-object). Differences between the hierarchy of objects may be detected. Differences may be navigated. Each displayed difference may be characterized as an insertion or a deletion.

Abstract: A testing system includes a controller, a device driver for the controller, and a processor. The controller is operable to control a device coupled thereto. The device driver is operable to provide a generic interface for data transfers to and from the controller. The processor is coupled to coupled to the controller and is operable to execute a test script having a plurality of script commands. Moreover, the processor is operable to transfer test data to the controller via the generic interface of the device driver in response to executing a first script command of the plurality of script commands. The processor is also operable to receive status information from the controller via the controller generic interface.

Abstract: A method of testing a device includes monitoring an output of the device, wherein the output is generated by the device in response to an applied test command; and resolving the output into atomic operations, wherein the atomic operations are substantially the smallest constituent operations which are substantially independent of the device. The method is used to provide a simple, comprehensive test environment that effectively tests 1394a and 1394-1995 designs, for example, in Verilog. The test environment contains rules which completely characterize the behavior of different 1394 bus protocols as defined by the IEEE specifications. The test environment provides portability between different devices under test and between different protocols, automated closed-loop reconciliation of test commands and protocol requirements, topology independence, and out-of-order execution of instructions or relative sequencing.

Abstract: A computerized method and system for automatically extracting an IEEE 1149.1 standard design from a netlist and performing compliance checking. The present invention receives the TAP (test access port) description and compliance enable ports of a netlist. The TAP controller is extracted and its state ports are identified, referenced in a boundary scan design database (BSDD) and its states are verified. The TAP controller is controlled so that the instruction register is located and referenced in the BSDD. The TAP controller is controlled so that the bypass register is found and the BSDD is updated. The TAP controller is controlled so that the shift and update cells of the boundary scan register (BSR) are found, the control, input and output BSR cells are characterized and the BSDD is updated. Primary input and output information is also inferred and the device_ID register is found. Frontier pins are used to locate signatures of the remaining instructions and their test data registers are found.

Abstract: A flexible test environment for automatic test equipment, whereby sequences of steps for developing and executing test programs are specified using hierarchical trees of nodes. The nodes in one tree include end leaves that correspond with the test program development steps, and the nodes in another tree include end leaves that correspond with the test program execution steps. Further, the end leaves in both trees have a plurality of associated properties, which are used for specifying test program flow and for indicating methods to be called when the steps are executed. The test environment can be easily adapted to a distributed tester architecture.

Abstract: A method for creating a computer program to be executed by a plurality of threads, in which the method utilizes a technique for execution synchronization referred to herein as a batch synchronization section. According to this technique, a plurality of threads may be associated with one another as a “batch” of threads. Each thread in the plurality (batch) of threads may execute the computer program simultaneously. The batch synchronization section may specify a portion of the computer program for which the execution of the portion by the plurality of threads is to be synchronized. In one embodiment different types of batch synchronization sections may be specified, wherein each type of batch synchronization section performs a different type of execution synchronization. In one embodiment the method may enable execution synchronization behavior for multiple concurrent executions of a test executive test sequence to be specified.

Abstract: The present invention relates to a method for testing the operability of a Peer to Peer Remote Copy (PPRC) data storage system in disaster situations. A PPRC data storage system includes a host processor, a primary storage subsystem and a secondary storage subsystem where the secondary storage subsystem is coupled to the primary storage subsystem for mirroring of data therebetween. A command is sent from the host processor directing the primary storage subsystem to simulate a disaster. Upon detection of the disaster, the host establishes direct communication with the secondary storage subsystem, and validates the integrity of the system by comparing data from the secondary storage subsystem to data from the primary storage subsystem.

Abstract: A computer is operated to generate electronic data defining a system model by loading into the computer a class definition defining instructions which are processed by the system, the definition including a set of functional methods to which the instruction is subjected by the system and a set of locations for members representing the instruction. A model execution program is then executed on the computer which calls the class definition for each instruction, invokes one of the functional methods and loads the locations of the entry with state information depending on the functional method to create a functional component. The functional component is loaded into memory and the state information of the functional component modified independence on a subsequently invoked functional method by the model execution program.

Abstract: A method and apparatus for concurrent testing of a plurality of microprocessors, each of which may have a different revision, by creating an abstract base class which specifies the names of a plurality of tests, creating a derived class for each revision and defining each of the tests appropriately for each of the derived classes, instantiating an object from one of the derived classes for each of the microprocessors and executing the tests by reference to the objects. A computer system is configured to be coupled to the microprocessors and, upon execution of a debug/test application on the computer system, the revision of each microprocessor is determined and an object is instantiated from the derived class corresponding to the revision. Each object is thereby dynamically bound to the tests defined for the derived class corresponding to the revision, and references to the tests via the objects automatically execute the test code specific to appropriate revision of the microprocessor.

Abstract: A system and method for circuitry design verification testing using a structure of interface independent classes to provide for rapid prototyping and design modification while maximizing test code re-use. A circuit simulation subsystem is interfaced with a test subsystem. The test subsystem employs a system transaction class for collecting common routines and pointers to device transactions. One or more configuration transaction classes derived from the system transaction class define transactions between functional models within the simulation subsystem and cause instantiation of the respective functional models. Operations are performed on the functional models via pointers to interface independent transaction classes which define interfaces to the devices. The operations are mapped to the current designs of the functional models by subclasses of the interface independent transaction classes.

Abstract: An apparatus and method for generating and maintaining a regression test case set directly and automatically from requirements models. A user operating the invention generates a representation of a system which captures temporal, functional, life cycle or other relationships between requirements. These relationships specify the circumstances under which a requirement applies and places the requirement in the context of other requirements. Selection techniques applied to the representation generates test cases. A regression test case set is a set of test cases that are used by the user to verify functionality in view of modifications or additions to the system. As the features of the system are adjusted over the life cycle of the system, the method of the invention automatically provides a regression test case set that affords optical coverage with respect to all features of the system. Any regression test cases that fail are automatically detected and graphically displayed.

Abstract: A test executive program which provides greatly improved configurability and modularity, thus simplifying the creation, modification and execution of test sequences. The test executive program includes process models for improved flexibility, modularity and configurability. Process models provide a modular and configurable entity for encapsulating operations and functionality associated with a class of test sequences. The process model thus encapsulates a “testing process” for a plurality of test sequences. The process model enables the user to write different test sequences without repeating standard testing operations in each sequence. The test executive program also includes step types for improved configurability. A step type is a modular, identifiable unit configured by the user which defines common properties and/or operations associated with a plurality of steps.

Abstract: A new control circuit for a watchdog timer which is incorporated on the same integrated circuit as a microprocessor or microcontroller. The control circuit permits either permanent or software enablement or disablement of the watchdog timer depending on the operating mode of the microprocessor.

Abstract: A method for supporting the end-user of a software application program is provided, wherein the end-user is provided the capability of communicating directly with the application program vendor and/or developer to request enhancement, provide comments, report defects and/or to ask questions. Moreover, this method provides an automatic communication that provides the developer of an application program such critical information as usage of program or document information, defects, and user comments. This invention provides a technique that minimizes the load and/or requirements for specially trained customer service personnel while simultaneously decreasing the feed-back lag time, thereby providing information, which is sufficiently timely to aid in the improvement of the quality of application software.

Abstract: A method and an apparatus for selecting test sequences that comprises preparing tree-structured state transition data associated with state transition weights from state transition data, extracting test sequences from the tree-structured state transition data, and repeating processes of determining averaged weight for each of the test sequences, selecting a test sequence by which the average is maximum, and decrementing the weights contained in the selected test sequence by one unit to prioritize the test sequences.

Abstract: A method and apparatus for testing one of a plurality of revisions of a microprocessor by creating an abstract base class which specifies the names of a plurality of tests or attributes, creating a derived class for each revision and defining each of the tests or attributes appropriately for each of the derived classes, instantiating an object from one of the derived classes and executing the tests by reference to the object. A computer system is configured to be coupled to a microprocessor which may be any one of a plurality of revisions. Upon execution of a debug/test application on the computer system, the revision of the microprocessor is determined and the object is instantiated from the derived class corresponding to the revision. The object is thereby dynamically bound to the tests defined for the derived class corresponding to the revision and references to each test via the object automatically execute the test code specific to the revision of the microprocessor.

Abstract: A test generator generates tests by randomly traversing a description of the interface of a program being tested, thereby generating tests that contain randomly selected actions and randomly generated data. When executed, these tests randomly manipulate the program being tested.

Abstract: A system and method perform a two-pass fault simulation on an original design representation including a software-modeled design element and a hardware-modeled design element. Logic simulation generates input stimulus for a port on the boundary of the software-modeled design element and the hardware-modeled design element, where such ports are output ports of the software-modeled design element and input ports of the hardware-modeled design element. The input stimulus is merged with test patterns for the original design representation. A modified design representation is generated by replacing the software-modeled design element with a nonfunctional block. Most or all possible faults in the hardware-modeled design representation are seeded. The modified design representation is fault simulated in a first pass using the merged input stimulus and test patterns.

Abstract: A test monitor loaded into a multiprocessor machine comprises a program (31) designed to interpret a script language for writing tests, a program (29) that constitutes a kernel part for conducting the tests according to the scripts, and a library (30) of functions that constitutes an application program interface with the firmware of the machine 1. This monitor implements a method for executing instruction sequences simultaneously in several processors (3, 4, 5) of a multiprocessor machine (1). The method comprises a first step (8) in which a single processor operating system is booted in a first processor (2) and a second step (9) in which the first processor (1) orders at least one other processor (3) of the machine, called an application processor, to execute one or more instruction sequences (17, 18, 19) under the control of said first processor.

Abstract: In this disclosure, among from verification (test) patterns deselected in conventional test pattern selection processing, one or more verification (test) patterns that can improve a fault coverage are selected, thereby achieving a fault coverage that is substantially identical to a value obtained in all verification (test) patterns by adding an almost minimum set of verification patterns. In this manner, test pattern selection processing with high efficiency can be achieved.

Abstract: A simulator simulates and verifies inter-chip functionality in a multi-chip computer system model. The chips within the multi-chip computer system model are characterized by a combination of detailed, low-level hardware models and generalized, high-level hardware emulators. As the simulator executes, inter-chip events are generated which are caused by interactions among and between the hardware models and the hardware emulators. An event processor processes events generated by the simulator, writing events to an event log file. An inter-chip event detector processes the event log file, filtering out inter-chip events caused by the hardware emulators, logging inter-chip events caused by the hardware models. Isolating inter-chip events caused by hardware models helps verification engineers direct the limited number of simulation cycles available during multi-chip verification, thus increasing the confidence level that the multi-chip computer system design is correct.

Abstract: A method and system are provided for invoking and monitoring of tests executed by multiple, diverse control systems/algorithms on a main controller which allows a single monitor system to be used to monitor the test results without requiring the monitor system to be programmed or individually tailored to process each unique output from the different possible tests. The present invention achieves independence between the monitor system and each of the specific test applications on the main controller by normalizing the unique results of each of the tests to a common value, thereby allowing the monitor system to generate expected results that can be used with any of the possible tests.

Abstract: A method and apparatus for automatically testing device drivers is disclosed. The invention is applicable to any system wherein software or a device driver receives primitive errors from a device and reports high level errors to a client. The invention generates error test data from tables in the software and creates a test data table. The invention then uses the test data table to simulate primitive errors in the device through hardware or software means. Finally, the invention ensures that the client has received the correct error messages from the software or device driver.

Abstract: This invention relates to a method for generating a test-instruction string for testing the pipeline mechanism of a processor, and which automatically generates from randomly generated instructions and instruction string which causes a pipeline interlock. This invention comprises a table (50 to 53) for notifying the subsequent instruction of the status of resource usage of the leading instruction, and by generating the resources used by the subsequent instruction according to the status of resource usage of the table (50 to 53), automatically generates a subsequent instruction that interferes with the leading instruction.

Abstract: A system that can test individual components having tolerances on a circuit board without complete access to every node on the board is disclosed. The system uses a method that develops test limits from a model of the board, component tolerances, and a list of accessible nodes. A method of reducing the complexity of the test problem by limiting the number of components under consideration is also disclosed. A method of reducing the complexity of the test problem by limiting the number of nodes under consideration is also disclosed. A method of picking nodes to apply stimulus to a board is also disclosed. Finally, a method of correcting for certain parasitics associated with tester hardware is disclosed.

Abstract: The PC SEDS database system provides a flexible database system that can encompass any type of Reliability and Maintainability (R&M) evaluation, while accurately storing and characterizing the collected data. The PC SEDS database system includes interfaces for entering, editing, processing, and analyzing the stored data. Front-end programs tailor themselves based upon information stored in a set of look-up tables. This allows the same programs to be used for a variety of projects without rewriting any code.

Abstract: A network access test system includes a test computer system having a modem bank coupled to a network access server via a telephone switch. The network access server is coupled to one or more host computer systems, preferably over a local area network. The test computer system is configured to simulate a large number of test users concurrently accessing the host computer systems via the network access server and performing typical data transfers. Preferably, each simulated test user employs unique network addresses assigned for the test computer system and the host computer system, respectively.

Abstract: A test specification generation system which utilizes a repository of design information entered in a design process so as to enhance operational efficiency of a testing process. A data analysis device reads statements written in the test configuration file, line by line, to thereby determine whether a read statement line is a fixed output line or a program output line. A test item determination device determines whether generation of a test item requires design information alone or not only the design information but also information from a standard test item file stored in a standard test item storage device. A first test item generation device generates test items based on information that the design information reading device obtained from tables of the design information, while a second test item generation device generates test items from the tables and the standard test item file.