Abstract: Functional coverage techniques during design verification using cross-product coverage models and hole analysis are enhanced by the use of coverage queries. After running a test suite, a core set of non-covered events is specified. A coverage query is then automatically constructed and executed on the test results to identify a hole in the functional coverage that satisfies conditions of the coverage query and includes the core set. The results of the query are presented as a simplified view of the coverage of the events in the cross-product space. Use of coverage queries allows a verification team to focus on specific areas of interest in the coverage space and to deal practically with highly complex coverage models. It also avoids the burden of producing and evaluating complete hole analysis reports.

Abstract: Apparatus for presentation of functional coverage, including one or more processors and a memory, wherein the memory stores software instructions including instructions for representing a set of attributes of a design under test as a multi-dimensional cross-product space, comprising events corresponding to combinations of values of the attributes to be tested, the events comprising legal and illegal events, instructions for running at least one test on the design, instructions for identifying, responsively to the at least one test, a first group of the legal events that were covered by the at least one test and a second group of the legal events that remain non-covered after the at least one test, instructions for grouping one or more of the illegal events with at least one of the first and second groups so as to generate a simplified model of the functional coverage of the events in the cross-product space and instructions for presenting the simplified model of the functional coverage on an output device.

Abstract: Methods, apparatus and systems are provided that enable the generation of random regression suites for verification of a hardware or software design to be formulated as optimization problems. Solution of the optimization problems using probabilistic methods provides information on which set of test specifications should be used, and how many tests should be generated from each specification. In one mode of operation regression suites are constructed that use the minimal number of tests required to achieve a specific coverage goal. In another mode of operation regression suites are constructed so as to maximize task coverage when a fixed number of tests are run or within a fixed cost.

Abstract: Functional coverage techniques during design verification using cross-product coverage models and hole analysis are enhanced by the use of coverage queries. After running a test suite, a core set of non-covered events is specified. A coverage query is then automatically constructed and executed on the test results to identify a hole in the functional coverage that satisfies conditions of the coverage query and includes the core set. The results of the query are presented as a simplified view of the coverage of the events in the cross-product space. Use of coverage queries allows a verification team to focus on specific areas of interest in the coverage space and to deal practically with highly complex coverage models. It also avoids the burden of producing and evaluating complete hole analysis reports.

Abstract: A method for presentation of functional coverage includes representing a set of attributes of a design under test as a multi-dimensional cross-product space, which includes events corresponding to combinations of values of the attributes to be tested, the events including legal and illegal events. At least one test is run on the design, and responsively to the at least one test, a first group of the legal events that were covered by the at least one test and a second group of the legal events that remain non-covered after the at least one test are identified. One or more of the illegal events are grouped with at least one of the first and second groups so as to present a simplified model of the coverage of the events in the cross-product space.

Abstract: A method for design verification includes running a simulation of a design in a simulation environment, which comprises a stimuli generator for providing inputs to the design during the simulation. Respective measures of quality are computed for at least some of the simulation states in a sequence of states generated by the environment. State data are saved with respect to at least one of the simulation states. The state data include indications both of the respective simulated state and of the respective environment state. Responsively to the respective measures of quality, the saved state data are recalled so as to restart the simulation from the at least one of the simulation states by returning the design to the respective simulated state and returning the simulation environment to the respective environment state.

Abstract: Test generation is improved by learning the relationship between an initial state vector for a stimuli generator and generation success. A stimuli generator for a design-under-verification is provided with information about the success probabilities of potential assignments to an initial state bit vector. Selection of initial states according to the success probabilities ensures a higher success rate than would be achieved without this knowledge. The approach for obtaining an initial state bit vector employs a CSP solver. A learning system is directed to model the behavior of possible initial state assignments. The learning system develops the structure and parameters of a Bayesian network that describes the relation between the initial state and generation success.

Abstract: Functional coverage techniques during design verification using cross-product coverage models and hole analysis are enhanced by the use of coverage queries. After running a test suite, a core set of non-covered events is specified. A coverage query is then automatically constructed and executed on the test results to identify a hole in the functional coverage that satisfies conditions of the coverage query and includes the core set. The results of the query are presented as a simplified view of the coverage of the events in the cross-product space. Use of coverage queries allows a verification team to focus on specific areas of interest in the coverage space and to deal practically with highly complex coverage models. It also avoids the burden of producing and evaluating complete hole analysis reports.

Abstract: A coverage-directed test generation technique for functional design verification relies on events that are clustered according to similarities in the way that the events are stimulated in a simulation environment, not necessarily related to the semantics of the events. The set of directives generated by a coverage-directed test generation engine for each event is analyzed and evaluated for similarities with sets of directives for other events. Identified similarities in the sets of directives provide the basis for defining event clusters. Once clusters have been defined, a common set of directives for the coverage-directed test generation engine is generated that attempts to cover all events in a given cluster.

Abstract: A Bayesian network correlating coverage data and input data to a test verification system for coverage directed test generation (CDG) of a device under test. In one embodiment, the Bayesian network is part of a CDG engine which also includes a data analyzer which analyzes coverage data from a current test run of a test verification system and from previous test runs to determine which coverage events from a coverage model have occurred therein, at what frequency and which ones have not yet occurred, a coverage model listing coverage events which define the goal of the test verification system and a task manager coupled to the data analyzer and the Bayesian network which refers to the coverage model and queries the Bayesian network to produce input data to achieve desired coverage events.

Abstract: Methods, apparatus and systems are provided that enable the generation of random regression suites for verification of a hardware or software design to be formulated as optimization problems. Solution of the optimization problems using probabilistic methods provides information on which set of test specifications should be used, and how many tests should be generated from each specification. In one mode of operation regression suites are constructed that use the minimal number of tests required to achieve a specific coverage goal. In another mode of operation regression suites are constructed so as to maximize task coverage when a fixed number of tests are run or within a fixed cost.

Abstract: Test generation is improved by learning the relationship between an initial state vector for a stimuli generator and generation success. A stimuli generator for a design-under-verification is provided with information about the success probabilities of potential assignments to an initial state bit vector. Selection of initial states according to the success probabilities ensures a higher success rate than would be achieved without this knowledge. The approach for obtaining an initial state bit vector employs a CSP solver. A learning system is directed to model the behavior of possible initial state assignments. The learning system develops the structure and parameters of a Bayesian network that describes the relation between the initial state and generation success.

Abstract: A method for presentation of functional coverage includes representing a set of attributes of a design under test as a multi-dimensional cross-product space, which includes events corresponding to combinations of values of the attributes to be tested, the events including legal and illegal events. At least one test is run on the design, and responsively to the at least one test, a first group of the legal events that were covered by the at least one test and a second group of the legal events that remain non-covered after the at least one test are identified. One or more of the illegal events are grouped with at least one of the first and second groups so as to present a simplified model of the coverage of the events in the cross-product space.

Abstract: A coverage-directed test generation technique for functional design verification relies on events that are clustered according to similarities in the way that the events are stimulated in a simulation environment, not necessarily related to the semantics of the events. The set of directives generated by a coverage-directed test generation engine for each event is analyzed and evaluated for similarities with sets of directives for other events. Identified similarities in the sets of directives provide the basis for defining event clusters. Once clusters have been defined, a common set of directives for the coverage-directed test generation engine is generated that attempts to cover all events in a given cluster.

Abstract: Methods and systems are provided that improve design verification by test generators by delaying assignment of values in the generated stimuli until these values are used in the design. Late binding allows the generator to have a more accurate view of the state of the design, and in order to choose correct values. Late binding can significantly improve test coverage with a reasonable performance penalty as measured by simulation time.

Abstract: A computer-implemented method and system for automatically invoking a predetermined debugger command at a desired location of a single thread of a program containing at least one thread. At the desired location of the program thread, there is embedded a utility which reads a trace file in which the predetermined debugger command has been previously embedded. Upon re-running the program, the trace file is read and upon reaching the predetermined debugger command, the debugger attaches itself to the running process and executes the process from its current program counter. The debugger is invoked only if there is a discrepancy between successive runs of the program.

Abstract: Methods and systems are provided that improve design verification by test generators by delaying assignment of values in the generated stimuli until these values are used in the design. Late binding allows the generator to have a more accurate view of the state of the design, and in order to choose correct values. Late binding can significantly improve test coverage with a reasonable performance penalty as measured by simulation time.

Abstract: A Bayesian network correlating coverage data and input data to a test verification system for coverage directed test generation (CDG) of a device under test. In one embodiment, the Bayesian network is part of a CDG engine which also includes a data analyzer which analyzes coverage data from a current test run of a test verification system and from previous test runs to determine which coverage events from a coverage model have occurred therein, at what frequency and which ones have not yet occurred, a coverage model listing coverage events which define the goal of the test verification system and a task manager coupled to the data analyzer and the Bayesian network which refers to the coverage model and queries the Bayesian network to produce input data to achieve desired coverage events.

Abstract: A method, apparatus and article of manufacture for persistent code coverage data collection are provided. Initially, a program for which code coverage data should be collected is identified and divided into code coverage tasks (i.e. basic blocks) and each code coverage task is given a unique name. Coverage points are then inserted into the program source code at the beginning of each coverage task to produce an instrumented program. The instrumented program is then compiled and link-edited with an appropriate library to produce a program executable. A set of test cases to be run for a persistent code coverage data collection purposes is identified next. Then, the code coverage database is created using the identified code coverage tasks and the test cases. The program executable is loaded and run with the set of identified test cases to write coverage point information into an output file.

Abstract: Coverage measurement tool enabling a user to create a specific coverage tool for a coverage model including a set of coverage tasks specified by the user for checking a design such as a hardware or software system, and being associated with a database and model definition that defines the coverage model. The tool includes an insertion engine for storing into the database a table containing traces resulting from multiple tests, a processing engine processing the traces in the database according to the model definition, and a coverage analyzer analyzing the measurement results from the processing engine and preparing coverage analysis reports according to the model definition.