Abstract: A system and method for determining concrete instances in traffic scenarios are provided. The method includes receiving a scenario in a scenario description language, wherein the scenario includes at least one sub-scenario; identifying at least one variable for the scenario and the at least one sub-scenario based on parsing of at least one actor and the received scenario; identifying at least one constraint relation derived from the scenario and the at least a sub-scenario; generating, from the at least one variable and at least one constraint, a constraint satisfaction problem; processing the constraint satisfaction problem to generate sequences of states for the at least one variable that comply with the at least one constraint, wherein the sequence of states defines the behavior of the at least one actor with time values; and determining at least one solution that includes the sequences of states.

Abstract: A system and methods thereof for monitoring proper behavior of an autonomous vehicle are provided. The method includes generating a plurality of agents, wherein each of the plurality of agents describes a physical object, wherein at least one of the plurality of agents is an agent for the DUT, generating a plurality of scenarios, wherein each scenario models a behavior of at least one of the plurality of agents, and monitoring an interaction between the plurality of agents and the DUT agent for a scenario modeling the respective agent.

Abstract: Embodiments provide methods, systems, and devices involving transaction correlation tools that may record a limited number of run attributes yet are likely to be important in the debugging process. Some embodiments may include novel tabular representations of the runs. Embodiments may allow the user to specify directives for the recording of the runs and the creation of these tables. Embodiments may include comparing sets of failing and passing runs, which may be generated at random. This approach is called statistical debugging, as it employs statistical tools to find attributes of the DVE that tend to co-occur with the failure.

Abstract: A method and system are provided for automatically generating executable system-level tests from an initial action or partially specified scenario by accumulating necessary complement actions and forming a set of constraints required by the initial action and the necessary complement actions. The set of constraints is solved by a constraint solving engine to provide an at least partial sequence of the actions and parameters thereto that satisfies the set of constraints. The sequence of actions that comply with the set of constraints are used to generate an executable system-level test code.

Abstract: A computerized method of characterizing a DUV includes executing in a verification environment (VE) a set of verification tests to stimulate the DUV to collect test results from the DUV. The method further includes collecting a set of failure data for the test results; and generating sets of common failures based on clusters of features of interest in the set of failure data. The method further includes generating a set of hints from the common failures; wherein the hints indicate a potential failure mode or a potential root cause failure of the DUV for the test results for the simplified set of tests; and generating a set of debug data from the clusters of features of interest. The method further includes transferring the set of hints and the set of debug data to a user computer for storage, display, and use in an interactive debug session of the DUV.

Abstract: A system and method for generalized scenarios, for automatically generating tests. The tests are generated from some underlying structure, such as one or more scenarios. The scenarios preferably include a plurality of constraints for generating a test program for generating input values to the test generation process. The scenarios provide a more generalized method for generating tests.

Abstract: Method, apparatus, and computer readable medium for functionally verifying a physical device under test (DUT) is described. In one example, verification test data is generated for the physical DUT using a constraint-based random test generation process. For example, the architecture, structure, and/or content of the verification test data may be defined in response to constraint data and an input/output data model. A first portion of the verification test data is applied to the physical DUT. Output data is captured from the physical DUT in response to application of the first portion of the verification test data. A second portion of the verification test data is selected in response to the output data. Expected output data for the physical DUT associated with the verification test data may be generated and compared with the output data captured from the DUT to functionally verify the design of the DUT.

Abstract: A system and method for generalized scenarios, for automatically generating tests. The tests are generated from some underlying structure, such as one or more scenarios. The scenarios preferably include a plurality of constraints for generating a test program for generating input values to the test generation process. The scenarios provide a more generalized method for generating tests.

Abstract: A system and method for testing the quality of a simulation model for the DUT (device under test) through temporal coverage of the testing and verification process. Temporal coverage examines the behavior of selected variables over time, according to a triggering event. Such a triggering event could be determined according to predefined sampling times and/or according to the behavior of another variable, for example. This information is collected during the testing/verification process, and is then analyzed in order to determine the behavior of these variables, as well as the quality of the simulation model for the DUT. For example, the temporal coverage information can be analyzed to search for a coverage hole, indicated by the absence of a particular value from a family of values.

Abstract: A method and apparatus are provided for functionally verifying an integrated circuit design. A hardware-oriented verification-specific object-oriented programming language is used to construct and customize verification tests. The language is extensible, and shaped to provide elements for stimulating and observing hardware device models. The invention is platform and simulator-independent, and is adapted for integration with Verilog, VHDL, and C functions. A modular system environment ensures interaction with any simulator through a unified system interface that supports multiple external types. A test generator module automatically creates verification tests from a functional description. A test suite can include any combination of statically and dynamically-generated tests. Directed generation constrains generated tests to specific functionalities. Test parameters are varied at any point during generation and random stability is supported.

Abstract: A method and apparatus are provided for functionally verifying an integrated circuit design. A hardware-oriented verification-specific object-oriented programming language is used to construct and customize verification tests. The language is extensible, and shaped to provide elements for stimulating and observing hardware device models. The invention is platform and simulator-independent, and is adapted for integration with Verilog, VHDL, and C functions. A modular system environment ensures interaction with any simulator through a unified system interface that supports multiple external types. A test generator module automatically creates verification tests from a functional description. A test suite can include any combination of statically and dynamically-generated tests. Directed generation constrains generated tests to specific functionalities. Test parameters are varied at any point during generation and random stability is supported.

Abstract: A method and apparatus are provided for functionally verifying an integrated circuit design. A hardware-oriented verification-specific object-oriented programming language is used to construct and customize verification tests. The language is extensible, and shaped to provide elements for stimulating and observing hardware device models. The invention is platform and simulator-independent, and is adapted for integration with Verilog, VHDL, and C functions. A modular system environment ensures interaction with any simulator through a unified system interface that supports multiple external types. A test generator module automatically creates verification tests from a functional description. A test suite can include any combination of statically and dynamically-generated tests. Directed generation constrains generated tests to specific functionalities. Test parameters are varied at any point during generation and random stability is supported.

Abstract: A method and apparatus are provided for functionally verifying an integrated circuit design. A hardware-oriented verification-specific object-oriented programming language is used to construct and customize verification tests. The language is extensible, and shaped to provide elements for stimulating and observing hardware device models. The invention is platform and simulator-independent, and is adapted for integration with Verilog, VHDL, and C functions. A modular system environment ensures interaction with any simulator through a unified system interface that supports multiple external types. A test generator module automatically creates verification tests from a functional description. A test suite can include any combination of statically and dynamically-generated tests. Directed generation constrains generated tests to specific functionalities. Test parameters are varied at any point during generation and random stability is supported.