Abstract: System and method for synchronizing software execution in a computer system. Threads or processes executing in the computer system are synchronized via various types of synchronization objects. User input indicating a desire to specify a synchronization object operation is received, specifying a particular type of synchronization object on which to perform the operation, in response to which a graphical user interface, is displayed which is useable to specify a plurality of synchronization object operations, e.g., that are specific to a specified type of synchronization object. The graphical user interface enables the user to select a desired operation to perform on a synchronization object. The user may also specify additional configuration information for the operation.

Abstract: A system and method for asynchronous execution of a test executive subsequence. A first test executive sequence (calling sequence) may be created and configured to asynchronously call a second test executive sequence (subsequence). In one embodiment, a “subsequence call” step may be included in the first test executive sequence, wherein the subsequence call step is operable to asynchronously call the second test executive sequence when the subsequence call step is executed. In response to executing the first test executive sequence, the second test executive sequence may also be executed, asynchronously from the first test executive sequence. The user may specify various options affecting execution of the second test executive sequence (the subsequence). For example, the user may specify various aspects of the execution environment or execution location for the subsequence.

Abstract: A test executive sequence may be created by including a plurality of test executive steps in the test executive sequence in response to user input to a sequence editor. At least a subset of the test executive steps in the test executive sequence may be configured to call user-supplied code modules. In one embodiment, one or more expressions may be configured for one or more steps in the test executive sequence, where the one or more expressions evaluate to result values to be logged when the step is executed during execution of the test executive sequence. In another embodiment, one or more parameters of one or more user-supplied code modules called by steps in the test executive sequence may be configured to be logged when the test executive sequence is executed.

Abstract: A system and method for synchronizing execution of multiple processes or threads executing to perform tests of one or more units under test. One or more synchronization steps may be included in a test executive sequence, in response to user input requesting inclusion of the synchronization steps. Each synchronization step may be configured to perform a synchronization operation, in response to user input specifying the synchronization operation. The test executive sequence may then be executed multiple times concurrently, e.g., by multiple threads or processes, and the one or more synchronization steps in the test executive sequence may coordinate the execution of the multiple threads or processes to test the unit(s) under test.

Abstract: A system and method for enabling execution stop and re-start of a test executive sequence or hierarchy of test executive sequences. Execution progress of a test executive sequence or test executive sequence hierarchy may be periodically stored. This may comprise performing or taking “snapshots” of the execution at various points during the execution. Performing a snapshot may comprise saving all data needed to restore and re-start the execution at the respective point. The criteria of when and where to perform the snapshots may be any of various criteria and may be specified in any of various ways.

Abstract: A system and method for isolating execution of user-supplied code modules that are called by steps of a test executive sequence. The test executive sequence may first be created by including a plurality of test executive steps in the test executive sequence and configuring at least a subset of the steps to call user-supplied code modules. The test executive sequence may then be executed on a host computer under control of a test executive engine. For each step that calls a user-supplied code module, the test executive engine may invoke the user-supplied code module for execution in a separate process other than the process in which the test executive engine executes.

Abstract: A system and method for detecting and/or preventing execution problems caused by user-supplied code modules that are called by steps of a test executive sequence. The test executive sequence may include a plurality of test executive steps. At least a subset of the steps may be configured to call user-supplied code modules. The test executive sequence may be executed on a host computer under control of a test executive engine. For each step that calls a user-supplied code module, the test executive engine may perform certain actions to detect and/or prevent execution problems caused by the user-supplied code module. For example, the test executive engine may be operable to automatically detect stack corruption caused by the user-supplied code module, may be operable to utilize stack safety buffers to avoid stack corruption problems caused by the user-supplied code module, and/or may include features for aiding a user in determining whether the user-supplied code module contains an uninitialized local variable.

Abstract: A system and method for automatically detecting heap corruption errors and memory leak errors caused by user-supplied code modules that are called by steps of a test executive sequence. The test executive sequence may first be created by including a plurality of test executive steps in the test executive sequence and configuring at least a subset of the steps to call user-supplied code modules. The test executive sequence may then be executed on a host computer under control of a test executive engine. For each step that calls a user-supplied code module, the test executive engine may perform certain actions to automatically detect whether the user-supplied code module causes a heap corruption error and/or automatically detect whether the user-supplied code module causes a memory leak error.

Abstract: A method for creating a computer program to be executed by a plurality of threads, in which the method utilizes a technique for execution synchronization referred to herein as a batch synchronization section. According to this technique, a plurality of threads may be associated with one another as a “batch” of threads. Each thread in the plurality (batch) of threads may execute the computer program simultaneously. The batch synchronization section may specify a portion of the computer program for which the execution of the portion by the plurality of threads is to be synchronized. In one embodiment different types of batch synchronization sections may be specified, wherein each type of batch synchronization section performs a different type of execution synchronization. In one embodiment the method may enable execution synchronization behavior for multiple concurrent executions of a test executive test sequence to be specified. The test sequence may include one or more batch synchronization sections.

Abstract: Individual test executive sequences may correspond to individual sub-components of a hierarchical system. To test a system sub-component directly without testing “upper” sub-components in the hierarchy, a first test executive sequence corresponding to that system sub-component may be executed directly, without requiring execution of test executive sequences above the first test executive sequence in the hierarchy, or requiring only partial execution of test executive sequences above the first test executive sequence in the hierarchy. Another embodiment of the invention comprises a system and method for enabling propagation or inheritance of test executive sequence local variables.

Abstract: A method for creating a computer program to be executed by a plurality of threads, in which the method utilizes a technique for execution synchronization referred to herein as a batch synchronization section. According to this technique, a plurality of threads may be associated with one another as a “batch” of threads. Each thread in the plurality (batch) of threads may execute the computer program simultaneously. The batch synchronization section may specify a portion of the computer program for which the execution of the portion by the plurality of threads is to be synchronized. In one embodiment different types of batch synchronization sections may be specified, wherein each type of batch synchronization section performs a different type of execution synchronization. In one embodiment the method may enable execution synchronization behavior for multiple concurrent executions of a test executive test sequence to be specified.

Abstract: A system and method for generating an Extensible Markup Language (XML) representation of a test executive object. In various embodiments, the test executive software may define various types of objects used in performing tests of a system. The test executive software may be operable to generate an XML representation for one or more of these test executive object types. In response to receiving a request for an XML representation of a specified test executive object, the test executive software may generate an XML representation of the specified test executive object. In one embodiment, the XML representation may include enough information about the object to enable an identical object to be re-instantiated in memory, e.g., on a different computer system. In other embodiments, the XML representation may include only a portion of the object information.

Abstract: A system and method for enabling Extensible Markup Language (XML) reporting for a test executive sequence. The test executive sequence may comprise a plurality of test executive steps operable to perform tests on a unit under test (UUT). In executing the test executive sequence, the test executive software may generate various test results, such as Boolean pass/fail results for various steps in the sequence, numeric limit values computed by various steps, etc. The test executive software may generate one or more XML files specifying the test results, i.e., the results of the execution of the test executive sequence may be represented in XML. An XSL stylesheet may then be used to transform and view the XML test results.

Abstract: A test executive program which provides greatly improved configurability and modularity, thus simplifying the creation, modification and execution of test sequences. The test executive program includes process models for improved flexibility, modularity and configurability. Process models provide a modular and configurable entity for encapsulating operations and functionality associated with a class of test sequences. The process model thus encapsulates a “testing process” for a plurality of test sequences. The process model enables the user to write different test sequences without repeating standard testing operations in each sequence. The test executive program also includes step types for improved configurability. A step type is a modular, identifiable unit configured by the user which defines common properties and/or operations associated with a plurality of steps.

Abstract: Individual test executive sequences may correspond to individual sub-components of a hierarchical system. To test a system sub-component directly without testing “upper” sub-components in the hierarchy, a first test executive sequence corresponding to that system sub-component may be executed directly, without requiring execution of test executive sequences above the first test executive sequence in the hierarchy, or requiring only partial execution of test executive sequences above the first test executive sequence in the hierarchy. Another embodiment of the invention comprises a system and method for enabling propagation or inheritance of test executive sequence local variables.

Abstract: A system and method for enabling execution stop and re-start of a test executive sequence or hierarchy of test executive sequences. Execution progress of a test executive sequence or test executive sequence hierarchy may be periodically stored. This may comprise performing or taking “snapshots” of the execution at various points during the execution. Performing a snapshot may comprise saving all data needed to restore and re-start the execution at the respective point. The criteria of when and where to perform the snapshots may be any of various criteria and may be specified in any of various ways.

Abstract: A system and method for asynchronous execution of a test executive subsequence. A first test executive sequence (calling sequence) may be created and configured to asynchronously call a second test executive sequence (subsequence). In one embodiment, a “subsequence call” step may be included in the first test executive sequence, wherein the subsequence call step is operable to asynchronously call the second test executive sequence when the subsequence call step is executed. In response to executing the first test executive sequence, the second test executive sequence may also be executed, asynchronously from the first test executive sequence. The user may specify various options affecting execution of the second test executive sequence (the subsequence). For example, the user may specify various aspects of the execution environment or execution location for the subsequence.

Abstract: A system and method for importing and exporting test executive values from or to a database. A test executive may provide the user various places, referred to as variables and properties, in which data values can be stored. These data values may affect execution of a test executive sequence. Variables may be global to a sequence file or local to a particular sequence. Each step in a sequence can have properties. The user may include a Property Loader step in a sequence, which is operable to dynamically, i.e., at run time, load property and/or variable values from a database. The Property Loader step may be placed in a setup group of the sequence, and the step(s) whose properties are configured may be placed in a main group of the sequence, so that when the sequence is executed the steps in the main group are configured with appropriate property values before running. The test executive may also enable a user to interactively, e.g.

Abstract: Abstract A test executive program which provides greatly improved configurability and modularity, thus simplifying the creation, modification and execution of test sequences. The test executive program includes process models for improved flexibility, modularity and configurability. Process models provide a modular and configurable entity for encapsulating operations and functionality associated with a class of test sequences. The process model thus encapsulates a “testing process” for a plurality of test sequences. The process model enables the user to write different test sequences without repeating standard testing operations in each sequence. The test executive program also includes step types for improved configurability. A step type is a modular, identifiable unit configured by the user which defines common properties and/or operations associated with a plurality of steps.

Abstract: A method for creating a computer program to be executed by a plurality of threads, in which the method utilizes a technique for execution synchronization referred to herein as a batch synchronization section. According to this technique, a plurality of threads may be associated with one another as a “batch” of threads. Each thread in the plurality (batch) of threads may execute the computer program simultaneously. The batch synchronization section may specify a portion of the computer program for which the execution of the portion by the plurality of threads is to be synchronized. In one embodiment different types of batch synchronization sections may be specified, wherein each type of batch synchronization section performs a different type of execution synchronization. In one embodiment the method may enable execution synchronization behavior for multiple concurrent executions of a test executive test sequence to be specified.