Abstract: Systems, methods, and computer-readable media are described for performing revision control for a System Under Test (SUT) such as a body of source code. Prior to committing code changes, a collection of breakpoints associated with the portion(s) of source code being changed are determined. Stored fingerprints corresponding to regression test cases are evaluated to identify a set of stored fingerprints that are cumulatively indicative of the collection of breakpoints. Attributes respectively stored in association with the set of stored fingerprints are determined and a corresponding group of regression test cases are determined. The code changes are committed responsive to successful execution of the selected group of regression test cases.

Abstract: A method for identifying optimal tests includes defining functional coverage by a test suite based on a functional coverage model of a System Under Test (SUT). The test suite includes a plurality of tests. The functional coverage model includes a plurality of attributes, each attribute having a set of possible values. The functional coverage model defines possible combinations of values of the attributes as covered by the test suite. A subset of the possible combinations of values is determined. A subset of the plurality of tests is selected that is operative to cover the determined subset of the possible combinations of values. A plurality of trees is generated to represent the selected subset of tests. Complexity of the generated trees is analyzed based on user-specified criteria. An optimal tree is selected from the subset of the plurality of trees to achieve the objective of the test suite.

Abstract: Systems, methods, and computer-readable media are described for utilizing breakpoint value-based fingerprints of failing regression test cases to determine specific components of a System Under Test (SUT) that are causing a fault such as specific lines of source code. A failing test case from a regression run is selected and fault localization and inverse combinatorics techniques are employed to generate a set of failing test cases around the selected failing test case. A set of test fingerprints corresponding to the set of failing test cases is compared to a set of test fingerprints corresponding to a set of passing test cases to determine breakpoints that are indicated as being encountered during execution of at least one failing test case and that are not encountered during execution of any of the passing test cases. Specific lines of source code that correspond to these breakpoints are then identified as causing the fault.

Abstract: Systems, methods, and computer-readable media are described for associating regression test cases with corresponding test fingerprints via a translation table or other suitable storage mechanism. A test fingerprint indicates a collection of breakpoints encountered as part of traversal of a code path during execution of a test case, and thus, provides an indication of an extent of code coverage of the test case. Test fingerprints can be evaluated to determine cumulative code coverage of sets of test cases. Specific sets of test cases can be selected based on the cumulative code coverage they provide. Sets of test cases that provide greater cumulative code coverage and/or that target specific areas of interest can be prioritized for execution particularly when having to adhere to execution constraints.

Abstract: A method for automatically avoiding fault paths in software code of a System Under Test (SUT) includes generating a plurality of fingerprints by executing a plurality of regression tests. Each of the plurality of fingerprints uniquely identifies a specific code path in the software code of the SUT. A critical error is detected during execution of the software code of the SUT. A fault code path in the software code of the SUT associated with the critical error is identified by analyzing the plurality of generated fingerprints. At least one fingerprint associated with the fault code path in the software code of the SUT is identified. During subsequent execution of the software code of the SUT, the identified fault code path in the software code of the SUT is automatically prevented from being executed based on the identified at least one fingerprint.

Abstract: A method for reducing test case infrastructure includes generating a first plurality of fingerprints for a first plurality of test cases. Each of the first plurality of fingerprints is associated with one of the first plurality of test cases. Each of the first plurality of fingerprints uniquely identifies a specific code path covered by a corresponding test case. A second plurality of test cases is generated based on a functional coverage model of the SUT. A second plurality of fingerprints is generated for the second plurality of test cases. Each of the second plurality of fingerprints is associated with one of the second plurality of test cases. The first plurality of fingerprints is compared to the second plurality of fingerprints to identify one or more matching fingerprints. One or more test cases is identified within the first plurality of test cases associated with the one or more matching fingerprints.

Abstract: Systems, methods, and computer-readable media are described for generating fingerprints for regression tests that identify code paths taken during execution of the regression tests. Breakpoint values are assigned to breakpoints encountered during execution of a regression test and a fingerprint is generated based on the assigned breakpoint values and a code path taken during execution of the regression test. Various breakpoint value assignment/fingerprint generation algorithms are described herein that generate a fingerprint from which a corresponding code path taken can be reconstructed including which breakpoints are encountered, the number of times each such breakpoint is encountered, and the order in which the breakpoints are encountered.

Abstract: A method includes defining functional coverage by a first test suite based on a first functional coverage model of a System Under Test (SUT). The first test suite includes a first plurality of tests. The first functional coverage model includes a first plurality of attributes. The first functional coverage model defines possible combinations of values of the first plurality of attributes. Functional coverage by a second test suite is defined based on a second functional coverage model which includes a second plurality of attributes. The second functional coverage model defines possible combinations of values of the second plurality of attributes. Subsets of the first and second plurality of attributes are determined. The subsets of attributes include exclusively common attributes between the first and the second plurality of attributes. A subset of the tests is selected. The selected subset is operative to cover the first and second subsets of the attributes.

Abstract: Systems, methods, and computer-readable media for identifying a champion test case that provides an increased likelihood of exposing a fault and expanding a set of test cases to include the champion test case are described. The fault may occur in a System Under Test (SUT), which may be a hardware system or a software system. A weight may be assigned to a champion test case that indicates the relative strength of the test case in detecting faults. The weight may be used to influence the selection of other test cases.

Abstract: Systems, methods, and computer-readable media are described for expanding test space coverage for testing performed on a System Under Test (SUT) through iterative test case generation from combinatoric pairwise outputs. At each test case generation iteration, a new set of test vectors is generated that provides complete pairwise coverage of the test space but that does not include any overlapping test vector with any previously generated set of test vectors. As such, cumulative m-wise test space coverage (where 2<m?n) is incrementally increased through each iteration until the iterative process ceases when a desired percentage of m-wise test space coverage is achieved.

Abstract: Systems, methods, and computer-readable media are described for performing fault detection and localization using Combinatorial Test Design (CTD) techniques and generating a regression bucket of test cases that expose a detected fault in a System Under Test (SUT). The SUT may be a hardware system or a software system. Further, the fault detection and localization may be performed while adhering to architectural restrictions on the SUT.

Abstract: Systems, methods, and computer-readable media are described for performing fault detection and localization using Combinatorial Test Design (CTD) techniques and generating a regression bucket of test cases that expose a detected fault in a System Under Test (SUT). The SUT may be a hardware system or a software system. Further, the fault detection and localization may be performed while adhering to architectural restrictions on the SUT.

Abstract: Method, apparatus and product for test selection based on domination criterion. In some embodiments, excluding from a test suite dominated tests, each of which is dominated by a predetermined number of dominating tests, wherein a dominated test is dominated by a dominating test if each target that is covered by the dominated test is also covered by the dominating test. In some embodiments, a reduced test suite is determined by excluding from a test suite each test that covers a dominated set of targets that is N-dominated by the reduced test suite, wherein a dominated set of targets is N-dominated by a set of tests if each target in the dominated set of targets is covered by at least N tests in the set of tests, wherein N is a predetermined number greater than one.

Abstract: Based on a functional coverage by a test suite, a functional coverage model of a System Under Test (SUT) may be defined to represent all covered combinations of functional attributes. Based on an n-wise combination criteria, a subset of the possible combinations of values may be determined. A subset of the test suite may be selected such that the selected subset is operative to cover the subset of the determined possible combinations of values. The disclosed subject matter may be used to reduce a size of the test suite while preserving the n-wise combinations coverage of the original test suite.

Abstract: A functional coverage model may be generated based on a trace. The functional coverage model may comprise values of attributes. The values may be extracted from one or more entries comprised by the trace. Based upon a selection of a field of an exemplary entry, content of the corresponding fields in other entries may be extracted and utilized. In some exemplary embodiments, names of attributes may be extracted from the trace. In some exemplary embodiments, a reference trace may be utilized as a comparative baseline coverage by generating a functional coverage model based upon the reference trace and comparing coverage of a second trace in respect to the functional coverage model with the coverage of the reference trace.

Abstract: A meta attribute useful for functional coverage is computed based on values comprised by two or more matching fields in a trace. The two or more matching fields may be comprised by a single entry of the trace or by a plurality of entries of the trace. A definition of the meta attribute may define which fields of the entries comprise values useful for computation of the meta attributes. The matching entries may be identified based on identifying values within the entries.

Abstract: Based on a functional coverage by a test suite, a functional coverage model of a System Under Test (SUT) may be defined to represent all covered combinations of functional attributes. Based on an n-wise combination criteria, a subset of the possible combinations of values may be determined A subset of the test suite may be selected such that the selected subset is operative to cover the subset of the determined possible combinations of values. The disclosed subject matter may be used to reduce a size of the test suite while preserving the n-wise combinations coverage of the original test suite.

Abstract: A functional coverage model may be generated based on a trace. The functional coverage model may comprise values of attributes. The values may be extracted from one or more entries comprised by the trace. Based upon a selection of a field of an exemplary entry, content of the corresponding fields in other entries may be extracted and utilized. In some exemplary embodiments, names of attributes may be extracted from the trace. In some exemplary embodiments, a reference trace may be utilized as a comparative baseline coverage by generating a functional coverage model based upon the reference trace and comparing coverage of a second trace in respect to the functional coverage model with the coverage of the reference trace.

Abstract: A meta attribute useful for functional coverage is computed based on values comprised by two or more matching fields in a trace. The two or more matching fields may be comprised by a single entry of the trace or by a plurality of entries of the trace. A definition of the meta attribute may define which fields of the entries comprise values useful for computation of the meta attributes. The matching entries may be identified based on identifying values within the entries.

Abstract: A simulator of WBEM/CIM indication providers conforming to the CIM Indication Provider object specification simulates both the CIM indication provider and the means to drive the associated CIM events. The simulator comprises three functionally unique pieces: one or more CIM indication provider drivers, one or more CIM event trigger drivers, and a control application. This modularization creates flexibility in configuring the simulator to stress test different aspects of an operating system's underlying support for CIM indications. Modularization also makes the simulator design operating system independent. Provision is made in the simulation for generation of additional CIM events as background activity on the operating system.