Abstract: Techniques for monitoring computer program test design are described herein. The techniques include a method that specifies a subset of tuples in a combinatorial test model as supervised. At least some of the supervised tuples are confirmed using expert data. The method may include determining a metric indicating progress of the combinatorial test modeling process based on the supervised tuples and the confirmed tuples.

Abstract: A method, apparatus, and computer-implemented method for optimized instrumentation based on functional coverage, the method comprising: receiving a set of states, the set of states defining required coverage of a program under test, each state of the set of states comprising a value of one or more controllable parameters and a value of one or more incontrollable parameters; executing the program under test and obtaining one or more runtime traces; comparing a value of the incontrollable parameters in the runtime trace to a value of the incontrollable parameters in one or more of the states; and responsive to the values of the incontrollable parameters in the runtime trace being equal to the values of the incontrollable parameters in any of the states, setting a value of one or more controllable parameters to the value of the one or more controllable parameters in the state.

Abstract: A computer-implemented method for test planning and test case generation, includes collecting, by a processor, a plurality of requirements, creating, by the processor, a plurality of content space specification files that includes the plurality of requirements, processing, by the processor, the plurality of content space specification files to generate a plurality of user stories, outputting, by the processor, the plurality of user stories and integrating, by the processor, the user stories into test planning and test case generation.

Abstract: A methods, apparatus and product for explaining partially illegal combinations in combinatorial models. The method comprising: obtaining a combinatorial model defining a legal test space, the combinatorial model comprising a set of attributes, a respective domain for each attribute defining possible values for the attribute, and a set of restrictions, wherein the restrictions define a combination of values of the attributes that are illegal and are excluded from the legal test case; obtaining a partially illegal combination defining value assignments to a portion of the attributes; automatically identifying an extension of the partially illegal combination, wherein the extension is excluded from the legal test space, wherein the extension can be modified to become legal by changing a portion of the value assignments defined by the partially illegal combination; and outputting the extension.

Abstract: A method executable on one or more processors for modeling a test space is provided. The method may include defining a coverage model including a set of variables. The method may also include selecting one or more variables within at least one subset of a plurality of subsets of the set of variables. The selection may be according to an interaction level requirement defined for at least one or more of the subsets, whereby the interaction level corresponds to a coverage of the test space that covers a plurality of possible combinations of the one or more variables at multiple levels. Furthermore, respective values for the one or more selected variables within the subset of the set of variables may be assigned. The method may also include one or more definitions for value combinations for said variables with assigned values.

Abstract: A method for modeling a test space is provided. The method comprises defining a coverage model including one or more attributes, wherein respective values for the attributes are assigned, one or more definitions of value properties for said attributes with assigned values, and one or more requirements that limit combination of attribute values that are legal for the model, wherein at least one of said requirements is defined using at least one value property.

Abstract: A method for refining a test plan is provided. The method comprises defining a coverage model including: one or more variables, wherein respective values for the variables are assigned, and one or more definitions for value combinations for said variables with assigned values, wherein zero or more of said value combinations are defined according to one or more restrictions for the purpose of generating a test plan to test a system for which the coverage model is constructed; determining zero or more uncovered value combinations in the test plan; and providing means to update the test plan.

Abstract: Techniques for monitoring computer program test design are described herein. The techniques include a method that specifies a subset of tuples in a combinatorial test model as supervised. At least some of the supervised tuples are confirmed using expert data. The method may include determining a metric indicating progress of the combinatorial test modeling process based on the supervised tuples and the confirmed tuples.

Abstract: A method, apparatus and computer-implemented method, the method comprising: receiving a combinatorial testing model comprising an attribute collection, the attribute collection comprising a predetermined number of initial attributes, one or more possible values for each of the initial attributes, a coverage requirement, and an order coverage requirement for the initial attributes, the order coverage requirement related to predetermined relative order in which the attribute values are to be applied; and providing two or more tests in which each of the initial attributes is assigned a value from the possible values, wherein the tests satisfy the order coverage requirement for the at least two attributes.

Abstract: A computer-implemented method for test planning and test case generation, includes collecting, by a processor, a plurality of requirements, creating, by the processor, a plurality of content space specification files that includes the plurality of requirements, processing, by the processor, the plurality of content space specification files to generate a plurality of user stories, outputting, by the processor, the plurality of user stories and integrating, by the processor, the user stories into test planning and test case generation.

Abstract: A methods, apparatus and product for explaining partially illegal combinations in combinatorial models. The method comprising: obtaining a combinatorial model defining a legal test space, the combinatorial model comprising a set of attributes, a respective domain for each attribute defining possible values for the attribute, and a set of restrictions, wherein the restrictions define a combination of values of the attributes that are illegal and are excluded from the legal test case; obtaining a partially illegal combination defining value assignments to a portion of the attributes; automatically identifying an extension of the partially illegal combination, wherein the extension is excluded from the legal test space, wherein the extension can be modified to become legal by changing a portion of the value assignments defined by the partially illegal combination; and outputting the extension.

Abstract: An apparatus and computer-implemented method for determining relevance of assignments in combinatorial models, the method comprising: receiving an attribute collection, the attribute collection comprising one or more attributes and one or more possible values for each of attributes; receiving pone or more restrictions, each restriction indicating one or more values for one or more attributes; receiving one or more assignments comprising one or more assigned values for one or more of the attributes; and determining whether the assignment is legal, illegal or partially-legal, wherein an illegal assignment is an assignment which violates a constraint by itself; a legal assignment is an assignment which is not illegal, and for every extension thereof which is illegal, a combination of values assigned to other attributes violates a constraint by itself; and a partially-legal assignment is an assignment which is neither legal nor illegal.

Abstract: A method, apparatus and computer-implemented method, the method comprising: receiving a combinatorial testing model comprising an attribute collection, the attribute collection comprising a predetermined number of initial attributes, one or more possible values for each of the initial attributes, a coverage requirement, and an order coverage requirement for the initial attributes, the order coverage requirement related to predetermined relative order in which the attribute values are to be applied; and providing two or more tests in which each of the initial attributes is assigned a value from the possible values, wherein the tests satisfy the order coverage requirement for the at least two attributes.

Abstract: A method for modeling a test space comprising defining a coverage model including: one or more variables, wherein respective values for the variables are assigned, and restrictions based on which valid variable value combinations are determined for the purpose of testing the model, wherein at least two values that are assignable to the one or more variables are merged to reduce number of variable values in the coverage model.

Abstract: A computer-implemented method for test planning and test case generation, includes collecting, by a processor, a plurality of requirements, creating, by the processor, a plurality of content space specification files that includes the plurality of requirements, processing, by the processor, the plurality of content space specification files to generate a plurality of user stories, outputting, by the processor, the plurality of user stories and integrating, by the processor, the user stories into test planning and test case generation.

Abstract: A computer-implemented method, computerized apparatus, and computer program product for test validation planning. The computer-implemented method, performed by a processor, comprising: having a test validation activity to be performed to validate results of two or more tests of a test suite; and automatically determining, by a processor, a subset of the two or more tests for which to perform the test validation activity; whereby avoiding performing duplicate validation activities. Optionally, for each test of the test suite a valuation of a set of functional attributes is available, and a subset of the functional attributes is deemed as relevant functional attributes with respect to the test validation activity. In such an embodiment, said determining is based on the valuation of the relevant functional attributes.

Abstract: A computer-implemented method for test planning and test case generation, includes collecting, by a processor, a plurality of requirements, creating, by the processor, a plurality of content space specification files that includes the plurality of requirements, processing, by the processor, the plurality of content space specification files to generate a plurality of user stories, outputting, by the processor, the plurality of user stories and integrating, by the processor, the user stories into test planning and test case generation.

Abstract: An apparatus and computer-implemented method for determining relevance of assignments in combinatorial models, the method comprising: receiving an attribute collection, the attribute collection comprising one or more attributes and one or more possible values for each of attributes; receiving pone or more restrictions, each restriction indicating one or more values for one or more attributes; receiving one or more assignments comprising one or more assigned values for one or more of the attributes; and determining whether the assignment is legal, illegal or partially-legal, wherein an illegal assignment is an assignment which violates a constraint by itself; a legal assignment is an assignment which is not illegal, and for every extension thereof which is illegal, a combination of values assigned to other attributes violates a constraint by itself; and a partially-legal assignment is an assignment which is neither legal nor illegal.

Abstract: A computer-implemented method, computerized apparatus, and computer program product for test validation planning. The computer-implemented method, performed by a processor, comprising: having a test validation activity to be performed to validate results of two or more tests of a test suite; and automatically determining, by a processor, a subset of the two or more tests for which to perform the test validation activity; whereby avoiding performing duplicate validation activities. Optionally, for each test of the test suite a valuation of a set of functional attributes is available, and a subset of the functional attributes is deemed as relevant functional attributes with respect to the test validation activity. In such an embodiment, said determining is based on the valuation of the relevant functional attributes.

Abstract: Obtaining a functional coverage model of a System Under Test (SUT) defining all functional coverage tasks of the SUT, wherein the functional coverage model defining a test-space with respect to functional attributes; obtaining a set of covered functional coverage tasks; encoding a covered Binary Decision Diagram (BDD) to represent the set of covered functional coverage tasks within the test-space; and manipulating the covered BDD to identify one or more coverage holes, wherein a coverage hole defines a set of coverage tasks in the test-space, all having a same combination of values to a subset of the functional attributes, that are not covered by the set of covered functional coverage task.