Abstract: An automated method of making a prediction regarding development of a software product includes receiving code changes information, build information, and failure information related to the software product. Entries are stored in a database, wherein each entry links a subset of the code changes information with a subset of the build information and with a subset of the failure information. A first matrix and a second matrix are generated using the entries in the database. Multi-target entropy calculations are performed based on the first matrix and the second matrix. The prediction regarding the development of the software product is performed based on the multi-target entropy calculations.

Abstract: An automated method of making a prediction regarding development of a software product includes receiving code changes information, build information, and failure information related to the software product. Entries are stored in a database, wherein each entry links a subset of the code changes information with a subset of the build information and with a subset of the failure information. A first matrix and a second matrix are generated using the entries in the database. Multi-target entropy calculations are performed based on the first matrix and the second matrix. The prediction regarding the development of the software product is performed based on the multi-target entropy calculations.

Abstract: A method of automatically selecting tests for execution on a software product includes generating a cost model based on test performance history data that is based on results of past executions of a plurality of tests on the software product, wherein the cost model provides, for each test in the plurality of tests, a first expected monetary cost value associated with executing the test and a second expected monetary cost value associated with skipping execution of the test. The method includes automatically selecting tests in the plurality of tests for future execution based on the first and second expected monetary cost values.

Abstract: Software debugging recommendation technique embodiments are presented that generally entails creating a database of characterized software bug descriptions and providing software debugging recommendations from the database in response to a query. This can employ a two-phased approach in the search for similar software bugs. The first is a search phase that takes a query as input and returns a ranked list of software bug descriptions that match the query. These bug descriptions can contain a mix of structured and unstructured data. The second phase is a related-information phase that uses the output of the first phase to retrieve a set of related recommendations such as for people, source files, functions and binaries.

Abstract: Software debugging recommendation technique embodiments are presented that generally entails creating a database of characterized software bug descriptions and providing software debugging recommendations from the database in response to a query. This can employ a two-phased approach in the search for similar software bugs. The first is a search phase that takes a query as input and returns a ranked list of software bug descriptions that match the query. These bug descriptions can contain a mix of structured and unstructured data. The second phase is a related-information phase that uses the output of the first phase to retrieve a set of related recommendations such as for people, source files, functions and binaries.

Abstract: Techniques described herein help determine dependencies and associations between CPEs in a computing system. These techniques track previous check-ins over a period of time in order to learn the dependencies and associations between CPEs. The previous check-ins are performed by a plurality of different computer programmers. In some embodiments, in response to receiving an indication that a CPE has either already been modified or is about to be modified by a computer programmer, the techniques provide the computer programmer with a recommendation indicating CPEs that are associated with the CPE being modified. This recommendation is based on the dependencies and associations determined from the previous check-ins performed by the plurality of different computer programmers.

Abstract: Test suites can be optimized for more efficient software testing. A software program is instrumented and test cases of a test suite are run against the instrumented target binaries. A set of metrics are identified that can be used to capture a test case's execution and behavior and allow pairs of test cases of a test suite to be compared in a quantifiable manner. Metric values for test case pairs are generated and combined to create one or more unique signature values. Signature values are compared to cluster analogous test cases, allowing for, e.g., the association of comparable test cases, the identification of redundant test cases, and the formation of a test suite subset that can effectively test under time constraints.

Abstract: A system and method for testing sequences of computer-related operations by using subsequences of operations of a given length. The subsequences are used to construct a suite of test cases that reduces the number of testing permutations while satisfying desired testing requirements. A mechanism generates test cases by listing the possible subsequences of a specified length, from which a subsequence is chosen to start a test case, and marked as covered. A selection algorithm may be used to choose a subsequence, such as based on a count of the number of subsequence elements starting still uncovered subsequences. Subsequences are added to the test case until a specified maximum length, with a preference for selecting uncovered subsequences. The process continues adding test cases until no subsequence remains uncovered. The resultant set of test cases is processed based on any specified constraints and preconditions, to produce a final suite of test cases.

Abstract: Provided is a system and method comprising a test suite generation mechanism for testing a domain (e.g., software program) via “black-box” testing. Parameters are defined, appropriate values for each are chosen, and via the system and method a suite of test cases are automatically generated that covers all N-way parameter combinations (e.g., pairs when N=2) while respecting any specified domain constraints. After receiving a model comprising the parameters and respective values to test, along with constraints in a form of logical predicates, the system and method produces the test cases. Based on the constraints, an algorithm removes invalid combinations in a first stage, and prevents any other invalid combinations from being used in a second stage during the test case generation process. A full set of statistically valuable test cases are thus efficiently and automatically generated, significantly improving testing while significantly reducing the resources needed to test.

Abstract: Provided is a system and method for black-box testing of software using positive and negative test cases with N-way combinations of parameter values. An original model comprising valid and invalid values is modified in a first phase, by generating exclusions (constraints) for pairs of invalid values. A first suite of test cases is generated from the modified model, and positive test cases eliminated, creating a first test suite with only negative tests. In a second phase, the original model is modified by eliminating invalid values, from which a second test suite having only positive test cases for all valid N-way combinations is generated. Merging the two test suites provides a suite of positive and negative test cases that test software with N-way combinations of values, in which each negative test case has only one invalid value.

Abstract: A system and method for testing sequences of computer-related operations by using subsequences of operations of a given length. The subsequences are used to construct a suite of test cases that reduces the number of testing permutations while satisfying desired testing requirements. A mechanism generates test cases by listing the possible subsequences of a specified length, from which a subsequence is chosen to start a test case, and marked as covered. A selection algorithm may be used to choose a subsequence, such as based on a count of the number of subsequence elements starting still uncovered subsequences. Subsequences are added to the test case until a specified maximum length, with a preference for selecting uncovered subsequences. The process continues adding test cases until no subsequence remains uncovered. The resultant set of test cases is processed based on any specified constraints and preconditions, to produce a final suite of test cases.

Abstract: Provided is a system and method for black-box testing of software using positive and negative test cases with N-way combinations of parameter values that solves the problem of input masking. An original model comprising valid and invalid values is modified in a first phase, by generating exclusions (constraints) for pairs of invalid values. A first suite of test cases is generated from the modified model, and positive test cases eliminated, creating a first test suite with only negative tests. In a second phase, the original model is modified by eliminating invalid values, from which a second test suite having only positive test cases for all valid N-way combinations is generated. Merging the two test suites provides a suite of positive and negative test cases that thoroughly test software with N-way combinations of values, in which each negative test case has only one invalid value, thereby preventing input masking.

Abstract: Provided is a system and method comprising a test suite generation mechanism for testing a domain (e.g., software program) via “black-box” testing. Parameters are defined, appropriate values for each are chosen, and via the system and method a suite of test cases are automatically generated that covers all N-way parameter combinations (e.g., pairs when N=2) while respecting any specified domain constraints. After receiving a model comprising the parameters and respective values to test, along with constraints in a form of logical predicates, the system and method produces the test cases. Based on the constraints, an algorithm removes invalid combinations in a first stage, and prevents any other invalid combinations from being used in a second stage during the test case generation process. A full set of statistically valuable test cases are thus efficiently and automatically generated, significantly improving testing while significantly reducing the resources needed to test.