Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.

Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.

Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.

Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, several tests for testing the SUT. The tests are generated based on a coverage model of the SUT, which includes multiple attributes. The method further includes creating, by the testing system, a minimal set of tests from the tests by selecting tests that do not exceed a predetermined performance threshold. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment. The soft failure occurs in the active environment during execution of the SUT based at least in part on a performance parameter of the active environment.

Abstract: A method for testing a system under test (SUT) in an active environment to identify cause of a soft failure includes recording a first difference vector by executing a set of test cases on a baseline system and monitoring performance parameters of the baseline system before and after executing the test cases. Each performance record represents differences in the performance parameters of the baseline system from before and after the execution of a corresponding test case. The method further includes, similarly, recording a second difference vector by executing the test cases on the SUT and monitoring performance parameters of the SUT before and after executing the test cases. The method further includes identifying an outlier performance record from the second difference vector by comparing the difference vectors and further, determining a root cause of the soft failure by analyzing a test case corresponding to the outlier.

Abstract: A method for testing a system under test (SUT) in an active environment includes generating, by a testing system, several tests for testing the SUT. The tests are generated based on a coverage model of the SUT, which includes multiple attributes. The method further includes creating, by the testing system, a minimal set of tests from the tests by selecting tests that do not exceed a predetermined performance threshold. The method further includes executing, by the testing system, the minimal set of tests on the SUT for analyzing a soft failure of the SUT in the active environment. The soft failure occurs in the active environment during execution of the SUT based at least in part on a performance parameter of the active environment.