Abstract: The present invention extends to methods, systems, and computer program products for isolating the evaluation of actual test results against expected test results from the test module that generates the actual test results. A results evaluation sub-system receives actual test results resulting from the execution of a test in a test environment. A results retrieval sub-system receives environmental data indicating that a test was performed in the test environment. The results retrieval sub-system selects one or more expected results from a results database based on the received environmental data. The results retrieval sub-system sends the selected one or more expected results to the results evaluation sub-system. The results evaluation sub-system receives the one or more selected results. The results evaluation sub-system evaluates the actual test results against the one or more expected results to determine if the test was successful.

Abstract: Methods, systems, and computer program products for testing software components without explicitly having to specify each test or test environment. A computer system for automated testing loads and parses and test file. The test file contains code sections that identify software components to test, locations sections that identify locations to run the software components, an expandable variation section that associates the software components with locations, and an expansion section that defines rules for expanding the expandable variation section. From the expandable variation section, the computer system creates expanded variation sections that associate particular software components with particular locations. Then, for each particular software component and locations, the computer system executes setup instructions to prepare for running the particular software component, executes the particular software component, and executes cleanup instructions.