Abstract: A generic testing framework is provided that allows components authored by third parties to be tested on a platform such as an operating system while mitigating exposure of implementation details of the third party components. In one aspect, a computerized test system is provided. The system includes at least one component test that operates on a component platform such as an operating system platform, for example. The component test can be specified in a generalized manner for sending and retrieving data from one or more third party components. At least one application programming interface (API) is provided that is associated with the third party component to enable data exchanges between the component test and the third party component on the component platform, where the API hides implementation details of the third party component.

Abstract: An automated and flexible architecture is provided that enables a plurality of component tests to be executed in various testing orders while imposing a test framework to mitigate adverse effects or results on previously developed tests. In one aspect, a computerized test system is provided. The system includes a test engine to execute a plurality of component tests. One or more application programming interfaces (API) are provided to selectively determine at a subset of the tests to execute, an execution order of the tests, or a fault response for the tests, if desired. The API determines a phase execution for the tests, where the phase execution can include a set up phase, an execution phase, a validation phase, a publishing phase, and a clean up phase, for example.