Abstract: A scalable test workflow service facilitates management, automated generation and execution of numerous test cases. Test definitions representing tasks to be executed to validate a capability, parameter providers that provide metadata options, and step executors that execute the individual steps of the tasks are all specified. In response to a request to perform a test, an iterative process builds, based on the test definitions and the corresponding parameters, test instances that are executed to obtain the test results. For example, a run tests workflow initiates a test initialization workflow for each test definition. The test initialization workflows call the parameter providers to obtain parameter combinations used to create test instances, and initiate test execution workflows for each parameter combination. Each test execution workflow calls a step executor for each step of the test definition using parameters for the test instance, and returns the results. Results are aggregated and returned.