Abstract: The technology described herein provides an automated software-testing platform that functions in an undefined action space. The technology described herein starts with an undefined action space but begins to learn about the action space through random exploration. Both the action taken during testing and the resulting state may be communicated to a centralized testing service. The technology described herein also mines the action telemetry data and state telemetry data to identify action patterns that produce a sought after result. Once a plurality of action patterns is identified and, at least, a partial model of the action space is built, the testing on the test machines may be split into random test mode, replay test mode, and a pioneering test mode.

Abstract: The technology described herein provides an automated software-testing platform that uses reinforcement learning to discover how to perform tasks used in testing. The technology described herein is able to perform quality testing even when prescribed paths to completing tasks are not provided. The reinforcement-learning agent is not directly supervised to take actions in any given situation, but rather learns which sequences of actions generate the most rewards through the observed states and rewards from the environment. In the software-testing environment, the state can be user interface features and actions are interactions with user interface elements. The testing system may recognize when a sought after state is achieved by comparing a new state to a reward criteria.

Abstract: The technology described herein trains a reinforcement-learning model in a simulated environment. A simulated environment contrasts with a live environment. A live environment is a computing environment with which the reinforcement-learning model will interact once it is deployed. In order to be effective, the simulated environment may provide inputs to the reinforcement-learning model in the same format as the reinforcement-learning model receives from the live environment. In aspects, the training in the simulated environment may act as pre-training for training in the live environment. Once pre-trained, the reinforcement-learning model may be deployed in a live environment and continue to learn how to perform the same task in different ways, learn how to perform additional tasks, and/or improve performance of a task learned in pre-training. In aspects, the reinforcement-learning model may be used to discover unhealthy conditions in software by performing the tasks it has learned.

Abstract: The technology described herein provides a cloud reinforcement-learning architecture that allows a single reinforcement-learning model to interact with multiple live software environments. The live software environments and the single reinforcement-learning model run in a distributed computing environment (e.g., cloud environment). The single reinforcement-learning model may run on a first computing device(s) with a graphical processing unit (GPU) to aid in training the single reinforcement-learning model. At a high level, the single reinforcement-learning model may receive state telemetry data from the multiple live environments. The single reinforcement-learning model selects an available action for each set of state telemetry data received and communicates the selection to appropriate the test agent. The test agent then facilitates completion of the action within the software instance being tested in the live environment. A reward is then determined for the action.

Abstract: Method and memory medium for generating a web service. A plurality of graphical data flow programs may be provided, and user input selecting one or more of plurality of graphical data flow programs for inclusion in a web service may be received, The web service may be generated based on the one or more graphical data flow programs. Each graphical data flow program may implement a respective web method, where each web method may implement or request a respective action. The web service may be deployable to a server for hosting, where the web service is invocable over a network to perform the corresponding one or more web methods.

Abstract: Method and memory medium for generating a web service. A plurality of graphical data flow programs may be provided, and user input selecting one or more of plurality of graphical data flow programs for inclusion in a web service may be received, The web service may be generated based on the one or more graphical data flow programs. Each graphical data flow program may implement a respective web method, where each web method may implement or request a respective action. The web service may be deployable to a server for hosting, where the web service is invocable over a network to perform the corresponding one or more web methods.