Abstract: A technique is introduced for generating automated tests based on analysis of end user interactions with an application that is to be tested. In an example embodiment, the introduced technique includes monitoring end user interaction with an application in a production environment to generate user interaction data. This user interaction data can then be analyzed to generate automated tests that are specifically tailored to the application. In some embodiments, the automated test can be executed in a separate testing environment to produce results such as screen captures of user interaction flows as well as detect bugs, errors, or other issues.

Abstract: A technique is introduced for mapping interactive elements in an application user interface as part of an automated testing process. In an example embodiment, the introduced technique includes crawling a graphical user interface (GUI) of a target application being tested, identifying interactive elements on multiple pages of the GUI based on the crawling, comparing properties of the interactive elements, and determining correspondences between interactive elements on different pages of the GUI based on the comparison of properties. In some embodiments, the introduced technique additionally includes generating element map data that is indicative of the identified correspondences between interactive elements. This element map data can then be used to, for example, detect errors in the GUI of the target application and/or detect when a test executor has returned to a previously accessed page in the GUI of the target application.

Abstract: A technique is introduced for detecting errors and other issues in an application graphical user interface (GUI) by applying machine learning to process screenshots of the GUI. In an example embodiment, the introduced technique includes crawling a GUI of a target application as part of an automated testing process. As part of the crawling, an executing computer system can interact with various interactive elements of the GUI and capture various screenshots of the GUI that depict the changing state of the GUI based on the interaction. These screenshots can then be processed using one or more machine learning models to detect errors and/or other issues with the GUI of the application. In some embodiments, the machine learning models can be trained using previously captured and labeled screenshots from other application GUIs.