Abstract: Disclosed are examples of systems, apparatuses, methods, and computer program products for generating synthesized user data. A method may involve receiving a data specification schema. A method may involve determining a number of test data objects to be generated. A method may involve defining the test data objects, the defining of each test data object including: determining, from the data specification schema, a number of fields of the test data object to be populated, the fields representing categories of simulated user data; and determining values for the fields, the values simulating user data. The method may involve storing the test data objects in a database. The method may involve generating a tabular data file including or identifying the test data objects, the tabular data file configured to be processed by one or more processors of a computing system during a user data testing procedure of the computing system.

Abstract: Methods and apparatus are described by which a rich, time-correlated information set is captured during automated testing of an application in a way that allows the application developer to understand the state of the application under test (AUT), the browser interacting with the AUT, and/or the device interacting with the AUT, as it/they changed over time. Mechanisms or features associated with browsers and/or device operating systems are exploited to capture such information, not only for the purpose of better understanding individual test runs, but also to enable the use of analytics over data sets.

Abstract: In one aspect, a computerized method useful for installing and managing a DevTool web-browser extension comprising: determining a web element selector using the DevTool web-browser extension; performing a specified action on a specified web element, wherein an address of the webpage element is ascertained using the specified selector output; with the DevTool web-browser extension, generating the address for specified web element automatically with a single click operation from a user into the web browser; with the address, generating a relative XPath selector, an absolute XPath selector or a Cascading Style Sheets (CSS) selector; and utilizing the DevTool web-browser extension as an editor tool for the relative XPath selector, the absolute XPath selector or the CSSs selector.

Abstract: Machine learning techniques are employed to model test runs of an automated test platform in ways that allow for reliable identification of various types of test behavior such as, for example, whether certain classes of failures can be characterized as test flake.

Abstract: In one aspect, a computerized method useful for automated software testing comprising: writing a test suite in a human-readable language; implementing an Artificial Intelligent (AI) process test suite; and creating a set of scripts, data, and execute tests.

Abstract: Methods and apparatus are described by which artificial intelligence (AI) is used to enable the rapid development of reliable test suites for web and mobile applications. An AI agent guided by reinforcement learning explores an application-under-test (AUT), interacting with the AUT to traverse the flows through the AUT by seeking novel application states. A subset of these flows is then identified as being representative of the functionality of the AUT. The interactions between the AI agent and the AUT that define these identified flows form the basis for the test suite.

Abstract: Methods and apparatus are described by which time-series data captured during the automated testing of software applications may be analyzed. Change-point detection is used to partition the time-series data, and an expected variance of data within each partition is determined. Because the partitioning of the test data provides a high level of confidence that the data points in a given partition conform to the same distribution, data points that represent meaningful changes in application performance can be more confidently and efficiently identified.

Abstract: Methods and apparatus are described by which a rich, time-correlated information set is captured during automated testing of an application in a way that allows the application developer to understand the state of the application under test (AUT), the browser interacting with the AUT, and/or the device interacting with the AUT, as it/they changed over time. Mechanisms or features associated with browsers and/or device operating systems are exploited to capture such information, not only for the purpose of better understanding individual test runs, but also to enable the use of analytics over data sets.

Abstract: A software verification system including a cloud service, a plurality of test nodes, a client manager, and an expediter is disclosed. The cloud service is a commercially available shared resource service having two or more physical machines, each of which is configured to host a plurality of virtual machines. Each test node is one virtual machine running a platform. The client manager manages a verification session between one of the clients and the verification system. The expediter recognizes the available platforms for use by the test nodes in a cloud service pool of servers and a local pool of servers. When a clients requests the verification session to use a specific platform, the expediter obtains a network address of an available test node running the specified platform and connects the requesting client to the available test node running the requested platform.

Abstract: Analytics of the recorded user interface operations at clients accessing a web application at a server are used to develop a test routine for testing the web application at a cloud service. A code module, which is either at the server or attached to the browser in each of the clients, records and correlates each of the user interface operations at each client to each action taken in the web application as it is being navigated. The module then transmits the correlated user interface operations to the test service. At the test service, analytics are performed on the correlated user interface operations to develop data from the user interface operations, from which data the test routine is constructed.

Abstract: A software verification system including a cloud service, a plurality of test nodes, a client manager, and an expediter is disclosed. The cloud service is a commercially available shared resource service having two or more physical machines, each of which is configured to host a plurality of virtual machines. Each test node is one virtual machine running a platform. The client manager manages a verification session between one of the clients and the verification system. The expediter recognizes the available platforms for use by the test nodes in a cloud service pool of servers and a local pool of servers. When a clients requests the verification session to use a specific platform, the expediter obtains a network address of an available test node running the specified platform and connects the requesting client to the available test node running the requested platform.