Abstract: Example implementations relate to comparable UI object identifications. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also automatically identify, based on the correlated data points, a set of comparable UI objects.

Abstract: Example implementations relate to determining potential test actions. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also include a test verification engine to determine, based on the correlation of the data points, a potential test action to perform during a future test execution of the application under test.

Abstract: Example implementations relate to test execution comparisons. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also automatically compare the test executions, based on the correlated data points, to identify commonalities.

Abstract: In one example of the disclosure, a user-defined success criterion for an application change is received. The criterion is provided to a computing system associated with a developer-user of the application. Evaluation code, for evaluating implementation of the change according to the criterion, is received from the computing system. The evaluation code is caused to execute responsive to receipt of a notice of production deployment of the change. A success rating for the change is determined based upon application performance data attained via execution of the evaluation code.

Abstract: Example implementations relate to determining a functional state of a system under test. For example, a system to determine a functional state of a system under test may include a system controller to execute a functional test of the system under test by invoking a subset of a plurality of functional agents to interact with the system under test. Further, the system may include an agent repository to interact with the system controller and store the plurality of functional agents. Also, the system may include a state module to determine a functional state for the system under test by querying each of the subset of functional agents and comparing aggregated results from the subset of functional agents against defined for the system under test.

Abstract: Example implementations relate to comparable UI object identifications. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also automatically identify, based on the correlated data points, a set of comparable UI objects.

Abstract: An application process can be executed based on an initialization instruction, where the application process includes instructions associated with a hook framework. A virtual machine configured to load the hook framework on the virtual machine based on instructions included in the application process can be initiated and the instructions associated with the hook framework can be executed upon initiation of the virtual machine to insert a hook on the virtual machine. A nascent process configured to initiate an additional virtual machine can be initiated based on a request to load an application, where the additional virtual machine is hooked via the hook inserted on the virtual machine.

Abstract: Example implementations relate to separating verifications from test executions. Some implementations may include a data capture engine that captures data points during test executions of the application under test. The data points may include, for example, application data, test data, and environment data. Additionally, some implementations may include a data correlation engine that correlates each of the data points with a particular test execution state of the application under test based on a sequence of events that occurred during the particular test execution state. Furthermore, some implementations may also include a test verification engine that, based on the correlation of the data points, verifies an actual behavior of the application under test separately from the particular test execution state.

Abstract: Example implementations relate to determining idle testing periods. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also include an idle testing period determination engine to determine, based on the correlation of the data points, idle testing periods of the test executions. The idle testing periods may be periods of time where both the test executions and the application under test are idle.

Abstract: Example implementations relate to automatically identifying regressions. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also include a regression identification engine to automatically identify, based on the correlated data points, a regression between a first version of the application under test and a second version of the application under test.

Abstract: Error data may be collected. The error data may represent a first plurality of errors of a first type and a second plurality of errors of a second type to occur in a plurality of instances of an application transaction. Visualization data may be generated. The visualization data may represent an error flow diagram to display on an output device. The error flow diagram may comprise a first block having a first visual property based on a first number of the first plurality of errors, a second block having a second visual property based on a second number of the second plurality of errors, and a first linkage between the first block and the second block.

Abstract: Example implementations relate to composing future tests. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also include a test composition engine to compose, based on an interaction with a visualization of results of a verification query of the correlated data points, a future test of the application under test.

Abstract: Example embodiments relate to determining code coverage based on production sampling. In example embodiments, a production execution data set that includes metrics for code units of a software application is obtained, where the metrics include input and output values for each of the code units and an average execution count for each of the code units. Further, application code execution is tracked during a testing procedure of the software application to determine executed lines of code. At this stage, production code coverage of the software application is determined based on the production execution data set and the executed lines of code.

Abstract: Examples disclosed herein relate to indicating a trait of a continuous delivery pipeline. Examples include accessing, for each of a plurality of continuous delivery (CD) pipelines, respective pipeline characteristics previously collected by a collection engine of a CD server for at least one of the CD pipelines, and indicating a trait of the pipeline characteristics of at least one of the CD pipelines.

Abstract: An example method for handling dependencies between feature flags can include defining, by a processing resource executing instructions, dependencies between a plurality of feature flags in a process executable by the processing resource. The method can include enforcing, by the processing resource executing instructions, the dependencies during activation of a first feature by a determination of validity of utilization of a feature flag as a switch for a second feature.

Abstract: Example implementations relate to creating a simulation environment. For example, a system for simulation environment creation may include a system controller. The system may also include a functional agent coupled to and executed by the system controller to create a simulation environment based on input comprising real user data and sensor data continuously received from a plurality of sources, perform tests within the simulation environment, continuously react to events occurring during the tests within the simulation environment, and continuously verify results of the tests.

Abstract: Example implementations relate to determining a functional state of a system under test. For example, a system to determine a functional state of a system under test may include a system controller to execute a functional test of the system under test by invoking a subset of a plurality of functional agents to interact with the system under test. Further, the system may include an agent repository to interact with the system controller and store the plurality of functional agents. Also, the system may include a state module to determine a functional state for the system under test by querying each of the subset of functional agents and comparing aggregated results from the subset of functional agents against defined for the system under test.

Abstract: Examples disclosed herein relate to acquiring identification of an application lifecycle management (ALM) entity associated with similar code. Examples include identifying a target code segment, and acquiring, from an ALM system, identification of an ALM entity associated with other code similar to the target code segment and identified by a code similarity system.

Abstract: Simulating sensors can include hooking an application associated with sensory data and associating the sensory data with an automation instruction. Simulating sensors can include providing the sensory data to a support device having an ability to modify the application and automatically causing the support device to simulate a sensory input using the sensory data by executing the automation instruction.

Abstract: Example implementations relate to test execution comparisons. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also automatically compare the test executions, based on the correlated data points, to identify commonalities.