Abstract: Quality indicators regarding a software project under development that has a plurality of functional areas are collected. Each quality indicator is normalized to a coverage of the functional area of the software project to which the quality indicator corresponds. The normalized quality indicators are correlated to previously identified anomalies of the software project, yielding an anomaly indicative value for each normalized quality indicator corresponding to a probability that the normalized quality indicator is revelatory of unidentified anomalies of the software project. A normal behavior for each normalized quality indicator is estimated. For each functional area of the software project, an anomaly score indicative of a likelihood of an unidentified anomaly within the functional area is calculated.

Abstract: Examples relate to load testing. The examples disclosed herein enable obtaining lines of code that are recorded as an application is executed in a client computing device, the lines of code being recorded in chronological order of the execution; determining whether a dependency on at least one variable exists in individual lines of the lines of code; in response to determining that the dependency exists, storing the dependency in a data storage; identifying, from the lines of code, a line of code including a network call statement that calls a called variable; and eliminating a first subset of the lines of code based on the called variable and dependencies stored in the data storage, wherein a second subset of the lines of code that remain after the elimination comprises user-entered parameter data.

Abstract: A technique includes accessing, by at least one hardware processor, a recorded request and a recorded response associated with an integration test involving a first computer system and a second computer system. The recorded request was previously issued by the first computer system to the second computer system to cause the second computer system to provide the recorded response. The technique includes, in a virtualized integration test involving the second computer system and initiated using the recorded request, comparing, by the hardware processor(s), the recorded response to a request produced by the second computer system in the virtualized integration test. The technique includes identifying, by the hardware processor(s), an action taken by the second computer system as being likely to be associated with a regression based on the comparison.

Abstract: Quality indicators regarding a software project under development that has a plurality of functional areas are collected. Each quality indicator is normalized to a coverage of the functional area of the software project to which the quality indicator corresponds. The normalized quality indicators are correlated to previously identified anomalies of the software project, yielding an anomaly indicative value for each normalized quality indicator corresponding to a probability that the normalized quality indicator is revelatory of unidentified anomalies of the software project. A normal behavior for each normalized quality indicator is estimated. For each functional area of the software project, an anomaly score indicative of a likelihood of an unidentified anomaly within the functional area is calculated.

Abstract: Example implementations relate to comparable UI object identifications. Some implementations may include a data capture engine to capture data points during test executions of the application under test. The data points may include, for example, test action data and application action data. Additionally, some implementations may include a data correlation engine to correlate each of the data points with a particular test execution of the test executions, and each of the data points may be correlated based on a sequence of events that occurred during the particular test execution. Furthermore, some implementations may also automatically identify, based on the correlated data points, a set of comparable UI objects.

Abstract: Automated test failures that result from automated testing of program code under development are windows to include just the automated test failures occurring for a first time and that are due to automated test code defects or program code defects. The automated test failures that remain after winnowing are clustered into automated test failure clusters that each individually corresponding to a different automated test code defect or a different program code defect. The automated test failure clusters are window to include just the automated test failure clusters that each individually correspond to a different program code defect. The automated test failure clusters that remain after winnowing are output.

Abstract: Automated test failures that result from automated testing of program code under development are windows to include just the automated test failures occurring for a first time and that are due to automated test code defects or program code defects. The automated test failures that remain after winnowing are clustered into automated test failure clusters that each individually corresponding to a different automated test code defect or a different program code defect. The automated test failure clusters are window to include just the automated test failure clusters that each individually correspond to a different program code defect. The automated test failure clusters that remain after winnowing are output.

Abstract: A technique includes accessing, by at least one hardware processor, a recorded request and a recorded response associated with an integration test involving a first computer system and a second computer system. The recorded request was previously issued by the first computer system to the second computer system to cause the second computer system to provide the recorded response. The technique includes, in a virtualized integration test involving the second computer system and initiated using the recorded request, comparing, by the hardware processor(s), the recorded response to a request produced by the second computer system in the virtualized integration test. The technique includes identifying, by the hardware processor(s), an action taken by the second computer system as being likely to be associated with a regression based on the comparison.

Abstract: Example implementations relate to determining visual testing coverages. Some implementations may include a principal application screen identification engine to identify, based on an image processing analysis of a set of screenshots of an application under test, principal application screens that represent a set of test executions of the application under test. Some implementations may also include a user interface element identification engine to identify user interface elements in the principal application screens. Some implementations may also include a visual testing coverage engine to automatically identify, based on an analysis of gaze data and the user interface elements, a visual testing coverage of the application under test.

Abstract: Example implementations relate to separating verifications from test executions. Some implementations may include a data capture engine that captures data points during test executions of the application under test. The data points may include, for example, application data, test data, and environment data. Additionally, some implementations may include a data correlation engine that correlates each of the data points with a particular test execution state of the application under test based on a sequence of events that occurred during the particular test execution state. Furthermore, some implementations may also include a test verification engine that, based on the correlation of the data points, verifies an actual behavior of the application under test separately from the particular test execution state.

Abstract: First object model data from a first application state generated in a first browser mode may be collected. The first object model data may comprise a code-based representation of a first layout element and a second layout element of the first application state. According to a layout detection rule, a first attribute of the first layout element may be compared with a second attribute of the first layout element or of a second layout element. A layout error corresponding to the first layout element may be identified based on the comparison of the first attribute with the second attribute.

Abstract: A first screenshot taken from a screen and having a first layout element may be determined to meet a first layout similarity threshold with a second screenshot having a second layout element. A first group comprising the first and second screenshot may be created in response to the first screenshot meeting the first layout similarity threshold with the second screenshot and it may be determined that a first feature of the first screenshot from the first group matches a second feature of a third screenshot from a second group. The second group may then be associated with a screen in response to the match determination.

Abstract: Examples disclosed herein relate to determining event and input coverage metrics for a graphical user interface (GUI) control instance. Examples include accessing event and input occurrence data identifying interactions performed on a GUI of an application, determining an event coverage metric for the given GUI control instance based on the event occurrence data, and determining an input coverage metric for the given GUI control instance based on the input occurrence data.

Abstract: An example device in accordance with an aspect of the present disclosure includes an identification engine, a clustering engine, and a weighting engine. The identification engine is to identify User Interface (UI) elements of an application under test based on an image processing analysis of screens of the application under test. The clustering engine is to cluster the UI elements into at least one cluster based on at least one feature of the UI elements. The weighting engine is to assign score weights to the UI elements of a given screen according to which of the at least one cluster that a given one of the UI elements is clustered in.

Abstract: An example device in accordance with an aspect of the present disclosure includes a screen identification engine to identify a screen(s) of a test run, an element identification engine to selectively identify an element(s) among available elements of the screen(s), and a signature building engine to build a signature corresponding to the test run. The signature incorporates the screen(s) and element(s) to capture an application flow of the test run, while excluding available elements that do not correspond to the application flow.

Abstract: Method and system of testing software using real time replication. At least some illustrative examples include interacting by a human tester with a first software program executed on a first computer system. The interacting causes an operation to be performed on the first software program and the operation is duplicated on a second software program executed on a second computer system. The duplication on the second computer system is done programmatically in real time with the interacting and the duplicating on the first computing system. A result of the operation on the first computer system against a result of the operation on the second computer system is programmatically analyzed on the second computing system. The human tester is notified when the result of the operation on the second computer system is unexpected.

Abstract: First object model data from a first application state generated in a first browser mode may be collected. The first object model data may comprise a code-based representation of a first layout element and a second layout element of the first application state. According to a layout detection rule, a first attribute of the first layout element may be compared with a second attribute of the first layout element or of a second layout element. A layout error corresponding to the first layout element may be identified based on the comparison of the first attribute with the second attribute.

Abstract: An example device in accordance with an aspect of the present disclosure includes an identification engine, a clustering engine, and a weighting engine. The identification engine is to identify User Interface (UI) elements of an application under test based on an image processing analysis of screens of the application under test. The clustering engine is to cluster the UI elements into at least one cluster based on at least one feature of the UI elements. The weighting engine is to assign score weights to the UI elements of a given screen according to which of the at least one cluster that a given one of the UI elements is clustered in.

Abstract: Methods, systems, and computer readable medium are disclosed to test a first business system and a second business system. A test of the first business system is performed, wherein the first business system is integrated with the second business system. One or more calls are recorded from the first business system to the second business system during the test of the first business system. The one or more calls from the first business system are identified for a test of the second business system.

Abstract: A first screenshot taken from a screen and having a first layout element may be determined to meet a first layout similarity threshold with a second screenshot having a second layout element. A first group comprising the first and second screenshot may be created in response to the first screenshot meeting the first layout similarity threshold with the second screenshot and it may be determined that a first feature of the first screenshot from the first group matches a second feature of a third screenshot from a second group. The second group may then be associated with a screen in response to the match determination.