Abstract: A processor-implemented method includes providing an analytic dashboard with a graphical user interface (GUI) that outputs aggregated results streaming in real-time of a load test performed on a target website. The load test consists of a plurality of virtual users simulated by one or more load servers that execute a test composition on one or more webpages of the website. A waterfall chart is produced on the analytic dashboard that includes a plurality of timeline bars. Each of the timeline bars represents an aggregated result for a resource of the one or more webpages across all of the virtual users. The timeline bars change in real-time as the load test progresses.

Abstract: A computer-implemented method includes scanning a clip of messages that includes message requests and message responses arranged in a sequence. The scanning is performed based on one or more search parameters and produces a list of one or more name/value pairs. The clip is utilized to perform a load test on a target website. Each name/value pair has a corresponding value. For each name/value pair in the list a message request in the clip is identified where the corresponding value is first used. Then, looking backwards in the sequence from the message request where the corresponding value is first used, prior message responses are located where the corresponding value is found. An extraction point is specified in the clip for the corresponding value as a latest message response in the sequence where the corresponding value was returned from the target website. The corresponding value is then stored as a property.

Abstract: A method for real-time capture and analysis of website performance metrics includes receiving, in each of a plurality of result servers, data values transmitted from a plurality of client devices, each data value representing a measurement of a performance metric associated with an experience of a user on a web page or using a web application, the measurement being made and the data value produced being transmitted in real-time while the user is on the web page or using the web application. For each data value received, a count in one of a set of N data buckets defined in a memory is incremented. All of the counts in each of the corresponding data buckets is aggregated to produce a histogram of the performance metric is rendered on an analytic dashboard.

Abstract: A method for real-time capture of analytics from real users of a native mobile application (app) includes storing a custom metric/timer definition for a native mobile application (app) in a configuration file on a server. The custom metric/timer definition includes one or more identifiers of an element or object of the native mobile app selected by touch gesture input via a user interface on a mobile device running the native mobile app in a special mode. The configuration file is downloaded from the server to real users running the native mobile app on associated mobile devices. Immediately thereafter, one or more collector servers receive data beaconed in real-time from the associated mobile devices of the real users. The beaconed data includes custom metric/timer data obtained from the custom metric/timer definition.

Abstract: A processor-implemented method includes providing an analytic dashboard with a graphical user interface (GUI) that outputs aggregated results streaming in real-time of a load test performed on a target website. Responsive to input of a user on the GUI, the input comprising selection of a source chart and a target chart, a single chart is automatically generated that represents either a combination or a statistical correlation of the source and target charts. The single chart has a left y-axis and an x-axis. The combination or the statistical correlation of the single chart changing in real-time as the load test progresses. A visual representation of the single chart is then produced on the analytic dashboard.

Abstract: A method for real-time capture of actual user experiences on a website, web application or mobile app includes receiving, in first servers, one or more beacons, each containing data items. Each beacon is generated in real-time from a user session. The data items are aggregated in one or more first sets of N data buckets associated with each of the first servers. Each of the first servers transmits the data items of each of the N data buckets to a second server over a network, which further aggregates the data items received in one or more sets of N data buckets corresponding to and identically configured as the data buckets of the first servers. The corresponding data contents of the one or more second sets of N data buckets is aggregated into one or more final sets of N data buckets used in generating a real-time analytic dashboard.

Abstract: A method includes providing a graphical user interface (GUI) that displays a view of the globe as from outer space. The view including a geographic distribution of beacons having one or more visual attributes. Each beacon corresponds to one or more real users who are accessing a website, web application or mobile app from a geographic location. The visual attributes include an indicator of a performance metric representative of an experience of the one or more real users of the website, web application, or mobile app. Each beacon is generated in real-time from aggregated data obtained in real-time from one or more corresponding real user sessions on the website, web application or mobile app. The beacons are periodically refreshed with the visual attributes changing in response to measured changes in the geographic distribution, a total number of real users at the location, or the performance indicator at the location.

Abstract: A method for real-time analysis of results from a load test performed on a target website includes periodically computing first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are computed from data received by each of the load server instances from the target website every first time interval. The first-level aggregated test results are then periodically sent from each of the load server instances to an associated one of a plurality of analytic server instances every second time interval. The first-level aggregated test results are aggregated by each of the analytic server instances to produce second-level aggregated test results, which test results may then further aggregated to produce third-level aggregated test results at a data storage instance in real-time.

Abstract: A method for cloud-based functional testing of a mobile application includes running a functional test program on a server. The functional test program provides a graphical user interface (GUI) that allows a user to select a mobile application and a mobile computing device having a touch-sensitive display screen for receiving user input. The mobile computing device is located remote to the server. The functional test program launches the mobile application on the mobile computing device via a wireless network connection. The server receives precision elements of each gesture-based input on the touch-sensitive display screen, the precision elements being captured and transmitted from the mobile computing device to the server during execution of the mobile application. The precision elements of each gesture-based input are then recorded in a test clip.

Abstract: A processor-implemented method includes providing an analytic dashboard with a graphical user interface (GUI) that outputs aggregated results streaming in real-time of a load test performed on a target website. The load test consists of a plurality of virtual users simulated by one or more load servers that execute a test composition on one or more webpages of the website. A waterfall chart is produced on the analytic dashboard that includes a plurality of timeline bars. Each of the timeline bars represents an aggregated result for a resource of the one or more webpages across all of the virtual users. The timeline bars change in real-time as the load test progresses.

Abstract: A method for real-time analysis of results from a load test performed on a target website includes calculating first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are calculated from data points received by each of the load server instances from the target website. The first-level aggregated test results include a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points. A standard deviation result (STDEV) is calculated and chart is generated on a display via a graphical user interface. The chart provides a visual representation of a performance metric for the load test based on the standard deviation result.

Abstract: A method for real-time analysis of results from a load test performed on a target website includes calculating first-level aggregated test results within each of a plurality of load server instances that generate a load on the target website. The first-level aggregated test results are calculated from data points received by each of the load server instances from the target website. The first-level aggregated test results include a sum of the data points, a count of the number of the data points, a sum of squares of the data points, and an average of the data points. A standard deviation result (STDEV) is calculated and chart is generated on a display via a graphical user interface. The chart provides a visual representation of a performance metric for the load test based on the standard deviation result.

Abstract: A processor-implemented method includes providing an analytic dashboard with a graphical user interface (GUI) that outputs aggregated results streaming in real-time of a load test performed on a target website. The load test consists of a plurality of virtual users simulated by one or more load servers that execute a test composition on one or more webpages of the website. A waterfall chart is produced on the analytic dashboard that includes a plurality of timeline bars. Each of the timeline bars represents an aggregated result for a resource of the one or more webpages across all of the virtual users. The timeline bars change in real-time as the load test progresses.

Abstract: A method for cloud-based functional testing of a mobile application includes running a functional test program on a server. The functional test program provides a graphical user interface (GUI) that allows a user to select a mobile application and a mobile computing device having a touch-sensitive display screen for receiving user input. The mobile computing device is located remote to the server. The functional test program launches the mobile application on the mobile computing device via a wireless network connection. The server receives precision elements of each gesture-based input on the touch-sensitive display screen, the precision elements being captured and transmitted from the mobile computing device to the server during execution of the mobile application. The precision elements of each gesture-based input are then recorded in a test clip.

Abstract: A method includes providing a graphical user interface (GUI) that allows a user to dynamically change virtual user (VU) distributions of a load test performed on a target website or web application during test run-time. The GUI allowing the user to make a change to a current ramp profile of one or more VU distributions responsive to application of first user input. The first input including dragging an icon to a position in a first graph, the position denoting a future number of VUs at a future time. Following application of the first input, the change to the current ramp profile is visually displayed on a second graph in real-time as the load test progresses.

Abstract: A method for real-time capture of actual user experiences on a website, web application or mobile app includes receiving, in first servers, one or more beacons, each containing data items. Each beacon is generated in real-time from a user session. The data items are aggregated in one or more first sets of N data buckets associated with each of the first servers. Each of the first servers transmits the data items of each of the N data buckets to a second server over a network, which further aggregates the data items received in one or more sets of N data buckets corresponding to and identically configured as the data buckets of the first servers. The corresponding data contents of the one or more second sets of N data buckets is aggregated into one or more final sets of N data buckets used in generating a real-time analytic dashboard.

Abstract: A method for real-time capture and analysis of website performance metrics includes receiving, in each of a plurality of result servers, data values transmitted from a plurality of client devices, each data value representing a measurement of a performance metric associated with an experience of a user on a web page or using a web application, the measurement being made and the data value produced being transmitted in real-time while the user is on the web page or using the web application. For each data value received, a count in one of a set of N data buckets defined in a memory is incremented. All of the counts in each of the corresponding data buckets is aggregated to produce a histogram of the performance metric is rendered on an analytic dashboard.

Abstract: A method includes providing a graphical user interface (GUI) that displays a view of the globe as from outer space. The view including a geographic distribution of beacons having one or more visual attributes. Each beacon corresponds to one or more real users who are accessing a website, web application or mobile app from a geographic location. The visual attributes include an indicator of a performance metric representative of an experience of the one or more real users of the website, web application, or mobile app. Each beacon is generated in real-time from aggregated data obtained in real-time from one or more corresponding real user sessions on the website, web application or mobile app. The beacons are periodically refreshed with the visual attributes changing in response to measured changes in the geographic distribution, a total number of real users at the location, or the performance indicator at the location.

Abstract: An automated method for provisioning a grid used to run a load test on a target website includes sending one or more requests in a multi-threaded manner to at least one cloud provider, the one or more requests for an allocation of N load server instances and M result server instances which comprise the grid. Requests received back from the cloud provider are also handled in a multi-threaded manner; any errors occurring during the allocation being corrected automatically. The N load server instances and the M result server instances are then verified to be operational and correctly running software deployed to provide defined test services. Errors identified during the verification are automatically corrected either by attempting to restart a failed instance or allocating a different instance.

Abstract: An automated method for provisioning a grid used to run a load test on a target website includes sending one or more requests in a multi-threaded manner to at least one cloud provider, the one or more requests for an allocation of N load server instances and M result server instances which comprise the grid. Requests received back from the cloud provider are also handled in a multi-threaded manner; any errors occurring during the allocation being corrected automatically. The N load server instances and the M result server instances are then verified to be operational and correctly running software deployed to provide defined test services. Errors identified during the verification are automatically corrected either by attempting to restart a failed instance or allocating a different instance.