Abstract: System, method, and non-transitory computer-readable medium for generating a test scenario template from runs of test scenarios run on software systems belonging to different organizations, including: clustering the runs to clusters comprising similar runs of test scenarios; selecting from the clusters a certain cluster comprising a first run of a first test scenario and a second run of a second test scenario belonging to different organizations. At least 95% of values used in the first run of the first test scenario are provided automatically by a test script. And generating a test scenario template representing the certain cluster, wherein the test scenario template identifies a transaction used in the first and second runs, and possible values for running the transaction.

Abstract: Methods and computer systems for improving the user friendliness of a non-executable human-readable test scenario by removing unessential test steps, including: receiving a non-executable human-readable test scenario including unessential test steps; identifying the unessential test steps; and generating a user friendlier non-executable human-readable test scenario by removing the identified unessential test steps.

Abstract: Described herein are systems, methods, and computer programs for generating candidate sequences of steps utilizing links between steps that are nonconsecutively performed. In one embodiment, a link generator module generates links between pairs of steps that are among steps belonging to streams of steps performed during interactions with instances of software systems; at least some of the links are from a first step to a second step, and the first and second steps are not consecutively performed steps in the same stream. A candidate generation module utilizes the links to generate candidate sequences from steps belonging to the streams; the candidate generation module provides the candidate sequences to a system that identifies which of the candidate sequences correspond to executions of a Business Process (BP).

Abstract: Described herein are systems, methods, and computer programs for extracting a seed comprising steps common in executions of a Business Process (BP) and utilizing the seed to identify other executions of the BP. In one embodiment, the seed is extracted from sequences of steps from streams of steps performed during interactions with instances of a software system. The sequences comprise first and second sequences corresponding to executions of the BP, which are associated with first and second organizations, respectively. The seed comprises two or more consecutively performed steps that appear in each of the sequences. Occurrences of the seed are identified in one or more streams of steps performed during interactions with an instance of the software system belonging to a third organization. The occurrences are extended in order to generate candidate sequences that are evaluated to determine whether they correspond to executions of the BP.

Abstract: Described herein are systems, methods, and computer programs that may be utilized to identify a sequence corresponding to an execution of a Business Processes (BP) using a machine learning-based model of the BP generated based on sequences corresponding to previous executions of the BP by a plurality of organizations. In one embodiment, a sequence parser module receives one or more streams of steps performed during interactions with an instance of a software system, which belongs to a certain organization, and selects, from among the one or more streams, candidate sequences of steps. A feature generator module generate, for each sequence from among the candidate sequences, a plurality of feature values. And a predictor module utilizes the model to calculate, based on an input comprising the plurality of feature values generated for the sequence, a value indicative of whether the sequence corresponds to an execution of the BP.

Abstract: Described herein are systems, methods, and computer programs that may be utilized to identify nonconsecutive executions of a Business Process (BP) utilizing a crowd-based model of the BP. In one embodiment, a BP model trainer module generates the crowd-based model of the BP based on sequences of steps corresponding to nonconsecutive executions of the BP, which are associated with at least first and second organizations. In one embodiment, the crowd-based model is utilized to identify nonconsecutive executions of the BP. A sequence parser module receives one or more streams of steps performed during interactions with an instance of the software system, which belongs to another organization, and to select, from among the one or more streams, candidate sequences of steps. Additionally, a BP-identifier module utilizes the crowd-based model to identify, from among the candidate sequences, a sequence of steps that corresponds to a nonconsecutive execution of the BP.

Abstract: Described herein are systems, methods, and computer programs that may be utilized perform ensemble-based identification of sequences of steps corresponding to executions of a Business Processes (BP). In one embodiment, a sequence parser module receives one or more streams of steps performed during interactions with an instance of a software system, which belongs to a certain organization, and selects, from among the one or more streams, candidate sequences of steps. A BP-scorer module utilizes models of the BP to calculate, for each sequence from among the candidate sequences, values indicative of whether the sequences correspond to executions of the BP. The models are generated based on sequences corresponding to executions of the BP that are associated with a plurality of organizations. An ensemble aggregator module utilizes the values calculated for each of the candidate sequences to identify among them one or more sequences that correspond to executions of the BP.

Abstract: Described herein are systems, methods, and computer programs that may be utilized to identify executions of a Business Process (BP) utilizing a crowd-based model of the BP. In one embodiment, a BP model trainer module generates the crowd-based model of the BP based on sequences of steps selected from among streams of steps performed during interactions with instances of a software system. Optionally, the sequences correspond to executions of the BP that are associated with a plurality of organizations. A sequence parser module is configured to receive one or more streams of steps performed during interactions with an instance of the software system, which belongs to another organization, and to select, from among the one or more streams, candidate sequences of steps. A BP-identifier module utilizes the crowd-based model to identify, from among the candidate sequences, one or more sequences of steps that correspond to executions of the BP.

Abstract: Described herein are systems, methods, and computer programs that may be utilized to identify a sequence corresponding to an execution of a Business Processes (BP) using an automaton. In one embodiment, a monitoring agent generates a stream of steps that were performed during interactions with an instance of a software system belonging to a certain organization, and a simulation module simulates a running of an automaton on an input comprising a sequence of steps. Optionally, parameters that define a behavior of the automaton are generated based on sequences of steps corresponding to executions of the BP associated with other organizations. Responsive to running the automaton on the stream, the automaton arrives at an accepting state at least once following a certain subsequence of steps that corresponds to a nonconsecutive execution of the BP, and the simulation module generates an indication indicative of a detection of an execution of the BP.

Abstract: Systems, methods, and computer programs for generating a model for linking between steps performed when executing a Business Process (BP). In one embodiment, a link example collector receives sequences of steps, each corresponding to an execution of the BP, and identifies pairs of nonconsecutively performed steps in the sequences. A sample generator module generates samples, each corresponding to a pair, which comprises one or more feature values describing properties of a link from a first step to a second step performed after the first step. A linkage model generator module generates the model based on training samples comprising: (i) positive samples generated by the sample generator module based on pairs, identified by the link example collector module, of first and second steps which were nonconsecutively performed, and (ii) negative samples generated by the sample generator module based on pairs of steps that are not nonconsecutively performed steps from the sequences.

Abstract: Described herein are systems, methods, and computer programs that may be utilized to identify executions of Business Processes (BPs) utilizing crowd-based patterns of the BPs. In one embodiment, each pattern of a BP is generated based on sequences corresponding to executions of the BP, which are associated with first and second organizations, respectively. A sequence parser module is configured to receive one or more streams of steps performed during interactions with an instance of a software system, which belongs to another organization, and to select, from among the one or more streams, candidate sequences of steps. A distance calculator module calculates distances between the candidate sequences and the patterns based on alignments of the candidate sequences and the patterns. An assignment module utilizes the distances to assign at least some of the candidate sequences with identifiers of the BPs.

Abstract: Described herein are systems, methods, and computer programs for generating a model for linking between steps performed when executing Business Processes (BPs). In one embodiment, a link example collector receives sequences of steps, each corresponding to an execution of a BP from among the BPs, and identifies pairs of nonconsecutively performed steps in the sequences. A linkage model generator module generates the model based on training samples comprising: (i) positive samples generated by the based on pairs, identified by the link example collector module, of first and second steps which were nonconsecutively performed, and (ii) negative samples generated by the sample generator module based on pairs of steps that are not nonconsecutively performed steps from the sequences. The positive samples comprise at least first and second samples generated based on first and second pairs in sequences corresponding to executions of respective first and second different BPs.

Abstract: System, method, and non-transitory computer-readable medium for generating a test scenario template from runs of test scenarios run on software systems belonging to different organizations, including: clustering the runs to clusters comprising similar runs of test scenarios; selecting from the clusters a certain cluster comprising a first run of a first test scenario and a second run of a second test scenario belonging to different organizations. At least 95% of values used in the first run of the first test scenario are provided automatically by a test script. And generating a test scenario template representing the certain cluster, wherein the test scenario template identifies a transaction used in the first and second runs, and possible values for running the transaction.

Abstract: Identifying missing test scenarios based on authorization policies, including: analyzing the authorization policies applied to non-super-users of a specific data-oriented large-scale software system in order to identify organization-specific in-use scenarios, whereby the non-super-users are usually unauthorized to irrelevant operations; comparing the identified organization-specific in-use scenarios with available test scenarios repairable by a non-programmer user, each available test scenario repairable by a non-programmer user tests at least two transactions; identifying organization-specific in-use scenarios that do not have corresponding test scenarios repairable by a non-programmer user, referred to as first set of missing test scenarios; and providing at least one of the missing test scenarios to a non-programmer user, having proper authorizations, for generating a test scenario repairable by a non-programmer user that covers the at least one missing test scenario.

Abstract: Automatic authorization-based routing of failed test scenarios, including: receiving descriptions of test scenarios that failed while executed by an automatic testing component on a data-oriented large-scale software system, wherein at least half of the test scenarios are expressed by user interaction activities and not code-related language; identifying the test scenarios that failed with abnormal termination as test scenarios to be forwarded to a first programmer authorized to change the data-oriented large-scale software system, wherein abnormal termination does not include normal termination with a value different from an expected value; and identifying the test scenarios that failed without abnormal termination as test scenarios to be forwarded to a second programmer unauthorized to change the data-oriented large-scale software system.

Abstract: Automatic processing of failed test scenarios, including: receiving descriptions of test scenarios that failed while executed by an automatic testing component, wherein at least half of the test scenarios are expressed by user interaction activities and not code-related language; identifying the test scenarios that failed with abnormal termination as test scenarios to be forwarded to a programmer for code change, wherein abnormal termination does not include normal termination with a value different from an expected value; and identifying the test scenarios that failed without abnormal termination as test scenarios to be forwarded to a non-programmer user for re-recording or manual evaluation.

Abstract: Processing failures of executing test scenarios over a data-oriented large-scale software system, including: identifying failures of the test scenarios due to logical errors in the failed test scenarios; forwarding the identified logical related failed test scenarios to a non-programmer user for re-recording or manual evaluation; identifying failures of the test scenarios due to code errors; and forwarding the identified code related failed test scenarios to a programmer for code change.

Abstract: Selecting a transaction for a certain user based on similarity of a profile of the certain user to profiles of users belonging to different organizations involves executing the following: receiving activity data obtained by monitoring activity of the users on software systems; identifying transactions executed by the users on the software systems; generating profiles of the users, based on the transactions, indicating transactions executed by the users; receiving a profile of the certain user, which indicates transactions executed by the certain user; and selecting, based on similarity of at least some of the profiles of the users to the profile of the certain user, a certain transaction that was executed by a user with a similar profile to the profile of the certain user.

Abstract: Utilizing data collected from users belonging to different organizations to select a test scenario template relevant for testing configuration elements involves the following: identifying runs of test scenarios run by users belonging to different organizations; clustering the runs of test scenarios to clusters that include similar runs; generating connections between the configuration elements and the clusters; weighting the connections based on the number of users that acted according to the connections; receiving certain configuration elements; selecting a certain cluster based on a subset of the certain configuration elements and the connections, where each element in the subset is connected to the certain cluster by a connection, and the sum of weights of connections between elements in the subset and the certain cluster, reaches a threshold; and selecting a test scenario template representing the certain cluster.

Abstract: System, method, and non-transitory medium for generating a test scenario template involving the steps of monitoring users belonging to different organizations to identify runs of test scenarios run on software systems belonging to the different organizations; clustering the runs to clusters of similar runs; selecting a certain cluster from the clusters; and generating, based on runs belonging to the certain cluster, a test scenario template. The template may represent the certain cluster and identify a transaction used in runs belonging to the certain cluster, and possible values for running the transaction.