Abstract: Systems and methods for mapping interactive UI (user interface) elements to an RPA (robotic process automation) object repository are provided. User input selecting a window of an application displayed on a display device is received. In response to receiving the user input selecting the window of the application, interactive UI elements in the window of the application are automatically identified. User input selecting one or more of the identified interactive UI elements in the window of the application is received. The one or more selected interactive UI elements are stored in an RPA object repository of an RPA system.

Abstract: In some embodiments, a robotic process automation (RPA) agent executing within a browser window/tab interacts with an RPA driver executing outside of the browser. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent exposes a robot design interface, while the RPA driver detects interactions of a user with a target user interface (e.g., an instance of a spreadsheet application, an email program, etc.) and transmits data characterizing the interactions to the RPA agent for constructing a robot specification.

Abstract: In some embodiments, a robotic process automation (RPA) agent executing within a browser window/tab interacts with an RPA driver executing outside of the browser. A bridge module establishes a communication channel between the RPA agent and the RPA driver. In one exemplary use case, the RPA agent receives a robot specification from a remote server, the specification indicating at least one RPA activity, and communicates details of the respective activity to the RPA driver via the communication channel. The RPA driver identifies a runtime target for the RPA activity within the target web page and executes the respective activity.

Abstract: Graphical element search technique selection, fuzzy logic selection for anchors and targets, and hierarchical graphical element identification for robotic process automation (RPA) are disclosed. The fuzzy logic selection of anchors and targets may be part of a larger, tiered, or hierarchical process for identifying graphical elements in the UI. When a selector for a UI element is not found with at least a confidence threshold, similar elements potentially corresponding to the selector for a UI element target may be searched based on fuzzy matching of the target and corresponding anchor(s). Geometric matching may also be employed between the target UI element and its respective anchor(s). The combination of fuzzy matching and geometric matching may allow for more flexible and accurate identification of the exact selector with which an RPA robot is attempting to interact.

Abstract: A system and a method for verification of execution of an activity are provided. The method comprises receiving a user input indicative of enablement of the verification, and displaying, in response to the reception of the user input, a target element comprising a menu for selecting an edit action. The method further comprises receiving, in response to the selection of the edit action, a verification element, and determining a status of the activity, wherein the status of the activity comprises either of successful execution of the activity or non-successful execution of the activity. Further, the method comprises generating a verification response based on the status of the activity and the verification element.

Abstract: Systems and methods for mapping interactive UI (user interface) elements to an RPA (robotic process automation) object repository are provided. User input selecting a window of an application displayed on a display device is received. In response to receiving the user input selecting the window of the application, interactive UI elements in the window of the application are automatically identified. User input selecting one or more of the identified interactive UI elements in the window of the application is received. The one or more selected interactive UI elements are stored in an RPA object repository of an RPA system.

Abstract: Semantic matching between a source screen or source data and a target screen using semantic artificial intelligence (AI) for robotic process automation (RPA) workflows is disclosed. The source data or source screen and the target screen are selected on a matching interface, semantic matching is performed between the source data/screen and the target screen using an artificial intelligence/machine learning (AI/ML) model, and matching graphical elements and unmatched graphical elements are highlighted, allowing the developer to see which graphical elements match and which do not. The matching interface may also provide a confidence score of the individual matches, provide an overall mapping score, and allow the developer to hide/unhide the matched/unmatched graphical elements. Activities of an RPA workflow may be automatically created based on the semantic mapping that can be executed to perform the automation.

Abstract: A system and a method for verification of execution of an activity are provided. The method comprises receiving a user input indicative of enablement of the verification, and displaying, in response to the reception of the user input, a target element comprising a menu for selecting an edit action. The method further comprises receiving, in response to the selection of the edit action, a verification element, and determining a status of the activity, wherein the status of the activity comprises either of successful execution of the activity or non-successful execution of the activity. Further, the method comprises generating a verification response based on the status of the activity and the verification element.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source-code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source-code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source-code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: Graphical element detection using a combined series and delayed parallel execution unified target technique that potentially uses a plurality of graphical element detection techniques, performs default user interface (UI) element detection technique configuration at the application and/or UI type level, or both, is disclosed. The unified target merges multiple techniques of identifying and automating UI elements into a single cohesive approach. A unified target descriptor chains together multiple types of UI descriptors in series, uses them in parallel, or uses at least one technique first for a period of time and then runs at least one other technique in parallel or alternatively if the first technique does not find a match within the time period.

Abstract: Graphical element search technique selection, fuzzy logic selection for anchors and targets, and hierarchical graphical element identification for robotic process automation (RPA) are disclosed. The fuzzy logic selection of anchors and targets may be part of a larger, tiered, or hierarchical process for identifying graphical elements in the UI. When a selector for a UI element is not found with at least a confidence threshold, similar elements potentially corresponding to the selector for a UI element target may be searched based on fuzzy matching of the target and corresponding anchor(s). Geometric matching may also be employed between the target UI element and its respective anchor(s). The combination of fuzzy matching and geometric matching may allow for more flexible and accurate identification of the exact selector with which an RPA robot is attempting to interact.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source-code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.

Abstract: A software robot is designed to carry out an activity (e.g., a mouse click, a text input, etc.) on a target element (e.g., a button, an input field, etc.) of a user interface. The robot's code specification is configured to include an on-screen image of the target element and a text displayed by the target element. The robot is configured to automatically identify the target element at runtime according to an element ID specified in the source code of the user interface, and when such identification fails, to identify the target element according to the text and image stored in the robot's code.