Abstract: Automatic anchor determination for target graphical element identification in user interface (UI) automation is disclosed. A context-based mechanism assists in discriminating between duplicate target UI element candidates. More specifically, additional anchors may be determined and automatically added for a target UI element that provide context and are visible in an area surrounding the target. During design time, a target UI element may be indicated by a user of a designer application and a corresponding anchor may be determined. When a pair of UI elements is found having the same or similar characteristics and/or relationships to the target-anchor pair, an additional anchor is automatically identified without requesting user input. The additional anchor may be selected from the UI elements within a radius of the target UI element.

Abstract: A user interface (UI) mapper for robotic process automation (RPA) is disclosed. The UI mapper may initially capture UI elements to fetch UI elements faster for later use and allow an RPA developer to “map” the UI elements for automating an application. This may enable subsequent developers who potentially do not have programming knowledge to build RPA workflows using these predefined “target” UI elements.

Abstract: Automatic anchor determination for target graphical element identification in user interface (UI) automation is disclosed. A context-based mechanism assists in discriminating between duplicate target UI element candidates. More specifically, additional anchors may be determined and automatically added for a target UI element that provide context and are visible in an area surrounding the target. During design time, a target UI element may be indicated by a user of a designer application and a corresponding anchor may be determined. When a pair of UI elements is found having the same or similar characteristics and/or relationships to the target-anchor pair, an additional anchor is automatically identified without requesting user input. The additional anchor may be selected from the UI elements within a radius of the target UI element.

Abstract: Multiple anchors may be utilized for robotic process automation (RPA) of a user interface (UI). The multiple anchors may be utilized to determine relationships between elements in the captured image of the UI for RPA. The results of the anchoring may be utilized for training or retraining of a machine learning (ML) component.

Abstract: Multiple anchors may be utilized for robotic process automation (RPA) of a user interface (UI). The multiple anchors may be utilized to determine relationships between elements in the captured image of the UI for RPA. The results of the anchoring may be utilized for training or retraining of a machine learning (ML) component.

Abstract: A Computer Vision (CV) model generated by a Machine Learning (ML) system may be retrained for more accurate computer image analysis in Robotic Process Automation (RPA). A designer application may receive a selection of a misidentified or non-identified graphical component in an image form a user, determine representative data of an area of the image that includes the selection, and transmit the representative data and the image to an image database. A reviewer may execute the CV model, or cause the CV model to be executed, to confirm that the error exists, and if so, send the image and a correct label to an ML system for retraining. While the CV model is being retrained, an alternative image recognition model may be used to identify the misidentified or non-identified graphical component.

Abstract: Multiple anchors may be utilized for robotic process automation (RPA) of a user interface (UI). The multiple anchors may be utilized to determine relationships between elements in the captured image of the UI for RPA. The results of the anchoring may be utilized for training or retraining of a machine learning (ML) component.