Abstract: Embodiments disclosed herein concern improved techniques for recognition of tabularized data within an image and construction of a computer-readable electronic table representing the recognized tabularized data. The image can be a screen image produced by a software application program operating on a computing device. The tabular data, such as pertaining to a table, can be recognized by recognizing rows, columns, text, controls and lines.

Abstract: Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process.

Abstract: A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.

Abstract: A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.