Abstract: A robotic process automation system processes task processing instructions which are operable to interact at a user level with user level application programs to process work items. A first server processor responds to a request to perform an automation task to automatically process without human user input, a work item by transmitting requests to an autologin program executing on a second server processor device to initiate a plurality of remote desktop protocol oriented processes within an operating system executing on the second server, where the remote desktop protocol oriented processes provide a graphical user interface to an authorized user. Credentials for the authorized users of the remote desktop protocol oriented processes are provided by one or more of the plurality of sets of task processing instructions, which upon logging into the corresponding remote desktop protocol oriented process, process the work items as encoded in the task processing instructions.

Abstract: Methods and systems that detect and define virtual objects in remote screens which do not expose objects. This permits simple and reliable automation of existing applications. In certain aspects a method for detecting objects from an application program that are displayed on a computer screen is disclosed. An image displayed on the computer screen is captured. The image is analyzed to identify blobs in the image. The identified blobs are filtered to identify a set of actionable objects within the image. Optical character recognition is performed on the image to detect text fields in the image. Each actionable object is linked to a text field positioned closest to a left or top side of the actionable object. The system automatically detects the virtual objects and links each actionable object such as textboxes, buttons, checkboxes, etc. to the nearest label object.