Abstract: A system comprising at least one computer hardware processor; and at least one computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one of a first application program and a second application program; controlling the first application program to perform a first sub-task of a task at least in part by using the object hierarchy; and controlling the second application program to perform a second sub-task of the task at least in part by using the object hierarchy, wherein the first and second application programs are configured to execute at least in part by using different GUI application libraries.

Abstract: A system comprising at least one hardware processor configured to perform: accessing a software robot computer program for controlling at least one application program to perform a task comprising a first sub-task to be performed by a first application program; generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program; and controlling the first application program to perform the first sub-task. The controlling includes identifying, using the software robot computer program, a first action to perform in furtherance of the first sub-task; automatically accessing, in the object hierarchy, a first object corresponding to a first active GUI element of the first application program, the accessing comprising refreshing the object hierarchy; and automatically using the first object to cause the first application program to at least partially perform the first action.

Abstract: A system comprising at least one computer hardware processor configured to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one application program; controlling the at least one application program to perform a task comprising a sequence of actions at least in part by using the object hierarchy to invoke actions in the sequence of actions; during performance of the sequence of actions, generating a log of already-performed actions in the sequence of actions; generating contextual information associated with the already performed actions, the contextual information containing at least some information not in the log and the log containing at least some information not in the contextual information; and providing to a user the log and the contextual information.