Abstract: A system for remotely controlling multiple application programs executing on multiple respective physical computing devices, the system comprising a first computing device, comprising a first processor, and configured to execute a first application program; a second computing device, comprising a second processor, and configured to execute a second application program; and a controller configured to perform: generating, based on first information obtained from the first computing device and second information obtained from the second computing device, a global object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program and the second application program; controlling the first application program to perform the first sub-task at least in part by using the global object hierarchy; and controlling the second application program to perform the second sub-task at least in part by using the global object hierarchy.

Abstract: A system comprising at least one hardware processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of a first application program, the generating comprising: obtaining a hierarchical representation of the active GUI elements of the application program; generating an object for each of at least some of the active GUI elements represented in the hierarchical representation; and organizing the generated objects into the object hierarchy based on relationships among active GUI elements represented in the hierarchical representation; and controlling, using software robot computer program and the generated object hierarchy, the first application program to perform the first sub-task.

Abstract: According to at least one aspect, a system for analyzing computer usage data of a user to identify an underlying task being performed by the user is provided. The system includes a hardware processor and a non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the hardware processor, cause the hardware processor to perform: receiving a plurality of events each indicative of an action performed by a user and contextual information associated with the action performed by the user, clustering the plurality of events to generate a plurality of clustered events, identifying a plurality of sub-tasks in the plurality of clustered events that each comprise a sequence of clustered events, identifying a task in the plurality of clustered events being performed by the user that comprises at least one sub-task, and generating a score for the task indicative of a difficulty of automating the task.

Abstract: A system for remotely controlling multiple application programs executing on multiple respective physical computing devices, the system comprising a first computing device, comprising a first processor, and configured to execute a first application program; a second computing device, comprising a second processor, and configured to execute a second application program; and a controller configured to perform: generating, based on first information obtained from the first computing device and second information obtained from the second computing device, a global object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program and the second application program; controlling the first application program to perform the first sub-task at least in part by using the global object hierarchy; and controlling the second application program to perform the second sub-task at least in part by using the global object hierarchy.

Abstract: A system comprising at least one hardware processor configured to perform: accessing a software robot computer program for controlling at least one application program to perform a task comprising a first sub-task to be performed by a first application program; generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program; and controlling the first application program to perform the first sub-task. The controlling includes identifying, using the software robot computer program, a first action to perform in furtherance of the first sub-task; automatically accessing, in the object hierarchy, a first object corresponding to a first active GUI element of the first application program, the accessing comprising refreshing the object hierarchy; and automatically using the first object to cause the first application program to at least partially perform the first action.

Abstract: A system comprising at least one hardware processor configured to perform: accessing a software robot computer program for controlling at least one application program to perform a task comprising a first sub-task to be performed by a first application program; generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of the first application program; and controlling the first application program to perform the first sub-task. The controlling includes identifying, using the software robot computer program, a first action to perform in furtherance of the first sub-task; automatically accessing, in the object hierarchy, a first object corresponding to a first active GUI element of the first application program, the accessing comprising refreshing the object hierarchy; and automatically using the first object to cause the first application program to at least partially perform the first action.

Abstract: A system comprising at least one computer hardware processor configured to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one application program; controlling the at least one application program to perform a task comprising a sequence of actions at least in part by using the object hierarchy to invoke actions in the sequence of actions.

Abstract: A system, comprising at least one computer hardware processor; and at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of a first application program; and controlling the first application program to perform a first sub-task. The controlling includes automatically accessing, in the object hierarchy, a first object corresponding to an active GUI element of the first application program, the accessing comprising searching for the first object in the object hierarchy, the searching comprising modifying the object hierarchy; and automatically using the first object to cause the first application program to at least partially perform a first action in furtherance of the first sub-task.

Abstract: A system comprising at least one computer hardware processor configured to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one application program; controlling the at least one application program to perform a task comprising a sequence of actions at least in part by using the object hierarchy to invoke actions in the sequence of actions.

Abstract: A system, comprising at least one computer hardware processor; and at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of a first application program; and controlling the first application program to perform a first sub-task. The controlling includes automatically accessing, in the object hierarchy, a first object corresponding to an active GUI element of the first application program, the accessing comprising searching for the first object in the object hierarchy, the searching comprising modifying the object hierarchy; and automatically using the first object to cause the first application program to at least partially perform a first action in furtherance of the first sub-task.

Abstract: A system comprising at least one hardware processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of a first application program, the generating comprising: obtaining a hierarchical representation of the active GUI elements of the application program; generating an object for each of at least some of the active GUI elements represented in the hierarchical representation; and organizing the generated objects into the object hierarchy based on relationships among active GUI elements represented in the hierarchical representation; and controlling, using software robot computer program and the generated object hierarchy, the first application program to perform the first sub-task.

Abstract: According to at least one aspect, a system for collecting computer usage information is provided. The system includes a hardware processor, a display coupled to the hardware processor to display a user interface, and a computer-readable storage medium storing processor-executable instructions that cause the hardware processor to receive an indication of an action being performed by a user on the system, gather contextual information associated with the action responsive to the action performed by the user, store information indicative of the action and the contextual information in a volatile memory as an event, determine whether at least one event stored in the volatile memory includes personal information of the user, and write the at least one event stored in the volatile memory to an event log in a non-volatile memory responsive to a determination that the at least one event does not include personal information of the user.

Abstract: A system comprising at least one hardware processor; at least one non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of a first application program, the generating comprising: obtaining a hierarchical representation of the active GUI elements of the application program; generating an object for each of at least some of the active GUI elements represented in the hierarchical representation; and organizing the generated objects into the object hierarchy based on relationships among active GUI elements represented in the hierarchical representation; and controlling, using software robot computer program and the generated object hierarchy, the first application program to perform the first sub-task.

Abstract: According to at least one aspect, a system is provided. The system comprises at least one hardware processor; and at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one hardware processor, cause the at least one hardware processor to perform: monitoring execution of a first computer program configured to control a second computer program to perform a task at least in part by controlling the second computer program, through a graphical user interface (GUI) and/or an application programming interface (API) for the second computer program, to perform at least one action in furtherance of the task; detecting at least one anomaly in the execution of the first computer program; and outputting the detected at least one anomaly in the execution of the first computer program.

Abstract: According to at least one aspect, a system for collecting computer usage information is provided. The system includes a hardware processor, a display coupled to the hardware processor to display a user interface, and a computer-readable storage medium storing processor-executable instructions that cause the hardware processor to receive an indication of an action being performed by a user on the system, cause the system to wait before updating the user interface in response to the action performed by the user, gather contextual information associated with the action while the system is waiting to process the action performed by the user to update the user interface, cause the system to update the user interface in response to the action performed by the user after gathering the contextual information, and store information indicative of the action and the contextual information.

Abstract: A system comprising at least one computer hardware processor configured to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one application program; controlling the at least one application program to perform a task comprising a sequence of actions at least in part by using the object hierarchy to invoke actions in the sequence of actions.

Abstract: Techniques for executing one or more instances of a computer program using virtual machines, the computer program comprising multiple computer program portions including a first computer program portion. The techniques include determining whether an instance of any of the multiple computer program portions is to be executed; when it is determined that a first instance of the first computer program portion is to be executed, accessing first information specifying a first set of one or more virtual machine resources required for executing the first instance of the first computer program portion; determining whether any one of the plurality of virtual machines has at least the first set of virtual machine resources available; and when it is determined that a first of the plurality of virtual machines has the first set virtual machine resources available, causing the first virtual machine to execute the first instance of the first computer program portion.

Abstract: According to at least one aspect, a system for collecting computer usage information is provided. The system includes a hardware processor, a display coupled to the hardware processor to display a user interface, and a computer-readable storage medium storing processor-executable instructions that cause the hardware processor to receive an indication of an action being performed by a user on the system, cause the system to wait before updating the user interface in response to the action performed by the user, gather contextual information associated with the action while the system is waiting to process the action performed by the user to update the user interface, cause the system to update the user interface in response to the action performed by the user after gathering the contextual information, and store information indicative of the action and the contextual information.

Abstract: A system comprising at least one computer hardware processor configured to perform: generating an object hierarchy comprising a plurality of objects corresponding to active graphical user interface (GUI) elements of at least one application program; controlling the at least one application program to perform a task comprising a sequence of actions at least in part by using the object hierarchy to invoke actions in the sequence of actions; during performance of the sequence of actions, generating a log of already-performed actions in the sequence of actions; generating contextual information associated with the already performed actions, the contextual information containing at least some information not in the log and the log containing at least some information not in the contextual information; and providing to a user the log and the contextual information.

Abstract: According to at least one aspect, a system for analyzing computer usage data of a user to identify an underlying task being performed by the user is provided. The system includes a hardware processor and a non-transitory computer-readable storage medium storing processor-executable instructions that, when executed by the hardware processor, cause the hardware processor to perform: receiving a plurality of events each indicative of an action performed by a user and contextual information associated with the action performed by the user, clustering the plurality of events to generate a plurality of clustered events, identifying a plurality of sub-tasks in the plurality of clustered events that each comprise a sequence of clustered events, identifying a task in the plurality of clustered events being performed by the user that comprises at least one sub-task, and generating a score for the task indicative of a difficulty of automating the task.