Abstract: Methods and systems are provided that leverage existing information displayed or presented to a user by an app without requiring the integration of new libraries or requiring a user to use a specific app. Rather, methods and systems presented herein identify a task, select an app or multiple apps capable of performing the task, obtain information that may be required by the app or apps, determine a contextual understanding of what is displayed by the selected app or apps, and finally generate and automate the interaction with the app or apps such that the interaction with the app or apps is handled by an automation agent, allowing a user to stay focused on one or more tasks that the user needs to accomplish.

Abstract: Systems and methods are provided for detecting an interruption during an automated workflow. An automated workflow may comprise a series of actions to be performed by or with the assistance of a computer. A workflow manager executes a workflow by progressing through a series of workflow states according to instructions associated with the workflow. When the workflow advances to a new state, an interruption detection engine determines whether the state contains an interruption by examining one or more attributes of the workflow state and/or the user interface associated therewith. An interruption detecting engine may examine a document object model and/or utilize computer vision to determine whether an interruption has occurred. When an interruption is detected, a workflow is paused until the interruption is resolved, such as by a user providing a required input. After an interruption has been resolved, the resumes and continues until completion of the workflow.

Abstract: Methods and systems are provided that leverage existing information displayed or presented to a user by an app without requiring the integration of new libraries or requiring a user to use a specific app. Rather, methods and systems presented herein identify a task, select an app or multiple apps capable of performing the task, obtain information that may be required by the app or apps, determine a contextual understanding of what is displayed by the selected app or apps, and finally generate and automate the interaction with the app or apps such that the interaction with the app or apps is handled by an automation agent, allowing a user to stay focused on one or more tasks that the user needs to accomplish.