Abstract: Robotic process automation (RPA) systems with improved playback capabilities. Certain embodiments can provide resilient playback of software automation processes by providing enhanced capability to locate user interface controls within a screen image of a user interface. In some embodiments, one or more of the user interface controls located within the screen image of the user interface can be used to manipulate the user interface so that other user interface controls become visible within the screen image. Other embodiments can, additionally or alternatively, provide resilient playback of software automation processes by providing the capability to record, compute and store parameters for user interface controls detected from a screen image of a user interface. These parameters can be used to assist in locating correct corresponding user interface controls within a screen image presented at playback of a software automation process.

Abstract: Computerized robotic process automation (RPA) methods and systems that increase the flexibility and lower the cost with which RPA systems may be deployed are disclosed herein. In one embodiment, an RPA system and method avoids the need for preinstalled RPA software on a device employed by a user to create and/or execute software robots to perform RPA. In another embodiment, an RPA system and method provides a capability to execute software robots that may have been encoded in one or more programming languages to execute on an operating system different than that employed by a server of the RPA system.

Abstract: Controls within images of a user interface of a computer application are detected by way of region-based R-FCN and Faster R-CNN engines. Datasets comprising images containing application control, wherein the application controls include images of application where width is greater than height, width is equal to height and height is greater than width are retrieved. Each of the datasets is processed with the R-FCN and Faster R-CNN engines to generate a software configured to recognize, from an input image, application controls wherein the application controls are characterized by dimensions where width is greater than height, where width is substantially equal to height, and where height is greater than width.

Abstract: A robotic process automation system employs centralized compilation to generate a platform independent executable version of a bot, which is encoded to perform user level operations. The system employs an extensible set of commands which can be user generated. The bots execute on devices that are separate and independent from a server processor that controls the system. The devices execute bots in an execution environment that is provided by the server processor. Change in a command in a bot requires recompilation of the bot which is then delivered upon request to a device. The execution environment does not require recompilation upon a change in a command.

Abstract: Improved techniques for combining a plurality of distinct recordings pertaining to user interactions with one or more software applications each of which concerns performing a task. Then, the combined recording can be used to form a software automation process for performing the task in an automated fashion. Advantageously, the improved techniques enable a robotic process automation (RPA) system to capture multiple recordings, combine (e.g., merge) the recordings, and form a software automation process to perform repetitive tasks in an automated manner, all with little or no assistance from a user.

Abstract: A computerized task automation system operates to permit automation of processes comprising one or more computer implemented tasks. Software robots may be created and retrieved to automate human user interaction with computer systems. The software robots each take one of three forms: (i) an attended software robot that is encoded to interact with one or more application programs and to accept one or more inputs from the user as required by the one or more application programs, (ii) an unattended software robot that is encoded to automatically interact with one or more application programs without any user input; and (iii) a cognitive software robot that is encoded to automatically interact with one or more application programs without any user input and is further encoded to automatically alter its interactions with the one or more application programs by way of a machine learning engine.

Abstract: A server responds to a request to perform a first automation task to process a work item from the plurality of work items, on a first computing device that is separate and independent from the server. The server receives a request from the first computing device to download the first automation task and queries a distribution information file to identify one or more other computing devices that have a copy of the first automation task. The server provides to the first computing device, an identifier for each of one or more other computing devices that has a copy of the first automation task. If the distribution information file does not contain an identification of any other device that has a copy of the first automation task, then the server processor causes the first automation task to be retrieved and to be provided to the first computing device.

Abstract: Computerized RPA methods and systems that increase the flexibility and lower the cost with which RPA systems may be deployed are disclosed herein. In one embodiment, an RPA system and method avoids the need for preinstalled RPA software on a device employed by a user to create and/or execute software robots to perform RPA. In another embodiment, an RPA system and method provides a capability to execute software robots that may have been encoded in one or more programming languages to execute on an operating system different than that employed by a server of the RPA system.

Abstract: Computerized detection of one or more user interface objects is performed by processing an image file containing one or more user interface objects of a user interface generated by an application program. Sub-control objects can be detected in the image file, where each sub-control object can form a portion of a user interface object that receives user input. Extraneous sub-control objects can be detected. Sub-control objects that overlap with or that are within a predetermined vicinity of an identified set of sub-control objects can be removed. Sub-control objects in the identified set of sub-control objects can be correlated to combine one or more of the sub-control objects in the identified set of sub-control objects to generate control objects that correspond to certain of the user interface objects of the user interface generated by the application program.

Abstract: A robotic process automation system provides a capability to deploy software robots (bots) by receiving from a deployment user a bot deployment request comprising a bot identification that identifies a specific preexisting bot and an authorized class of user to execute the specific preexisting bot. Credentials of the deployment user are checked. An execution device upon which the specific preexisting bot will execute is identified from a set of available devices. An authorization token is issued for the execution device to uniquely identify the execution device and to authorize the execution device to communicate with the robotic process automation system. In response to a request by the execution device the specific preexisting bot and credentials corresponding to the authorized class of user are provided, wherein the specific preexisting bot executes on the execution device automatically without input from any individual corresponding to the authorized class of user.

Abstract: A robotic process automation (RPA) system provides bots that interact with and provide user credentials to applications that require multi-factor authentication (MFA). First user credentials associated with MFA are retrieved by the bots from credential storage. Second user credentials that correspond to questions posed to a user of an application are retrieved from credential storage. Second user credentials that correspond to a one-time password are generated by the RPA system. The second user credentials may also be generated by a third-party authentication service that provides the credentials via a secondary channel such as email or SMS, which are then retrieved for presentation to the application.

Abstract: A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.

Abstract: Task automation is enabled by recording, over a period of time, inputs of a computing device user to generate a log of inputs by the user in connection with one or more task applications. The user inputs are stored along with information pertaining to the one or more task applications. The log is processed to identify the one or more task applications to generate a user task file. The log is further processed to identify the fields in the task applications with which the user entered inputs and the identified fields are stored to the task file. The task file is processed to identify one or more tasks performed by the user. An automated software robot which is encoded with instructions to perform automatically, when invoked, one or more of the tasks performed by the user may be automatically generated.

Abstract: A computerized method for processing a set of robotic process automation (RPA) tasks receives service level requirement inputs that specify a first set of RPA tasks to be performed within a specified period of time. A response to the service level requirement inputs is computed to determine a number of computing resources required to perform the first set of RPA tasks in the specified period of time. Availability of computing resources from a set of computing resources is determined to generate an allocated set of computing resources. The allocated set of computing resources are deployed. A subset of the first set of RPA tasks is queued for each computing resource and each computing resource is monitored and redeployed as it completes tasks in its queue. Quality of Service (QOS) is achieved by prioritizing certain tasks above others.

Abstract: Image encoded documents are identified by recognizing known objects in each document with an object recognizer. The objects in each page are filtered to remove lower order objects. Known features in the objects are recognized by sequentially organizing each object in each filtered page into a one-dimensional array, where each object is positioned in a corresponding one-dimensional array as a function of location in the corresponding filtered page. The one-dimensional array is then compared to known arrays to classify the image document corresponding to the one-dimensional array.

Abstract: A robotic process automation system employs centralized compilation to generate a platform independent executable version of a bot, which is encoded to perform user level operations. The system employs an extensible set of commands which can be user generated. The bots execute on devices that are separate and independent from a server processor that controls the system. The devices execute bots in an execution environment that is provided by the server processor. Change in a command in a bot requires recompilation of the bot which is then delivered upon request to a device. The execution environment does not require recompilation upon a change in a command.

Abstract: A computer system and computerized method that groups documents with similar image layout together. A document similarity metric based on locally connected subgraphs is employed. Region adjacency graphs are generated from word segments extracted from document images. Fuzzy attributed graph isomorphism is performed on subgraphs checking node and edge attribute similarity. Document similarity is then calculated on a normalized score between matching subgraphs of different documents. Unsupervised clustering of document layouts is performed to generate clusters of documents with similar structure.

Abstract: Computerized robotic process automation (RPA) methods and systems that increase the flexibility and lower the cost with which RPA systems may be deployed are disclosed herein. In one embodiment, an RPA system and method avoids the need for preinstalled RPA software on a device employed by a user to create and/or execute software robots to perform RPA. In another embodiment, an RPA system and method provides a capability to execute software robots that may have been encoded in one or more programming languages to execute on an operating system different than that employed by a server of the RPA system.

Abstract: Information contained in tables in a digitized document is extracted by retrieving table layout data regarding bounding boxes, each being auto-generated by the system and/or (re)generated by a user to the digitized image of a sample document. A row template is used to identify a first table, by automatically scanning within the document. Upon detecting a possible row in the input image, a Row Possibility Confidence Value (RPCV) is generated that indicates a likelihood that the possible row corresponds to an actual row in the first table. The possible row is regarded as an actual row if the RPCV exceeds a predetermined threshold value. For repeated tables in a document only the first table needs to be identified via bounding boxes. Also, related tables can be linked to permit linked data to be extracted to a structured file. Also, only the primary column in a readable and existent table header is required to extract table values across columns.

Abstract: Computerized detection of one or more user interface objects is performed by processing an image file containing one or more user interface objects of a user interface generated by an application program. Sub-control objects are detected in the image file, where each sub-control object forms a portion of a user interface object that receives user input. Extraneous sub-control objects are detected. Sub-control objects that overlap with or that are within a predetermined vicinity of an identified set of sub-control objects are removed. Sub-control objects in the identified set of sub-control objects are correlated to combine one or more of the sub-control objects in the identified set of sub-control objects to generate control objects that correspond to certain of the user interface objects of the user interface generated by the application program.