Abstract: Information contained in tables in a digitized document is extracted by retrieving table layout data regarding bounding boxes, each being auto-generated by the system and/or (re)generated by a user to the digitized image of a sample document. A row template is used to identify a first table, by automatically scanning within the document. Upon detecting a possible row in the input image, a Row Possibility Confidence Value (RPCV) is generated that indicates a likelihood that the possible row corresponds to an actual row in the first table. The possible row is regarded as an actual row if the RPCV exceeds a predetermined threshold value. For repeated tables in a document only the first table needs to be identified via bounding boxes. Also, related tables can be linked to permit linked data to be extracted to a structured file. Also, only the primary column in a readable and existent table header is required to extract table values across columns.

Abstract: Automation controls and associated text in images displayed by a computer application are automatically detected by way of region-based R-FCN and Faster R-CNN engines. Datasets comprising images containing application controls, where the application controls include images of application where width is greater than height, width is equal to height and height is greater than width are retrieved and each dataset is processed with the R-FCN and Faster R-CNN engines to generate a software robot configured to recognize corresponding application controls. Text is recognized by an optical character recognition system that employs a deep learning system trained to process a plurality of images to identify images representing text within each image and to convert the images representing text to textually encoded data. The deep learning system is trained with training data generated from a corpus of real-life text segments that are generated by a plurality of OCR modules.

Abstract: A User Interface (UI) interface object detection system employs an initial dataset comprising a set of images, that may include synthesized images, to train a Machine Learning (ML) engine to generate an initial trained model. A data point generator is employed to generate an updated synthesized image set which is used to further train the ML engine. The data point generator may employ images generated by an application program as a reference by which to generate the updated synthesized image set. The images generated by the application program may be tagged in advance. Alternatively, or in addition, the images generated by the application program may be captured dynamically by a user using the application program.

Abstract: A system that automatically recognizes checkboxes within a document and extracts the contents thereof. A user views a digitized document and annotates the document by identifying checkboxes contained in the document, by way of visually perceptible bounding boxes. The annotated document is processed by a machine learning engine that employs multiple convolutional operations followed by a global average pooling layer, a fully connected layer with 1024 node and ‘ReLU’ activation, a fully connected layer with 2 node and ‘softmax’ activation. The identified checkboxes and their contents are stored as label-value pairs, where the label identifies the checkbox and the value identifies the value of the checkbox, which can be either Yes, No, or No checkbox found.

Abstract: A robotic process automation system includes a server processor that performs an automation task to process a work item, by initiating a java virtual machine on a second device. A first user session that employs credentials of a first user for managing execution of the automation task is also initiated on the second device. The server processor loads into the java virtual machine, with a platform class loader, one or more modules, such as logging and security, that perform functions common to the sets of task processing instructions. A first class loader a first set of task processing instructions is also loaded. Then each instruction in the first set of task processing instructions is loaded with a separate class loader. The server processor causes execution, under control of the first user session, on the second device, the task processing instructions that correspond to the work item.

Abstract: A server responds to a request to perform a first automation task to process a work item from the plurality of work items, on a first computing device that is separate and independent from the server. The server receives a request from the first computing device to download the first automation task and queries a distribution information file to identify one or more other computing devices that have a copy of the first automation task. The server provides to the first computing device, an identifier for each of one or more other computing devices that has a copy of the first automation task. If the distribution information file does not contain an identification of any other device that has a copy of the first automation task, then the server processor causes the first automation task to be retrieved and to be provided to the first computing device.

Abstract: Image encoded documents are identified by recognizing known objects in each document with an object recognizer. The objects in each page are filtered to remove lower order objects. Known features in the objects are recognized by sequentially organizing each object in each filtered page into a one-dimensional array, where each object is positioned in a corresponding one-dimensional array as a function of location in the corresponding filtered page. The one-dimensional array is then compared to known arrays to classify the image document corresponding to the one-dimensional array.

Abstract: Robotic process automation (RPA) tasks for operating on data within a productivity program may be initiated by single user action from within the productivity program. A user device is provided with a plugin program that provides an automation user interface within the productivity program. A request to perform an automation task on data in a productivity file is received along with identification of a software robot to perform the automation task. The request also specifies data from the productivity file. The request is provided to a control room server that controls execution of a plurality of different automation tasks by a plurality of different software robots. Results of the automation task are received from the control room server and are provided to the plugin program, which provides the results of the automation task to the user within a productivity file accessible by the user.

Abstract: A computerized method and system for adding distortions to a computer-generated image of a document stored in an image file. An original computer-generated image file is selected and is processed to generate one or more distorted image files for each original computer-generated image file by selecting one or more augmentation modules from a set of augmentation modules to form an augmentation sub-system. The original computer-generated image file is processed with the augmentation sub-system to generate an augmented image file by altering the original computer-generated image file to add distortions that simulate distortions introduced during scanning of a paper-based representation of a document represented in the original computer-generated image file.

Abstract: Image documents that have a visually perceptible geometric structure and a plurality of visually perceptible key-value pairs are grouped. The image documents are processed to generate a corresponding textually encoded document. The textually encoded documents are each assigned into one of a plurality of layout groups, wherein all textually encoded documents in a particular layout group share a visually perceptible layout that is substantially similar. Triplets are selected from the layout groups, where two documents are from the same layout group and one document is from a different layout group. The triplets are processed with a convolutional neural network to generate a trained neural network that may be used to classify documents in a production environment such that a template designed on one image document in a group permits an extraction engine to extract all relevant fields on all image documents within the group.

Abstract: A computer implemented method and system for correcting error produced by Optical Character Recognition (OCR) of text contained in an image encoded document. An error model representing frequency and type of errors produced by Optical Character Recognition Engine is generated. An OCR character string generated by OCR is retrieved. A user-defined pattern of a plurality of character strings is retrieved, where each character string represents a possible correct representation of characters in the OCR character string. The OCR character string is compared to each of the above generated character strings and a ‘likelihood score’ is calculated based on the information from the error model. The character string with the highest ‘likelihood score’ is presumed to be the corrected version of the OCR character string.

Abstract: A robotic process automation system processes task processing instructions which are operable to interact at a user level with user level application programs to process work items. A first server processor responds to a request to perform an automation task to automatically process without human user input, a work item by transmitting requests to an autologin program executing on a second server processor device to initiate a plurality of remote desktop protocol oriented processes within an operating system executing on the second server, where the remote desktop protocol oriented processes provide a graphical user interface to an authorized user. Credentials for the authorized users of the remote desktop protocol oriented processes are provided by one or more of the plurality of sets of task processing instructions, which upon logging into the corresponding remote desktop protocol oriented process, process the work items as encoded in the task processing instructions.

Abstract: A robotic process automation (RPA) system receives task prioritization inputs that specify prioritization for processing of a set of RPA tasks. The tasks are performed in accordance with the specified priorities. The RPA system also receives queue orchestration commands that specify conditions under which tasks processed from a first queue are sent to another queue for subsequent processing. The RPA system also provides service level automation in accordance with specified parameters. Further task prioritization may be specified to provide quality of service performance.

Abstract: Key/Value pairs, each comprising a keyword string and an associated value, are extracted automatically from a document image. Each document image has a plurality of pixels with each pixel having a plurality of bits. A first subset of the plurality of bits for each pixel represents information corresponding to the document image. The document image is processed to add information to a second subset of the plurality of bits for each pixel. The information added to the second subset alters the appearance of the document image in a manner that facilitates semantic recognition of textually encoded segments within the document image by a Deep Neural Network (DNN) trained to recognize images within image documents. The DNN detects groupings of text segments within detected spatial templates within the document image. The text segments are mapped to known string values to generate the keyword strings and associated values.

Abstract: A robotic process automation system operates to generate a plurality of bots, each bot comprising one or more configurable commands arranged to perform assigned tasks. A processor is configured to execute instructions that when executed cause the processor to provide to a user a plurality of recorders to permit the user to create one or more bots for storage in data storage. The instructions implement a plurality of recorders to permit capture of user interaction in a variety of application environments. A secure recording mode is selectable by an administrator of the robotic process automation system, for preventing display to a user of an entire screen of information generated by an application while the user is generating a bot. The administrator can employ the robotic process automation system permit display to the user of only selected fields of information, along with associated labels.

Abstract: Computerized RPA methods and systems that increase the flexibility and lower the cost with which RPA systems may be deployed are disclosed herein. In one embodiment, an RPA system and method avoids the need for preinstalled RPA software on a device employed by a user to create and/or execute software robots to perform RPA. In another embodiment, an RPA system and method provides a capability to execute software robots that may have been encoded in one or more programming languages to execute on an operating system different than that employed by a server of the RPA system.

Abstract: Computerized RPA methods and systems that increase the flexibility and lower the cost with which RPA systems may be deployed are disclosed herein. In one embodiment, an RPA system and method avoids the need for preinstalled RPA software on a device employed by a user to create and/or execute software robots to perform RPA. In another embodiment, an RPA system and method provides a capability to execute software robots that may have been encoded in one or more programming languages to execute on an operating system different than that employed by a server of the RPA system.

Abstract: A robotic process automation system includes a server processor that performs an automation task to process a work item, by initiating a java virtual machine on a second device. A first user session that employs credentials of a first user for managing execution of the automation task is also initiated on the second device. The server processor loads into the java virtual machine, with a platform class loader, one or more modules, such as logging and security, that perform functions common to the sets of task processing instructions. A first class loader a first set of task processing instructions is also loaded. Then each instruction in the first set of task processing instructions is loaded with a separate class loader. The server processor causes execution, under control of the first user session, on the second device, the task processing instructions that correspond to the work item.

Abstract: A robotic process automation system provides a capability to deploy software robots (bots) by receiving from a deployment user a bot deployment request comprising a bot identification that identifies a specific preexisting bot and an authorized class of user to execute the specific preexisting bot. Credentials of the deployment user are checked. An execution device upon which the specific preexisting bot will execute is identified from a set of available devices. An authorization token is issued for the execution device to uniquely identify the execution device and to authorize the execution device to communicate with the robotic process automation system. In response to a request by the execution device the specific preexisting bot and credentials corresponding to the authorized class of user are provided, wherein the specific preexisting bot executes on the execution device automatically without input from any individual corresponding to the authorized class of user.

Abstract: Methods and systems that detect and define virtual objects in remote screens which do not expose objects. This permits simple and reliable automation of existing applications. In certain aspects a method for detecting objects from an application program that are displayed on a computer screen is disclosed. An image displayed on the computer screen is captured. The image is analyzed to identify blobs in the image. The identified blobs are filtered to identify a set of actionable objects within the image. Optical character recognition is performed on the image to detect text fields in the image. Each actionable object is linked to a text field positioned closest to a left or top side of the actionable object. The system automatically detects the virtual objects and links each actionable object such as textboxes, buttons, checkboxes, etc. to the nearest label object.