Abstract: Controls within images of a user interface of a computer application are detected by way of region-based R-FCN and Faster R-CNN engines. Datasets comprising images containing application control, wherein the application controls include images of application where width is greater than height, width is equal to height and height is greater than width are retrieved. Each of the datasets is processed with the R-FCN and Faster R-CNN engines to generate a software configured to recognize, from an input image, application controls wherein the application controls are characterized by dimensions where width is greater than height, where width is substantially equal to height, and where height is greater than width.

Abstract: Information contained in tables in a digitized document is extracted by retrieving table layout data regarding bounding boxes, each being auto-generated by the system and/or (re)generated by a user to the digitized image of a sample document. A row template is used to identify a first table, by automatically scanning within the document. Upon detecting a possible row in the input image, a Row Possibility Confidence Value (RPCV) is generated that indicates a likelihood that the possible row corresponds to an actual row in the first table. The possible row is regarded as an actual row if the RPCV exceeds a predetermined threshold value. For repeated tables in a document only the first table needs to be identified via bounding boxes. Also, related tables can be linked to permit linked data to be extracted to a structured file. Also, only the primary column in a readable and existent table header is required to extract table values across columns.

Abstract: Automation controls and associated text in images displayed by a computer application are automatically detected by way of region-based R-FCN and Faster R-CNN engines. Datasets comprising images containing application controls, where the application controls include images of application where width is greater than height, width is equal to height and height is greater than width are retrieved and each dataset is processed with the R-FCN and Faster R-CNN engines to generate a software robot configured to recognize corresponding application controls. Text is recognized by an optical character recognition system that employs a deep learning system trained to process a plurality of images to identify images representing text within each image and to convert the images representing text to textually encoded data. The deep learning system is trained with training data generated from a corpus of real-life text segments that are generated by a plurality of OCR modules.

Abstract: A system that automatically recognizes checkboxes within a document and extracts the contents thereof. A user views a digitized document and annotates the document by identifying checkboxes contained in the document, by way of visually perceptible bounding boxes. The annotated document is processed by a machine learning engine that employs multiple convolutional operations followed by a global average pooling layer, a fully connected layer with 1024 node and ‘ReLU’ activation, a fully connected layer with 2 node and ‘softmax’ activation. The identified checkboxes and their contents are stored as label-value pairs, where the label identifies the checkbox and the value identifies the value of the checkbox, which can be either Yes, No, or No checkbox found.