Abstract: This disclosure relates generally to program synthesis for weakly-supervised multimodal question answering using filtered iterative back-translation (FIBT). Existing approaches for chart question answering mainly address structural, visual, relational, or simple data retrieval queries with fixed-vocabulary answers. The present disclosure implements a two-stage approach where, in first stage, a computer vision pipeline is employed to extract data from chart images and store in a generic schema. In second stage, SQL programs for Natural Language (NL) queries are generated in dataset by using FIBT. To adapt forward and backward models to required NL queries, a Probabilistic Context-Free Grammar is defined, whose probabilities are set to be inversely proportional to SQL programs in training data and sample programs from it.

Abstract: The need for extracting information trapped in unstructured document images is becoming more acute. A major hurdle to this objective is that these images often contain information in the form of tables and extracting data from tabular sub-images presents a unique set of challenges. Embodiments of the present disclosure provide systems and methods that implement a deep learning network for both table detection and structure recognition, wherein interdependence between table detection and table structure recognition are exploited to segment out the table and column regions. This is followed by semantic rule-based row extraction from the identified tabular sub-regions.

Abstract: Traditional systems that enable extracting information from Piping and Instrumentation Diagrams (P&IDs) lack accuracy due to existing noise in the images or require a significant volume of annotated symbols for training if deep learning models that provide good accuracy are utilized. Conventional few-shot/one-shot learning approaches require a significant number of training tasks for meta-training prior. The present disclosure provides a method and system that utilizes the one-shot learning approach that enables symbol recognition using a single instance per symbol class which is represented as a graph with points (pixels) sampled along the boundaries of different symbols present in the P&ID and subsequently, utilizes a Graph Convolutional Neural Network (GCNN) or a GCNN appended to a Convolutional Neural Network (CNN) for symbol classification. Accordingly, given a clean symbol image for each symbol class, all instances of the symbol class may be recognized from noisy and crowded P&IDs.

Abstract: The need for extracting information trapped in unstructured document images is becoming more acute. A major hurdle to this objective is that these images often contain information in the form of tables and extracting data from tabular sub-images presents a unique set of challenges. Embodiments of the present disclosure provide systems and methods that implement a deep learning network for both table detection and structure recognition, wherein interdependence between table detection and table structure recognition are exploited to segment out the table and column regions. This is followed by semantic rule-based row extraction from the identified tabular sub-regions.