Abstract: A method of document image processing comprises, based on at least a document page image, generating a plurality of semantic tokens that includes a plurality of word tokens and a plurality of special tokens. Each special token among the plurality of special tokens represents a non-textual semantic element of the document image, and generating the plurality of semantic tokens includes predicting, for each special token among the plurality of special tokens, a token type of the special token. The method also comprises generating, for each semantic token among the plurality of semantic tokens, a corresponding semantic token embedding among a plurality of semantic token embeddings; and applying a trained model to process an input that is based on the plurality of semantic token embeddings and a plurality of visual token embeddings based on at least the document page image to generate a semantic processing result.

Abstract: In some embodiments, techniques for document entity extraction are provided. For example, a process may involve processing document images to detect a plurality of regions of interest that includes text objects and non-text objects; for each of the plurality of regions of interest, producing a corresponding text string; and processing the text strings to identify entities. Processing the document images may involve applying a text object detection model to the document images to detect the text objects; and applying at least one non-text object detection model to the document images to detect the non-text objects. Prior to processing the document images, at least two object detection models among the text object detection model and the at least one non-text object detection model were generated by fine-tuning respective instances of a pre-trained object detection model.

Abstract: A computer-implemented method includes accessing, by a processor, an unstructured document. The method also includes performing, by the processor, box detection on the unstructured document to generate a box graph of the unstructured document. Further, the method includes integrating, by the processor, identified text of the unstructured document with the box graph of the unstructured document to build a text graph. Furthermore, the method includes generating, by the processor, a structured text representation of the unstructured document using the text graph.