Abstract: Multimodal multitask machine learning system for document intelligence tasks includes a feature extractor processing token values obtained from a document to obtain features, and a token extraction head classifying, using the features, the token values to obtain classified tokens. The classified tokens are aggregated into entities. A document classification model is executed on the features to classify the document and obtain a document label prediction. Further a confidence head model applying the document label prediction processes the entities to obtain a result.

Abstract: A method may include extracting, from a document, a first key-value pair including a key and a first value and corresponding to a first confidence score, extracting a second key-value pair including the key and a second value corresponding to a second confidence score, classifying a first match probability for the first key-value pair and a second match probability for the second key-value pair, generating a first calibrated confidence score for the first confidence score and a second calibrated confidence score for the second confidence score by transforming, using precision lookup tables constructed from training records, the first match probability to the first calibrated confidence score and the second match probability to second calibrated confidence score, selecting, using the first and second calibrated confidence scores, one of the first key-value pair and the second key-value pair, and presenting, in a graphical user interface (GUI), the selected key-value pair.