Abstract: A method including receiving an initial dataset of data points and a sampled dataset including a sampling of data in a second dataset. Identified are a first number of subpopulations in the initial dataset and a second number of subpopulations in the sampled dataset corresponding to the first number of subpopulations. A measure of representativeness is identified using the first number of subpopulations in the initial dataset and the second number of subpopulations in the sampled dataset. A measure of coverage of the sampled dataset is identified relative to the initial dataset. An initial machine learning model is executed on the sampled dataset based on thresholds to generate a number of outputs corresponding to the subpopulations. A number of accuracies of the initial machine learning model are determined for the second number of subpopulations. The sampled dataset is remediated. The initial machine learning model is retrained.

Abstract: Systems and methods for training an encoder-decoder model are disclosed.

Abstract: Certain aspects of the disclosure provide techniques for bounding box annotation for automated information extraction. A method generally includes obtaining an extracted field value for a field key of a document using optical character recognition (OCR) data generated based on the document, wherein: the OCR data comprises OCR tokens and bounding boxes, each associated with one OCR token; generating a value union bounding box surrounding value bounding box(es) of the bounding boxes, wherein: the value bounding box(es) satisfy a first threshold; and the value bounding box(es) are associated with first OCR token(s) of the plurality of OCR tokens that satisfy a second threshold when compared to one or more field value tokens of the extracted field value; and generating an output bounding box for display on a computing device with the document based on relative coordinates of the value union bounding box with respect to known dimensions of the document.

Abstract: Certain aspects of the disclosure provide techniques for automated information extraction. A method generally includes performing optical character recognition (OCR) on a document to generate OCR data; iteratively, for one or more field keys of the document: generating a prompt comprising the OCR data and a field key of the one or more field keys; prompting a large language model (LLM) with the prompt to extract a field value corresponding to the field key; and receiving, from the LLM, an extracted field value from the document; and providing one or more extracted field values from the document to an application for further processing.

Abstract: Systems and methods for training an encoder-decoder model are disclosed.

Abstract: Systems and methods for training machine learning models are disclosed. An example method includes receiving a plurality of first outputs and a ground truth value for each first output, each first output including an extracted string and a raw confidence score, determining, for each first output, an accuracy metric based at least in part on the extracted string and its corresponding ground truth value, for each extracted string: determining a similarity metric between the respective extracted string and each other extracted string of the plurality of first outputs, and determining a pseudo-accuracy based at least in part on the determined similarity metrics and the determined accuracy metrics, generating training data based at least in part on the determined pseudo-accuracies and the plurality of first outputs, and training the machine learning model, based on the training data, to predict pseudo-accuracies associated with subsequent outputs from a document extraction model.