Abstract: In an embodiment, a computer-implemented process comprises accessing a plurality of digitally stored, unstructured medical diagnostic data; digitally displaying a first subset of the medical diagnostic data, the first subset of the medical diagnostic data including at least a first set of diagnostic reports, using a computer display device, concurrently with digitally displaying one or more quality control checklists that are specific to a medical discipline represented in the first set of diagnostic reports; receiving digital input specifying one or more errors in the first set of diagnostic reports and digitally storing the digital input in association with the first subset of medical diagnostic data; training a hierarchical Bayesian machine learning model using the digital input and the first subset of medical diagnostic data; evaluating the hierarchical Bayesian machine learning model, after training, for a second subset of the medical diagnostic data, the second subset being different from the first subs

Abstract: Described are techniques for determining an accurate ground-truth Bayesian Inter-Reviewer Agreement Rate (BIRAR) from unreliable sources. For instance, a process can include obtaining an initial set of diagnostic imaging exams, wherein each diagnostic imaging exam includes a severity grade associated with an initial radiologist. For each diagnostic imaging exam, two or more secondary quality assurance (QA) reviews can be obtained for each diagnostic imaging exam, wherein the secondary QA reviews are associated with QA'ing radiologists different than the initial radiologist. One or more inter-reviewer agreement rates can be determined for the QA'ing radiologists, based on the secondary QA reviews associated with the QA'ing radiologists.

Abstract: In an embodiment, a computer-implemented process comprises accessing a plurality of digitally stored, unstructured medical diagnostic data; digitally displaying a first subset of the medical diagnostic data, the first subset of the medical diagnostic data including at least a first set of diagnostic reports, using a computer display device, concurrently with digitally displaying one or more quality control checklists that are specific to a medical discipline represented in the first set of diagnostic reports; receiving digital input specifying one or more errors in the first set of diagnostic reports and digitally storing the digital input in association with the first subset of medical diagnostic data; training a hierarchical Bayesian machine learning model using the digital input and the first subset of medical diagnostic data; evaluating the hierarchical Bayesian machine learning model, after training, for a second subset of the medical diagnostic data, the second subset being different from the first subs