System and method for determining veracity of patient diagnoses within one or more electronic health records
System and methods are disclosed which use one or more dimensions that assesses the veracity of a diagnosis in a health care record. The veracity of the diagnosis is based on a set of dimension values and a set of weightings. A user interface for healthcare providers/caregivers is provided to review the assessment and provide feedback so that the weighting of the dimensions may be adjusted to improve the assessment of the diagnosis.
Latest Humana Inc. Patents:
- Cloud platform based data mesh architecture for data pipelines
- Integrated development environment for development and continuous delivery of cloud-based applications
- Cloud platform based architecture for continuous deployment and execution of modular data pipelines
- Infrastructure quality assurance framework
- System for providing a data market for health data and for providing rewards to data market participants
This application is a continuation of U.S. application Ser. No. 14/216,230 filed Mar. 17, 2014 which claims priority to provisional application 61/789,834, filed on Mar. 15, 2013, the disclosures of each of which are hereby incorporated by reference in their entireties.
FIELDThe disclosure relates generally to electronic health records and in particular to a system and method for verifying the accuracy of one or more diagnoses in an electronic health record.
BACKGROUND AND SUMMARY OF THE INVENTIONElectronic health records typically contain codified diagnoses (frequently using standard coding schemes such as ICD9 or SnoMed) such records may also comprise patient clinical problem lists that are required for billing purposes, as well as for maintaining a patient clinical problem list, for subsequent data mining, and for clinical decision support. In a patient data sharing healthcare ecosystem, however, there is co-mingling of diagnoses from multiple providers (who may not know that they may all be treating the same patient), and diagnoses from insurers, which typically include diagnoses amalgamated from suppliers, laboratories, hospitals, nursing homes, imaging facilities, pharmacies, and other entities whose products and services require a diagnosis for payment.
There is therefore a difficulty in determining in a single patient which diagnoses are real and/or currently active when data is shared. These include diagnoses: (1) that are presumptive and created for reimbursement of legitimate services (presumptive diagnosis), (2) diagnoses that are created for services not actually rendered (fraud), (3) diagnoses that do not represent the most severe manifestation of a disease (known as “undercoding”) because of lack of caregiver time to encode the most specific condition (e.g., plain “diabetes” vs. “Uncontrolled Type II diabetes with renal manifestations”), (4) diagnoses captured in health records by inadequately trained personnel across the entire spectrum of caregivers (from a generalist physician, to a specialist, to a nurse, to a pharmacist, to a technical assisting with documentation, to self-reported diagnoses by the patient), and finally (5) diagnoses that were once true (e.g., knee sprain, influenza, or routine urinary tract infection from 1 year ago) but would be expected to have time-expired due to the natural history of the illness, and no longer present in a patient's active diagnoses today.
In the case of presumptive diagnosis, by way of example, an imaging facility or laboratory typically records what is known as a ‘presumptive’ or ‘rule-out’ diagnosis when it submits a bill for payment. For example, a chest x-ray is commonly associated with a diagnosis of ‘pneumonia’ when in fact that it is a presumptive diagnosis to include with the image. In many, if not most, instances when a chest x-ray is performed, pneumonia is not found, yet the presumptive diagnosis persists in the patient's medical history.
In electronic health record (EHR) sharing environments, all of these diagnoses can become humanly impossible to sift through in the few minutes a doctor has to treat a new patient, for example.
Therefore, it is desirable to have a system and method in which the veracity of the diagnoses (stored in a patient medical record) can be determined across one or more amalgamated sources of diagnosis data (an example of an amalgamated source of diagnosis date may be a multi-provider health care record or health care records from multiple health care providers) at the current point of care and it is to this end that the disclosure is directed.
Various care data concerning a patient is weighted and processed to provide a likelihood indicator of whether the diagnosis is accurate. The invention can also be used to provide alternative diagnosis for a care provider to consider, ongoing feedback may be used to enhance the reliability of the invention output.
The system and method may be particularly applicable to a web-based system that delivers diagnosis veracity assessments to one or more health care provider systems over the Internet and it is in this context that the system and method will be described. One skilled in the art will understand that: 1) the system and method may be implemented in various manners that are within the scope of the invention and therefore, that the system and method are not limited to the example web-based system described below; and 2) the system and method may use different dimensions, different coefficients, different formulas, etc. than those described below and as such, the system and method are not limited to the examples provided below.
In general, embodiments of the system and method, using one or more dimensions, assesses the veracity of each diagnosis found in a health care record (when such a health care record may be an electronic medical record) based on a set of dimension values and a set of weightings, and provides a user interface to health care providers/caregivers to review and verify the assessment made by the system. Embodiments of the system may have a feedback mechanism so that the system may adjust the weighting of the dimensions to improve the assessment of the diagnosis made by the system. The system and method may have a set of dimensions, a weighting of those dimensions, and a veracity determining formula stored/available for each diagnosis (for example, diabetes, cancer, etc.). Since each diagnosis may have unique dimensions/dimension values, unique weighting and/or a unique veracity determining function that may be fine-tuned to perform the best possible assessment of the veracity determining formula, weighting for any particular diagnoses may be adjusted over time to fine tune the assessment. In one embodiment, the system and method also may infer a diagnosis for a patient based on the dimensions, the weighting of the dimensions and the function of a particular diagnosis of a particular patient. For example, although a particular patient has not been diagnosed with diabetes, certain tests, lab results, etc. may allow the system to infer that the patient has diabetes. Now, various examples of the embodiments of the system are described in more detail.
Each health care provider unit 102 may be a computerized device comprised of a computer processor that has sufficient processing power, memory, display capabilities and wireless/wired connectivity circuitry to interact with the diagnosis veracity assessment unit 104. For example, each health care provider unit 102 may be a personal computer, a terminal, a computer server of a health care provider, a multiple provider health care record storage system, a laptop computer, or any number of mobile devices, a pocket PC device, a smartphone (RIM Blackberry, Apple iPhone, etc.), tablet computer, a mobile phone, a mobile email device, etc. Each health care provider unit 102 may also include a typical browser software application 111, such as units 111a, 111b, . . . , 111n, that may be, in an exemplary web-based client/server implementation, a web browser (or a plurality of lines of computer code stored in the health care provider unit and executed by the processing unit of the health care provider unit) that interacts with the diagnosis veracity assessment unit 104 and generates display images with information from the diagnosis veracity assessment unit 104.
The diagnosis veracity assessment unit 104, in one implementation may be implemented as one or more computer servers that may execute one or more software programs. In the web-based example illustrated in
The system for diagnosis verification 109 may further comprise a data store 114, implemented as one or more databases hosted on one or more database servers in the illustrated implementation. In embodiments of the invention, such a data store may be part of the diagnosis veracity assessment unit 104 or remotely located from the diagnosis veracity assessment unit 104. Such a data store may or may not be owned or controlled by the owner of unit 104. Such a data store 114 may include a plurality of health records 106 for a plurality of patients. Such health records may also be stored in an electronic medical record (EMR) system that is remote from the diagnosis verification system 109. Such a data store may also compare a dimensions, weighting and rule store 108 that stores the dimensions, weighting of those dimensions, and a function to determine the veracity for a particular diagnosis for a plurality of diagnosis and the assessments for each patient, for each diagnosis (and any diagnosis that may be inferred for a patient).
The plurality of health records 106 may also be one or more health records from one or more health care providers (that may be stored in different locations) for a particular patient so that the system can retrieve all of the health care records for a patient. Access to such a plurality of records may enable the system to analyze a greater range of data when determining the veracity of each diagnosis in the health care records for a patient. In addition, access to records from each care provider to a patient may be provided so that a current health care provider can rapidly determine which diagnosis are correct and which are not accurate. In addition, using the feedback (described later herein), the dimensions, weighting, and functions for any particular diagnosis may be updated and then stored in the data store 108.
The system 109 may also include a user data store 116 that may be used to store various pieces of information about the users of the system. For example, the user data store may have a record associated with each health care provider and its associated users of the system. Such a record may comprise, for example, the preferences for each health care provider and its associated users. Each of the units/portions of the system 109 may be implemented in software, hardware, or a combination of software and hardware, and embodiments of the system should not be considered to be limited to any particular implementation of the system and its units.
Although a typical client/server architecture using web pages is illustrated in
The diagnosis assessment unit 113 may comprise a veracity score unit 120 that, using a predetermined set of dimensions, weighting and functions, determine a diagnosis veracity value for the diagnosis of the particular patient. The diagnosis assessment unit may be configured to perform diagnosis veracity value calculations for a plurality of different diagnosis for a plurality of different patients. The diagnosis assessment unit 113 may also have a visualization dashboard generator 122 that generate a diagnosis dashboard for a particular diagnosis that may be returned to the caregiver/health care provider who is treating a patient so that the caregiver/health care provider can view the assessment and confirm the diagnosis assessment of the system. The system may receive such a user confirmation and then “learn” and improve its assessment based on such caregiver/health care provider feedback. To accomplish this, the diagnosis assessment unit 113 may also have a feedback processing system 124 that is closed loop and thus enable machine learning by modifying automatically, or by user or expert input, the dimensions, weighting of the dimensions and/or function used to assess each diagnosis. Such automatic modification may also be applied to the dimensions, weighting of the dimensions and function used to infer a diagnosis by the system. Such feedback enables the system assessments and inferences to be improved in part based on the caregiver or health care provider confirmation of the diagnosis assessments provided by the system over time.
An example of the dimensions, weightings and functions for a particular diagnosis whose veracity can be assessed by the system will now be described. The flow chart of
Each diagnosis that is assessed by the system may be evaluated using a plurality of dimensions. In one implementation, seven dimensions may be used. The dimensions may be: (1) the type of entity which recorded the diagnosis (e.g., institution, patient, actual caregiver, and if a caregiver the level of training or specialization of the caregiver), (2) the physical location of the diagnosis (e.g., a nursing home, hospital, laboratory, imaging facility), (3) the date of the diagnosis, (4) related similar diagnoses from different entities, (5) medications taken to date that suggest a particular diagnosis, (6) procedures performed to date that suggest a particular diagnosis, and (7) lab values known in the patient's medical history that suggest a particular diagnosis.
The type of entity and physical location dimensions are hierarchical dimensions because there is a hierarchy for the values associate with that dimension. For example, the type of entity dimension may have values that are higher for a physician specialist—Endocrinologist (for a diabetes diagnosis) who is given a higher value (72, for example) than a family practitioner (30 for example), who has a higher value than a nurse (20, for example) who is given a higher value than a patient (10, for example), etc., so that the type of entity value is hierarchical and reflective of the decreasing likelihood (in this example) that each type of entity will make the proper diagnosis. Similarly, the physical location dimension may have values such that a doctor's office is given a higher value (90, for example) than a nursing home (80, for example) which has a higher value than an ambulance (50, for example), which has a higher value than a laboratory (40, for example) that has a higher value than a medical equipment supplier (20, for example), etc., so that the location of diagnosis value is also hierarchical and reflective of the likelihood that each location will have a proper diagnosis made by the person at such a location.
The date of the diagnosis dimension is time value with more recent diagnosis being given a higher value to indicate that a more recent diagnosis is more likely to be accurate. Related similar diagnoses from different entities, medications taken to date that suggest a particular diagnosis, procedures performed to date that suggest a particular diagnosis, and lab values known in the patient's medical history that suggest a particular diagnosis are given numerical values with the number assigned to each type of information being used to assess the veracity of a diagnosis.
Once the dimensions for a diagnosis are determined, the system retrieves the weighting for each dimension for the particular diagnosis. The weighing of each dimension for the particular diagnosis may have a default value, but as was previously described, such default values may be adjusted by the system due to closed loop feedback. The weighting of each dimension may be accomplished by a coefficient. In an exemplary embodiment, the function for a particular diagnosis may have the form of:
Veracity Score for Diagnosis #1=f(a1*Dimension1*b1Type+Count,a2*Dimension2b2*Type+Count,a3*Dimension3*b3Type+Count, . . . a7*Dimension7b7 time)
wherein f is a function and a1, a2, a3, . . . a7 are a weighting coefficient for each corresponding dimension and b1, b2, b3, . . . b7 are each a weighting coefficient to invoke the hierarchy of the particular dimension (if any), and “Count” represents the frequency count of a particular variable. Thus, the final probability (or veracity score) for a particular diagnosis may be a function of the 7 variables and their frequency counts and the time of each event. It is understood that due to iterative feedback over time, the coefficients may be modified, even empirically, such that the likelihood of the diagnosis receiving confirmation as being correct is maximized. Embodiments of the invention may use various different functions and should not be considered as limited to any particular function. For example, the system may use a logistic regression or other weighted, exponential formula (asymptotic, constrains from 0 to 1 for infinite counts and values, etc.).
The system may determine the above described veracity scores for each diagnosis that is part of the health record. The system thus generates a veracity score for each diagnosis (148). The visualization dashboard generator 122 may generate the visualization for each generated diagnosis veracity score with the score, each dimension (and optionally a description of each dimension), and the values associated with each dimension. An example of such a visualization dashboard is shown in
The visualization dashboard should provide the caregiver/health care provider with enough information to allow such a caregiver/health care provider to consider the data fully in order to prevent caregiver/health care provider bias just before casting a vote. For example, the caregiver/health care provider may not believe that a patient has diabetes, however, he/she might be confronted with data (which can be from another caregiver) that shows persistent elevated blood sugar, ophthalmic complications specific to diabetes, and/or repetitive use of medications that treat diabetes and resulted in a therapeutic response, such as a lowering of blood sugar values. When provided with such data, the caregiver may vote for the correctness of the diagnoses, when without such information, he or she otherwise may not have done so. By way of example, a table similar to what is shown in the visualization dashboard illustrated in
In addition to the generating of the diagnosis veracity score as described above, the system may also identify previously unidentified diagnosis in a health record. In particular, based on the same dimensions, weighting and formulas described above, the system can infer certain diagnoses of a patient's condition. For example, the system may retrieve data from the patient's health care record that indicates a persistent elevated blood sugar, ophthalmic complications specific to diabetes, and/or repetitive use of medications that treat diabetes and resulted in a therapeutic response, such as a lowering of blood sugar values. These values of the dimensions can be used by the system to infer that the patient has diabetes even though there is not a diabetes diagnosis in the patient's health record. For example, if the veracity score for a particular previously unreported diagnosis is about a predetermined threshold (such as 90%), then the diagnosis becomes an inferred diagnosis. Conversely, patient data trending in the opposite direction would indicate that a diabetes diagnoses is incorrect. While the foregoing has been with reference to a particular embodiment of the invention, it will be appreciated by those skilled in the art that changes in this embodiment may be made without departing from the principles and spirit of the disclosure, the scope of which is defined by the appended claims.
Claims
1. A method for generating and optimizing veracity scores for diagnoses in patient records, said method comprising the steps of:
- electronically receiving a health record for a patient, wherein said health record comprises data representing a diagnosis for the patient;
- electronically retrieving data representing values of dimensions for the diagnosis, wherein each of said dimensions are specific to the diagnosis;
- electronically retrieving data representing a weighting value for each of the dimensions, wherein each of the weighting values are specific to the diagnosis;
- electronically retrieving a function for the diagnosis, wherein said function is selected from a plurality of functions, each comprises variables for each of the dimensions and each of the weighting values, and each specific to one of a number of diagnoses comprising the diagnosis;
- electronically applying the weighting values and values of the dimensions retrieved to the function retrieved to determine a veracity score for the diagnosis;
- displaying a visualization dashboard at an electronic device associated with a healthcare provider, said visualization dashboard comprising the veracity score and the diagnosis as well as supporting reasoning provided in tabular form comprising each of the values of dimensions applied to arrive at the score and each of the weighting values applied to arrive at the score separately displayed;
- displaying, at the electronic device and in association with the visualization dashboard, a veracity feedback tool comprising user input options for agreement or disagreement with the diagnoses and the supporting reasoning;
- receiving healthcare provider input at the veracity feedback tool indicating selection of the agreement or disagreement option with the diagnosis;
- automatically adjusting the weighting values in response to the healthcare provider input received at the veracity feedback tool; and
- using the adjusted weighting values in a subsequent veracity score determination;
- wherein the values of the dimensions comprise at least one non-numerical descriptor.
2. The method of claim 1 wherein:
- the feedback tool is displayed at the visualization dashboard adjacent to the score, the diagnosis, and the supporting reasoning.
3. The method of claim 1 wherein:
- the feedback tool comprises a true option and a false option.
4. The method of claim 1 wherein:
- said health record comprises a type of entity which recorded the diagnosis; and
- said value of said dimensions comprise the type of entity which recorded the diagnosis.
5. The method of claim 4 wherein:
- said type of entity which recorded the diagnosis comprises one of an institution, the patient, and a caregiver associated with a level of training.
6. The method of claim 1 wherein:
- said health record comprises a physical location where the diagnosis was made; and
- said value of said dimensions comprise the physical location where the diagnosis was made.
7. The method of claim 6 wherein:
- said physical location comprises one of: a nursing home, a hospital, a laboratory, and an imaging facility.
8. The method of claim 1 wherein:
- said health record comprises a date the diagnosis was made; and
- said value of said dimensions comprise the date the diagnosis was made.
9. The method of claim 1 wherein:
- said health record comprises other, different diagnoses for the patient; and
- said value of said dimensions comprise one or more related diagnoses from different entities from the other, different diagnoses.
10. The method of claim 1 wherein:
- said health record comprises one or more medications taken by the patient; and
- said value of said dimensions comprise medications from the one or more medications that suggest the diagnosis.
11. The method of claim 1 wherein:
- said health record comprises one or more procedures performed on the patient; and
- said value of said dimensions comprise procedures from the one or more procedures performed on the patient that suggest the diagnosis.
12. The method of claim 1 wherein:
- said health record comprises one or more lab results for the patient; and
- said value of said dimensions comprise lab values that suggest the diagnosis from the one or more lab results.
13. The method of claim 1 wherein:
- the functions comprise a logistic regression, a weighted function, and an exponential function.
14. The method of claim 1 further comprising the steps of:
- determining that a new diagnosis not already in the health record for the patient has a veracity score above a predetermined threshold; and
- adding the new diagnosis to the health record for the patient.
15. A system for generating and optimizing veracity scores for diagnoses in patient records, said system comprising:
- health care provider devices;
- one or more data stores comprising: a plurality of health records, each of which is associated with a patient and comprises one or more diagnoses for the associated patient; functions, each of which is associated with, and specific to, one or more diagnoses; dimensions, each of which is associated with, and specific to, one or more diagnoses and includes a value, wherein said values of said dimensions comprise a type of entity which recorded the diagnosis, including one of an institution, the patient, and a caregiver associated with a level of training, the physical location of the diagnosis, including one of a nursing home, a hospital, a laboratory, and an imaging facility, the date of the diagnosis, related similar but different diagnoses from different entities, medication taken which suggest the particular diagnosis, medical procedures performed which suggest the particular diagnosis, and lab values which suggest the particular diagnosis; and weighting values, each of which is associated with, and specific to, one or more of the dimensions; and a diagnosis assessment unit comprising: a diagnosis veracity score unit configured to, on a diagnosis and patient specific basis: retrieve a health record from the one or more data stores for a particular patient; retrieve a function from the functions, wherein the retrieved function is associated with a particular diagnosis contained within the retrieved health record; retrieve each of the values of the dimensions and each of the weighting values from the one or more data stores associated with the particular diagnosis; and apply each of the retrieved values of the dimensions and each of the retrieved weighting values to the retrieved function to arrive at a veracity score; a visualization dashboard generator configured to display, at one or more of the healthcare provider devices, the particular diagnosis, the veracity score, and supporting reasoning comprising the retrieved values of the dimensions and each of the weighting values provided separately at a table; and a feedback processing system configured to: display, at the one or more of the healthcare providers devices, a prompt requesting user input regarding the veracity of the particular diagnosis, said prompt comprising a confirm indication and a disavow indication displayed in association with the table; receive user input regarding the veracity of the particular diagnosis in the form of selection of one of the confirm indication and the disavow indication; and send an updated weighting value to the one or more data stores with instruction to replace at least one of the one or more weighting values associated with the particular diagnosis with the updated weighting value; wherein the updated weighting value is set to be greater than the weighting values to be replaced wherein the received user input indicated selection of the affirmation option; and wherein the updated weighting value is set to be less than the weighting values to be replaced wherein the received user input indicated selection of the disavow option; wherein the values of the dimensions comprise at least one non-numerical descriptor; wherein the one or more data stores are configured to replace the at least one of the one or more weighting values associated with the particular diagnosis with the updated weighting value following receipt of the instructions.
16. The system of claim 15 wherein:
- said diagnosis veracity score unit is configured to: determine that a new diagnosis not already in the health record for the patient has a veracity score above a predetermined threshold; and add the new diagnosis to the health record for the patient.
9495515 | November 15, 2016 | Kennedy |
20060089812 | April 27, 2006 | Jacquez |
20070055552 | March 8, 2007 | St. Clair |
20070250352 | October 25, 2007 | Tawil |
20110119212 | May 19, 2011 | De Bruin |
20110257988 | October 20, 2011 | Denekamp |
20120130743 | May 24, 2012 | Gotthardt |
20120290319 | November 15, 2012 | Saria et al. |
20160012349 | January 14, 2016 | Lai |
WO-9963320 | December 1999 | WO |
- Goraya et al., Validation of death certificate diagnosis of out-of-hospital coronary heart disease deaths in Olmsted County, Minnesota, Jul. 2000, Mayo Clinic Proceedings, pp. 681-687. (Year: 2000).
Type: Grant
Filed: Mar 25, 2020
Date of Patent: Jan 16, 2024
Patent Publication Number: 20200243200
Assignee: Humana Inc. (Louisville, KY)
Inventor: Ahmed Ghouri (San Diego, CA)
Primary Examiner: Christopher L Gilligan
Application Number: 16/829,575
International Classification: G16H 50/70 (20180101); G16H 50/30 (20180101); G16H 10/60 (20180101); G06N 3/08 (20230101); G16H 15/00 (20180101); G16H 50/20 (20180101);