Pursuing integration of performance measures into electronic medical records: beta-adrenergic receptor antagonist medications
http://www.100md.com
《健康质量安全杂志》
1 Indiana University Center for Aging Research, Indianapolis, Indiana, USA
2 Regenstrief Institute Inc, Indianapolis, Indiana, USA
3 Department of Medicine, Indiana University, Indianapolis, Indiana, USA
4 Health Care Excel Inc, Medicare Quality Improvement Organization, Indianapolis, Indiana, USA
ABSTRACT
Objective: Electronic medical records seldom integrate performance indicators into daily operations. Assessing quality indicators traditionally requires resource intensive chart reviews of small samples. We sought to use an electronic medical record to assess use of -adrenergic antagonist medications (-blockers) following myocardial infarction, to compare a standardized manual assessment with assessment using electronic medical records, and to discuss potential for future integration of performance indicators into electronic records.
Design: Cross-sectional data analysis.
Setting: An urban academic medical center.
Participants: US Medicare beneficiaries 65 years of age or older, admitted to hospital with myocardial infarction between 1995 and 1999.
Measurements and main results: Manual chart review was compared with a computer driven assessment of electronic records. Administration of -blockers and cases excluded from use of -blockers were measured, based on Medicare criteria. Among 4490 older adults, 391 (4%) of 9018 hospital admissions contained codes for myocardial infarction. In 323 (83%) of the 391 hospital admissions, criteria for excluding -blockers were met; 235 (60%) were excluded due to heart failure. Of 68 hospital admissions for myocardial infarction that did not meet exclusion criteria, physicians prescribed -blockers in 49 (72%) on admission and 42 (62%) at discharge. Compared with manual chart review, electronic review had a sensitivity of 83–100% and led to fewer false negative findings.
Conclusions: An electronic medical records system can be used instead of chart review to measure use of -blockers after myocardial infarction. This should lead to integration of real time automated performance measurement into electronic medical records.
Abbreviations: AMI, acute myocardial infarction; CHF, chronic heart failure; COPD, chronic obstructive pulmonary disease; EMR, electronic medical record; QIO, Quality Improvement Organizations
Keywords: electronic medical records; acute myocardial infarction; beta-blockers; performance measures
In many countries objective assessments of quality of medical care show deficiencies in numerous areas such as preventive care and care of cardiovascular diseases, diabetes mellitus, and mental illness. Acute myocardial infarction (AMI) has received special attention, especially due to available interventions that prolong survival.1,2 In a comparison of data from Australia, Canada, New Zealand, England, and the United States, Hussey and colleagues reported that 30 day in-hospital case fatality rates for AMI are highest in Canada where they exceed the rate in Australia by about 50%, regardless of age.3 In the US, major organizations involved with accreditation4–6 and national quality assurance7 include among important measures of quality of health care the use of -adrenergic receptor antagonists (-blockers) in treating AMI. A recently published report from South Wales, UK showed that only 58% of patients admitted for AMI were discharged on -blockers.8 Although a fraction of the other 42% met exclusion criteria, some exclusion criteria are questionable and have been debated. In any case, suboptimal quality of care9 poses challenges for all health systems.
To develop programs and policies to improve health care, medical institutions and clinicians need timely and accurate data about the quality of care that they are providing. Much of the available data comes in the form of reports generated from research rather than from routines in daily practice. In many countries assessment of performance occurs by manual chart review. In the US, for example, government contracted Quality Improvement Organizations (QIOs) help to assess and promote quality for many patients such as nearly all those over 65 years of age who qualify for Medicare national health insurance (box 1). The QIOs assess quality indicators by requesting a sample of individual paper charts from health institutions and manually reviewing each chart in the sample. In a recent 3 year period about 8000 charts per state have been reviewed in a national assessment of two samples (4000 charts per state per sample), and many QIOs perform additional reviews to increase sample size for feedback to hospitals. Although the QIO reimburses hospitals for copying medical records for these audits, a hospital’s expenditure for copying often exceeds the reimbursement provided. The total cost of copying and reviewing 8000 charts manually is estimated at more than US$10 million. With such a burden, the degree to which an individual chart can be evaluated is limited10 and national assessments of state level indicators may be possible only every few years.11 Two thirds of US hospitals paid by the two largest government insurance programs—Medicare and Medicaid—do not report any quality measures.12 As a result, applying performance data at the point of care—a major goal of efforts at improving quality—is often impossible.
Box 1 US Medicare
Federal health insurance program created under the Social Security Act of 1965, originally to ensure access to health care for retired older Americans.
Covers nearly all older Americans and now also certain younger people with disabilities or end-stage renal disease.
Covers hospital care, home health care, hospice care, and up to 100 days in a skilled nursing facility without additional premiums; supplemental coverage for outpatient care, physicians’ services, and durable medical equipment can be purchased separately.
Program defines several measures of quality, including prescription of -blockers following acute myocardial infarction (AMI).
Contracts with Quality Improvement Organizations for aspects of administration, as well as assessment and improvement of quality of care.
Uses manual chart reviews of national samples to assess quality.
The need for better management of health information9,13 prompts new methods of collecting and using data about performance. Computerization of medical records offers many advantages not only for care of individual patients but for care of populations. For example, interactive electronic medical records (EMR) can prompt clinicians automatically with reminders about care and flags of potentially adverse outcomes. These forms of active decision support have also been proved to change physicians’ practices and improve health outcomes.14–22 Leaders of medical organizations who receive information from these systems can then also take proactive approaches in the timely development of programs to improve quality of care,23 including providing financial or other incentives based directly on performance.24–26 The government of the UK has already implemented a national plan to reward performance27 and, in the US, the Medicare Prescription Drug, Improvement and Modernization Act of 2003 increases payments to hospitals that publicly report data about quality.
The problems associated with assessing quality of health care on a large scale beg for more affordable and smarter uses of EMRs. Institutions with computer based approaches to quality improvement still provide a minority of health care. Finding inexpensive, efficient, and accurate ways for individual medical institutions to assess their own quality indicators may lower governmental and institutional costs and lead to more local programs that can directly improve quality of care.28 The US Secretary of the Department of Health and Human Services has recently commissioned the design of a freely distributable standardized model of an EMR.29 In the meantime, vendors that create and sell commercial EMRs compete in a narrow marketplace by providing divergent, proprietary products. Many vendors would welcome but lack standardized approaches to automating performance measures, while a clear solution to the problem requires direct integration of performance measures into these products. We sought to use an EMR to assess the use of -adrenergic antagonist medications (-blockers) following myocardial infarction, to compare a standardized manual assessment with assessment using EMR, and discuss the potential for future integration of performance indicators into electronic records.
METHODS
Electronic medical records (EMRs)
The main analysis was conducted using EMRs of the Regenstrief Medical Records System (RMRS).30–32 RMRS registers and stores data for all patients with inpatient or outpatient encounters in Wishard Health Services, an urban public medical institution of Marion County, Indiana. It captures inpatient and outpatient registrations, scheduling, clinicians’ orders, diagnoses, notes, documentation of allergies, laboratory results, electrocardiograms, radiology reports and images, and charges. Like many pharmacy systems nationwide, information about all medications ordered or dispensed through the pharmacies is recorded electronically. Medications prescribed but not administered at discharge are also recorded in this system. Adjacent to approximately half of the hospital’s beds are machines that automatically capture blood pressure and pulse.
Defining cases of AMI and a subgroup eligible to receive -blockers
After a patient’s discharge, hospital personnel identify major events, procedures, and diagnoses, along with appropriate diagnostic codes from the International Classification of Diseases (ICD), 9th Revision, Clinical Modification,33,34 which is universal to all hospitals. In this study only cases of AMI were reviewed. To identify patients discharged with AMI we screened the ICD codes of all Medicare covered hospital admissions between January 1995 and December 1999. AMI was defined to occur if a primary or secondary discharge ICD code of 410.x (AMI), 410.x0 (episode unspecified), or 410.x1 (initial episode) was identified. Excluded were hospital admissions for patients under 65 years of age and those with a discharge ICD code of 410.x2 (subsequent episode) or 411.0 (post-infarction syndrome).
The Centers for Medicare & Medicaid Services (CMS) defines two measures that reflect the use of -blockers in treating Medicare beneficiaries admitted to hospital with AMI: administration within 24 hours of arrival at the hospital and prescription at discharge. Patients eligible to receive -blockers—that is, those in the denominator—are defined as Medicare beneficiaries of any age admitted to an acute care hospital with AMI (discharge ICD code 410.xx) and discharged alive. Excluded from the denominator are patients who have received initial treatment for AMI and are readmitted for further observation, evaluation, or additional treatment within 8 weeks (ICD code 410.x2), and patients with any of the following which the Medicare program has considered a contraindication to -blockers: allergy to -blocker, chronic heart failure (CHF), shock, chronic obstructive pulmonary disease (COPD), asthma, or peripheral vascular disease (ICD codes 440.21, 440.22, 440.23, or 440.24). According to CMS criteria, a pulse of less than 60 on admission, when the patient was not on a -blocker prior to admission, excludes patients from the admission indicator; a last pulse of less than 50 excludes patients from the discharge indicator. Similarly, systolic blood pressure of less than 100 mm Hg on arrival excludes patients from the admission indicator and last systolic blood pressure in the same range excludes patients from the discharge indicator. Also excluded from the denominator are patients enrolled in a Medicare+Choice plan and patients transferred from another acute care hospital or emergency department. The numerator includes patients in the denominator who are prescribed a -blocker within 24 hours of arrival at the hospital (for the admission indicator) or upon discharge (for the discharge indicator).
Using the EMR to assess performance
We sought a process of using electronic data that would match the QIO’s manual process as closely as possible. To assess use of -blockers and to match the QIO’s inclusion and exclusion criteria, we used electronic data pertaining to vital signs, discharge ICD codes, laboratory test results, administration of drugs in the hospital, and discharge orders. Because an exact time of admission is not recorded in the data system, we used the dispensation of -blockers on the day of admission or the following day as a proxy for dispensation within 24 hours of the admitting time.
We extracted the relevant factors from the data system, including information about demographic data, drugs, diagnoses, and vital signs for the period of interest. We searched the pharmacy data files for occurrences of all -blockers (acebutolol, atenolol, bisoprolol, carvedilol, esmolol, labetalol, metoprolol, nadolol, penbutolol, pindolol, propranolol, sotalol, and timolol) administered non-topically. We included all available prescription drug data pertaining to eligible participants, separating inpatient from outpatient data according to whether dates of drug administration were included in periods of hospitalization.
Manual chart review by the QIO
To provide a measure of comparison against a known "gold standard", we manually reviewed a sample of paper records. We sought to simulate a usual manual review by the QIO. To review paper charts and assess the use of -blockers after AMI, the QIO typically requests a sample of charts from a hospital in their region. The hospital is responsible for copying and delivering the charts to the QIO, as described above. At Wishard Memorial Hospital, providing a complete record of the encounter on paper for the QIO requires special printing of some electronic data, such as laboratory test results, which are not routinely stored in paper form. QIO staff then review seven indicators of quality of care including use of -blockers in AMI.
We used a computer generated random number sequence to identify, for manual review, a random sample of 170 hospitalizations (just over 40% of all cases) for AMI. The QIO requested the charts from the hospital whose staff copied the charts and delivered them to the QIO. Upon receipt of records, two QIO staff members proceeded to review the charts for use of -blockers and for any contraindications, according to the schematic shown in fig 1. As shown, duplicate reviews of many charts were included to assess reliability between the two reviewers. Although most charts were reviewed only for use of -blockers, a subsample was also reviewed for the seven standard quality indicators in case reviewing more indicators led to different results for -blockers than when only the -blockers were assessed. The number of charts selected for review allowed us to detect meaningful differences between reviewers and types of review. One hundred and forty of the charts underwent primary review for the two indicators pertaining to -blockers.
Analysis of participants and comparison of manual and electronic methods
The basic characteristics of patients hospitalized for AMI were reported. To assess whether prescription was associated with patients’ characteristics, we compared cases in which -blockers were prescribed with those in which they were not prescribed (regardless of eligibility), with both hospitalization and participant as the unit of analysis. 2 or t tests were used to compare participants’ characteristics according to whether -blockers were prescribed. To compare manual and electronic methods among the subgroup that underwent manual review we created a series of 2x2 tables indicating whether the methods showed that -blockers were prescribed. Using manual review as a standard, we assessed the sensitivity and specificity of the electronic method. A kappa statistic was calculated for each comparison.
The university’s institutional review board approved the study.
RESULTS
Cases of AMI and a subgroup eligible to receive -blockers
We identified 15 610 Medicare covered hospital admissions during 1995–99 corresponding to 7251 patients. Among 4490 adults aged 65 years or more, 391 (4%) of their 9018 hospital admissions contained a primary or secondary discharge ICD code for AMI.
Electronic chart review
Table 1 shows the demographic characteristics of the participants, whether they had CHF or COPD, and whether they received -blockers according to electronic review. Electronic review of all cases of AMI revealed that physicians prescribed -blockers in 259 (66%) of the 391 hospital admissions at any time during the admission. Ignoring exclusion criteria, -blockers were prescribed at admission in 218 (56%) of the 391 hospitalizations for AMI and at discharge in 176 (45%). Compared with those who did not receive -blockers during their first admission for AMI, those who did were significantly less likely (p<0.05) to die in hospital or have COPD.
Manual chart review
Of 140 charts that underwent primary analysis in comparing manual with electronic review, 27 were not reviewed. Of these, 25 (18% of all 140 cases) had an unconfirmed diagnosis of AMI on manual review; the other two had died, were discharged, or transferred on the date of admission. Of the 113 cases remaining, one was not reviewed for the admission indicator and 25 were not reviewed for the discharge indicator because of death (N = 22) or transfer to another acute care facility. Of the 113 cases reviewed, 78 (70%) of 112 (18+94) met the exclusion criteria for the admission indicator and 52 (59%) of 88 (21+67) were excluded from the discharge indicator. Of cases eligible for the indicator, a -blocker was given to 28 (82%) of 34 (2+32) on admission and to 32 (89%) of 36 (3+33) at discharge. The two QIO reviewers always agreed independently about whether -blockers should be administered or met the exclusion criteria.
Manual versus electronic review
Cases reviewed using manual and electronic methods are compared in table 2 using manual review as the standard for comparison. The sensitivity of electronic review ranged from 83% to 100% and the specificity from 17% to 75%. When electronic review indicated that an exclusion criterion was present, manual review often did not (19 cases on admission; 24 at discharge).
Re-review of discrepancies between methods
Cases in which the QIO review indicated exclusion but RMRS indicated none were reviewed individually. For exclusion on admission there were 22 disagreements between QIO and RMRS. The reasons given by the QIO for exclusion of three cases were bifascicular block on electrocardiogram, history of COPD, and shock on arrival. Of the remaining 19 disagreements, diagnosis of CHF was a cause of disagreement in 10. In two cases, re-review acknowledged an initial QIO error due to overlooking a diagnosis of CHF, asthma, or shock. For exclusion from the discharge indicator there were 33 disagreements between QIO and RMRS. The reasons given by the QIO for exclusion of nine cases were hypotension or shock, bifascicular block on electrocardiogram, history of COPD, or CHF. Of the remaining 24 disagreements, CHF was a cause of disagreement in 18. In eight cases, re-review acknowledged an initial QIO error due to erroneous assessment of pulse rate or diagnosis of CHF.
Validity of ICD code for CHF
Of the 391 hospital admissions for AMI, an ICD code for CHF was present in the index admission in 209 (53%) and a code for CHF was present any time during the study period in an additional 26 (7%). Because this large number of hospital admissions met Medicare’s exclusion criteria for administration of -blockers, we assessed validity of the ICD code for CHF by comparing ICD codes with echocardiography results documented during hospital admission for AMI (table 3). Of 146 hospital admissions for AMI in which echocardiography was performed and an ICD code for CHF was present, 111 (15+96; 76%) had an echocardiogram consistent with systolic heart failure, defined as a cardiologist’s recording of left ventricular systolic dysfunction or fractional shortening of 0.2 or less. Fractional shortening is a measure made from a two-dimensional echocardiogram and represents the percentage of shortening of the diameter of the left ventricle during the cardiac cycle. The fractional shortening is calculated as the difference between left ventricular end diastolic and end systolic dimensions, divided by the end diastolic dimension. It may be considered the two-dimensional analogue of the ejection fraction.
DISCUSSION
This study of AMI among older adults in an urban public hospital over 5 years reveals for the first time that assessment of use of -blockers using an electronic medical record is comparable to manual review, has high sensitivity, and has advantages over manual methods. Electronic review also caught the prescriptions of several hospitalized patients that were not detected by manual review, raising the possibility of improved accuracy through the use of valid electronic systems. Since the institution’s EMR captures information about all medications ordered or dispensed through the pharmacies, a manual process for identifying routine administration of -blockers could not be better than the electronic review. The manual method used here might even overestimate an average of similar methods used elsewhere, since paper charts in our hospital contain not only individual drug orders but also printed summaries which may facilitate manual chart review.
Despite the recognized therapeutic benefit of -blockers, including decreased morbidity and mortality in patients with CHF35–37 and decreased mortality,1,2 costs,38 magnitude of infarction, and incidence of complications and morbidity in patients with AMI,39 these drugs are underused internationally.40–42 According to a study by Jencks et al11 in which a systematic random sample of 850 or fewer records per US state were reviewed, US Medicare beneficiaries with AMI received -blockers within 24 hours in only about 64% of 1998–99 hospital admissions (judged by median state). Use of critical pathways in treating AMI has led to disappointing results,43 and even a multifaceted intervention incorporating guideline oriented tools had modest results.44 Our study results are consistent with the notion that, regardless of the criteria that Medicare has considered exclusive, overall death rates are lower when -blockers are administered. In the US, prescribing has been especially low for female,45 indigent, or African-American patients46 and by non-internists and physicians in rural, southern, or western US regions.47,48 Krumholz, Bates, and colleagues have highlighted the need to use informatics and real-time decision support to improve quality of care in this area.49,50 Computers can be exploited more aggressively to improve the quality of health care in hospitals and clinics.
Integrating performance measurement into EMRs could reduce errors and limitations in reporting that occur through human review. We identified many instances in which reviewers made errors in reviewing charts, primarily by overlooking diagnoses or related inclusion or exclusion criteria. CHF, one of US Medicare’s exclusion criteria for administration of -blockers, is a prime example, especially since half of the 101 hospital admissions without any diagnosis of CHF but with echocardiographic testing showed echocardiographic results consistent with CHF. It is worth pointing out that the converse situation—that is, the diagnosis of CHF without echocardiographic evidence of systolic dysfunction—might usually be explained by diastolic dysfunction not easily appreciated by echocardiography. In any case, the related errors occurring through manual review of these records may be caused by fatigue, inattention, difficult to read handwriting in charts, buried information, inconsistent construction of charts, or missing data. Electronic assessment of performance indicators may overcome difficulties with accuracy or inconsistency found among reviews by QIOs.51
A primary advantage of electronic review is more accurate, systematic, and efficient ability to identify chronic disease by easily including cumulative assessments of archived data. In the case of CHF as an exclusion criterion for prescription of -blockers, Medicare defines CHF based only on the index hospital admission. This is sensible from the standpoint of a manual review. CHF, however, was often coded in an earlier hospital admission instead of the index admission. The ability of EMR to detect a prior diagnosis is evident in its low specificity, which is misleading since it found CHF in many cases where the QIO did not; in other words, most of these cases represent false negatives (undetected CHF) under manual review. Thus, the information based power of automated systems will itself narrow the gap between factors that are measurable and those that are targeted by medical evidence, leading to more accurate assessments of performance. Answering clinical questions in more than superficial ways, of course, requires rich data systems.
Rapid developments and changes in clinical guidelines demand easily modifiable information systems. Medicare guidelines indicating that patients with CHF should not receive -blockers were created some 20 years ago and remain "valid" with respect to the performance indicator for -blockers, despite many recent years of medical evidence to the contrary. Using Medicare guidelines, most participants in this study were ultimately excluded due to presence of CHF, regardless of whether electronic or manual review is performed. A review from all acute care hospitals in Alabama, Connecticut, Iowa, and Wisconsin also showed that 70% of patients with AMI in 1992–3 met exclusion criteria for administration of -blockers.52 Many of these participants may have had only mild left ventricular systolic failure—when -blockers may not pose a threat. Indeed, others have also shown that patients with AMI and relative contraindications to these drugs, including those with mild asthma or COPD,53 are still likely to benefit from -blocker therapy.54–57 Automated assessments of performance measurement can more readily facilitate changes to clinical guidelines.
Key messages
Real time performance improvement can benefit from timely and accurate feedback about previous performance.
Manual copying and auditing of paper charts is expensive and time consuming.
Electronic review is directly comparable to manual chart review in assessing prescription of -blockers following acute myocardial infarction.
Electronic review can efficiently summarize medications, chronic diseases, and exclusion criteria for certain performance indicators, especially as documented from previous medical encounters.
ICD coding in AMI is especially prone to error, probably due to "rule out" cases. Iezzoni noted that AMI did not occur in up to 42% of hospital admissions with ICD codes that indicated AMI; almost a third of cases had suffered the infarction 5–8 weeks before admission.58,59 Others have found the false positive codes to occur in only 12% of cases.60 In our study, chart review suggested that AMI may not have occurred in 18% of hospital admissions with ICD codes indicating AMI. Although manual review was used as the standard with which electronic review was compared, this study makes clear that each method has unique advantages—although the electronic method appears to have the edge. When rich clinical electronic data—such as those reflecting bedside vital signs—are available, electronic performance assessment will become more accurate through use of robust diagnostic rules that can automatically cross check and verify diagnoses using multiple criteria. With an appropriately designed data system we expect that electronic review would serve as a desirable and valid proxy for manual review.
Although we examined all cases of AMI in the study period, the sample is small and manual chart reviews may yield different results elsewhere. Nevertheless, development of an automated system has produced transportable guidelines for data handling. We have assessed only a single performance indicator rather than a large series, to provide an example of how computers can be used to design automated features that can lead to timely feedback about population based performance.
In conclusion, EMRs may be used in place of expensive, time consuming manual reviews to assess the use of -blockers in patients admitted to hospital with AMI. We have provided evidence suggesting that appropriately designed computer systems can generate data and reports that should serve as a new gold standard in efforts to measure and improve performance. Compared with manual methods, electronic systems may more readily identify prescribed medications and the presence of some chronic diseases such as CHF. The hospital whose records were studied is now securely providing continuous, real time, electronic data about performance directly to CMS through the Internet in a pilot demonstration of a National Health Information Infrastructure.61 All countries can consider the importance of creating regional and national networks for exchanging real time digital data about quality of care. The next steps should involve broader study of computer based performance measures, adaptation of existing guidelines in developing portable modules for integration into disparate EMRs, and studying the uses of real time, population based performance feedback in clinical practice.
ACKNOWLEDGEMENTS
The analyses upon which this publication is based were performed under Contract Number 500-99-IN03 entitled "Utilization and Quality Control Peer Review Organization for the State of Indiana" sponsored by the Centers for Medicare & Medicaid Services, Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government.
FOOTNOTES
This work has been supported by Regenstrief Institute Inc, Health Care Excel Inc, and Centers for Medicare and Medicaid Services. Dr Weiner is supported by grant number 5K23AG020088-02 from the National Institute on Aging (US). Dr Callahan is supported by NIA grant number AG 00868.
The authors have no conflicts of interest in this work.
The authors assume full responsibility for the accuracy and completeness of the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by the Centers for Medicare & Medicaid Services which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of this contractor. Ideas and contributions to the author concerning experience in engaging with issues presented are welcomed.
REFERENCES
Chen J, Radford MJ, Wang Y, et al. Are beta-blockers effective in elderly patients who undergo coronary revascularization after acute myocardial infarction Arch Intern Med 2000;160:947–52.
Avezum A, Tsuyuki RT, Pogue J, et al. Beta-blocker therapy for congestive heart failure: a systemic overview and critical appraisal of the published trials. Can J Cardiol 1998;14:1045–53.
Hussey PS, Anderson GF, Osborn R, et al. How does the quality of care compare in five countries Health Aff (Millwood) 2004;23:89–99.
Joint Commission on Accreditation of Healthcare Organizations. Measurement information form, performance measure identifier 14232 (AMI-5): beta blocker prescribed at discharge. 2002.
Joint Commission on Accreditation of Healthcare Organizations. Measurement information form, performance measure identifier 14234 (AMI-6): beta blocker at arrival. 2002.
Joint Commission on Accreditation of Healthcare Organizations. Overview of the acute myocardial infarction (AMI) core measure set. 2002.
National Committee for Quality Assurance. HEDIS 2000: technical specifications. Washington, DC (NCQA item #10235-100-00), 1999.
Underwood P, Beck P. Secondary prevention following myocardial infarction: evidence from an audit in South Wales that the National Service Framework for coronary heart disease does not address all the issues. Qual Saf Health Care 2002;11:230–2.
Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001.
DiSalvo TG, Normand SL, Hauptman PJ, et al. Pitfalls in assessing the quality of care for patients with cardiovascular disease. Am J Med 2001;111:297–303.
Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to Medicare beneficiaries: a profile at state and national levels. JAMA 2000;284:1670–6.
Anon. More hospitals voluntarily report quality data. Caring for the ages 2004:65.
The Leapfrog Group. Survey results. 2003.
Dexter PR, Perkins S, Overhage JM, et al. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med 2001;345:965–70.
Overhage JM, Tierney WM, Zhou XH, et al. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Informatics Assoc 1997;4:364–75.
McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976;295:1351–5.
Dexter PR, Wolinsky FD, Gramelspacher GP, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med 1998;128:102–10.
Tierney WM, McDonald CJ, Martin DK, et al. Computerized display of past test results. Effect on outpatient testing. Ann Intern Med 1987;107:569–74.
McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med 1984;100:130–8.
Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care 1986;24:659–66.
Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Informatics Assoc 1999;6:313–21.
Weiner M, Callahan CM, Tierney WM, et al. Using information technology to improve the healthcare of older adults. Ann Intern Med 2003;139:430–6.
Berwick DM. Developing and testing changes in delivery of care. Ann Intern Med 1998;128:651–6.
Medicare Payment Advisory Commission. Improving quality through Medicare payment policy. Washington, DC: Medicare Payment Advisory Commission, 2004.
Epstein AM, Lee TH, Hamel MB. Paying physicians for high-quality care. N Engl J Med 2004;350:406–10.
Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies and an analysis. Health Aff (Millwood) 2003;22:17–30.
Marshall M, Smith P. Rewarding results: using financial incentives to improve quality. Qual Saf Health Care 2003;12:397–8.
Medicare Payment Advisory Commission. Report to the Congress: applying quality improvement standards in Medicare. Washington, DC: Medicare Payment Advisory Commission, 2002.
Department of Health & Human Services. HHS launches new efforts to promote paperless health care system. 2003.
McDonald CJ, Overhage JM, Suico JG, et al. The Regenstrief Medical Record System 2000: expanding the breadth and depth of a community wide EMR. Los Angeles, CA: AMIA 2000 Annual Symposium, 2000.
McDonald CJ, Blevins L, Tierney WM, et al. The Regenstrief medical records. MD Comput 1988;5:34–47.
McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Informatics 1999;54:225–53.
American Medical Association. International classification of diseases, 9th revision, clinical modification (1996). Dover, DE: American Medical Association, 1995.
Centers for Disease Control and Prevention. Classifications of diseases and functioning and disability. 2002.
Goldstein S . Benefits of beta-blocker therapy for heart failure: weighing the evidence. Arch Intern Med 2002;162:641–8.
Brophy JM, Joseph L, Rouleau JL. Beta-blockers in congestive heart failure. A Bayesian meta-analysis. Ann Intern Med 2001;134:550–60.
Foody JM, Farrell MH, Krumholz HM. Beta-blocker therapy in heart failure: scientific review. JAMA 2002;287:883–9.
Phillips KA, Shlipak MG, Coxson P, et al. Health and economic benefits of increased beta-blocker use following myocardial infarction. JAMA 2000;284:2748–54.
Ryan TJ, Anderson JL, Antman EM, et al. ACC/AHA guidelines for the management of patients with acute myocardial infarction. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Management of Acute Myocardial Infarction). J Am Coll Cardiol 1996;28:1328–428.
Krumholz HM, Radford MJ, Wang Y, et al. National use and effectiveness of beta-blockers for the treatment of elderly patients after acute myocardial infarction: National Cooperative Cardiovascular Project. JAMA 1998;280:623–9.
Krumholz HM, Radford MJ, Wang Y, et al. Early beta-blocker therapy for acute myocardial infarction in elderly patients. Ann Intern Med 1999;131:648–54.
Brotons C, Permanyer G, Pacheco V, et al. Prophylactic treatment after myocardial infarction in primary care: how far can we go Family Practice 2003;20:32–5.
Holmboe ES, Meehan TP, Radford MJ, et al. Use of critical pathways to improve the care of patients with acute myocardial infarction. Am J Med 1999;107:324–31.
Mehta RH, Montoye CK, Gallogly M, et al. Improving quality of care for acute myocardial infarction: The Guidelines Applied in Practice (GAP) Initiative. JAMA 2002;287:1269–76.
Sial SH, Malone M, Freeman JL, et al. Beta blocker use in the treatment of community hospital patients discharged after myocardial infarction. J Gen Intern Med 1994;9:599–605.
Rathore SS, Berger AK, Weinfurt KP, et al. Race, sex, poverty, and the medical treatment of acute myocardial infarction in the elderly. Circulation 2000;102:642–8.
Fehrenbach SN, Budnitz DS, Gazmararian JA, et al. Physician characteristics and the initiation of beta-adrenergic blocking agent therapy after acute myocardial infarction in a managed care population. Am J Manag Care 2001;7:717–23.
Sheikh K, Bullock C. Urban-rural differences in the quality of care for Medicare patients with acute myocardial infarction. Arch Intern Med 2001;161:737–43.
Krumholz HM. Key challenges in measuring quality of care: what needs to be done Am J Med 2001;111:324–5.
Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Informatics Assoc 2001;8:299–308.
Rubin HR, Rogers WH, Kahn KL, et al. Watching the doctor-watchers. How well do peer review organization methods detect hospital care quality problems JAMA 1992;267:2349–54.
Ellerbeck EF, Jencks SF, Radford MJ, et al. Quality of care for Medicare patients with acute myocardial infarction. A four-state pilot study from the Cooperative Cardiovascular Project. JAMA 1995;273:1509–14.
Chen J, Radford MJ, Wang Y, et al. Effectiveness of beta-blocker therapy after acute myocardial infarction in elderly patients with chronic obstructive pulmonary disease or asthma. J Am Coll Cardiol 2001;37:1950–6.
Radford MJ, Krumholz HM. Beta-blockers after myocardial infarction—for few patients, or many N Engl J Med 1998;339:551–3.
Gottlieb SS, McCarter RJ, Vogel RA. Effect of beta-blockade on mortality among high-risk and low-risk patients after myocardial infarction. N Engl J Med 1998;339:489–97.
Farrell MH, Foody JM, Krumholz HM. Beta-blockers in heart failure: clinical applications. JAMA 2002;287:890–7.
Howard PA, Ellerbeck EF. Optimizing beta-blocker use after myocardial infarction. Am Fam Physician 2000;62:1853–60, 18656.
Iezzoni LI. Using administrative diagnostic data to assess the quality of hospital care. Pitfalls and potential of ICD-9-CM. Int J Technol Assess Health Care 1990;6:272–81.
Iezzoni LI, Burnside S, Sickles L, et al. Coding of acute myocardial infarction. Clinical and policy implications. Ann Intern Med 1988;109:745–51.
Pladevall M, Goff DC, Nichaman MZ, et al. An assessment of the validity of ICD code 410 to identify hospital admissions for myocardial infarction: The Corpus Christi Heart Project. Int J Epidemiol 1996;25:948–52.
eHealth Initiative. Healthcare Collaboration Network: a national demonstration project. 2003.(M Weiner, T E Stump C M C)
2 Regenstrief Institute Inc, Indianapolis, Indiana, USA
3 Department of Medicine, Indiana University, Indianapolis, Indiana, USA
4 Health Care Excel Inc, Medicare Quality Improvement Organization, Indianapolis, Indiana, USA
ABSTRACT
Objective: Electronic medical records seldom integrate performance indicators into daily operations. Assessing quality indicators traditionally requires resource intensive chart reviews of small samples. We sought to use an electronic medical record to assess use of -adrenergic antagonist medications (-blockers) following myocardial infarction, to compare a standardized manual assessment with assessment using electronic medical records, and to discuss potential for future integration of performance indicators into electronic records.
Design: Cross-sectional data analysis.
Setting: An urban academic medical center.
Participants: US Medicare beneficiaries 65 years of age or older, admitted to hospital with myocardial infarction between 1995 and 1999.
Measurements and main results: Manual chart review was compared with a computer driven assessment of electronic records. Administration of -blockers and cases excluded from use of -blockers were measured, based on Medicare criteria. Among 4490 older adults, 391 (4%) of 9018 hospital admissions contained codes for myocardial infarction. In 323 (83%) of the 391 hospital admissions, criteria for excluding -blockers were met; 235 (60%) were excluded due to heart failure. Of 68 hospital admissions for myocardial infarction that did not meet exclusion criteria, physicians prescribed -blockers in 49 (72%) on admission and 42 (62%) at discharge. Compared with manual chart review, electronic review had a sensitivity of 83–100% and led to fewer false negative findings.
Conclusions: An electronic medical records system can be used instead of chart review to measure use of -blockers after myocardial infarction. This should lead to integration of real time automated performance measurement into electronic medical records.
Abbreviations: AMI, acute myocardial infarction; CHF, chronic heart failure; COPD, chronic obstructive pulmonary disease; EMR, electronic medical record; QIO, Quality Improvement Organizations
Keywords: electronic medical records; acute myocardial infarction; beta-blockers; performance measures
In many countries objective assessments of quality of medical care show deficiencies in numerous areas such as preventive care and care of cardiovascular diseases, diabetes mellitus, and mental illness. Acute myocardial infarction (AMI) has received special attention, especially due to available interventions that prolong survival.1,2 In a comparison of data from Australia, Canada, New Zealand, England, and the United States, Hussey and colleagues reported that 30 day in-hospital case fatality rates for AMI are highest in Canada where they exceed the rate in Australia by about 50%, regardless of age.3 In the US, major organizations involved with accreditation4–6 and national quality assurance7 include among important measures of quality of health care the use of -adrenergic receptor antagonists (-blockers) in treating AMI. A recently published report from South Wales, UK showed that only 58% of patients admitted for AMI were discharged on -blockers.8 Although a fraction of the other 42% met exclusion criteria, some exclusion criteria are questionable and have been debated. In any case, suboptimal quality of care9 poses challenges for all health systems.
To develop programs and policies to improve health care, medical institutions and clinicians need timely and accurate data about the quality of care that they are providing. Much of the available data comes in the form of reports generated from research rather than from routines in daily practice. In many countries assessment of performance occurs by manual chart review. In the US, for example, government contracted Quality Improvement Organizations (QIOs) help to assess and promote quality for many patients such as nearly all those over 65 years of age who qualify for Medicare national health insurance (box 1). The QIOs assess quality indicators by requesting a sample of individual paper charts from health institutions and manually reviewing each chart in the sample. In a recent 3 year period about 8000 charts per state have been reviewed in a national assessment of two samples (4000 charts per state per sample), and many QIOs perform additional reviews to increase sample size for feedback to hospitals. Although the QIO reimburses hospitals for copying medical records for these audits, a hospital’s expenditure for copying often exceeds the reimbursement provided. The total cost of copying and reviewing 8000 charts manually is estimated at more than US$10 million. With such a burden, the degree to which an individual chart can be evaluated is limited10 and national assessments of state level indicators may be possible only every few years.11 Two thirds of US hospitals paid by the two largest government insurance programs—Medicare and Medicaid—do not report any quality measures.12 As a result, applying performance data at the point of care—a major goal of efforts at improving quality—is often impossible.
Box 1 US Medicare
Federal health insurance program created under the Social Security Act of 1965, originally to ensure access to health care for retired older Americans.
Covers nearly all older Americans and now also certain younger people with disabilities or end-stage renal disease.
Covers hospital care, home health care, hospice care, and up to 100 days in a skilled nursing facility without additional premiums; supplemental coverage for outpatient care, physicians’ services, and durable medical equipment can be purchased separately.
Program defines several measures of quality, including prescription of -blockers following acute myocardial infarction (AMI).
Contracts with Quality Improvement Organizations for aspects of administration, as well as assessment and improvement of quality of care.
Uses manual chart reviews of national samples to assess quality.
The need for better management of health information9,13 prompts new methods of collecting and using data about performance. Computerization of medical records offers many advantages not only for care of individual patients but for care of populations. For example, interactive electronic medical records (EMR) can prompt clinicians automatically with reminders about care and flags of potentially adverse outcomes. These forms of active decision support have also been proved to change physicians’ practices and improve health outcomes.14–22 Leaders of medical organizations who receive information from these systems can then also take proactive approaches in the timely development of programs to improve quality of care,23 including providing financial or other incentives based directly on performance.24–26 The government of the UK has already implemented a national plan to reward performance27 and, in the US, the Medicare Prescription Drug, Improvement and Modernization Act of 2003 increases payments to hospitals that publicly report data about quality.
The problems associated with assessing quality of health care on a large scale beg for more affordable and smarter uses of EMRs. Institutions with computer based approaches to quality improvement still provide a minority of health care. Finding inexpensive, efficient, and accurate ways for individual medical institutions to assess their own quality indicators may lower governmental and institutional costs and lead to more local programs that can directly improve quality of care.28 The US Secretary of the Department of Health and Human Services has recently commissioned the design of a freely distributable standardized model of an EMR.29 In the meantime, vendors that create and sell commercial EMRs compete in a narrow marketplace by providing divergent, proprietary products. Many vendors would welcome but lack standardized approaches to automating performance measures, while a clear solution to the problem requires direct integration of performance measures into these products. We sought to use an EMR to assess the use of -adrenergic antagonist medications (-blockers) following myocardial infarction, to compare a standardized manual assessment with assessment using EMR, and discuss the potential for future integration of performance indicators into electronic records.
METHODS
Electronic medical records (EMRs)
The main analysis was conducted using EMRs of the Regenstrief Medical Records System (RMRS).30–32 RMRS registers and stores data for all patients with inpatient or outpatient encounters in Wishard Health Services, an urban public medical institution of Marion County, Indiana. It captures inpatient and outpatient registrations, scheduling, clinicians’ orders, diagnoses, notes, documentation of allergies, laboratory results, electrocardiograms, radiology reports and images, and charges. Like many pharmacy systems nationwide, information about all medications ordered or dispensed through the pharmacies is recorded electronically. Medications prescribed but not administered at discharge are also recorded in this system. Adjacent to approximately half of the hospital’s beds are machines that automatically capture blood pressure and pulse.
Defining cases of AMI and a subgroup eligible to receive -blockers
After a patient’s discharge, hospital personnel identify major events, procedures, and diagnoses, along with appropriate diagnostic codes from the International Classification of Diseases (ICD), 9th Revision, Clinical Modification,33,34 which is universal to all hospitals. In this study only cases of AMI were reviewed. To identify patients discharged with AMI we screened the ICD codes of all Medicare covered hospital admissions between January 1995 and December 1999. AMI was defined to occur if a primary or secondary discharge ICD code of 410.x (AMI), 410.x0 (episode unspecified), or 410.x1 (initial episode) was identified. Excluded were hospital admissions for patients under 65 years of age and those with a discharge ICD code of 410.x2 (subsequent episode) or 411.0 (post-infarction syndrome).
The Centers for Medicare & Medicaid Services (CMS) defines two measures that reflect the use of -blockers in treating Medicare beneficiaries admitted to hospital with AMI: administration within 24 hours of arrival at the hospital and prescription at discharge. Patients eligible to receive -blockers—that is, those in the denominator—are defined as Medicare beneficiaries of any age admitted to an acute care hospital with AMI (discharge ICD code 410.xx) and discharged alive. Excluded from the denominator are patients who have received initial treatment for AMI and are readmitted for further observation, evaluation, or additional treatment within 8 weeks (ICD code 410.x2), and patients with any of the following which the Medicare program has considered a contraindication to -blockers: allergy to -blocker, chronic heart failure (CHF), shock, chronic obstructive pulmonary disease (COPD), asthma, or peripheral vascular disease (ICD codes 440.21, 440.22, 440.23, or 440.24). According to CMS criteria, a pulse of less than 60 on admission, when the patient was not on a -blocker prior to admission, excludes patients from the admission indicator; a last pulse of less than 50 excludes patients from the discharge indicator. Similarly, systolic blood pressure of less than 100 mm Hg on arrival excludes patients from the admission indicator and last systolic blood pressure in the same range excludes patients from the discharge indicator. Also excluded from the denominator are patients enrolled in a Medicare+Choice plan and patients transferred from another acute care hospital or emergency department. The numerator includes patients in the denominator who are prescribed a -blocker within 24 hours of arrival at the hospital (for the admission indicator) or upon discharge (for the discharge indicator).
Using the EMR to assess performance
We sought a process of using electronic data that would match the QIO’s manual process as closely as possible. To assess use of -blockers and to match the QIO’s inclusion and exclusion criteria, we used electronic data pertaining to vital signs, discharge ICD codes, laboratory test results, administration of drugs in the hospital, and discharge orders. Because an exact time of admission is not recorded in the data system, we used the dispensation of -blockers on the day of admission or the following day as a proxy for dispensation within 24 hours of the admitting time.
We extracted the relevant factors from the data system, including information about demographic data, drugs, diagnoses, and vital signs for the period of interest. We searched the pharmacy data files for occurrences of all -blockers (acebutolol, atenolol, bisoprolol, carvedilol, esmolol, labetalol, metoprolol, nadolol, penbutolol, pindolol, propranolol, sotalol, and timolol) administered non-topically. We included all available prescription drug data pertaining to eligible participants, separating inpatient from outpatient data according to whether dates of drug administration were included in periods of hospitalization.
Manual chart review by the QIO
To provide a measure of comparison against a known "gold standard", we manually reviewed a sample of paper records. We sought to simulate a usual manual review by the QIO. To review paper charts and assess the use of -blockers after AMI, the QIO typically requests a sample of charts from a hospital in their region. The hospital is responsible for copying and delivering the charts to the QIO, as described above. At Wishard Memorial Hospital, providing a complete record of the encounter on paper for the QIO requires special printing of some electronic data, such as laboratory test results, which are not routinely stored in paper form. QIO staff then review seven indicators of quality of care including use of -blockers in AMI.
We used a computer generated random number sequence to identify, for manual review, a random sample of 170 hospitalizations (just over 40% of all cases) for AMI. The QIO requested the charts from the hospital whose staff copied the charts and delivered them to the QIO. Upon receipt of records, two QIO staff members proceeded to review the charts for use of -blockers and for any contraindications, according to the schematic shown in fig 1. As shown, duplicate reviews of many charts were included to assess reliability between the two reviewers. Although most charts were reviewed only for use of -blockers, a subsample was also reviewed for the seven standard quality indicators in case reviewing more indicators led to different results for -blockers than when only the -blockers were assessed. The number of charts selected for review allowed us to detect meaningful differences between reviewers and types of review. One hundred and forty of the charts underwent primary review for the two indicators pertaining to -blockers.
Analysis of participants and comparison of manual and electronic methods
The basic characteristics of patients hospitalized for AMI were reported. To assess whether prescription was associated with patients’ characteristics, we compared cases in which -blockers were prescribed with those in which they were not prescribed (regardless of eligibility), with both hospitalization and participant as the unit of analysis. 2 or t tests were used to compare participants’ characteristics according to whether -blockers were prescribed. To compare manual and electronic methods among the subgroup that underwent manual review we created a series of 2x2 tables indicating whether the methods showed that -blockers were prescribed. Using manual review as a standard, we assessed the sensitivity and specificity of the electronic method. A kappa statistic was calculated for each comparison.
The university’s institutional review board approved the study.
RESULTS
Cases of AMI and a subgroup eligible to receive -blockers
We identified 15 610 Medicare covered hospital admissions during 1995–99 corresponding to 7251 patients. Among 4490 adults aged 65 years or more, 391 (4%) of their 9018 hospital admissions contained a primary or secondary discharge ICD code for AMI.
Electronic chart review
Table 1 shows the demographic characteristics of the participants, whether they had CHF or COPD, and whether they received -blockers according to electronic review. Electronic review of all cases of AMI revealed that physicians prescribed -blockers in 259 (66%) of the 391 hospital admissions at any time during the admission. Ignoring exclusion criteria, -blockers were prescribed at admission in 218 (56%) of the 391 hospitalizations for AMI and at discharge in 176 (45%). Compared with those who did not receive -blockers during their first admission for AMI, those who did were significantly less likely (p<0.05) to die in hospital or have COPD.
Manual chart review
Of 140 charts that underwent primary analysis in comparing manual with electronic review, 27 were not reviewed. Of these, 25 (18% of all 140 cases) had an unconfirmed diagnosis of AMI on manual review; the other two had died, were discharged, or transferred on the date of admission. Of the 113 cases remaining, one was not reviewed for the admission indicator and 25 were not reviewed for the discharge indicator because of death (N = 22) or transfer to another acute care facility. Of the 113 cases reviewed, 78 (70%) of 112 (18+94) met the exclusion criteria for the admission indicator and 52 (59%) of 88 (21+67) were excluded from the discharge indicator. Of cases eligible for the indicator, a -blocker was given to 28 (82%) of 34 (2+32) on admission and to 32 (89%) of 36 (3+33) at discharge. The two QIO reviewers always agreed independently about whether -blockers should be administered or met the exclusion criteria.
Manual versus electronic review
Cases reviewed using manual and electronic methods are compared in table 2 using manual review as the standard for comparison. The sensitivity of electronic review ranged from 83% to 100% and the specificity from 17% to 75%. When electronic review indicated that an exclusion criterion was present, manual review often did not (19 cases on admission; 24 at discharge).
Re-review of discrepancies between methods
Cases in which the QIO review indicated exclusion but RMRS indicated none were reviewed individually. For exclusion on admission there were 22 disagreements between QIO and RMRS. The reasons given by the QIO for exclusion of three cases were bifascicular block on electrocardiogram, history of COPD, and shock on arrival. Of the remaining 19 disagreements, diagnosis of CHF was a cause of disagreement in 10. In two cases, re-review acknowledged an initial QIO error due to overlooking a diagnosis of CHF, asthma, or shock. For exclusion from the discharge indicator there were 33 disagreements between QIO and RMRS. The reasons given by the QIO for exclusion of nine cases were hypotension or shock, bifascicular block on electrocardiogram, history of COPD, or CHF. Of the remaining 24 disagreements, CHF was a cause of disagreement in 18. In eight cases, re-review acknowledged an initial QIO error due to erroneous assessment of pulse rate or diagnosis of CHF.
Validity of ICD code for CHF
Of the 391 hospital admissions for AMI, an ICD code for CHF was present in the index admission in 209 (53%) and a code for CHF was present any time during the study period in an additional 26 (7%). Because this large number of hospital admissions met Medicare’s exclusion criteria for administration of -blockers, we assessed validity of the ICD code for CHF by comparing ICD codes with echocardiography results documented during hospital admission for AMI (table 3). Of 146 hospital admissions for AMI in which echocardiography was performed and an ICD code for CHF was present, 111 (15+96; 76%) had an echocardiogram consistent with systolic heart failure, defined as a cardiologist’s recording of left ventricular systolic dysfunction or fractional shortening of 0.2 or less. Fractional shortening is a measure made from a two-dimensional echocardiogram and represents the percentage of shortening of the diameter of the left ventricle during the cardiac cycle. The fractional shortening is calculated as the difference between left ventricular end diastolic and end systolic dimensions, divided by the end diastolic dimension. It may be considered the two-dimensional analogue of the ejection fraction.
DISCUSSION
This study of AMI among older adults in an urban public hospital over 5 years reveals for the first time that assessment of use of -blockers using an electronic medical record is comparable to manual review, has high sensitivity, and has advantages over manual methods. Electronic review also caught the prescriptions of several hospitalized patients that were not detected by manual review, raising the possibility of improved accuracy through the use of valid electronic systems. Since the institution’s EMR captures information about all medications ordered or dispensed through the pharmacies, a manual process for identifying routine administration of -blockers could not be better than the electronic review. The manual method used here might even overestimate an average of similar methods used elsewhere, since paper charts in our hospital contain not only individual drug orders but also printed summaries which may facilitate manual chart review.
Despite the recognized therapeutic benefit of -blockers, including decreased morbidity and mortality in patients with CHF35–37 and decreased mortality,1,2 costs,38 magnitude of infarction, and incidence of complications and morbidity in patients with AMI,39 these drugs are underused internationally.40–42 According to a study by Jencks et al11 in which a systematic random sample of 850 or fewer records per US state were reviewed, US Medicare beneficiaries with AMI received -blockers within 24 hours in only about 64% of 1998–99 hospital admissions (judged by median state). Use of critical pathways in treating AMI has led to disappointing results,43 and even a multifaceted intervention incorporating guideline oriented tools had modest results.44 Our study results are consistent with the notion that, regardless of the criteria that Medicare has considered exclusive, overall death rates are lower when -blockers are administered. In the US, prescribing has been especially low for female,45 indigent, or African-American patients46 and by non-internists and physicians in rural, southern, or western US regions.47,48 Krumholz, Bates, and colleagues have highlighted the need to use informatics and real-time decision support to improve quality of care in this area.49,50 Computers can be exploited more aggressively to improve the quality of health care in hospitals and clinics.
Integrating performance measurement into EMRs could reduce errors and limitations in reporting that occur through human review. We identified many instances in which reviewers made errors in reviewing charts, primarily by overlooking diagnoses or related inclusion or exclusion criteria. CHF, one of US Medicare’s exclusion criteria for administration of -blockers, is a prime example, especially since half of the 101 hospital admissions without any diagnosis of CHF but with echocardiographic testing showed echocardiographic results consistent with CHF. It is worth pointing out that the converse situation—that is, the diagnosis of CHF without echocardiographic evidence of systolic dysfunction—might usually be explained by diastolic dysfunction not easily appreciated by echocardiography. In any case, the related errors occurring through manual review of these records may be caused by fatigue, inattention, difficult to read handwriting in charts, buried information, inconsistent construction of charts, or missing data. Electronic assessment of performance indicators may overcome difficulties with accuracy or inconsistency found among reviews by QIOs.51
A primary advantage of electronic review is more accurate, systematic, and efficient ability to identify chronic disease by easily including cumulative assessments of archived data. In the case of CHF as an exclusion criterion for prescription of -blockers, Medicare defines CHF based only on the index hospital admission. This is sensible from the standpoint of a manual review. CHF, however, was often coded in an earlier hospital admission instead of the index admission. The ability of EMR to detect a prior diagnosis is evident in its low specificity, which is misleading since it found CHF in many cases where the QIO did not; in other words, most of these cases represent false negatives (undetected CHF) under manual review. Thus, the information based power of automated systems will itself narrow the gap between factors that are measurable and those that are targeted by medical evidence, leading to more accurate assessments of performance. Answering clinical questions in more than superficial ways, of course, requires rich data systems.
Rapid developments and changes in clinical guidelines demand easily modifiable information systems. Medicare guidelines indicating that patients with CHF should not receive -blockers were created some 20 years ago and remain "valid" with respect to the performance indicator for -blockers, despite many recent years of medical evidence to the contrary. Using Medicare guidelines, most participants in this study were ultimately excluded due to presence of CHF, regardless of whether electronic or manual review is performed. A review from all acute care hospitals in Alabama, Connecticut, Iowa, and Wisconsin also showed that 70% of patients with AMI in 1992–3 met exclusion criteria for administration of -blockers.52 Many of these participants may have had only mild left ventricular systolic failure—when -blockers may not pose a threat. Indeed, others have also shown that patients with AMI and relative contraindications to these drugs, including those with mild asthma or COPD,53 are still likely to benefit from -blocker therapy.54–57 Automated assessments of performance measurement can more readily facilitate changes to clinical guidelines.
Key messages
Real time performance improvement can benefit from timely and accurate feedback about previous performance.
Manual copying and auditing of paper charts is expensive and time consuming.
Electronic review is directly comparable to manual chart review in assessing prescription of -blockers following acute myocardial infarction.
Electronic review can efficiently summarize medications, chronic diseases, and exclusion criteria for certain performance indicators, especially as documented from previous medical encounters.
ICD coding in AMI is especially prone to error, probably due to "rule out" cases. Iezzoni noted that AMI did not occur in up to 42% of hospital admissions with ICD codes that indicated AMI; almost a third of cases had suffered the infarction 5–8 weeks before admission.58,59 Others have found the false positive codes to occur in only 12% of cases.60 In our study, chart review suggested that AMI may not have occurred in 18% of hospital admissions with ICD codes indicating AMI. Although manual review was used as the standard with which electronic review was compared, this study makes clear that each method has unique advantages—although the electronic method appears to have the edge. When rich clinical electronic data—such as those reflecting bedside vital signs—are available, electronic performance assessment will become more accurate through use of robust diagnostic rules that can automatically cross check and verify diagnoses using multiple criteria. With an appropriately designed data system we expect that electronic review would serve as a desirable and valid proxy for manual review.
Although we examined all cases of AMI in the study period, the sample is small and manual chart reviews may yield different results elsewhere. Nevertheless, development of an automated system has produced transportable guidelines for data handling. We have assessed only a single performance indicator rather than a large series, to provide an example of how computers can be used to design automated features that can lead to timely feedback about population based performance.
In conclusion, EMRs may be used in place of expensive, time consuming manual reviews to assess the use of -blockers in patients admitted to hospital with AMI. We have provided evidence suggesting that appropriately designed computer systems can generate data and reports that should serve as a new gold standard in efforts to measure and improve performance. Compared with manual methods, electronic systems may more readily identify prescribed medications and the presence of some chronic diseases such as CHF. The hospital whose records were studied is now securely providing continuous, real time, electronic data about performance directly to CMS through the Internet in a pilot demonstration of a National Health Information Infrastructure.61 All countries can consider the importance of creating regional and national networks for exchanging real time digital data about quality of care. The next steps should involve broader study of computer based performance measures, adaptation of existing guidelines in developing portable modules for integration into disparate EMRs, and studying the uses of real time, population based performance feedback in clinical practice.
ACKNOWLEDGEMENTS
The analyses upon which this publication is based were performed under Contract Number 500-99-IN03 entitled "Utilization and Quality Control Peer Review Organization for the State of Indiana" sponsored by the Centers for Medicare & Medicaid Services, Department of Health and Human Services. The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does mention of trade names, commercial products, or organizations imply endorsement by the US Government.
FOOTNOTES
This work has been supported by Regenstrief Institute Inc, Health Care Excel Inc, and Centers for Medicare and Medicaid Services. Dr Weiner is supported by grant number 5K23AG020088-02 from the National Institute on Aging (US). Dr Callahan is supported by NIA grant number AG 00868.
The authors have no conflicts of interest in this work.
The authors assume full responsibility for the accuracy and completeness of the ideas presented. This article is a direct result of the Health Care Quality Improvement Program initiated by the Centers for Medicare & Medicaid Services which has encouraged identification of quality improvement projects derived from analysis of patterns of care, and therefore required no special funding on the part of this contractor. Ideas and contributions to the author concerning experience in engaging with issues presented are welcomed.
REFERENCES
Chen J, Radford MJ, Wang Y, et al. Are beta-blockers effective in elderly patients who undergo coronary revascularization after acute myocardial infarction Arch Intern Med 2000;160:947–52.
Avezum A, Tsuyuki RT, Pogue J, et al. Beta-blocker therapy for congestive heart failure: a systemic overview and critical appraisal of the published trials. Can J Cardiol 1998;14:1045–53.
Hussey PS, Anderson GF, Osborn R, et al. How does the quality of care compare in five countries Health Aff (Millwood) 2004;23:89–99.
Joint Commission on Accreditation of Healthcare Organizations. Measurement information form, performance measure identifier 14232 (AMI-5): beta blocker prescribed at discharge. 2002.
Joint Commission on Accreditation of Healthcare Organizations. Measurement information form, performance measure identifier 14234 (AMI-6): beta blocker at arrival. 2002.
Joint Commission on Accreditation of Healthcare Organizations. Overview of the acute myocardial infarction (AMI) core measure set. 2002.
National Committee for Quality Assurance. HEDIS 2000: technical specifications. Washington, DC (NCQA item #10235-100-00), 1999.
Underwood P, Beck P. Secondary prevention following myocardial infarction: evidence from an audit in South Wales that the National Service Framework for coronary heart disease does not address all the issues. Qual Saf Health Care 2002;11:230–2.
Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: National Academy Press, 2001.
DiSalvo TG, Normand SL, Hauptman PJ, et al. Pitfalls in assessing the quality of care for patients with cardiovascular disease. Am J Med 2001;111:297–303.
Jencks SF, Cuerdon T, Burwen DR, et al. Quality of medical care delivered to Medicare beneficiaries: a profile at state and national levels. JAMA 2000;284:1670–6.
Anon. More hospitals voluntarily report quality data. Caring for the ages 2004:65.
The Leapfrog Group. Survey results. 2003.
Dexter PR, Perkins S, Overhage JM, et al. A computerized reminder system to increase the use of preventive care for hospitalized patients. N Engl J Med 2001;345:965–70.
Overhage JM, Tierney WM, Zhou XH, et al. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Informatics Assoc 1997;4:364–75.
McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976;295:1351–5.
Dexter PR, Wolinsky FD, Gramelspacher GP, et al. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med 1998;128:102–10.
Tierney WM, McDonald CJ, Martin DK, et al. Computerized display of past test results. Effect on outpatient testing. Ann Intern Med 1987;107:569–74.
McDonald CJ, Hui SL, Smith DM, et al. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med 1984;100:130–8.
Tierney WM, Hui SL, McDonald CJ. Delayed feedback of physician performance versus immediate reminders to perform preventive care. Effects on physician compliance. Med Care 1986;24:659–66.
Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Informatics Assoc 1999;6:313–21.
Weiner M, Callahan CM, Tierney WM, et al. Using information technology to improve the healthcare of older adults. Ann Intern Med 2003;139:430–6.
Berwick DM. Developing and testing changes in delivery of care. Ann Intern Med 1998;128:651–6.
Medicare Payment Advisory Commission. Improving quality through Medicare payment policy. Washington, DC: Medicare Payment Advisory Commission, 2004.
Epstein AM, Lee TH, Hamel MB. Paying physicians for high-quality care. N Engl J Med 2004;350:406–10.
Leatherman S, Berwick D, Iles D, et al. The business case for quality: case studies and an analysis. Health Aff (Millwood) 2003;22:17–30.
Marshall M, Smith P. Rewarding results: using financial incentives to improve quality. Qual Saf Health Care 2003;12:397–8.
Medicare Payment Advisory Commission. Report to the Congress: applying quality improvement standards in Medicare. Washington, DC: Medicare Payment Advisory Commission, 2002.
Department of Health & Human Services. HHS launches new efforts to promote paperless health care system. 2003.
McDonald CJ, Overhage JM, Suico JG, et al. The Regenstrief Medical Record System 2000: expanding the breadth and depth of a community wide EMR. Los Angeles, CA: AMIA 2000 Annual Symposium, 2000.
McDonald CJ, Blevins L, Tierney WM, et al. The Regenstrief medical records. MD Comput 1988;5:34–47.
McDonald CJ, Overhage JM, Tierney WM, et al. The Regenstrief Medical Record System: a quarter century experience. Int J Med Informatics 1999;54:225–53.
American Medical Association. International classification of diseases, 9th revision, clinical modification (1996). Dover, DE: American Medical Association, 1995.
Centers for Disease Control and Prevention. Classifications of diseases and functioning and disability. 2002.
Goldstein S . Benefits of beta-blocker therapy for heart failure: weighing the evidence. Arch Intern Med 2002;162:641–8.
Brophy JM, Joseph L, Rouleau JL. Beta-blockers in congestive heart failure. A Bayesian meta-analysis. Ann Intern Med 2001;134:550–60.
Foody JM, Farrell MH, Krumholz HM. Beta-blocker therapy in heart failure: scientific review. JAMA 2002;287:883–9.
Phillips KA, Shlipak MG, Coxson P, et al. Health and economic benefits of increased beta-blocker use following myocardial infarction. JAMA 2000;284:2748–54.
Ryan TJ, Anderson JL, Antman EM, et al. ACC/AHA guidelines for the management of patients with acute myocardial infarction. A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee on Management of Acute Myocardial Infarction). J Am Coll Cardiol 1996;28:1328–428.
Krumholz HM, Radford MJ, Wang Y, et al. National use and effectiveness of beta-blockers for the treatment of elderly patients after acute myocardial infarction: National Cooperative Cardiovascular Project. JAMA 1998;280:623–9.
Krumholz HM, Radford MJ, Wang Y, et al. Early beta-blocker therapy for acute myocardial infarction in elderly patients. Ann Intern Med 1999;131:648–54.
Brotons C, Permanyer G, Pacheco V, et al. Prophylactic treatment after myocardial infarction in primary care: how far can we go Family Practice 2003;20:32–5.
Holmboe ES, Meehan TP, Radford MJ, et al. Use of critical pathways to improve the care of patients with acute myocardial infarction. Am J Med 1999;107:324–31.
Mehta RH, Montoye CK, Gallogly M, et al. Improving quality of care for acute myocardial infarction: The Guidelines Applied in Practice (GAP) Initiative. JAMA 2002;287:1269–76.
Sial SH, Malone M, Freeman JL, et al. Beta blocker use in the treatment of community hospital patients discharged after myocardial infarction. J Gen Intern Med 1994;9:599–605.
Rathore SS, Berger AK, Weinfurt KP, et al. Race, sex, poverty, and the medical treatment of acute myocardial infarction in the elderly. Circulation 2000;102:642–8.
Fehrenbach SN, Budnitz DS, Gazmararian JA, et al. Physician characteristics and the initiation of beta-adrenergic blocking agent therapy after acute myocardial infarction in a managed care population. Am J Manag Care 2001;7:717–23.
Sheikh K, Bullock C. Urban-rural differences in the quality of care for Medicare patients with acute myocardial infarction. Arch Intern Med 2001;161:737–43.
Krumholz HM. Key challenges in measuring quality of care: what needs to be done Am J Med 2001;111:324–5.
Bates DW, Cohen M, Leape LL, et al. Reducing the frequency of errors in medicine using information technology. J Am Med Informatics Assoc 2001;8:299–308.
Rubin HR, Rogers WH, Kahn KL, et al. Watching the doctor-watchers. How well do peer review organization methods detect hospital care quality problems JAMA 1992;267:2349–54.
Ellerbeck EF, Jencks SF, Radford MJ, et al. Quality of care for Medicare patients with acute myocardial infarction. A four-state pilot study from the Cooperative Cardiovascular Project. JAMA 1995;273:1509–14.
Chen J, Radford MJ, Wang Y, et al. Effectiveness of beta-blocker therapy after acute myocardial infarction in elderly patients with chronic obstructive pulmonary disease or asthma. J Am Coll Cardiol 2001;37:1950–6.
Radford MJ, Krumholz HM. Beta-blockers after myocardial infarction—for few patients, or many N Engl J Med 1998;339:551–3.
Gottlieb SS, McCarter RJ, Vogel RA. Effect of beta-blockade on mortality among high-risk and low-risk patients after myocardial infarction. N Engl J Med 1998;339:489–97.
Farrell MH, Foody JM, Krumholz HM. Beta-blockers in heart failure: clinical applications. JAMA 2002;287:890–7.
Howard PA, Ellerbeck EF. Optimizing beta-blocker use after myocardial infarction. Am Fam Physician 2000;62:1853–60, 18656.
Iezzoni LI. Using administrative diagnostic data to assess the quality of hospital care. Pitfalls and potential of ICD-9-CM. Int J Technol Assess Health Care 1990;6:272–81.
Iezzoni LI, Burnside S, Sickles L, et al. Coding of acute myocardial infarction. Clinical and policy implications. Ann Intern Med 1988;109:745–51.
Pladevall M, Goff DC, Nichaman MZ, et al. An assessment of the validity of ICD code 410 to identify hospital admissions for myocardial infarction: The Corpus Christi Heart Project. Int J Epidemiol 1996;25:948–52.
eHealth Initiative. Healthcare Collaboration Network: a national demonstration project. 2003.(M Weiner, T E Stump C M C)