The legacy of Bristol: public disclosure of individual surgeons' results
http://www.100md.com
《英国医生杂志》
1 Society of Cardiothoracic Surgeons of Great Britain and Ireland, Royal College of Surgeons of England, London WC2A 3PE, 2 The Limes, Charfield, Wotton-under-Edge GL12 8SR
Correspondence to: B Keogh Bruce.Keogh@uclh.org
Measurement of outcomes from medical or surgical interventions is part of good practice, but publication of individual doctors' results remains controversial. The authors discuss this issue in the context of cardiothoracic surgery
Introduction
A detailed analysis by the Nuffield Trust has shown that the arguments for and against publication are finely balanced.5 The reason for publication determines the way such data are presented. The two key reasons are either to facilitate patient choice or to demonstrate safety. Publishing for patient choice requires detailed, risk adjusted tables of outcome published in a comparative fashion. Publishing to indicate whether a surgeon is safe or not requires agreeing a threshold of unacceptable mortality and then showing where each individual surgeon's results lie relative to that threshold. This is analogous to the blood alcohol level test for driving—a driver is either above or below the agreed or legal limit.
The comparative cardiac surgery reporting programmes in Pennsylvania, New Jersey, and New York have been well publicised. The claims are that these systems are transparent and that in New York the associated scrutiny has resulted in a demonstrable reduction in post operative mortality.6-9 Counter claims suggest that this reduction in mortality is no greater than that seen across the rest of the United States and that in a litigious climate the data required protracted, detailed auditing and validation with the result that, when finally published three years later, the data are no longer relevant. Furthermore, there is a feeling in the US cardiac surgery community that an unintended negative consequence of public disclosure is that surgeons may be protecting their results by avoiding higher risk cases if they feel that their results are drifting into a range that might attract unnecessary yet easily avoidable scrutiny.10-13 The improvement in mortality is easy to show. The avoidance of high risk surgery is less easy to show because of the subjective and immeasurable nature of the clinical decision making process in these complex patients. This is a real irony because the evidence suggests that patients are the one group who pay little attention to these data. What they really want is an operation in a hospital close to home and as soon as possible.14-16
Although the surgeon plays an important role in surgical outcome, so does the anaesthetist, the intensive care physician, and the intensive care nurse. Surgical results are also influenced by the socioeconomic status of the local population; severity of cardiac illness; prevalence of comorbidities; threshold of referral from both the general practitioner and the cardiologist; threshold of acceptance by the surgeons; standards of anaesthesia, surgery, and intensive care; adequacy of facilities and staffing levels; attitude to training; interpersonal relationships between staff; and the geographical layout of the unit (for example, in some units the wards are so far from the theatre and intensive care unit that surgeons have no time to check up on ward patients between surgery cases). So the concept of blaming the surgeon was perceived as unfair.17
These concerns have been reflected in the decision by the Veterans Administration (the biggest US healthcare provider) to discourage the generation of surgeon specific outcomes. The administration believes the performance of a surgeon cannot be separated from that of his or her institution as quality is highly dependent on institutional systems.18 19 Others argue that it is the doctors who are best placed to change institutional processes that influence outcome and they are therefore a logical target.20
Public disclosure of hospital and surgeon specific data in other specialties has not been well publicised but will gain increasing prominence.21 22
We thought carefully about ways to present the data in the United Kingdom to avoid some of the pitfalls of the US models. We agreed we would base any risk adjusted comparative analyses on lower risk cases alone, leaving surgeons able to tackle more complex and difficult cases without unnecessary apprehension. The wisdom of this strategy was recently highlighted by a study in the BMJ confirming that risk stratification systems that may be good at predicting risk in large institutional groups of patients are much less reliable in high risk cases at the level of an individual surgeon because they tend to "under-predict" for higher risk groups. More importantly this study defined the level of predicted risk above which we should exclude patients from comparative analyses.23
The national service framework for coronary heart disease
Learning from Bristol: the report of the public inquiry into children's heart surgery at the Bristol Royal Infirmary 1984-1995. www.bristolinquiry.org.uk/final_report/index.htm (accessed 9 August 2004).
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. New Engl J Med 1996;334: 394-8.
Keogh B, Dussek J, Watson D, Magee P, Wheatley D. Public confidence and cardiac surgical outcome. BMJ 1998;316: 1759-60.
Treasure T, Utley M, Bailey A. Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study. BMJ 2003;327: 73-5.
Marshall M, Sheklle P, Brook R, Leatherman S. Dying to know: public release of information about quality of healthcare. London: Nuffield Trust and Rand, 2000.
Hannan EL, Kilburn H Jr, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York state. JAMA 1994;271: 761-6.
Hannan EL, Siu AL, Kumar D, Kilburn H Jr, Chassin MR. The decline in coronary artery bypass graft surgery mortality in New York state: the role of surgeon volume. JAMA 1995;273: 209-13.
Hannan EL, Kumar D, Racz M, Siu AL, Chassin MR. New York state's cardiac surgery reporting system: four years later. Ann Thorac Surg 1994;58: 1852-7.
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. New Engl J Med 1996;334: 394-8.
Omoigui N, Annan K, Brown K, Miller D, Cosgrove D, Loop F. Potential explanation for decreased CABG related mortality in New York state: outmigration to Ohio. Circulation 1994;90: I93.
Schneider E, Epstein A. Influence of cardiac surgery performance reports on referral practices and access to care. New Engl J Med 1996;335: 251-6.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of report cards on health care providers. National Bureau of Economic Research. (Working paper 8697.) www.nber.org/papers/w8697 (accessed 9 August 2004).
Burack J, Impellizzeri P, Homel P, Cunningham J. Public reporting of surgical mortality: a survey of New York state cardiothoracic surgeons. Ann Thorac Surg 199;68: 1195-200.
Schneider E, Epstein A. Use of public performance reports. A survey of patients undergoing cardiac surgery. JAMA 1998;279: 1638-42.
Shahian D, Yip W, Westcott G, Johnson J. Selection of a cardiac surgery provider in the managed care era. J Thorac Cardiovasc Surg 2000;120: 978-89.
Schneider EC, Lieberman T. Publicly disclosed information about the quality of healthcare: response of the US public. Qual Health Care 2001;10: 96-103.
Keogh B. Facts of life the figures can hide. Times 2001 Nov 19.
Khuri SK. Quality, advocacy, healthcare policy and the surgeon. Ann Thorac Surg 2002;74: 641-94.
Albert A, Walter J, Arnrich B, Hassanein W, Rosendahl U, Bauer S, et al. On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery. Eur J Cardiothorac Surg 2004;25: 312-9.
Chassin MR. Improving the quality of care. Part 3: improving the quality of care. N Engl J Med 1996;335: 1060-3.
Marshall M, Sheklle P, Leatherman S, Brook R. The public release of performance data. What do we expect to gain? A review of the evidence. JAMA 2000;283: 1866-74.
Hannan E, Radzyner M, Rubin D, Dougherty J, Brennan M. The influence of hospital and surgeon volume on in-hospital mortality for colectomy, gastrectomy and lung lobectomy in patients with cancer Surgery 2002;131: 6-15.
Bridgewater B, Grayson A, Jackson M, Brooks N, Grotte G, Keenan D, et al. Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ 2003;327: 13-7.
Department of Health. Coronary heart disease: national service framework for coronary heart disease: modern standards and service models. London: DoH, 2000. (Search at www.dh.gov.uk/).
Department of Health. Coronary heart disease information strategy. London: DoH, 2001. www.dh.gov.uk/PolicyAndGuidance/InformationPolic y/InformationSupportingNSFAndNCASP/InformationPolic yCoronaryHeartDisease/fs/en?CONTENT_ID=4015656&chk=HUnWUQ (accessed 9 August 2004).
National clinical audit support programme: support for national service frameworks and clinical govenance. www.dh.gov.uk/assetRoot/04/05/95/61/04059561.pdf (accessed 9 August 2004).
Fine L, Keogh B, Orlando M, Cretin S, Gould M. Improving the credibility of information on healthcare outcomes. London: Nuffield Trust, 2003.
Fine L, Keogh B, Cretin S, Orlando M, Gould M. How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience. BMJ 2003;326: 25-8.
Sergeant P, Blackstone E, Meyns B. Validation and interdependence with patient-variables of the influence of procedural variables on early and late survival after CABG. KU Leuven coronary surgery program. Eur J Cardiothorac Surg 1997;12: 1-19.
Healthcare Commission. Clinical indicator. Deaths following a heart bypass operation. 2004. http://ratings.healthcarecommission.org.uk/indicators_2004/trust/indicato r/indicatordescriptionshort.asp?indicatorid=1400 (accessed 9 August 2004).
Shahian D, Normand S Torchiana D, Lewis SM, Pastore JO, Kuntz RE, et al. Cardiac surgery report cards; comprehensive review and statistical critique. Ann Thorac Surg 2001;72: 2155-68.
Iezzoni L. The risks of risk adjustment. JAMA 1997;278: 1600-11.(Bruce Keogh, president el)
Correspondence to: B Keogh Bruce.Keogh@uclh.org
Measurement of outcomes from medical or surgical interventions is part of good practice, but publication of individual doctors' results remains controversial. The authors discuss this issue in the context of cardiothoracic surgery
Introduction
A detailed analysis by the Nuffield Trust has shown that the arguments for and against publication are finely balanced.5 The reason for publication determines the way such data are presented. The two key reasons are either to facilitate patient choice or to demonstrate safety. Publishing for patient choice requires detailed, risk adjusted tables of outcome published in a comparative fashion. Publishing to indicate whether a surgeon is safe or not requires agreeing a threshold of unacceptable mortality and then showing where each individual surgeon's results lie relative to that threshold. This is analogous to the blood alcohol level test for driving—a driver is either above or below the agreed or legal limit.
The comparative cardiac surgery reporting programmes in Pennsylvania, New Jersey, and New York have been well publicised. The claims are that these systems are transparent and that in New York the associated scrutiny has resulted in a demonstrable reduction in post operative mortality.6-9 Counter claims suggest that this reduction in mortality is no greater than that seen across the rest of the United States and that in a litigious climate the data required protracted, detailed auditing and validation with the result that, when finally published three years later, the data are no longer relevant. Furthermore, there is a feeling in the US cardiac surgery community that an unintended negative consequence of public disclosure is that surgeons may be protecting their results by avoiding higher risk cases if they feel that their results are drifting into a range that might attract unnecessary yet easily avoidable scrutiny.10-13 The improvement in mortality is easy to show. The avoidance of high risk surgery is less easy to show because of the subjective and immeasurable nature of the clinical decision making process in these complex patients. This is a real irony because the evidence suggests that patients are the one group who pay little attention to these data. What they really want is an operation in a hospital close to home and as soon as possible.14-16
Although the surgeon plays an important role in surgical outcome, so does the anaesthetist, the intensive care physician, and the intensive care nurse. Surgical results are also influenced by the socioeconomic status of the local population; severity of cardiac illness; prevalence of comorbidities; threshold of referral from both the general practitioner and the cardiologist; threshold of acceptance by the surgeons; standards of anaesthesia, surgery, and intensive care; adequacy of facilities and staffing levels; attitude to training; interpersonal relationships between staff; and the geographical layout of the unit (for example, in some units the wards are so far from the theatre and intensive care unit that surgeons have no time to check up on ward patients between surgery cases). So the concept of blaming the surgeon was perceived as unfair.17
These concerns have been reflected in the decision by the Veterans Administration (the biggest US healthcare provider) to discourage the generation of surgeon specific outcomes. The administration believes the performance of a surgeon cannot be separated from that of his or her institution as quality is highly dependent on institutional systems.18 19 Others argue that it is the doctors who are best placed to change institutional processes that influence outcome and they are therefore a logical target.20
Public disclosure of hospital and surgeon specific data in other specialties has not been well publicised but will gain increasing prominence.21 22
We thought carefully about ways to present the data in the United Kingdom to avoid some of the pitfalls of the US models. We agreed we would base any risk adjusted comparative analyses on lower risk cases alone, leaving surgeons able to tackle more complex and difficult cases without unnecessary apprehension. The wisdom of this strategy was recently highlighted by a study in the BMJ confirming that risk stratification systems that may be good at predicting risk in large institutional groups of patients are much less reliable in high risk cases at the level of an individual surgeon because they tend to "under-predict" for higher risk groups. More importantly this study defined the level of predicted risk above which we should exclude patients from comparative analyses.23
The national service framework for coronary heart disease
Learning from Bristol: the report of the public inquiry into children's heart surgery at the Bristol Royal Infirmary 1984-1995. www.bristolinquiry.org.uk/final_report/index.htm (accessed 9 August 2004).
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. New Engl J Med 1996;334: 394-8.
Keogh B, Dussek J, Watson D, Magee P, Wheatley D. Public confidence and cardiac surgical outcome. BMJ 1998;316: 1759-60.
Treasure T, Utley M, Bailey A. Assessment of whether in-hospital mortality for lobectomy is a useful standard for the quality of lung cancer surgery: retrospective study. BMJ 2003;327: 73-5.
Marshall M, Sheklle P, Brook R, Leatherman S. Dying to know: public release of information about quality of healthcare. London: Nuffield Trust and Rand, 2000.
Hannan EL, Kilburn H Jr, Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York state. JAMA 1994;271: 761-6.
Hannan EL, Siu AL, Kumar D, Kilburn H Jr, Chassin MR. The decline in coronary artery bypass graft surgery mortality in New York state: the role of surgeon volume. JAMA 1995;273: 209-13.
Hannan EL, Kumar D, Racz M, Siu AL, Chassin MR. New York state's cardiac surgery reporting system: four years later. Ann Thorac Surg 1994;58: 1852-7.
Chassin MR, Hannan EL, DeBuono BA. Benefits and hazards of reporting medical outcomes publicly. New Engl J Med 1996;334: 394-8.
Omoigui N, Annan K, Brown K, Miller D, Cosgrove D, Loop F. Potential explanation for decreased CABG related mortality in New York state: outmigration to Ohio. Circulation 1994;90: I93.
Schneider E, Epstein A. Influence of cardiac surgery performance reports on referral practices and access to care. New Engl J Med 1996;335: 251-6.
Dranove D, Kessler D, McClellan M, Satterthwaite M. Is more information better? The effects of report cards on health care providers. National Bureau of Economic Research. (Working paper 8697.) www.nber.org/papers/w8697 (accessed 9 August 2004).
Burack J, Impellizzeri P, Homel P, Cunningham J. Public reporting of surgical mortality: a survey of New York state cardiothoracic surgeons. Ann Thorac Surg 199;68: 1195-200.
Schneider E, Epstein A. Use of public performance reports. A survey of patients undergoing cardiac surgery. JAMA 1998;279: 1638-42.
Shahian D, Yip W, Westcott G, Johnson J. Selection of a cardiac surgery provider in the managed care era. J Thorac Cardiovasc Surg 2000;120: 978-89.
Schneider EC, Lieberman T. Publicly disclosed information about the quality of healthcare: response of the US public. Qual Health Care 2001;10: 96-103.
Keogh B. Facts of life the figures can hide. Times 2001 Nov 19.
Khuri SK. Quality, advocacy, healthcare policy and the surgeon. Ann Thorac Surg 2002;74: 641-94.
Albert A, Walter J, Arnrich B, Hassanein W, Rosendahl U, Bauer S, et al. On-line variable live-adjusted displays with internal and external risk-adjusted mortalities. A valuable method for benchmarking and early detection of unfavourable trends in cardiac surgery. Eur J Cardiothorac Surg 2004;25: 312-9.
Chassin MR. Improving the quality of care. Part 3: improving the quality of care. N Engl J Med 1996;335: 1060-3.
Marshall M, Sheklle P, Leatherman S, Brook R. The public release of performance data. What do we expect to gain? A review of the evidence. JAMA 2000;283: 1866-74.
Hannan E, Radzyner M, Rubin D, Dougherty J, Brennan M. The influence of hospital and surgeon volume on in-hospital mortality for colectomy, gastrectomy and lung lobectomy in patients with cancer Surgery 2002;131: 6-15.
Bridgewater B, Grayson A, Jackson M, Brooks N, Grotte G, Keenan D, et al. Surgeon specific mortality in adult cardiac surgery: comparison between crude and risk stratified data. BMJ 2003;327: 13-7.
Department of Health. Coronary heart disease: national service framework for coronary heart disease: modern standards and service models. London: DoH, 2000. (Search at www.dh.gov.uk/).
Department of Health. Coronary heart disease information strategy. London: DoH, 2001. www.dh.gov.uk/PolicyAndGuidance/InformationPolic y/InformationSupportingNSFAndNCASP/InformationPolic yCoronaryHeartDisease/fs/en?CONTENT_ID=4015656&chk=HUnWUQ (accessed 9 August 2004).
National clinical audit support programme: support for national service frameworks and clinical govenance. www.dh.gov.uk/assetRoot/04/05/95/61/04059561.pdf (accessed 9 August 2004).
Fine L, Keogh B, Orlando M, Cretin S, Gould M. Improving the credibility of information on healthcare outcomes. London: Nuffield Trust, 2003.
Fine L, Keogh B, Cretin S, Orlando M, Gould M. How to evaluate and improve the quality and credibility of an outcomes database: validation and feedback study on the UK Cardiac Surgery Experience. BMJ 2003;326: 25-8.
Sergeant P, Blackstone E, Meyns B. Validation and interdependence with patient-variables of the influence of procedural variables on early and late survival after CABG. KU Leuven coronary surgery program. Eur J Cardiothorac Surg 1997;12: 1-19.
Healthcare Commission. Clinical indicator. Deaths following a heart bypass operation. 2004. http://ratings.healthcarecommission.org.uk/indicators_2004/trust/indicato r/indicatordescriptionshort.asp?indicatorid=1400 (accessed 9 August 2004).
Shahian D, Normand S Torchiana D, Lewis SM, Pastore JO, Kuntz RE, et al. Cardiac surgery report cards; comprehensive review and statistical critique. Ann Thorac Surg 2001;72: 2155-68.
Iezzoni L. The risks of risk adjustment. JAMA 1997;278: 1600-11.(Bruce Keogh, president el)