Self regulation must be made to work
http://www.100md.com
《英国医生杂志》
1 Medical Council of Canada, PO Box 8234, Ottawa, ON, Canada, K1G 3H7 dale.dauphinee@mcc.ca
Self regulation is not magic: it requires substantive, defensible, and valid processes. The US and Canada show the importance of validation of all applications, every time they are used
Several high profile cases of poor medical practice in the United Kingdom have brought self regulation into question. Pressure for change is great, but experience from other Western countries suggests that abandoning the existing systems is not the best way forward. The United States and Canada both have well established systems of self regulation that are being gradually adapted to include a requirement for regular recertification. Although the healthcare environment in the UK is somewhat different, North America presents important lessons on how to ensure doctors are competent.
Should we abandon self regulation?
Stacey, the distinguished British sociologist, has written extensively about self regulation over the past two decades. In 1997, she concluded that given the alternatives, it is, on balance, the best option.1 The prominent American sociologist Friedson has come to similar conclusions, after having a different view several years previously.2 He argues that ideological attacks on professionalism have distracted us from the fact that we are profoundly dependent on organised bodies of specialised knowledge and technique. This assault has created a sense of distrust that further weakens the credibility of professional institutions' ability to offer an independent and moral viewpoint. Friedson suggests that the monopoly of expert authorities cannot be avoided as it is essential for nurturing specialised knowledge.
The political philosopher Onora O'Neill also raised the importance of trust in the BBC Reith lecture in 2002.3 She pointed out that it cannot be legislated or managed by numbers alone:
Intelligent accountability, I suspect, requires more attention to good governance and fewer fantasies about total control. Good governance is possible only if institutions are allowed some margin for self-governance of a form appropriate to their particular tasks, within a framework of financial and other reporting... Real accountability provides substantive and knowledgeable independent judgment of an institution's or professional's work.3
North America and Australasia use self regulation of doctors, but with greater emphasis on proving the validity of the processes used in each setting. Validity is not something to be shown once and then assumed to be inherent in that system.4 It exists only in the execution of each attempt of any approach. New Zealand has also experienced loss of trust after a healthcare disaster; the response was to redesign self regulation with new understanding and attention to processes.5
We can therefore argue that the focus in the UK should not be away from self regulation but to work towards better processes and the validation of all applications, every time they are used. The regulatory framework should state explicitly the assumptions on which the regulatory processes will be built. Ultimately, a coherent reporting system must be implemented and managed effectively.
Experience of revalidation
What is the state of revalidation elsewhere? In Canada, Australia, and the United States the constitutional frameworks delegate education and health matters, including licensure of practitioners, to states or provinces. This makes setting national standards challenging. But it does provide numerous opportunities to develop valid systems for assessment for licensure and revalidation or recertification through local mechanisms, which, as Irvine has noted, are critical to any accountability framework.6
The US and Canada both have established licensing and certifying bodies with large resources for assessment. But the healthcare environment in Canada is less competitive than in the US and more like the UK. The existence of one major employer of doctors in Canada and the UK makes it easier for these countries to integrate the employer into revalidation or other assessment process, such as licensure or specialty certification. In the US, that model is really only applicable nationwide in organisations such as the military or veterans affairs' health systems. That said, neither Canada nor the US has a nationwide revalidation or certification process. But there are national specialty specific systems in which certification is awarded for a limited time and has to be renewed.
The National Board of Medical Examiners in the early 20th century
Credit: NATIONAL LIBRARY OF MEDICINE
Box 1: Core competencies of American Board of Medical Specialties
Medical knowledge
Patient care
Interpersonal and communication skills
Professionalism
Practice based learning and improvement
Systems based practice
Progress in the US
Revalidation in the US is evolving around national assessment organisations and not licensing bodies. National assessment bodies (such as the National Board of Medical Examiners) have long been established at undergraduate and postgraduate levels. Recent developments have focused on the certifying bodies that operate independently of the medical associations and specialty societies, as well as the state licensing bodies. The inherent value that they place on assessment procedures and their existing infrastructural processes have been crucial. The benefits of the system include:
Large national assessment bodies have processes, in-house expertise, and measurement standards that are independent of professional advocacy roles at the licensure and postgraduate training levels
Major national certifying bodies operate under the umbrella of the American Board of Medical Specialties, which has promoted national standards and policies that it can use to encourage member boards to adopt new methods and measurement approaches, again independent of direct professional advocacy
Intramural expertise in assessment abounds, and in many cases there is an excellent track record of development and research to provide evidence for the initial validity of assessment processes before they are adopted by national licensing examinations and certifying bodies
Extensive attention to due process and the knowledge that the courts will intervene challenges these bodies to avoid conflicts of interest and to ensure fairness and equality of applications.
Key US agencies have established national definitions of required competencies in response to the national quality agenda.7 In turn, this has directed interest in developing better assessment methods and clear statements about what is to be assessed in recertification. The six competencies framework produced by the American Board of Medical Specialties was developed incrementally through deliberative planning over several years (box 1).8
To assess these competencies the board recommends multisource feedback and practice based assessment modules such as those used for maintenance of certification in internal medicine.7 The American Board of Internal Medicine's website provides tools for self evaluation of knowledge and performance as part of this process.9
Similarly, the recent policies of the Accreditation Council on Graduate Medical Education are also fully compatible with the six competencies framework. These policies apply to programme accreditation and the assessment of trainees, such that time limited certification and lifelong maintenance will begin with day one of postgraduate clinical education.10 These developments are analogous to those existing in Canada.11
The other interesting development in the US has been the Veterans Affairs medical network's implementation of the electronic record. The record integrates local quality of care assessments and processes for monitoring use of services. This allows national standards to be applied, including revalidation processes (Davoren JB. Annual meeting of the Society of Medical Administrators. Carefree, Arizona, 12 January, 2004).
Canada's story
Canada has also seen a stepwise evolution of revalidation, but, unlike the US, this has been driven by the licensure community rather than the certifying bodies.11 After three workshop conferences with input from all other national medical bodies in 1994-6, the Federation of Medical Regulatory Authorities of Canada developed a Canadian model of monitoring and enhancement of physician performance (box 2). Its full implementation remains the goal.11
Regional and provincial programmes to assess training needs continue to expand, with a new programme developing in Atlantic Canada. The goal of an inexpensive monitoring system to screen all doctors, however, remains more elusive. One model is Alberta's multisource feedback physician achievement programme, which is continuing to be developed by the College of Physicians and Surgeons in association with the University of Calgary measurement group.12 Although the programme is being used to screen all doctors practising in Alberta, its predictive validity and the accuracy of longer term measures of successful outcome are unproved.13
Ontario's regulatory college has recently announced plans to adopt the three step model and Quebec is also developing a programme.14 The costs are substantial, however, despite the fact that Quebec, in particular, has been able to take full advantage of existing databases to carry out part of these analyses. Until electronic record and data systems are more widely implemented, such that pre-existing data can be used without introducing redundant infrastructure costs, screening of all doctors will need to rely on rating systems such as multisource feedback.
Box 2: Canadian model for monitoring and enhancement of physician performance
Step 1—Monitoring of all doctors with feedback to focus their educational programmes
Step 2—Further assessment of doctors at some to moderate risk with face to face or supportive feedback and follow-up re-enhancement
Step 3—Detailed assessment for doctors at high risk with feedback to define "best solution" or a prescribed enhancement programme
Summary points
Recertification and maintenance of competencies in the US is being introduced only after stepwise evaluation and validation of the assessment methods
North America has a long tradition of assessment organisations that are separate from physician advocacy groups
US standards bodies have focused on key competences for training accreditation and maintenance of certification
Canada has found the cost of screening all doctors a barrier but is making progress towards valid multisource feedback as a first step procedure
Self regulation should not be abandoned
Based on these developments, will a Canadian integrated assessment network with common standards and approaches emerge, as it has in the US? Canada has drawn up a list of essential competencies (CanMEDS). The recent move of the Royal College of Physicians and Surgeons of Canada to a competency based framework using this list (N Mikhael, personal communication) and its joint initiative with the College of Family Physicians of Canada to share the competency framework is the first step to national standards.15 The Medical Council of Canada also plans to use the approach in its national assessment procedures.
Lessons for the UK
Experience in the United States and Canada shows that self regulation can be effective and maintain the public's trust. The required infrastructure and tradition of excellence in assessment have been part of the North American medical culture for decades. The separation of assessment bodies from other national bodies with advocacy roles is a major advantage for North American certifying bodies. Changes have been implemented over a long period without the current pressure in the UK to do something immediately. This has given US bodies the opportunity to win acceptance and, perhaps most importantly, to take advantage of past reputations for doing their jobs at an arm's length from the professional associations. The challenge for the UK is to avoid any focus on reactionary stances that punish rather than reward. Although the public wants change to optimise health, it will not want to commit to unproved approaches and reactionary strategies that could end up doing more harm than good.
This is the fifth article in a series examining regulation of doctors
I thank Carole White for editorial help.
Contributors and sources: WDD was a trustee on the Foundation of the American Board of Internal Medicine until June 2004 and has been a consultant on issues of maintenance of competence and continuing professional development in Australasia, Ireland, and the Caribbean. His current research activities focus on clinical outcomes and the predictive validity of assessment procedures. Information was gathered from a search of published literature, supplemented with personal knowledge of developments in the field in North America.
Competing interests: None declared.
References
Stacey M. The case for and against medical self-regulation. Fed Bull 1997;84: 17-25.
Friedson E. Professionalism reborn: theory, prophecy and policy. Cambridge: Policy Press, 1994.
O'Neill O. A question of trust. Cambridge: Cambridge University Press, 2002.
Dauphinee WD. Licensure and certification. In: Norman GF, van der Vleuten CPM, Newble DI, eds. International handbook of research in medical education. Dordrecht: Kluwer Academic, 2002: 835-82.
Paul C. Internal and external morality of medicine: lessons from New Zealand. BMJ 2000;320: 499-503.
Irvine D. The performance of doctors. II. Maintaining good practice, protecting patients from poor performance. BMJ 1997;314: 1613-5.
Brennan TA, Horowitz RI, Duffy FD, Cassel CK, Goode, LD, Lipner RS. The role of physician specialty certification status in the quality movement. JAMA 2004;292: 1038-43.
Horowitz SD, Miller SH, Miles PV. Board certification and physician quality. Med Educ 2004;38: 10-1.
American Board of Internal Medicine. Maintenance of certification. www.abim.org/moc/moc_new_knowledge.shtm (accessed 5 Apr 2005).
Leach DC. Competence is a habit. JAMA 2002;287: 243-4.
Dauphinee WD. Revalidation of doctors in Canada. BMJ 1999;319: 1188-90.
Violato C, Lockyer J, Fidler H. Multi-source feedback: a method of assessing surgical practice. BMJ 2003;326: 546-6.
Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. BMJ 2004;328:1240-3 (see full version on bmj.com).
Jacques A. Maintaining competence: a professional challenge. Bull Kuwait Inst Med Specialization 2003;2:5-10. http://www.kims.org.kw/bulletin/Issues/issue3/MaintenanceCompetence.pdf (accessed 17 May 2005).
Frank J. The CanMEDS project: the Royal College of Physicians and Surgeons of Canada moves medical education into the 21st century. In: Dinsdale H, Hurteau G, eds. The evolution of specialty medicine: 1979-2004. Ottawa: Royal College of Physicians and Surgeons of Canada, 2004: 109-24.(W Dale Dauphinee, executive director1)
Self regulation is not magic: it requires substantive, defensible, and valid processes. The US and Canada show the importance of validation of all applications, every time they are used
Several high profile cases of poor medical practice in the United Kingdom have brought self regulation into question. Pressure for change is great, but experience from other Western countries suggests that abandoning the existing systems is not the best way forward. The United States and Canada both have well established systems of self regulation that are being gradually adapted to include a requirement for regular recertification. Although the healthcare environment in the UK is somewhat different, North America presents important lessons on how to ensure doctors are competent.
Should we abandon self regulation?
Stacey, the distinguished British sociologist, has written extensively about self regulation over the past two decades. In 1997, she concluded that given the alternatives, it is, on balance, the best option.1 The prominent American sociologist Friedson has come to similar conclusions, after having a different view several years previously.2 He argues that ideological attacks on professionalism have distracted us from the fact that we are profoundly dependent on organised bodies of specialised knowledge and technique. This assault has created a sense of distrust that further weakens the credibility of professional institutions' ability to offer an independent and moral viewpoint. Friedson suggests that the monopoly of expert authorities cannot be avoided as it is essential for nurturing specialised knowledge.
The political philosopher Onora O'Neill also raised the importance of trust in the BBC Reith lecture in 2002.3 She pointed out that it cannot be legislated or managed by numbers alone:
Intelligent accountability, I suspect, requires more attention to good governance and fewer fantasies about total control. Good governance is possible only if institutions are allowed some margin for self-governance of a form appropriate to their particular tasks, within a framework of financial and other reporting... Real accountability provides substantive and knowledgeable independent judgment of an institution's or professional's work.3
North America and Australasia use self regulation of doctors, but with greater emphasis on proving the validity of the processes used in each setting. Validity is not something to be shown once and then assumed to be inherent in that system.4 It exists only in the execution of each attempt of any approach. New Zealand has also experienced loss of trust after a healthcare disaster; the response was to redesign self regulation with new understanding and attention to processes.5
We can therefore argue that the focus in the UK should not be away from self regulation but to work towards better processes and the validation of all applications, every time they are used. The regulatory framework should state explicitly the assumptions on which the regulatory processes will be built. Ultimately, a coherent reporting system must be implemented and managed effectively.
Experience of revalidation
What is the state of revalidation elsewhere? In Canada, Australia, and the United States the constitutional frameworks delegate education and health matters, including licensure of practitioners, to states or provinces. This makes setting national standards challenging. But it does provide numerous opportunities to develop valid systems for assessment for licensure and revalidation or recertification through local mechanisms, which, as Irvine has noted, are critical to any accountability framework.6
The US and Canada both have established licensing and certifying bodies with large resources for assessment. But the healthcare environment in Canada is less competitive than in the US and more like the UK. The existence of one major employer of doctors in Canada and the UK makes it easier for these countries to integrate the employer into revalidation or other assessment process, such as licensure or specialty certification. In the US, that model is really only applicable nationwide in organisations such as the military or veterans affairs' health systems. That said, neither Canada nor the US has a nationwide revalidation or certification process. But there are national specialty specific systems in which certification is awarded for a limited time and has to be renewed.
The National Board of Medical Examiners in the early 20th century
Credit: NATIONAL LIBRARY OF MEDICINE
Box 1: Core competencies of American Board of Medical Specialties
Medical knowledge
Patient care
Interpersonal and communication skills
Professionalism
Practice based learning and improvement
Systems based practice
Progress in the US
Revalidation in the US is evolving around national assessment organisations and not licensing bodies. National assessment bodies (such as the National Board of Medical Examiners) have long been established at undergraduate and postgraduate levels. Recent developments have focused on the certifying bodies that operate independently of the medical associations and specialty societies, as well as the state licensing bodies. The inherent value that they place on assessment procedures and their existing infrastructural processes have been crucial. The benefits of the system include:
Large national assessment bodies have processes, in-house expertise, and measurement standards that are independent of professional advocacy roles at the licensure and postgraduate training levels
Major national certifying bodies operate under the umbrella of the American Board of Medical Specialties, which has promoted national standards and policies that it can use to encourage member boards to adopt new methods and measurement approaches, again independent of direct professional advocacy
Intramural expertise in assessment abounds, and in many cases there is an excellent track record of development and research to provide evidence for the initial validity of assessment processes before they are adopted by national licensing examinations and certifying bodies
Extensive attention to due process and the knowledge that the courts will intervene challenges these bodies to avoid conflicts of interest and to ensure fairness and equality of applications.
Key US agencies have established national definitions of required competencies in response to the national quality agenda.7 In turn, this has directed interest in developing better assessment methods and clear statements about what is to be assessed in recertification. The six competencies framework produced by the American Board of Medical Specialties was developed incrementally through deliberative planning over several years (box 1).8
To assess these competencies the board recommends multisource feedback and practice based assessment modules such as those used for maintenance of certification in internal medicine.7 The American Board of Internal Medicine's website provides tools for self evaluation of knowledge and performance as part of this process.9
Similarly, the recent policies of the Accreditation Council on Graduate Medical Education are also fully compatible with the six competencies framework. These policies apply to programme accreditation and the assessment of trainees, such that time limited certification and lifelong maintenance will begin with day one of postgraduate clinical education.10 These developments are analogous to those existing in Canada.11
The other interesting development in the US has been the Veterans Affairs medical network's implementation of the electronic record. The record integrates local quality of care assessments and processes for monitoring use of services. This allows national standards to be applied, including revalidation processes (Davoren JB. Annual meeting of the Society of Medical Administrators. Carefree, Arizona, 12 January, 2004).
Canada's story
Canada has also seen a stepwise evolution of revalidation, but, unlike the US, this has been driven by the licensure community rather than the certifying bodies.11 After three workshop conferences with input from all other national medical bodies in 1994-6, the Federation of Medical Regulatory Authorities of Canada developed a Canadian model of monitoring and enhancement of physician performance (box 2). Its full implementation remains the goal.11
Regional and provincial programmes to assess training needs continue to expand, with a new programme developing in Atlantic Canada. The goal of an inexpensive monitoring system to screen all doctors, however, remains more elusive. One model is Alberta's multisource feedback physician achievement programme, which is continuing to be developed by the College of Physicians and Surgeons in association with the University of Calgary measurement group.12 Although the programme is being used to screen all doctors practising in Alberta, its predictive validity and the accuracy of longer term measures of successful outcome are unproved.13
Ontario's regulatory college has recently announced plans to adopt the three step model and Quebec is also developing a programme.14 The costs are substantial, however, despite the fact that Quebec, in particular, has been able to take full advantage of existing databases to carry out part of these analyses. Until electronic record and data systems are more widely implemented, such that pre-existing data can be used without introducing redundant infrastructure costs, screening of all doctors will need to rely on rating systems such as multisource feedback.
Box 2: Canadian model for monitoring and enhancement of physician performance
Step 1—Monitoring of all doctors with feedback to focus their educational programmes
Step 2—Further assessment of doctors at some to moderate risk with face to face or supportive feedback and follow-up re-enhancement
Step 3—Detailed assessment for doctors at high risk with feedback to define "best solution" or a prescribed enhancement programme
Summary points
Recertification and maintenance of competencies in the US is being introduced only after stepwise evaluation and validation of the assessment methods
North America has a long tradition of assessment organisations that are separate from physician advocacy groups
US standards bodies have focused on key competences for training accreditation and maintenance of certification
Canada has found the cost of screening all doctors a barrier but is making progress towards valid multisource feedback as a first step procedure
Self regulation should not be abandoned
Based on these developments, will a Canadian integrated assessment network with common standards and approaches emerge, as it has in the US? Canada has drawn up a list of essential competencies (CanMEDS). The recent move of the Royal College of Physicians and Surgeons of Canada to a competency based framework using this list (N Mikhael, personal communication) and its joint initiative with the College of Family Physicians of Canada to share the competency framework is the first step to national standards.15 The Medical Council of Canada also plans to use the approach in its national assessment procedures.
Lessons for the UK
Experience in the United States and Canada shows that self regulation can be effective and maintain the public's trust. The required infrastructure and tradition of excellence in assessment have been part of the North American medical culture for decades. The separation of assessment bodies from other national bodies with advocacy roles is a major advantage for North American certifying bodies. Changes have been implemented over a long period without the current pressure in the UK to do something immediately. This has given US bodies the opportunity to win acceptance and, perhaps most importantly, to take advantage of past reputations for doing their jobs at an arm's length from the professional associations. The challenge for the UK is to avoid any focus on reactionary stances that punish rather than reward. Although the public wants change to optimise health, it will not want to commit to unproved approaches and reactionary strategies that could end up doing more harm than good.
This is the fifth article in a series examining regulation of doctors
I thank Carole White for editorial help.
Contributors and sources: WDD was a trustee on the Foundation of the American Board of Internal Medicine until June 2004 and has been a consultant on issues of maintenance of competence and continuing professional development in Australasia, Ireland, and the Caribbean. His current research activities focus on clinical outcomes and the predictive validity of assessment procedures. Information was gathered from a search of published literature, supplemented with personal knowledge of developments in the field in North America.
Competing interests: None declared.
References
Stacey M. The case for and against medical self-regulation. Fed Bull 1997;84: 17-25.
Friedson E. Professionalism reborn: theory, prophecy and policy. Cambridge: Policy Press, 1994.
O'Neill O. A question of trust. Cambridge: Cambridge University Press, 2002.
Dauphinee WD. Licensure and certification. In: Norman GF, van der Vleuten CPM, Newble DI, eds. International handbook of research in medical education. Dordrecht: Kluwer Academic, 2002: 835-82.
Paul C. Internal and external morality of medicine: lessons from New Zealand. BMJ 2000;320: 499-503.
Irvine D. The performance of doctors. II. Maintaining good practice, protecting patients from poor performance. BMJ 1997;314: 1613-5.
Brennan TA, Horowitz RI, Duffy FD, Cassel CK, Goode, LD, Lipner RS. The role of physician specialty certification status in the quality movement. JAMA 2004;292: 1038-43.
Horowitz SD, Miller SH, Miles PV. Board certification and physician quality. Med Educ 2004;38: 10-1.
American Board of Internal Medicine. Maintenance of certification. www.abim.org/moc/moc_new_knowledge.shtm (accessed 5 Apr 2005).
Leach DC. Competence is a habit. JAMA 2002;287: 243-4.
Dauphinee WD. Revalidation of doctors in Canada. BMJ 1999;319: 1188-90.
Violato C, Lockyer J, Fidler H. Multi-source feedback: a method of assessing surgical practice. BMJ 2003;326: 546-6.
Evans R, Elwyn G, Edwards A. Review of instruments for peer assessment of physicians. BMJ 2004;328:1240-3 (see full version on bmj.com).
Jacques A. Maintaining competence: a professional challenge. Bull Kuwait Inst Med Specialization 2003;2:5-10. http://www.kims.org.kw/bulletin/Issues/issue3/MaintenanceCompetence.pdf (accessed 17 May 2005).
Frank J. The CanMEDS project: the Royal College of Physicians and Surgeons of Canada moves medical education into the 21st century. In: Dinsdale H, Hurteau G, eds. The evolution of specialty medicine: 1979-2004. Ottawa: Royal College of Physicians and Surgeons of Canada, 2004: 109-24.(W Dale Dauphinee, executive director1)