Assessing and scaling the knowledge pyramid: the good-guideline guide
http://www.100md.com
《加拿大医疗协会学报》
Physicians and other health care professionals are increasingly exposed to an overwhelming amount of medical literature. Thousands of primary research studies form the base of an enormous pyramid of knowledge, unscalable by the individual clinician, however industrious, critical and self-directed.1,2,3,4
Recognizing the need for a trustworthy source of predigested medical literature, the Cochrane Collaborative5,6 has used robust methodologies to develop systematic reviews of selected topic areas. Furthermore, Haynes7 has called for extending the application of such distillations of best evidence by linking them to systems of care. These efforts are intended to shrink the volume of clinical reading and at the same time enhance the value and validity of that information for clinical practice.
In this process of making sense of the enormous pyramid of research studies, scientific reviews, consensus documents and other papers in the published literature, where do clinical practice guidelines (CPGs) fit? More importantly, to what extent can we be assured of their validity and freedom from bias — commercial or otherwise?
CPGs take the next step of translating evidence into practice recommendations. They have been defined as "systematically developed statements to assist practitioners and patient decisions about appropriate health care for specific circumstances."8 CPGs can be valuable instruments in the hands of clinicians, if they in fact reflect "best" evidence and are useful in enhancing the effectiveness or efficiency of clinical practice.
Intuitively, the clinician understands that some guidelines will be better than others. Some factors in this judgment are objective, such as the quality of the evidence that supports the recommendations. Other factors are subjective, such as the perspectives that the authors of CPGs bring to bear. Although these subjective factors may seem to harm the validity of a guideline, they actually provide the experienced human judgment necessary to make decisions easier at the practice level. Understanding that subjectivity should not be eliminated from the guideline development process,9 organizations such as the Canadian Task Force on Preventive Health Care and the US Preventive Services Task Force have identified and tried to minimize factors that might lead to bias in the recommendation development process.
The Appraisal of Guidelines, Research and Evaluation (AGREE) Collaboration10 has created and validated tools for clinicians to rate guidelines according to their process of development by identifying the factors that are considered important in judging their quality. Within the AGREE instrument, the reader of a guideline is asked to consider, among other things, the scope and purpose of the guideline, including its objectives and the patient population to whom the guideline will be applied; the involvement of all relevant stakeholders necessary to the development of the guidelines; the format used to present the guideline, including clarity of the wording of its recommendations; and the applicability of the guideline in the setting in which its use is being reviewed, including organizational and cost barriers to implementation.
The threat of bias is addressed by the AGREE instrument in 2 additional domains: the rigour and the editorial independence of the development process. The measures of rigour address such questions as, Was the literature search on which the guideline is based thorough and systematic? Are the criteria for selection of articles clearly described? How have the recommendations been formatted? Is each recommendation clearly and explicitly linked to supporting sources of evidence, and is the grade of the evidence noted? Have health benefits, side effects and risks been considered in formulating the recommendations?
The question of editorial independence, also a prominent feature of the AGREE instrument, addresses the issue of whether and how the guideline development process, and the formulation of recommendations contained in the CPG, have identified and maintained independence from funding sources and other conflicts of interest. This is an issue on which editors, guideline dissemination groups and practitioners must maintain a constant vigilance.
Still, the clinician is left with a time-consuming task. When applying an instrument such as AGREE's, it helps to have some training in critical appraisal. Guidelines, like the primary research studies and systemic reviews that have spawned them, have proliferated exponentially. For example, a literature search of MEDLINE (performed Dec. 6, 2005) for guidelines for managing patients with cholesterol problems yielded more than 300 CPGs in English published in the past decade (1995–2005) alone!
Assisting physicians with the problem of sifting and distilling the quality of numerous available guidelines has been the work of the Guidelines Advisory Committee (GAC),11 a joint initiative of the Ontario Medical Association and the Ministry of Health and Long-Term Care. For selected clinical topics relevant to clinicians, patients and the health care system, the GAC identifies, rates and endorses the best available guideline. The GAC process includes a systematic search of published guidelines, from which evidence-based CPGs are selected. This smaller list of CPGs is then reviewed by practising physicians trained in the use of the AGREE guideline assessment instrument. The GAC has adopted the AGREE instrument as a foundation for its efforts to review, endorse and summarize the recommendations of guidelines. The GAC then considers the final AGREE quality rating as well as the relevance of the guideline to the provincial practice environment before it endorses and summarizes the key messages of the guideline for Ontario physicians. In this manner, practising clinicians need not be trained in critical appraisal nor invest a great deal of time to have access to concise, evidence-based guidance.
Not all guidelines that are published document all the process steps and measures identified by the AGREE instrument, as yet. Clearer and more regular declarations are necessary that pertain to the editorial independence of the guideline development process, and to commercial bias in producing the guidelines.12,13,14 Needed as well are further critiques and refinements of guideline assessment instruments.15,16
The GAC and its partners do not suggest that its process and its use of the AGREE instrument are perfect answers to the challenges of information overload and the elimination of commercial bias in the development or endorsement of guidelines. Nevertheless, their results do provide a "leg up" in scaling the pyramid of knowledge.
@ See related articles pages 333 and 335
Footnotes
Contributors: All authors contributed equally to the conception and preparation of this commentary and support the final version.
Acknowledgements: We thank Laure Perrier, Information Specialist, Department of Continuing Education, Faculty of Medicine, University of Toronto, who carries out literature searches for the Guidelines Advisory Committee.
Competing interests: None declared.
REFERENCES
Barnett GO, Barry MJ, Robb-Nicholson C, et al. Overcoming information overload: an information system for the primary care physician. Medinfo 2004;11:273-6.
Hall A, Walton G. Information overload within the health care system: a literature review. Health Info Libr J 2004;21:102-8.
Cartwright J, de Sylva S, Glasgow M, et al. Inaccessible information is useless information: addressing the knowledge gap. J Med Pract Manage 2002;18:36-41.
Davenport TH, Glaser J. Just-in-time delivery comes to knowledge management. Harv Bus Rev 2002 Jul;80(7):107-11, 126.
Grimshaw J. So what has the Cochrane Collaboration ever done for us? A report card on the first 10 years. CMAJ 2004;171(7):747-9.
Mowatt G, Grimshaw JM, Davis DA, et al. Getting evidence into practice: the work of the Cochrane Effective Practice and Organization of Care group (EPOC). J Contin Educ Health Prof 2001;21:55-60.
Haynes RB. Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence. Evid Based Ment Health 2001;4:37-9.
c Health Service on Clinical Practice Guidelines, Institute of Medicine. Clinical practice guidelines: directions for a new program. Washington: The National Academies Press, 1990.
Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl 3):21-35.
AGREE Collaboration. Appraisal of guidelines, research, and evaluation. Available: www.agreecollaboration.org (accessed 2005 Dec 5).
Guidelines Advisory Committee. Recommended clinical practice guidelines. Available: http://gacguidelines.ca (accessed 2005 Dec 5).
Clinical practice guidelines and conflict of interest [editorial]. CMAJ 2005; 173 (11): 1297.
Abramson J, Starfield B. The effect of conflict of interest on biomedical research and clinical practice guidelines: Can we trust the evidence in evidence-based medicine? J Am Board Fam Pract 2005;18:414-8.
Choudhry NK, Stelfox HT, Detsky AS. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. JAMA 2002;287:612-7.
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines. JAMA 1999; 281: 1900-5.
Graham ID, Calder LA, Herbert PC, et al. A comparison of clinical practice guideline appraisal instruments. Int J Technol Assess Health Care 2000;18:1024-38.(Dave Davis, Valerie Palda)
Recognizing the need for a trustworthy source of predigested medical literature, the Cochrane Collaborative5,6 has used robust methodologies to develop systematic reviews of selected topic areas. Furthermore, Haynes7 has called for extending the application of such distillations of best evidence by linking them to systems of care. These efforts are intended to shrink the volume of clinical reading and at the same time enhance the value and validity of that information for clinical practice.
In this process of making sense of the enormous pyramid of research studies, scientific reviews, consensus documents and other papers in the published literature, where do clinical practice guidelines (CPGs) fit? More importantly, to what extent can we be assured of their validity and freedom from bias — commercial or otherwise?
CPGs take the next step of translating evidence into practice recommendations. They have been defined as "systematically developed statements to assist practitioners and patient decisions about appropriate health care for specific circumstances."8 CPGs can be valuable instruments in the hands of clinicians, if they in fact reflect "best" evidence and are useful in enhancing the effectiveness or efficiency of clinical practice.
Intuitively, the clinician understands that some guidelines will be better than others. Some factors in this judgment are objective, such as the quality of the evidence that supports the recommendations. Other factors are subjective, such as the perspectives that the authors of CPGs bring to bear. Although these subjective factors may seem to harm the validity of a guideline, they actually provide the experienced human judgment necessary to make decisions easier at the practice level. Understanding that subjectivity should not be eliminated from the guideline development process,9 organizations such as the Canadian Task Force on Preventive Health Care and the US Preventive Services Task Force have identified and tried to minimize factors that might lead to bias in the recommendation development process.
The Appraisal of Guidelines, Research and Evaluation (AGREE) Collaboration10 has created and validated tools for clinicians to rate guidelines according to their process of development by identifying the factors that are considered important in judging their quality. Within the AGREE instrument, the reader of a guideline is asked to consider, among other things, the scope and purpose of the guideline, including its objectives and the patient population to whom the guideline will be applied; the involvement of all relevant stakeholders necessary to the development of the guidelines; the format used to present the guideline, including clarity of the wording of its recommendations; and the applicability of the guideline in the setting in which its use is being reviewed, including organizational and cost barriers to implementation.
The threat of bias is addressed by the AGREE instrument in 2 additional domains: the rigour and the editorial independence of the development process. The measures of rigour address such questions as, Was the literature search on which the guideline is based thorough and systematic? Are the criteria for selection of articles clearly described? How have the recommendations been formatted? Is each recommendation clearly and explicitly linked to supporting sources of evidence, and is the grade of the evidence noted? Have health benefits, side effects and risks been considered in formulating the recommendations?
The question of editorial independence, also a prominent feature of the AGREE instrument, addresses the issue of whether and how the guideline development process, and the formulation of recommendations contained in the CPG, have identified and maintained independence from funding sources and other conflicts of interest. This is an issue on which editors, guideline dissemination groups and practitioners must maintain a constant vigilance.
Still, the clinician is left with a time-consuming task. When applying an instrument such as AGREE's, it helps to have some training in critical appraisal. Guidelines, like the primary research studies and systemic reviews that have spawned them, have proliferated exponentially. For example, a literature search of MEDLINE (performed Dec. 6, 2005) for guidelines for managing patients with cholesterol problems yielded more than 300 CPGs in English published in the past decade (1995–2005) alone!
Assisting physicians with the problem of sifting and distilling the quality of numerous available guidelines has been the work of the Guidelines Advisory Committee (GAC),11 a joint initiative of the Ontario Medical Association and the Ministry of Health and Long-Term Care. For selected clinical topics relevant to clinicians, patients and the health care system, the GAC identifies, rates and endorses the best available guideline. The GAC process includes a systematic search of published guidelines, from which evidence-based CPGs are selected. This smaller list of CPGs is then reviewed by practising physicians trained in the use of the AGREE guideline assessment instrument. The GAC has adopted the AGREE instrument as a foundation for its efforts to review, endorse and summarize the recommendations of guidelines. The GAC then considers the final AGREE quality rating as well as the relevance of the guideline to the provincial practice environment before it endorses and summarizes the key messages of the guideline for Ontario physicians. In this manner, practising clinicians need not be trained in critical appraisal nor invest a great deal of time to have access to concise, evidence-based guidance.
Not all guidelines that are published document all the process steps and measures identified by the AGREE instrument, as yet. Clearer and more regular declarations are necessary that pertain to the editorial independence of the guideline development process, and to commercial bias in producing the guidelines.12,13,14 Needed as well are further critiques and refinements of guideline assessment instruments.15,16
The GAC and its partners do not suggest that its process and its use of the AGREE instrument are perfect answers to the challenges of information overload and the elimination of commercial bias in the development or endorsement of guidelines. Nevertheless, their results do provide a "leg up" in scaling the pyramid of knowledge.
@ See related articles pages 333 and 335
Footnotes
Contributors: All authors contributed equally to the conception and preparation of this commentary and support the final version.
Acknowledgements: We thank Laure Perrier, Information Specialist, Department of Continuing Education, Faculty of Medicine, University of Toronto, who carries out literature searches for the Guidelines Advisory Committee.
Competing interests: None declared.
REFERENCES
Barnett GO, Barry MJ, Robb-Nicholson C, et al. Overcoming information overload: an information system for the primary care physician. Medinfo 2004;11:273-6.
Hall A, Walton G. Information overload within the health care system: a literature review. Health Info Libr J 2004;21:102-8.
Cartwright J, de Sylva S, Glasgow M, et al. Inaccessible information is useless information: addressing the knowledge gap. J Med Pract Manage 2002;18:36-41.
Davenport TH, Glaser J. Just-in-time delivery comes to knowledge management. Harv Bus Rev 2002 Jul;80(7):107-11, 126.
Grimshaw J. So what has the Cochrane Collaboration ever done for us? A report card on the first 10 years. CMAJ 2004;171(7):747-9.
Mowatt G, Grimshaw JM, Davis DA, et al. Getting evidence into practice: the work of the Cochrane Effective Practice and Organization of Care group (EPOC). J Contin Educ Health Prof 2001;21:55-60.
Haynes RB. Of studies, summaries, synopses, and systems: the "4S" evolution of services for finding current best evidence. Evid Based Ment Health 2001;4:37-9.
c Health Service on Clinical Practice Guidelines, Institute of Medicine. Clinical practice guidelines: directions for a new program. Washington: The National Academies Press, 1990.
Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20(Suppl 3):21-35.
AGREE Collaboration. Appraisal of guidelines, research, and evaluation. Available: www.agreecollaboration.org (accessed 2005 Dec 5).
Guidelines Advisory Committee. Recommended clinical practice guidelines. Available: http://gacguidelines.ca (accessed 2005 Dec 5).
Clinical practice guidelines and conflict of interest [editorial]. CMAJ 2005; 173 (11): 1297.
Abramson J, Starfield B. The effect of conflict of interest on biomedical research and clinical practice guidelines: Can we trust the evidence in evidence-based medicine? J Am Board Fam Pract 2005;18:414-8.
Choudhry NK, Stelfox HT, Detsky AS. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. JAMA 2002;287:612-7.
Shaneyfelt TM, Mayo-Smith MF, Rothwangl J. Are guidelines following guidelines? The methodological quality of clinical practice guidelines. JAMA 1999; 281: 1900-5.
Graham ID, Calder LA, Herbert PC, et al. A comparison of clinical practice guideline appraisal instruments. Int J Technol Assess Health Care 2000;18:1024-38.(Dave Davis, Valerie Palda)