Clever searching for evidence
http://www.100md.com
《英国医生杂志》
New search filters can help to find the needle in the haystack
Do people who write about and propagate evidence based medicine use its principles properly? A paper in this week's BMJ and two other recent BMJ papers report on such authors' abilities to find the best evidence.1-3 On the way, they have provided excellent search strategies, using filters they call "hedges" (as in hedging one's bets) that help to separate the wheat (scientifically strong studies of diagnosis, treatment, and systematic reviews) from the chaff (less rigorous ones) in one of the most frequently accessed medical literature databases, Medline.
Why is this important? We still often need to search large databases such as Medline to find original research data because reviews may not cover our questions, may be out of date, and may not be relevant enough to real clinical problems. Databases of primary research are staggeringly large (there are more than 12 million citations in Medline, and 7 million in Embase). Most research papers are written as communications from scientist to scientist and relatively few have immediate clinical relevance. Most of the remainder are not rigorous enough to warrant applying clinically.4 Subsequently, the proportion of useful information is very small.w1
This lack of high quality, clinically relevant research studies leads to insecurity about the results of literature searches (table). Finding evidence can often seem easy, but those searching may either be blissfully unaware that they have not found the best evidence, or might suspect that there is still better to be found.5 But when a search retrieves nothing worth while, was it because there is none, or is it that the person looking just did not find it?
Lack of security about the results of a search
Skills limitations—both perceived and real—are important barriers to effective searching.5 Efforts have been made to simplify use of the large databases by allowing searchers to search with simple statements and minimal knowledge of the technicalities of the way the database works.w2 But, until this approach has been found effective in pragmatic trials, we have to deal with some associated consequences. Simple searching using free text can give the impression that searching in the medical literature databases is easy and can be performed in the same manner as a general internet search. But this is wrong.w3 We still have to know how to formulate a search in each particular database. Even the best filters1-3 cannot compensate for a poorly structured content search strategy.
For example, in Pubmed a searcher may enter the term "Myocardial infarct*", thinking that the truncation will improve their search by retrieving articles using the terms myocardial infarction, myocardial infarctions, and myocardial infarcts. But the truncation (*) removes the automatic term mapping feature, which searches for articles using both the text word and MeSH term, reducing retrieval and potentially missing articles.
Furthermore, many clinicians do not search at all6 7 w4 or look for answers in easily located but less evidence based sources.7 8 To delay clinical activity to look for the best evidence is not even considered by most doctors. There seems to be a "90 second rule"—that is, clinicians don't even try to find information unless they think they can do it in a minute and a half.9 10 Yet, with practice and access to certain sources, this is possible during routine ward rounds.11
Searching is only one necessary skill. Whenever we run evidence based practice workshops, we are struck by the overemphasis that clinicians place on searching for and accessing evidence and their dismay at finding they also need to acquire other skills—asking an initial useful question, appraising, and applying evidence—as well.
Is using evidence in these ways too hard, and a waste of time for people who are only ever going to be amateurs, and who could refer their questions to experts? Is this analogous to pathology and medical imaging where clinicians need to know the principles of how investigations are done, but do not have to do the tests themselves?
Ideally, a literature searching service would be woven into the medical record, or made available at some point even closer to the clinical decision. Although this is still a dream for most clinicians, literature search services are available that provide evaluated and summarised evidence, and answer specific questions.12 Efforts to nationally coordinate and develop these services are currently under way in the United Kingdom.w5
Another alternative to clinicians searching is to use compendiums of the evidence. An increasing number of excellent resources is available that either synthesise primary research (such as the Cochrane Library), or summarise it (such as Clinical Evidence, Bandolier, TRIP Database Plus, and evidence based clinical guidelines). Valuable sources of ed and appraised clinically relevant primary studies (such as Best Evidence) are also available. But these ed services are not always easy to access, tend to focus on interventions rather than diagnosis or prognosis, are more prone to being out of date—and are now so prolific that they, in turn, have to be searched for and appraised.
Clinicians risk missing something important if they do not dip their fingers into the mess of searching for, finding, and using the best evidence, at least for some of their questions. Being able to weigh the evidence in one's hand, to first feel the heft of it, could lead to a greater ability to use it well. Using the evidence in processed form potentially loses so much in understanding issues of quality, the quantity of an effect, and generalisability. Finding the needle will then be only a part of the battle with the haystack.
Sharon Sanders, senior research officer
Discipline of General Practice, University of Queensland, Herston, QLD 4006, Australia (s.sanders@uq.edu.au)
Chris Del Mar, dean, Health Sciences and Medicine
Bond University, Gold Coast, QLD 4229 Australia (cdelmar@bond.edu.au)
Additional references w1-w5 are on bmj.com
Papers p 1179
Competing interests: None declared.
References
Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2005;330: 1179-82.
Haynes RB, Wilczynski NL. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2004;328: 1040-4.
Montori VM, Wilczynski NL, Morgan D, Haynes RB. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005;330: 68-73. (8 January.)
Haynes B, Glasziou P, Straus S. Advances in evidence-based information resources for clinical practice. ACP J Club 2000;132: A11-4.
Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med 2005;80: 176-82.
Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Making 1995;15: 113-9.
Green ML, Ciampi MA, Ellis PJ. Residents' medical information needs in clinic: are they being met? Am J Med 2000;109: 218-23.
Montori VM, Tabini CC, Ebbert JO. A qualitative assessment of 1st-year internal medicine residents' perceptions of evidence-based clinical decision making. Teach Learn Med 2002;14: 114-8.
Ramos K, Linscheid R, Schafer S. Real-time information-seeking behavior of residency physicians. Fam Med 2003;35: 257-60.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319: 358-61.
Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: the "evidence cart." JAMA 1998;280: 1336-8.
Del Mar CB, Silagy CA, Glasziou P, Weller D, Spinks AB, Bernath V, et al. Feasibility of an evidence-based literature search service for general practitioners. Med J Aust 2001;175: 134-7.
Do people who write about and propagate evidence based medicine use its principles properly? A paper in this week's BMJ and two other recent BMJ papers report on such authors' abilities to find the best evidence.1-3 On the way, they have provided excellent search strategies, using filters they call "hedges" (as in hedging one's bets) that help to separate the wheat (scientifically strong studies of diagnosis, treatment, and systematic reviews) from the chaff (less rigorous ones) in one of the most frequently accessed medical literature databases, Medline.
Why is this important? We still often need to search large databases such as Medline to find original research data because reviews may not cover our questions, may be out of date, and may not be relevant enough to real clinical problems. Databases of primary research are staggeringly large (there are more than 12 million citations in Medline, and 7 million in Embase). Most research papers are written as communications from scientist to scientist and relatively few have immediate clinical relevance. Most of the remainder are not rigorous enough to warrant applying clinically.4 Subsequently, the proportion of useful information is very small.w1
This lack of high quality, clinically relevant research studies leads to insecurity about the results of literature searches (table). Finding evidence can often seem easy, but those searching may either be blissfully unaware that they have not found the best evidence, or might suspect that there is still better to be found.5 But when a search retrieves nothing worth while, was it because there is none, or is it that the person looking just did not find it?
Lack of security about the results of a search
Skills limitations—both perceived and real—are important barriers to effective searching.5 Efforts have been made to simplify use of the large databases by allowing searchers to search with simple statements and minimal knowledge of the technicalities of the way the database works.w2 But, until this approach has been found effective in pragmatic trials, we have to deal with some associated consequences. Simple searching using free text can give the impression that searching in the medical literature databases is easy and can be performed in the same manner as a general internet search. But this is wrong.w3 We still have to know how to formulate a search in each particular database. Even the best filters1-3 cannot compensate for a poorly structured content search strategy.
For example, in Pubmed a searcher may enter the term "Myocardial infarct*", thinking that the truncation will improve their search by retrieving articles using the terms myocardial infarction, myocardial infarctions, and myocardial infarcts. But the truncation (*) removes the automatic term mapping feature, which searches for articles using both the text word and MeSH term, reducing retrieval and potentially missing articles.
Furthermore, many clinicians do not search at all6 7 w4 or look for answers in easily located but less evidence based sources.7 8 To delay clinical activity to look for the best evidence is not even considered by most doctors. There seems to be a "90 second rule"—that is, clinicians don't even try to find information unless they think they can do it in a minute and a half.9 10 Yet, with practice and access to certain sources, this is possible during routine ward rounds.11
Searching is only one necessary skill. Whenever we run evidence based practice workshops, we are struck by the overemphasis that clinicians place on searching for and accessing evidence and their dismay at finding they also need to acquire other skills—asking an initial useful question, appraising, and applying evidence—as well.
Is using evidence in these ways too hard, and a waste of time for people who are only ever going to be amateurs, and who could refer their questions to experts? Is this analogous to pathology and medical imaging where clinicians need to know the principles of how investigations are done, but do not have to do the tests themselves?
Ideally, a literature searching service would be woven into the medical record, or made available at some point even closer to the clinical decision. Although this is still a dream for most clinicians, literature search services are available that provide evaluated and summarised evidence, and answer specific questions.12 Efforts to nationally coordinate and develop these services are currently under way in the United Kingdom.w5
Another alternative to clinicians searching is to use compendiums of the evidence. An increasing number of excellent resources is available that either synthesise primary research (such as the Cochrane Library), or summarise it (such as Clinical Evidence, Bandolier, TRIP Database Plus, and evidence based clinical guidelines). Valuable sources of ed and appraised clinically relevant primary studies (such as Best Evidence) are also available. But these ed services are not always easy to access, tend to focus on interventions rather than diagnosis or prognosis, are more prone to being out of date—and are now so prolific that they, in turn, have to be searched for and appraised.
Clinicians risk missing something important if they do not dip their fingers into the mess of searching for, finding, and using the best evidence, at least for some of their questions. Being able to weigh the evidence in one's hand, to first feel the heft of it, could lead to a greater ability to use it well. Using the evidence in processed form potentially loses so much in understanding issues of quality, the quantity of an effect, and generalisability. Finding the needle will then be only a part of the battle with the haystack.
Sharon Sanders, senior research officer
Discipline of General Practice, University of Queensland, Herston, QLD 4006, Australia (s.sanders@uq.edu.au)
Chris Del Mar, dean, Health Sciences and Medicine
Bond University, Gold Coast, QLD 4229 Australia (cdelmar@bond.edu.au)
Additional references w1-w5 are on bmj.com
Papers p 1179
Competing interests: None declared.
References
Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2005;330: 1179-82.
Haynes RB, Wilczynski NL. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2004;328: 1040-4.
Montori VM, Wilczynski NL, Morgan D, Haynes RB. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005;330: 68-73. (8 January.)
Haynes B, Glasziou P, Straus S. Advances in evidence-based information resources for clinical practice. ACP J Club 2000;132: A11-4.
Green ML, Ruff TR. Why do residents fail to answer their clinical questions? A qualitative study of barriers to practicing evidence-based medicine. Acad Med 2005;80: 176-82.
Gorman PN, Helfand M. Information seeking in primary care: how physicians choose which clinical questions to pursue and which to leave unanswered. Med Decis Making 1995;15: 113-9.
Green ML, Ciampi MA, Ellis PJ. Residents' medical information needs in clinic: are they being met? Am J Med 2000;109: 218-23.
Montori VM, Tabini CC, Ebbert JO. A qualitative assessment of 1st-year internal medicine residents' perceptions of evidence-based clinical decision making. Teach Learn Med 2002;14: 114-8.
Ramos K, Linscheid R, Schafer S. Real-time information-seeking behavior of residency physicians. Fam Med 2003;35: 257-60.
Ely JW, Osheroff JA, Ebell MH, Bergus GR, Levy BT, Chambliss ML, et al. Analysis of questions asked by family doctors regarding patient care. BMJ 1999;319: 358-61.
Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: the "evidence cart." JAMA 1998;280: 1336-8.
Del Mar CB, Silagy CA, Glasziou P, Weller D, Spinks AB, Bernath V, et al. Feasibility of an evidence-based literature search service for general practitioners. Med J Aust 2001;175: 134-7.