How to Get Started
Andrew D. Oxman, David L. Sackett, Gordon H. Guyatt and the Evidence Based Medicine Working Group
Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (1993;270(17):2093-2095). Copyright 1995, American Medical Association.
- Clinical Scenario
- Introduction
- Need for the Users' Guides Series
- Asking questions that are pertinent and answerable
- Tracking down articles
- Deciding if an article is likely to provide valid results
- Conclusion
- Guides for selecting articles that are most likely to provide valid results
- References
Clinical Scenario
You are a primary care physician inspired by a recent article about lifelong learning [1]. You decide to use some of the time you normally take for continuing medical education conferences for "practice-based education" tailored to your own practice. You begin by setting aside two hours every week to read about relevant clinical problems.
It is now Friday morning and you have two hours to spend in the hospital library. You review a one page list of questions you have generated from the patients you've seen in the prior week. Your questions include:
- What should you tell a 33 year old woman with migraine headaches who has asked for a prescription for sumatriptan after reading a magazine article about it?
- Should you be screening older men in your practice for prostate cancer?
- What should you tell the mother of a six month old boy who had a febrile seizure about his risk of developing epilepsy?
- Should you try to reduce a 25 year old asthmatic man's reliance on inhaled beta-agonists?
- What should you tell a 50 year old menopausal woman asking about hormone replacement?
Introduction
This series will help you translate the results of medical research into clinical practice. We've written them from the perspective of the busy clinician who wants to provide effective medical care but is sharply restricted in time for reading. We do not attempt a course in research methods; the series is about using, not doing, research. It is designed to help provide our patients with care that is based on the best evidence currently available - "evidence-based medicine" [2]. Evidence-based medicine emphasizes the need to move beyond clinical experience and physiological principles to rigorous evaluations of the consequences of clinical actions. Knowing how to use the clinical literature is imperative for ensuring we are providing optimal patient care.
In this section we will present a general approach to using one's clinical reading time effectively, and some specific suggestions for deciding which clinical articles to read. In subsequent sections we will go into more detail on how this approach can contribute to solving clinical problems in the treatment, prevention, diagnosis, and prognosis of disease.
Need for the Users' Guides Series
Clinical information comes from two principal sources, the individual patient and research. To provide effective care both types of information are needed. Information about the individual patient is elicited through a careful history, physical examination and other investigations. The ways in which information from scientific research is obtained by clinicians is less clear, but of no less importance to the quality of care that patients receive.
To the extent that clinicians rely on community standards or opinion leaders to guide their practice, there is an implicit assumption that their needs for scientific information are being met through these means; i.e. that community standards and the recommendations of clinical experts (opinion leaders) reflect the best available scientific information. However, the ways in which experts' opinions and "standard practice" evolve are complex [3]. Variation in clinical practice, comparisons of practice with evidence-based standards and evaluations of the recommendations of clinical experts suggest that expert opinion and "standard practice" do not provide adequate mechanisms for the transfer of scientific information into clinical decision-making [4] [5]. Expert opinion often lags far behind the evidence, and is not infrequently inconsistent with evidence [6]. This is not to say that expert opinion may not be important and useful, but it is clearly not sufficient.
In the editorial that preceded this series, we reviewed the reasons why clinicians need tools to evaluate and use the medical literature in their day-to-day clinical practice [7]. The present series is designed to fill that need.
For reasons of both logic and efficiency, we have sought uniformity in presentation of the Users' Guides by organizing each set into three basic questions:
- Are the results of the study valid?
- What are the results?
- Will the results help me in caring for my patients?
"Yes" and "no" are often not adequate answers to these questions. This may contrast with readers' intuitive approach. After all, the Users' Guides are designed to help clinicians make decisions, and most clinical decisions are black and white: for example, we either start a treatment or we do not. It is understandable, therefore, that we seek black-or-white answers from the clinical literature. The article is right or wrong; the treatment works or it does not; the results apply to my patient or they do not. Unfortunately, evidence comes in shades of grey. Often, results may be valid, perhaps demonstrate an important effect, and they might improve patient care.
The goal of the users' guides presented in this series of articles is to help clinicians sift through these shades of grey and make appropriate decisions, recognizing the "level" of certainty (or strength of inference) underlying those decisions. The first key question - "Are the results valid?" - and the last - "Will the results help me in caring for my patients?" - reflect the need to make a decision, despite the fact that the strength of the inferences that can be made based on a study span a spectrum from strong to weak. Since this is a series on how to use research in taking care of patients, not how to do research, we will focus on flaws in study design or implementation that are most likely to weaken the strength of inference in ways that seriously distort clinical decisions based on them.
In the remainder of this article, we will introduce strategies for
- framing clinical questions that are pertinent and answerable,
- tracking down articles, and
- deciding which articles to read, and which to believe.
Asking questions that are pertinent and answerable
Clinical questions arise continuously in the course of providing routine medical care, but must be clearly formulated to ensure clear answers. Most clinical questions can be formulated in terms of a simple relationship between the patient, some "exposure" (to a treatment, a diagnostic test, or a potentially harmful agent), and one or more specific outcomes of interest, as shown in the following modifications of the questions from the scenario at the beginning of this article:
- Would sumatriptan (exposure) reduce the severity of headache pain (outcome) in this woman with frequent migraine attacks (patient)? - a question of therapy.
- Would a prostate-specific antigen test (exposure), if performed in this symptomless elderly man (patient), decrease his risk of dying from prostate cancer (outcome)? - a question of secondary prevention through early diagnosis.
- Does the febrile seizure (exposure) that this six month old infant (patient) just had increase the likelihood that he will develop epilepsy (outcome)? - a question of prognosis.
- Do b-agonists (exposure) increase the risk of death (outcome) in this asthmatic man (patient)? - a question of harm.
The importance of such focused questions can be quickly assessed, and priority given to problems that are seen routinely and have practically important consequences. In general, those questions that are clearly related to a clinical decision about whether to use a therapeutic, preventive or diagnostic intervention are the ones that warrant the most time. Focusing the question clarifies the target of the literature search and permits use of the appropriate guides for assessing validity in screening the titles and abstracts of the articles that are located.
For example, the question posed in the scenario at the beginning of this article about hormone replacement, while likely to be important in most primary care practices, is not well focused. It is worthwhile to clarify the type of patient and the outcomes of interest before beginning to look for an answer. Is the woman seeking treatment for hot flashes or is she asymptomatic? If the woman is asymptomatic and is wondering if she should take estrogen to prevent osteoporosis, clinically important outcomes that might be considered include hip fracture, cardiovascular disease, breast and endometrial cancer, and vaginal bleeding. In this case, a good approach might be to start by looking for published clinical practice guidelines instead of tracking down the evidence for each outcome. Later in this series we will present guides for how to critically appraise practice guidelines.
Tracking down articles
Having posed a pertinent, answerable clinical question, one proceeds to track down the best available evidence. There are four routes for doing this: asking someone, checking reference lists in textbooks, finding a relevant article in your own reprint file, and using a bibliographic database such as MEDLINE. Asking a colleague or consultant is highly efficient, and makes most sense when the question concerns an exposure or treatment or patient you are unlikely to encounter again. If a recent textbook is at hand (published or updated within the previous year), you can follow your reading of the appropriate passage by checking the references cited by the author. Because a text book is only as up to date as its most recent reference, all are at least partly out of date even before they are published. A new type of "subscription" textbook addresses this problem by providing periodic updates, and often cites the evidence used in making its changes [8] [9]. While frequent updates help protect against being out of date, they do not ensure that the conclusions of the clinical experts writing textbook chapters are valid. Prototypes of textbooks that are based on systematic reviews of validated evidence are available for obstetrical [10] and neonatal problems [11], but most textbooks and review articles do not qualify as scientific overviews [12].
A third starting point may be an article in your personal reprint file. Since the amount of time required to maintain an up-to-date file of clinical articles is formidable, you are unlikely to have the key article at hand. New methods for retrieving the current medical literature are rendering personal filing systems nonessential, if not obsolete.
The final route, conducting electronic searches of the medical literature, is fast becoming a basic skill for practising modern, evidence-based medicine. Electronic access to MEDLINE is readily available in North America in a variety of online and CD-ROM formats. Clinicians can easily acquire the basic skills [13] and learn to retrieve the same number of relevant citations as librarians, even if their searches remain a bit messier [14]. The addition of structured abstracts to MEDLINE and the development of databases that have screened articles for their validity and clinical relevance, such as the Oxford Database of Perinatal Trials [15] and an electronic version of the ACP Journal Club, promise to make the task of retrieving information from the medical literature even easier. You can seek a review article (often the best place to start) by adding, to whatever Medical Subject Heading (MeSH) terms are used to identify the disorder and "exposure", in your MEDLINE search, the search term: REVIEW (PT) (PT stands for publication type). You are more likely to find a methodologically sound review article by using the term META-ANALYSIS (PT) instead of REVIEW. Another potential place to start is with practice guidelines, which now have their own search term: PRACTICE GUIDELINE (PT). Recruiting a librarian to help you with your first few searches may help you learn to avoid searches that are too broad and unfocused, or too narrow and thus risk missing key articles. Increasing numbers of physicians are finding that MEDLINE searches can help them solve clinical problems, and improve patient care and clinical outcomes [16].
Deciding if an article is likely to provide valid results
The first question applied to any article tracked down in an effort to find an answer for a clinical problem concerns its closeness to the truth: are the results of this article valid? Table 1 presents two key guides to assess validity for primary studies (those that provide original data on a topic) and integrative studies (those that summarize data from primary studies). For each type of integrative study, the first criterion has to do with whether the question is appropriately framed, and the second with whether the evidence was appropriately collected and summarized. The clinician can use these most important criteria to rapidly screen an abstract to determine whether it warrants the additional time required to read it in detail. The busy clinician who has tracked down a number of articles on a question can use the guides to choose the one or two papers most likely to provide a valid answer. These criteria can also be used to reduce the clinical literature to a manageable size when trying to keep up with new advances that are pertinent to one's practice. If a more detailed review of a paper's methods reveals that these "validity" guides are met, readers can turn their attention to the other guides designed to help them answer the next two key questions: what are the results and will they benefit my patient care?
Conclusion
Subsequent articles in this series will describe strategies for efficiently selecting and using each of the types of articles in Table 1. In doing so, they will describe the justification and application of guides for determining whether the results of an article are valid and applicable to the clinical decisions you must make.
Readers should be warned that the guides do not come with definitive answers. Learning to apply them can be challenging. However, it can also be extremely gratifying. More importantly, it is only by translating good evidence into good clinical decisions that we can be sure that we do more good than harm for our patients.
Table 1. Guides for selecting articles that are most likely to provide valid results
* Each of these guides makes an implicit or explicit reference to primary studies. Investigators need to evaluate the validity of the studies that they are reviewing to produce their integrative article. The validity criteria one would use in making this evaluation would depend on the area being addressed (therapy, diagnosis, prognosis or causation), and are those which are presented in the part of the table dealing with primary articles. |
References
1. Manning PR, DeBakey L. Lifelong learning tailored to individual clinical practice. JAMA. 1992 Sep 2;268(9):1135-6.
2. Evidence-Based Medicine Working Group. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA. 1992 Nov 4;268(17):2420-5.
3. Eddy DM. Clinical policies and the quality of clinical practice. N Engl J Med. 1982 Aug 5;307(6):343-7.
4. Stross JK, Harlan WR. The dissemination of new medical information. JAMA. 1979 Jun 15;241(24):2622-4.
5. Williamson JW, German PS, Weiss R, Skinner EA, Bowes F III. Health science information management and continuing education of physicians. A survey of U.S. primary care practitioners and their opinion leaders. Ann Intern Med. 1989 Jan 15;110(2):151-60.
6. Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts. Treatments for myocardial infarction. JAMA. 1992 Jul 8;268(2):240-8.
7. Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA. 1993 Nov 3;270(17):2096-7.
8. Rubenstein E, Federman D, eds. Medicine. New York: Scientific American Medicine; 1993.
9. Rubenstein E, Federman D, eds. Care of the Surgical Patient. New York: Scientific American Medicine; 1993.
10. Chalmers I, Enkin M, Keirse MJNC, eds. Evaluating the Effects of Care During Pregnancy and Childbirth. Oxford: Oxford University Press; 1989:3-38.
11. Sinclair JC, Bracken MB, eds. Effective Care of the Newborn Infant. Oxford: Oxford University Press; 1992.
12. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987 Mar;106(3):485-8.
13. Haynes RB, McKibbon KA, Fitzgerald D, Guyatt GH, Walker CJ, Sackett DL. How to keep up with the medical literature: V. Access by personal computer to the medical literature. Ann Intern Med. 1986 Nov;105(5):810-6.
14. Haynes RB, McKibbon KA, Walker CJ, Ryan N, Fitzgerald D, Ramsden MF. Online access to MEDLINE in clinical settings. A study of use and usefulness. Ann Intern Med. 1990 Jan 1;112(1):78-84.
15. Chalmers I, ed. Oxford Database of Perinatal Trials. Oxford: Oxford Electronic Publishing, Oxford University Press; 1992.
16. Lindberg DA, Siegel ER, Rapp BA, Wallingford KT, Wilson SR. Use of MEDLINE by physicians for clinical problem solving. JAMA. 1993 Jun 23-30;269(24):3124-9.
© 2001 Evidence-Based Medicine Informatics Project
