Centre for Health Evidence: Home » Users' Guides to EBP

Evidence-Based Medicine: A New Approach to Teaching the Practice of Medicine

Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (1992 Nov 4;268(17):2420-5) Copyright 1992, American Medical Association.

• Abstract
• Introduction
• A Paradigm Shift
• Requirements for the Practice of Evidence-Based Medicine
• Evidence-based Medicine in a Medical Residency
• Effective Teaching of Evidence-based Medicine
• Methods for Scaling the Barriers to the Dissemination of Evidence-based Medicine
• Barriers to Teaching Evidence-Based Medicine
• Barriers to Practising Evidence-Based Medicine
• Does teaching and learning evidence-based medicine improve patient outcomes
• Conclusion
• References

Abstract

A new paradigm for medical practice is emerging. Evidence-based medicine de-emphasizes intuition, unsystematic clinical experience, and pathophysiologic rationale as sufficient grounds for clinical decision-making, and stresses the examination of evidence from clinical research. Evidence-based medicine requires new skills of the physician, including efficient literature-searching, and the application of formal rules of evidence in evaluating the clinical literature.

An important goal of our medical residency program is to educate physicians in the practice of evidence-based medicine. Strategies include a weekly formal academic half-day for residents, devoted to learning the necessary skills; recruitment into teaching roles of physicians who practice evidence-based medicine; sharing among faculty of approaches to teaching evidence-based medicine; and providing faculty with feedback on their performance as role-models and teachers of evidence-based medicine.

The influence of evidence-based medicine on clinical practice and medical education is increasing.

Introduction

A junior medical resident working in a teaching hospital admits a 43 year old previously well man who experienced a witnessed grand mal seizure. He had never had a seizure before and had not had any recent head trauma. He drank alcohol once or twice a week and had not had alcohol on the day of the seizure. Physical examination is negative. The patient is given an loading dose of phenytoin intravenously and the drug is continued orally. A computed tomographic head scan is completely normal and an electroencephalogram shows only nonspecific findings. The patient is very concerned about his risk of seizure recurrence.

How might she proceed?

The way of the past

Faced with this situation as a clinical clerk, the resident was told by her senior resident (who was supported in his view by the attending physician) that the risk is high (though he couldn't put an exact number on it) and that was the information that should be conveyed to the patient. She now follows this path, emphasizing to the patient the need not to drive, to continue his medication, and to see his family physician in follow-up. The patient leaves in a state of vague trepidation about his risk of subsequent seizure.

The way of the future

The resident asks herself whether she knows the prognosis of a first seizure, and realizes she does not. She proceeds to the library and, using the Grateful Med program [1], conducts a computerized literature search. She uses the Medical Subject Headings (MeSH) terms "epilepsy" and "prognosis" and "recurrence" and finds 25 citations. Surveying the titles, one [2] appears most directly relevant. She reviews the paper, finds that it meets criteria she has previously learned for a valid investigation of prognosis [3], and that the results are applicable to her patient. The search cost the resident $2.68, and the entire process (including the trip to the library and the time to make a photocopy of the article) took half an hour.

The results of the relevant study show that the patients risk of recurrence at one year is between 30% and 43%, and at three years is between 51% and 60%. After a seizure-free period of 18 months his risk of recurrence would likely be under 20%. She conveys this information to the patient, along with a recommendation that he take his medication, see his family doctor regularly, and have a review of his need for medication if he remains seizure-free for 18 months. The patient leaves with a clear idea of his prognosis.

A Paradigm Shift

Thomas Kuhn has described scientific paradigms as ways of looking at the world which define both the problems which can legitimately be addressed and the range of admissible evidence which may bear on their solution [4]. When defects in an existing paradigm accumulate to the extent that the paradigm is no longer tenable, the paradigm is challenged and replaced by a new way of looking at the world. Medical practice is changing, and the change which involves using the medical literature more effectively in guiding medical practice is profound enough that it can appropriately be called a paradigm shift.

The foundations of the paradigm shift lie in developments in clinical research over the last 30 years. In 1960, the randomized clinical trial was an oddity. It is now accepted that virtually no drug can enter clinical practice without a demonstration of its efficacy in clinical trials. Moreover, the same randomized trial method increasingly is being applied to surgical therapies [5], and diagnostic tests [6]. Meta-analysis is gaining increasing acceptance as a method of summarizing the results of a number of randomized trials, and ultimately may have as profound an effect on setting treatment policy as have randomized trials themselves [7]. While less dramatic, crucial methodological advances have also been made in other areas, such as the assessment of diagnostic tests [8] [9] and prognosis [2].

A new philosophy of medical practice and teaching has followed these methodological advances. This paradigm shift is manifested in a number of ways. There has been a profusion of articles instructing clinicians on how to access [10], evaluate [11], and interpret [12] the medical literature. Proposals to apply the principles of clinical epidemiology to day-to-day clinical practice have been put forward [3]. A number of major medical journals have adopted a more informative structured abstract format which incorporates issues of methods and design into the portion of an article the reader sees first [13]. The American College of Physicians has launched a journal, ACP Journal Club, that summarizes new publications of high relevance and methodologic rigour [14]. Textbooks which provide a rigorous review of available evidence, including a methods section describing both the methodologic criteria used to systematically evaluate the validity of the clinical evidence and the quantitative techniques used for summarizing the evidence, have begun to appear [15][16]. Practice guidelines based on rigorous methodological review of the available evidence are increasingly common [17]. A final manifestation is the growing demand for courses and seminars which instruct physicians on how to make more effective use of the medical literature in their day-to-day patient care [3].

We call the newparadigm "evidence-based medicine" [18]. In this paper, we describe how this approach differs from prior practice, and briefly outline how we are building a residency program in which a key goal is to practice, role-model, teach, and help residents become highly adept in evidence-based medicine. We will also present some of the problems educators and medical practitioners face in implementing the new paradigm.

The Former Paradigm

The former paradigm was based on the following assumptions about the knowledge required to guide clinical practice.

• Unsystematic observations from clinical experience are a valid way of building and maintaining one's knowledge about patient prognosis, the value of diagnostic tests, and the efficacy of treatment.
• The study and understanding of basic mechanisms of disease and pathophysiologic principles is a sufficient guide for clinical practice.
• A combination of thorough traditional medical training and common sense is sufficient to allow one to evaluate new tests and treatment.
• Content expertise and clinical experience are a sufficient base from which to generate valid guidelines for clinical practice.

According to this paradigm clinicians have a number of options for sorting out clinical problems they face. They can reflect on their own clinical experience, reflect on the underlying biology, go to a textbook, or ask a local expert. The "Introduction" and "Discussion" sections of a paper could be considered an appropriate way of gaining the relevant information from a current journal.

It should be noted that this paradigm puts a high value on traditional scientific authority and adherence to standard approaches, and answers are frequently sought from direct contact with local experts, or reference to the writings of international experts [19].

The New Paradigm

The assumptions of the new paradigm are as follows.

• Clinical experience, and the development of clinical instincts (particularly with respect to diagnosis), are crucial and necessary parts of becoming a competent physician. Many aspects of clinical practice cannot, or will not, ever be adequately tested. Clinical experience, and its lessons, are particularly important in these situations. At the same time, systematic attempts to record observations in a reproducible and unbiased fashion markedly increase the confidence one can have in knowledge about patient prognosis, the value of diagnostic tests, and the efficacy of treatment. In the absence of systematic observation one must be cautious in the interpretation of information derived from clinical experience and intuition, for it may at times be misleading.
• The study and understanding of basic mechanisms of disease are necessary but insufficient guides for clinical practice. The rationales for diagnosis and treatment which follow from basic pathophysiologic principles may in fact be incorrect, leading to inaccurate predictions about the performance of diagnostic tests and the efficacy of treatments.
• Understanding certain rules of evidence is necessary to correctly interpret literature on causation, prognosis, diagnostic tests, and treatment strategy.

It follows that clinicians should regularly consult the original literature (and read and be able to critically appraise the "Methods" and "Results" sections) in solving clinical problems and providing optimal patient care. It also follows that clinicians must be ready to accept and live with uncertainty, and to acknowledge that management decisions are often made in the face of relative ignorance of their true impact.

The new paradigm puts a much lower value on authority [20]. The underlying belief is that physicians can gain the skills to make independent assessments of evidence, and thus evaluate the credibility of opinions being offered by experts. The decreased emphasis on authority does not imply a rejection of what one can learn from colleagues and teachers whose years of experience have provided them with insight into methods of history-taking, physical examination, and diagnostic strategies which can never be gained from formal scientific investigation. A final assumption of the new paradigm is that physicians whose practice is based on an understanding of the underlying evidence will provide superior patient care.

Requirements for the Practice of Evidence-Based Medicine

The role-modelling, practice, and teaching of evidence-based medicine requires skills that are not traditionally part of medical training. These include precisely defining a patient problem, and what information is required to resolve the problem; conducting an efficient search of the literature; selecting the best of the relevant studies, and applying rules of evidence to determine their validity [3]; being able to present to colleagues in a succinct fashion the content of the article, and its strengths and weaknesses; extracting the clinical message, and applying it to the patient problem. We will refer to this process as the "critical appraisal exercise."

Evidence-based medicine also involves applying traditional skills of medical training. A sound understanding of pathophysiology is necessary to interpret and apply the results of clinical research. For instance, most patients to whom we would like to generalize the results of randomized trials would, for one reason or another, not have been enroled in the most relevant study. The patient may be too old, be too sick, have other underlying illness, or be uncooperative. Understanding the underlying pathophysiology allows the clinician to better judge whether the results are applicable to the patient at hand. Understanding of pathophysiology also has a crucial role as a conceptual and memory aid.

A second traditional skill required of the evidence-based physician is a sensitivity to patients' emotional needs. Understanding patients' suffering [21], and how that suffering can be ameliorated by the caring and compassionate physician, are fundamental requirements for medical practice. These skills can be acquired through careful observation of patients and of physician role-models. Here too, though, the need for systematic study, and the limitations of the present evidence, must be considered. The new paradigm would call for using the techniques of behavioral science to determine what patients are really looking for from their physicians [22], and how physician and patient behavior affects the outcome of care [23]. Ultimately, randomized trials of different strategies for interacting with patients (such as the randomized trial conducted by Greenfield and colleagues that demonstrated the positive effects of increasing patients' involvement with their care [24]) may be appropriate.

Since evidence-based medicine involves skills of problem definition, searching, evaluating, and applying original medical literature, it is incumbent on residency programs to teach these skills. Understanding the barriers to educating physicians-in-training in evidence-based medicine can lead to more effective teaching strategies.

Evidence-based Medicine in a Medical Residency

The Internal Medicine Residency Program at McMaster University has an explicit commitment to producing practitioners of evidence-based medicine. While other clinical departments at McMaster have devoted themselves to teaching evidence-based medicine, the commitment is strongest in the Department of Medicine. We will therefore focus on the Internal Medicine Residency in our discussion, and briefly outline some of the strategies we are using in implementing the paradigm shift.

1. The residents spend each Wednesday afternoon at an "academic half-day." At the beginning of each new academic year, the rules of evidence which relate to articles concerning therapy, diagnosis, prognosis, and overviews are reviewed. In subsequent sessions, the discussion is built around a clinical case, and two original articles which bear on the problem are presented. The residents are responsible for critically appraising the articles, and arriving at "bottom lines" regarding the strength of evidence and how it bears on the clinical problem. They learn to present the methods and results in a succinct fashion, emphasizing only the key points. A wide-ranging discussion, including issues of underlying pathophysiology and related questions of diagnosis and management, follows presentation of the articles.
   The second part of the half-day is devoted to the physical examination. Clinical teachers present optimal techniques of examination with attention to what is known about their reproducibility and accuracy.
2. Facilities for computerized literature searching are available on the teaching medical ward in each of the four teaching hospitals. Costs of searching are absorbed by the Residency Program. Residents not familiar with computer searching, or the Grateful Med program we use, are instructed at the beginning of the rotation. Research in our institution has shown that MEDLINE searching from clinical settings is feasible with brief training [25]. A subsequent investigation demonstrated that internal medicine house staff who have computer access on the ward and feedback concerning their searching do an average of over 3.6 searches per month [26]. House staff believe that over 90% of their searches that are stimulated by a patient problem lead to some improvement in patient care [25].
3. Assessment of searching and critical appraisal skills is being incorporated into the evaluation of residents.
4. We believe that the new paradigm will remain an academic mirage with little relation to the world of day-to-day clinical practice unless physicians-in-training are exposed to role models who practice evidence-based medicine. As a result, the Residency Program has placed major emphasis on ensuring this exposure.
   a) One focus of recruitment for our Department of Medicine has been internists with training in clinical epidemiology. These individuals have the skills, and commitment, to practice evidence-based medicine. The Residency Program works to ensure they have clinical teaching roles available to them.
   b) A program of more rigorous evaluation of attending physicians has been instituted. One of the areas evaluated is the extent to which attending physicians are effective in teaching evidence-based medicine. The relevant items from the evaluation form are reproduced in Figure 1.
   c) Because it is new to both teachers and learners, and because most clinical teachers have observed few role models and have not received formal training, teaching evidence-based medicine is not easy. To help attending physicians improve their skills in this area, we have encouraged them to form partnerships which involve attending the partner's clinical rounds, making observations, and providing formal feedback. One learns through observation and through criticisms of one's performance. A number of faculty members have participated in this program.

To further facilitate attending physicians improving their skills, the Department of Medicine held a retreat devoted to sharing strategies for effective clinical teaching. Part of the workshop, attended by over 30 faculty members, was devoted to teaching evidence-based medicine. Some of the strategies that were adduced are briefly summarized in the next section.

Effective Teaching of Evidence-based Medicine

• Attending physicians must be enthusiastic, effective role-models for the practice of evidence-based medicine, even in high-pressure clinical settings such as ICUs. Providing a model goes a long way toward inculcating attitudes that lead learners to develop skills in critical appraisal. Acting as a role model involves specifying the strength of evidence that supports clinical decisions. In one case, the teacher can point to a number of large randomized trials, rigorously reviewed and included in a meta-analysis which allows one to say how many patients one must treat to prevent a death. In other cases, the best evidence may come from accepted practice or one's clinical experience and instincts. The clinical teacher should make it clear to learners on what basis decisions are being made. This can be done efficiently. For instance, "Prospective studies suggest that Mr. Jones' risk of a major vascular event in the first year after his infarct is 4%; a meta-analysis of randomized trials of aspirin in this situation suggests a risk reduction of 25%; we'd have to treat 100 such patients to prevent an event [27]; given the minimal expense and toxicity of low-dose enteric-coated aspirin, treating Mr. Jones is clearly warranted." Or, "How long to treat with antibiotics following pneumonia has not been systematically studied, so my recommendation that we give Mrs. Smith three days of intravenous antibiotics and treat her for a total of ten days is arbitrary; somewhat shorter or longer courses of treatment would be equally reasonable." In the latter type of situation, dogmatic or rigid insistence on following a particular course of action would not be appropriate.
• It is crucial that critical appraisal issues arise from patient problems which the learner is currently confronting, demonstrating that critical appraisal is a pragmatic and central, not an "academic" or tangential element of optimal patient care. The problem selected for critical appraisal must be one that the learners recognize as important, about which they are uncertain, and on which they do not fully trust expert opinion; in other words, they must feel it is worth the effort to find out what the literature says on a topic. The likeliest candidate topics are common problems where learners have been exposed to divergent opinions (and thus there is disagreement and/or uncertainty among the learners). The clinical teacher should keep these requirements in mind when considering questions to encourage the learners to address. It can be useful to ask each member of the group their opinion about the clinical problem at hand. One can then ensure that the problem is appropriate for a critical appraisal exercise by asking the group the following questions:
  • a) It seems the group is uncertain about the optimal approach. Is that right?
  • b) Do you feel it's important for us to sort out this question by going to the original literature?
• Criteria for methodologic rigour must be few and simple. Most published criteria can be overwhelming for the novice. Suggested criteria for studies of diagnosis, treatment, and review articles follow:
  • Diagnosis
    • Has the diagnostic test been evaluated in a patient sample that included an appropriate spectrum of mild and severe, treated and untreated disease, plus individuals with different but commonly confused disorders [28]?
    • Was there an independent, blind comparison with a "gold standard" of diagnosis [28]?
  • Treatment
    • Was the assignment of patients to treatments randomized [29]?
    • Were all patients who entered the study accounted for at its conclusion [29]?
  • Review Articles
    • Were explicit methods used to determine which articles to include in the review [30]?

As learners become more sophisticated, additional criteria can be introduced. The criteria should not be presented in such a way that fosters nihilism (if the study is not randomized, it's useless and provides no valuable information), but as a way of helping arrive at the strength of inference associated with the clinical decision. Teachers can point out instances in which criteria can be violated without reducing the strength of inference.

Methods for Scaling the Barriers to the Dissemination of Evidence-based Medicine

Misapprehensions about Evidence-based Medicine

In developing the practice and teaching of evidence-based medicine at our institution we have found that the nature of the new paradigm is sometimes misinterpreted. Recognizing the limitations of intuition, experience, and understanding of pathophysiology in permitting strong inferences may be misinterpreted as rejecting these routes to knowledge. Specific misinterpretations of evidence-based medicine, and their corrections, follow.

Misinterpretation #1) Evidence-based medicine ignores clinical experience and clinical intuition.

• Correction: On the contrary, it is important to expose learners to exceptional clinicians who have a gift for intuitive diagnosis, a talent for precise observation, and excellent judgement in making difficult management decisions. Untested signs and symptoms should not be rejected out of hand. They may prove extremely useful, and ultimately be proved valid through rigorous testing. The more experienced clinicians can dissect the process they use in diagnosis [31], and clearly present it to learners, the greater the benefit. Similarly, the gain for students will be greatest when clues to optimal diagnosis and treatment are culled from the barrage of clinical information in a systematic and reproducible fashion.
  Institutional experience can also provide important insights. Diagnostic tests may differ in their accuracy depending on the skill of the practitioner. A local expert in, for instance, diagnostic ultrasound, may produce far better results that the average from the published literature. The effectiveness and complications associated with therapeutic interventions, particularly surgical procedures, may also differ across institutions. When optimal care is taken to both record observations reproducibly and avoid bias, clinical and institutional experience evolves into the systematic search for knowledge that forms the core of evidence-based medicine [32].

Misinterpretation #2) Understanding of basic investigation and pathophysiology plays no part in evidence-based medicine.

• Correction: The dearth of adequate evidence demands that clinical problem-solving must rely on an understanding of underlying pathophysiology. Moreover, a good understanding of pathophysiology is necessary for interpreting clinical observations and for appropriate interpretation of evidence (especially in deciding on its generalizability).

Misinterpretation #3) Evidence-based medicine ignores standard aspects of clinical training such as the physical examination.

• Correction: A careful history and physical examination provides much, and often the best, evidence for diagnosis and directs treatment decisions. The clinical teacher of evidence-based medicine must give considerable attention to teaching the methods of history and clinical examination, with particular attention to which items have demonstrated validity and to strategies to enhance observer agreement.

Barriers to Teaching Evidence-Based Medicine

Difficulties we have encountered in teaching evidence-based medicine include the following.

• Many house staff start with rudimentary critical appraisal skills and the topic may be threatening for them.
• People like quick and easy answers. Cookbook medicine has its appeal. Critical appraisal involves additional time and effort, and may be perceived as inefficient and distracting from the real goal (to provide optimal care for patients).
• For many clinical questions, high quality evidence is lacking. If such questions predominate in attempts to introduce critical appraisal, a sense of futility can result.
• The concepts of evidence-based medicine are met with scepticism by many faculty members who are therefore unenthusiastic about modifying their teaching and practice in accordance with its dictates.

These problems can be ameliorated by use of the strategies described in the previous section on "effective teaching of evidence-based medicine." Threat can be reduced by beginning with a contract with the residents which sets out modest and achievable goals, and further reduced by the attending physician role-modelling the practice of evidence-based medicine. Inefficiency can be reduced by teaching effective searching skills and simple guidelines for assessing the validity of the papers. In addition, one can emphasize that critical appraisal as a strategy for solving clinical problems is most appropriate when the problems are common in one's own practice. Futility can be reduced by, particularly initially, targeting critical appraisal exercises to areas in which there is likely to be high-quality evidence that will affect clinical decisions. Scepticism on the part of faculty members can be reduced by the availability of "quick and dirty" (as well as more sophisticated) courses on critical appraisal of evidence, and by the teaching partnerships and teaching workshops described earlier.

Many problems in the practice and teaching of evidence-based medicine remain. Many physicians, including both residents and faculty members, are still sceptical about the tenets of the new paradigm. A medical residency is full of competing demands, and the appropriate balance between goals is not always evident. At the same time, we are buoyed by the number of residents and faculty who have enthusiastically adopted the new approach, and found ways to integrate it into their learning and practice.

Barriers to Practising Evidence-Based Medicine

Even if our Residency Program is successful in producing graduates who enter the world of clinical practice enthusiastic to apply what they have learned about evidence-based medicine, they will face difficult challenges. Economic constraints and counter-productive incentives may compete with the dictates of evidence as determinants of clinical decisions. The relevant literature may not be readily accessible. Time may be insufficient to carefully review the evidence (which may be voluminous) relevant to a pressing clinical problem.

Some solutions to these problems are already available. Optimal integration of computer technology into clinical practice facilitates finding and accessing evidence. Reference to literature overviews meeting scientific principles [30] [33] and collections of methodologically sound and highly relevant articles [14] can markedly increase efficiency. Other solutions will emerge over time. Health educators will continue to find better ways of role-modelling and teaching evidence-based medicine. Standards in writing reviews and texts are likely to change, with a greater focus on methodologic rigour [15] [16]. Evidence-based summaries will therefore become increasingly available. Practical approaches to making evidence-based easier to apply in clinical practice, many based on computer technology, will be developed and expanded. As described earlier, we are already using computer searching on the ward. In the future, the results of diagnostic tests may be provided with the associated sensitivity, specificity, and likelihood ratios. Health policy-makers may find that the structure of medical practice must be shifted in basic ways to facilitate the practice of evidence-based medicine. Increasingly, scientific overviews will be systematically integrated with information regarding toxicity and side effects, cost, and the consequences of alternative courses of action to develop clinical policy guidelines [34]. The prospects for these developments are both bright and exciting.

Does teaching and learning evidence-based medicine improve patient outcomes?

The proof of the pudding of evidence-based medicine lies in whether patients cared for in this fashion enjoy better health. This proof is no more achievable for the new paradigm than it is for the old, for no long-term randomized trials of traditional and evidence-based medical education are likely to be carried out. What we do have are a number of short-term studies which confirm that the skills of evidence-based medicine can be taught to medical students [35] and medical residents [36]. In addition, a study compared the graduates a medical school that operates under the new paradigm (McMaster) to graduates of a traditional school. A random sample of McMaster graduates who had chosen careers in family medicine were more knowledgeable with respect to current therapeutic guidelines in the treatment of hypertension than were the graduates of the traditional school [37]. These results suggest that the teaching of evidence-based medicine may help graduates stay up to date. Further evaluation of the evidence-based medicine approach is necessary.

Our advocating evidence-based medicine in the absence of definitive evidence of its superiority in improving patient outcomes may appear to be an internal contradiction. As has been pointed out, however, evidence-based medicine does not advocate a rejection of all innovations in the absence of definitive evidence. When definitive evidence is not available, one must fall back on weaker evidence (such as the comparison of graduates of two medical schools which use different approaches cited above), and on biologic rationale. The rationale in this case is that physicians who are up-to-date as a function of their ability to read the current literature critically, and are able to distinguish strong from weaker evidence are likely to be more judicious in the therapy they recommend. Physicians who understand the properties of diagnostic tests and are able to use a quantitative approach to those tests are likely to make more accurate diagnoses. While this rationale appears compelling to us, compelling rationales have often proved misleading. Until more definitive evidence is adduced, adoption of evidence-based medicine should appropriately be restricted to three groups. One group is those who find the rationale compelling, and thus believe that use of the evidence-based medicine approach is likely to improve clinical care. A second group is those who have the energy, enthusiasm, and resources to test evidence-based medicine in educational trials. A final group include those who, while sceptical of improvements in patient outcome, believe it is very unlikely that deterioration in care results from the evidence-based approach and who find that the practice of medicine in the new paradigm is more exciting and fun.

Conclusion

Based on an awareness of the limitations of traditional determinants of clinical decisions, a new paradigm for medical practice has arisen. Evidence-based medicine deals directly with the uncertainties of clinical medicine and has the potential for transforming the education and practice of the next generation of physicians. These physicians will continue to face an exploding volume of literature, rapid introduction of new technologies, deepening concern about burgeoning medical costs, and increasing attention to the quality and outcomes of medical care. The likelihood that evidence-based medicine can help ameliorate these problems should encourage its dissemination.

Evidence-based medicine will require new skills for the physician, skills which residency programs should be equipped to teach. While strategies for inculcating the principles of evidence-based medicine remain to be refined, initial experience has revealed a number of effective approaches. Incorporating these practices into postgraduate medical education, and continuing to work on their further development, will result in more rapid dissemination and integration of the new paradigm into medical practice.

References

1. Lindberg DA. Information systems to support medical practice and scientific discovery. Methods Inf Med. 1989 Nov;28(4):202-6.

2. Hart YM, Sander JW, Johnson AL, Shorvon SD. National General Practice Study of Epilepsy: recurrence after a first seizure. Lancet. 1990 Nov 24;336(8726):1271-4.

3. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology, a Basic Science for Clinical Medicine. 2nd ed. Boston: Little Brown & Co Inc; 1991: 218.

4. Kuhn TS. The Structure of Scientific Revolutions. Chicago: University of Chicago Press; 1970.

5. MRC European Carotid Surgery Trial: interim results for symptomatic patients with severe (70-99%) or with mild (0-29%) carotid stenosis. European Carotid Surgery Trialists' Collaborative Group. Lancet. 1991 May 25;337(8752):1235-43.

6. Larsen ML, Horder M, Mogensen EF. Effect of long-term monitoring of glycosylated hemoglobin levels in insulin-dependent diabetes mellitus. N Engl J Med. 1990 Oct 11;323(15):1021-5.

7. L'Abbe KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med. 1987 Aug;107(2):224-33. Review.

8. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978 Oct 26;299(17):926-30.

9. Nierenberg AA, Feinstein AR. How to evaluate a diagnostic marker test. Lessons from the rise and fall of dexamethasone suppression test. JAMA. 1988 Mar 18;259(11):1699-702. Review.

10. Haynes RB, McKibbon KA, Fitzgerald D, Guyatt GH, Walker CJ, Sackett DL. How to keep up with the medical literature: V. Access by personal computer to the medical literature. Ann Intern Med. 1986 Nov;105(5):810-6.

11. Bulpitt CJ. Confidence intervals. Lancet. 1987 Feb 28;1(8531):494-7.

12. Godfrey K. Simple linear regression in medical research. N Engl J Med. 1985 Dec 26;313(26):1629-36.

13. Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited. Ann Intern Med. 1990 Jul 1;113(1):69-76. Review.

14. Haynes RB. The Origins and Aspirations of ACP Journal Club. [Editorial]. ACP J Club. 1991 Jan-Feb:A18. Ann Intern Med 1991:114S1.

15. Chalmers I, Enkin M, Keirse MJNC, eds. Evaluating the Effects of Care During Pregnancy and Childbirth. Oxford: Oxford University Press; 1989:3-38.

16. Sinclair JC, Bracken MB, eds. Effective Care of the Newborn Infant. Oxford: Oxford University Press; 1992.

17. Audet AM, Greenfield S, Field M. Medical practice guidelines: current activities and future directions. Ann Intern Med. 1990 Nov 1;113(9):709-14.

18. Guyatt GH. Evidence-based medicine. ACP J Club. 1991;114(suppl 2):A-16

19. Light D Jr. Uncertainty and control in professional training. J Health Soc Behav. 1979 Dec;20(4):310-22.

20. Chalmers I. Scientific inquiry and authoritarianism in perinatal care and education. Birth. 1983 Fall;10(3):151-66.

21. Cassell EJ. The nature of suffering and the goals of medicine. N Engl J Med. 1982 Mar 18;306(11):639-45.

22. Ende J, Kazis L, Ash A, Moskowitz MA. Measuring patients' desire for autonomy: decision making and information-seeking preferences among medical patients. J Gen Intern Med. 1989 Jan-Feb;4(1):23-30.

23. Carter WB, Inui TS, Kukull WA, Haigh VH. Outcome-based doctor-patient interaction analysis: II. Identifying effective provider and patient behavior. Med Care. 1982 Jun;20(6):550-66.

24. Greenfield S, Kaplan S, Ware JE Jr. Expanding patient involvement in care. Effects on patient outcomes. Ann Intern Med. 1985 Apr;102(4):520-8.

25. Haynes RB, McKibbon KA, Walker CJ, Ryan N, Fitzgerald D, Ramsden MF. Online access to MEDLINE in clinical settings. A study of use and usefulness. Ann Intern Med. 1990 Jan 1;112(1):78-84.

26. McKibbon KA, Haynes RB, Johnston ME, Walker CJ. A study to enhance clinical end-user MEDLINE search skills: design and baseline findings. Proc Annu Symp Comput Appl Med Care. 1991;:73-7.

27. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988 Jun 30;318(26):1728-33.

28. Department of Clinical Epidemiology and Biostatistics. How to read clinical journals: II. To learn about a diagnostic test. Can Med Assoc J. 1981 Mar 15;124(6):703-10.

29. Department of Clinical Epidemiology and Biostatistics. How to read clinical journals: V: To distinguish useful from useless or even harmful therapy. Can Med Assoc J. 1981 May 1;124(9):1156-62.

30. Oxman AD, Guyatt GH. Guidelines for reading literature reviews. CMAJ. 1988 Apr 15;138(8):697-703.

31. Campbell EJ. The diagnosing mind. Lancet. 1987 Apr 11;1(8537):849-51.

32. Feinstein AR. What kind of basic science for clinical medicine? N Engl J Med. 1970 Oct 15;283(16):847-52.

33. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987 Mar;106(3):485-8.

34. Eddy DM. Clinical decision making: from theory to practice. Guidelines for policy statements: the explicit approach. JAMA. 1990 Apr 25;263(16):2239-40, 2243.

35. Bennett KJ, Sackett DL, Haynes RB, Neufeld VR, Tugwell P, Roberts R. A controlled trial of teaching critical appraisal of the clinical literature to medical students. JAMA. 1987 May 8;257(18):2451-4.

36. Kitchens JM, Pfeifer MP. Teaching residents to read the medical literature: a controlled trial of a curriculum in critical appraisal/clinical epidemiology. J Gen Intern Med. 1989 Sep-Oct;4(5):384-7.

37. Shin J, Haynes RB. Does a problem-based, self-directed undergraduate medical curriculum promote continuing clinical competence? Clin Res. 1991;39:143A.

© 2001 Evidence-Based Medicine Informatics Project

© 2001 Centre for Health Evidence.
Home. Users' Guides to EBP. Webmaster. Disclaimer.