How to Use an Article About Disease Probability for Differential Diagnosis
W. Scott Richardson, Mark C. Wilson, Gordon H. Guyatt, Deborah J. Cook, James Nishikawa, and the Evidence Based Medicine Working Group
Based on the Users Guides to Evidence-based Medicine and reproduced with permission from JAMA. (1999 Apr 7; 281(13):1214-9). Copyright 1999, American Medical Association.
- Clinical Scenario
- The Search
- Introduction
- I. Are the results of the study valid?
- II. What are the results?
- III. Will the results help me in caring for my patients?
- Resolution of the Scenario
- References
Clinical Scenario
You are an experienced clinician working at a hospital emergency department. One morning, you see a 33 year-old man who presents with palpitations. He describes the new onset of episodes of fast, regular chest pounding, which come on gradually, last one to two minutes, and occur several times a day. He reports no relation of symptoms to activities, and no change in exercise tolerance. He is very anxious, and tells you that he fears heart disease. He has no other symptoms, no personal or family history of heart disease, and takes no medications. You find his heart rate is 90 and regular, and physical examination of his eyes, thyroid gland, lungs and heart is normal, without click, murmur or gallop. His 12-lead ECG is normal, without arrhythmia or signs of pre-excitation.
You suspect his anxiety explains his palpitations, that they are mediated by hyperventilation, and are possibly part of a panic attack. While he has no findings of cardiac arrhythmia or hyperthyroidism, you wonder if these disorders are common enough in the emergency department setting to consider seriously. You reject pheochromocytoma as too unlikely. Thus, you can list causes of palpitations, but want more information about the frequency of these causes to choose a diagnostic work-up. You ask the question: "In patients presenting with palpitations, what is the frequency of underlying disorders?"
The Search
Your emergency department computer networks with the medical library, where the MedLine is on CD-ROM. In the MedLine file for current years, you enter three text words: “palpitations” (89 citations), “differential diagnosis” (7039 citations) and “(cause or causes)” (71848 citations). You combine these sets, yielding 17 citations. Reviewing the titles and abstracts on-screen, you see a paper by Weber et al that promises to have what you want. [1] With a keystroke and a mouse click, you review this article’s full text on-screen.
Introduction
Sick persons seldom present with the diagnosis already made; instead, they present with one or more symptoms. These symptoms prompt the clinician to gather information through history and physical examination, identifying clinical findings that suggest explanations for the symptom(s). For example, in an older woman presenting with generalized pruritis, the clinician could identify recent anorexia and weight loss, along with jaundice and the absence of a rash. For most symptoms, the clinician must consider multiple causes for the patient’s findings.
Differential diagnosis is the method by which the clinician considers the possible causes of a patient’s clinical findings before making a final diagnosis. [2] [3] Experienced clinicians often group the findings into meaningful clusters, summarized in brief phrases about the symptom, body location or organ system involved, such as “generalized pruritis”, “painless jaundice” and “constitutional symptoms” for the older woman mentioned earlier. We call these clusters ‘clinical problems’, [3] [4] and include problems of biologic, psychologic or sociologic origin. [5] It is for these clinical problems, rather than for the final diagnosis, that the clinician selects a patient’s differential diagnosis.
When considering a patient’s differential diagnosis, how is the clinician to decide which disorders to pursue? If the clinician were to consider all known causes equally likely and test for them all simultaneously (the ‘possibilistic’ approach), then the patient would undergo unnecessary testing. Instead, the experienced clinician is selective, considering first those disorders that are more likely (a ‘probabilistic’ approach), more serious if left undiagnosed and untreated (a ‘prognostic’ approach) or more responsive to treatment if offered (a ‘pragmatic’ approach).
Wisely selecting a patient’s differential diagnosis involves all three considerations (probabilistic, prognostic and pragmatic), as depicted in Table 1. The clinician’s single best explanation for the patient’s clinical problem(s) can be termed the ‘leading hypothesis’ or ‘working diagnosis’ (see #1 in Table 1). A few (usually 1 to 5) other diagnoses, termed ‘active alternatives’ (group #2 in Table 1), may be worth considering further at the time of initial work-up, because of their likelihood, seriousness if undiagnosed and untreated, or responsiveness to treatment. Additional causes of the clinical problem(s), termed ‘other hypotheses’ (group #3 in Table 1), may be too unlikely to consider at the time of initial diagnostic work-up, but remain possible and could be considered further if the working diagnosis and active alternatives are later disproved. Using this framework for the patient with palpitations in the scenario, you are considering anxiety as the working diagnosis, and you are wondering whether cardiac arrhythmias, hyperthyroidism or pheochromocytoma belong in the active alternatives (group 2 in Table 1) or the other hypotheses (group 3 in Table 1).
Table 1: Selecting a patient-specific differential diagnosis
* LR+ denotes likelihood ratio for a positive result; LR- denotes likelihood ratio for a negative result. |
Selecting a patient-specific differential diagnosis has implications for both diagnostic testing and initial therapy. For the leading hypothesis, the clinician may choose to confirm the diagnosis, using a highly specific test with a high likelihood ratio for a positive result. [10] [11] For the active alternatives, the clinician would choose to exclude these diagnoses, using highly sensitive tests with low likelihood ratios for negative results. Usually, the clinician would not order tests initially for the other hypotheses. The clinician may start initial therapy for both the working diagnosis and for one or more of the active alternatives, depending on circumstances.
How can information about disease probability help clinicians select patients’ differential diagnoses? We’ll illustrate with some brief cases. First, consider a patient who presents with a painful eruption of grouped vesicles in the distribution of a single dermatome. In an instant, an experienced clinician would make a diagnosis of herpes zoster and turn to thinking about whether to offer the patient therapy. In the framework of Table 1, the working diagnosis is zoster and there are no active alternatives. In other words, the probability of zoster is so high (near 1.0 or 100%) that it is above a threshold where no further testing is required.
Next, consider a previously healthy athlete who presents with lateral rib cage pain after being accidentally struck by an errant baseball pitch. Again, the experienced clinician might rapidly recognize the clinical problem (post-traumatic lateral chest pain), and quickly list a leading hypothesis (rib contusion) and an active alternative (rib fracture), and plan a test (radiograph) to exclude the latter. If asked, the clinician could also list disorders that are too unlikely to consider further (such as myocardial infarction). In other words, while not as likely as rib contusion, the probability of a rib fracture is above a threshold for testing, while the probability of myocardial infarction is below the threshold for testing.
These cases illustrate how clinicians can estimate the probability of disease from the patient’s clinical findings, risk factors, exposures, etc., and then compare disease probabilities to two thresholds. The probability above which the diagnosis is sufficiently likely to warrant therapy defines the upper threshold. This threshold is termed the ‘test-treatment’ or simply the ‘treatment’ threshold. [12] In the case of shingles above, the clinician judged the diagnosis of zoster to be above this treatment threshold of probability. The probability below which the clinician a diagnosis warrants no further consideration defines the llower threshold. This treatment is termed the ‘no test-test’ or simply the ‘test’ threshold. In the case of post-traumatic torso pain above, the diagnosis of rib fracture fell above, and the diagnosis of myocardial infarction below, the test threshold.
Clinicians begin with pre-test estimates of disease probability, and then adjust the probability as new diagnostic information arrives. Test results are useful when they move our pre-test probabilities across one of these two thresholds. For a disorder with a pre-test probability above the treatment threshold, a confirmatory test that raises the probability further would not aid diagnostically. On the other end of the scale, for a disorder with a pre-test probability below the test threshold, an exclusionary test that lowers the probability further would not aid diagnostically. When the clinician believes the pre-test probability is high enough to test for and not high enough to begin treatment (i.e. between the two thresholds), a test could be diagnostically useful if it moves the probability across either threshold.
How do clinicians arrive at pre-test estimates of disease probability? They remember prior cases with the same clinical problem(s), so that disorders diagnosed frequently have higher probability than diagnoses made less frequently. Remembered cases are easily and quickly available, and are calibrated to our local practices. Yet our memories are imperfect, and the probabilities that result are subject to biases and errors. [13] [14] [15]
Knowing memory is prone to bias, clinicians can consult other sources including population prevalence statistics and original research. Inclusion of the entire population, rather than persons with a given clinical problem, limit the usefulness of population surveys. [16] Inconsistency of how diagnoses are made and recorded may further limit survey usefulness.
Original research constitutes another source of information about disease probability. For example, in a study of diagnostic tests for anemia in the aged, investigators compared blood tests with bone marrow results in 259 elderly persons, finding iron deficiency in 94 (36%) patients. [17] Thus, while this study focused on evaluating tests for iron deficiency, it also provides information about disease frequency. Some research studies focus more directly on the frequency of diseases that cause symptoms, [18] like the Weber article on palpitations from the scenario. [1] This Users’ Guide will help you understand direct studies of disease probability, judge their validity and decide whether to use them for estimating pre-test probability for your own patients (see Table 2).
Table 2: Disease Probability for Differential Diagnosis
|
I. Are the results of the study valid?
Did the study patients represent the full spectrum of those who present with this clinical problem?
This question asks about two related issues. First, how do the investigators define the clinical problem under study? The definition of the clinical problem for study determines the population from whom the study patients should be. Thus, investigators studying ‘hematuria’ might include patients with microscopic and gross hematuria, with or without symptoms. On the other hand, investigators studying ‘asymptomatic, microscopic hematuria’ would exclude those with symptoms or with gross hematuria.
Such differing definitions of the clinical problem can yield different frequencies of underlying diseases. Including patients with gross hematuria or urinary symptoms might raise the frequency of acute infection as the underlying cause relative to those without symptoms. So, assessing the validity of an article about differential diagnosis begins with a search for a clear definition of the clinical problem.
Having defined the target population by clinical problem statement, investigators next assemble a patient sample. Ideally, the study sample mirrors the target population in all important ways, so that the frequency of underlying diseases in the sample approximates that of the target population. Such a patient sample is termed ‘representative’, and the more representative the sample is, the more accurate the resulting disease probabilities. Investigators seldom are able to utilize the strongest method of assuring representativeness, obtaining a random sample of the entire population of patients with the clinical problem. The next strongest methods are either to include all patients with the clinical problem from a defined geographic area, or to include a consecutive series of all patients with the clinical problem who receive care at the investigators’ institution(s). To the extent that a non-consecutive case series opens the study to the differential inclusion of patients with different underlying disorders it compromises study validity.
You can judge the representativeness of the sample by examining the setting from which patients come. Patients with ostensibly the same clinical problem can present to different clinical settings, resulting in different services seeing different types of patients. Typically, patients in secondary or tertiary care settings have higher proportions of more serious diseases or more uncommon diseases than those patients seen in primary care. [19] For instance, in a study of coronary artery disease in 1074 patients with chest pain, a higher proportion of referral practice patients had coronary artery disease than the primary care practice patients, even in patients with similar clinical histories. [20]
To evaluate representativeness, you can also note the methods by which patients were recruited. By considering how investigators identified their patients, how carefully they avoided missing patients, and noting who was included and who was excluded, a clinician can judge whether important subgroups appear to be missing. The wider the spectrum of patients in the sample, the more representative the sample should be of the whole population, and therefore the more valid the results. For example, in a study of Clostridium difficile colitis in 609 patients with diarrhea, the patient sample consisted of adult inpatients whose diarrheal stools were tested for cytotoxin, thereby excluding any patients whose clinicians chose not to test. [21] The inclusion of only those tested is likely to raise the probability of C. difficile in relation to the entire population of patients with diarrhea.
Weber et al [1] defined palpitations broadly, as any one of several patient complaints (e.g. fast heartbeats, skipped heartbeats, etc.) and included patients with new and recurrent palpitations. They obtained patients from three clinical settings (emergency department, inpatient floors and a medical clinic) in one university medical center in a middle-sized North American city. Of the 229 adult patients presenting consecutively for care of palpitations at their center during the study period, 39 refused participation, and the investigators included the remaining 190 patients, including 62 from emergency. No important subgroups appear to have been excluded, so these 190 patients probably represent the full spectrum of patients presenting with palpitations.
Were the criteria for each final diagnosis explicit and credible?
Excellent clinicians will often disagree about the diagnosis for an individual patient. Such clinical disagreement about final diagnosis could threaten the validity of a study’s conclusions about disease frequency. To minimize this threat, investigators can use a set of explicit criteria when assigning each of the final diagnoses. Ideally, these sets of criteria should include not only the findings needed to confirm each diagnosis, but also those findings useful for rejecting each diagnosis. For example, published diagnostic criteria for infective endocarditis include both criteria for verifying the infection and criteria for rejecting it. [22] [23] Investigators can then classify study patients into diagnostic groups that are mutually exclusive, with the exception of patients whose symptoms are from more than one etiology. This allows clinicians to understand which diagnoses remain possible for any undiagnosed patients, after the investigators have ruled out alternatives.
Ideally, studies of disease probability would not only use high quality criteria, but also measure the agreement beyond chance for the clinicians assigning diagnoses, as was done in a study of causes of dizziness. [24] The goal of explicit, credible criteria is to ensure reproducible diagnosis; the ultimate test, then, is a formal agreement evaluation.
While reviewing the diagnostic criteria, keep in mind that “lesion finding” is not necessarily the same thing as “illness explaining”. In other words, using explicit and credible criteria a patient may be found to have two or more disorders that might explain the clinical problem, causing some doubt as to which disorder is the culprit. Better studies of disease probability will include some assurance that the disorders found actually do explain the patients’ illnesses. For example in a sequence of studies of syncope, investigators required that the symptoms occur simultaneously with an arrhythmia before that arrhythmia was judged to be the cause. [25] Alternatively, in a study of chronic cough, investigators gave cause-specific therapy and used positive responses to this to strengthen the case for these disorders actually causing the chronic cough. [26]
Weber et al developed a priori explicit and credible criteria for confirming each possible disorder causing palpitations and listed their criteria in an appendix, along with supporting citations. They evaluated study patients prospectively, and assigned final diagnoses using the explicit criteria. Wherever relevant, such as for cardiac arrhythmias, they required that the palpitations occur at the same time as the arrhythmias in order for that cause to be diagnosed. They do not report on agreement for these assignments.
Was the diagnostic workup comprehensive and consistently applied?
This criterion addresses two closely related issues. First, have the investigators evaluated their patients thoroughly enough to have detected any of the important causes of this clinical problem? Within reason and ethics, the more comprehensive the work up, the lower the chance that invalid conclusions about disease frequency will be reached. For example, in a retrospective study of stroke in 127 patients with mental status changes, the diagnostic workup was reported to include neurological examination and neuro-imaging; a comprehensive search for other causes of delirium was not described, and 118 cases remained unexplained. [27] As the investigators do not describe a complete and systematic search for causes of delirium, the disease probabilities appear less credible.
The second issue is how consistently the diagnostic work up was applied. This does not mean that every patient must undergo every test. Instead, for many clinical problems, the clinician performs a detailed yet focused history, a problem-oriented examination of the involved organ systems, and a few initial tests. Then, depending on the diagnostic clues that result from this information, further inquiry proceeds down one of multiple branching pathways. Ideally, investigators would evaluate all patients with the same initial work up, and then ‘follow the clues’ using pre-specified testing sequences. Once a definitive test result confirms a final diagnosis, then further confirmatory testing is unnecessary and unethical.
You may find it easy deciding whether the patients’ illnesses have been well investigated if they were evaluated prospectively using a pre-determined diagnostic approach. It becomes harder to judge when patients are studied only after their investigation is complete, or when investigation is not standardized. For example, in a study of precipitating factors in 101 patients with decompensated heart failure, while all patients underwent history and physical examination, the lack of standardization of subsequent testing makes it difficult to judge the accuracy of the disease probabilities. [28]
Weber et al evaluated their patients’ palpitations prospectively using two principal means, a structured interview completed by one of the investigators, and the combined diagnostic evaluation (i.e., history, examination, testing) chosen by the individual physician seeing the patient at the index visit. In addition, all patients completed self-administered questionnaires designed to assist in detecting various psychiatric disorders. A majority of patients (166 of 190) had EKGs obtained, and large numbers had other testing for cardiac disease as well. Thus, the diagnostic workup was reasonably comprehensive for common disease categories, although not exhaustive. Since the subsequent testing ordered by the individual physicians was not fully standardized, some inconsistency may have been introduced, although it does not appear likely to have distorted the probabilities of common disease categories such as psychiatric or cardiac causes.
For initially undiagnosed patients, was follow-up sufficiently long and complete?
Even when investigators use explicit diagnostic criteria after a comprehensive evaluation that is consistently applied, some patients’ clinical problems may remain unexplained. The higher the number of undiagnosed patients, the greater the chance of error in the estimates of disease probability. For example, in a retrospective study of various causes of dizziness in 1194 patients in an ENT clinic, about 27% remained undiagnosed. [29] With more than a quarter of patients’ illnesses unexplained the disease probabilities for the overall sample might be inaccurate.
If the study evaluation leaves any patients undiagnosed, investigators can follow these patients over time, searching for additional clues leading to eventual diagnoses, and observing the prognosis. The longer and more complete this follow-up is, the greater our confidence in the benign nature of the condition in patients who remain undiagnosed yet unharmed at the study’s end. How long is long enough? No single answer would correctly fit all clinical problems, but we’d suggest 1 to 6 months for symptoms that are acute and self-limited and 1 to 5 years for chronically recurring or progressive symptoms.
Weber et al identified a diagnosable etiology of palpitations in all but 31 (16.3%) of their 190 patients. The investigators followed nearly all of the study patients (96%) for at least a year, during which one additional diagnosis was made in those initially undiagnosed (symptomatic correlation with ventricular premature beats). None of the 31 undiagnosed patients had a stroke or died.
II. What are the Results?
What were the diagnoses and their probabilities?
The authors of many studies of disease probability display the main results in a table listing the diagnoses made, along with the numbers and percentages of patients with those diagnoses. For some symptoms, patients may have more than one underlying disease coexisting and presumably contributing to the clinical problem. In these situations, authors often identify the major diagnosis for such patients and separately tabulate contributing causes. Alternatively, authors sometimes identify a separate, ‘multiple etiology’ group.
Weber et al present a table that tells us that 58 (31%) were diagnosed with psychiatric causes, 82 (43%) had cardiac disorders, while 5 (2.6%) were found to have thyrotoxicosis and none had pheochromocytoma. This distribution differed across clinical settings: for instance, cardiac disorders were over twice as likely in patients presenting to the emergency department compared to patients presenting to the outpatient clinic.
How precise are these estimates of disease probability?
Even when valid, these disease probabilities are only estimates of the true frequencies. You can examine the precision of these estimates using their confidence intervals (CI’s). If the authors don’t provide the CI’s for you, you can calculate them yourself using the following formula (for 95% CI’s):
95%CI = p + 1.96 ?? (p (1 - p))/n
Where p is the proportion of patients with the etiology of interest, and n is the number of patients in the sample. [30] This formula becomes inaccurate when the number of cases is 5 or fewer, and approximations are available for this situation. [31] [32]
For instance, consider the category of psychiatric causes of palpitations in the Weber study. Using the above formula, we would start with p = 0.31, (1 - p) = 0.69, and n = 190. Working through the arithmetic, we find the CI to be 0.31 + 0.066. Thus, while the most likely true proportion is 31%, it may range between 24.4% and 37.6%.
Whether you will consider the confidence intervals sufficiently precise depends on where the estimated proportion and confidence interval fall in relation to your test or treatment thresholds. If both the estimate and the entire 95% confidence interval are on the same side of your threshold, then the result is precise enough to allow firm conclusions about disease probability for use in planning tests or treatments. Conversely, if the confidence limit around the estimate crosses your threshold, the result may not be precise enough for definitive conclusions about disease probability. You might still use a valid but imprecise probability result, while keeping in mind the uncertainty and what it might mean for testing or treatment.
Weber et al do not provide the 95% CI’s for the probabilities they found. However, as we just illustrated, if you were concerned about how near the probabilities were to your thresholds, you could calculate the 95% CI’s yourself.
III. Will the results help me in caring for my patients?
Are the study patients similar to my own?
This question concerns whether the clinical setting and patient characteristics are similar enough to yours to allow you to extrapolate the results to your practice. The closer the match, the more confident you can be in applying the results. We suggest you ask yourself whether the setting or patients are so different from yours that you should discard the results. [33] For instance, consider whether your patients come from areas where one or more of the underlying disorders are endemic, which could make these disorders much more likely in your patients than was found in the study. Also, consider whether your patients have different cultural patterns of illness behavior or health practices that might cause important differences in the disease probabilities, when compared with the patients in the study.
Weber et al recruited the 190 palpitation patients from those presenting to the outpatient clinics, the inpatient medical and surgical services and the emergency department (62 of the 190) in one university medical center in a middle-sized North American city. Thus, these patients are likely to be similar to the patients seen in your hospital emergency department, and you can use the study results to help inform the pre-test probabilities for the patient in the scenario.
Is it unlikely that the disease possibilities or probabilities have changed since this evidence was gathered?
As time passes, evidence about disease frequency can become obsolete. Old diseases can be controlled or, as in the case of smallpox, eliminated. [34] ‘New’ diseases, or at least, new epidemics of disease can arise. Such events can so alter the spectrum of possible diseases or their likelihood that previously valid and applicable studies may no longer be applicable to current practice. For example, consider how much the arrival of HIV disease has transformed the list of possibilities for such clinical problems as generalized lymphadenopathy, chronic diarrhea and unexplained weight loss, to name a few.
Similar changes can occur as the result of progress in medical science or public health. For instance, in studies of fever of unknown origin, new diagnostic technologies have substantially altered the proportions of patients who are found to have malignancy or whose fevers remain unexplained. [35] [36] [37] Treatment advances that improve survival, such as chemotherapy for childhood leukemia, can bring about shifts in disease likelihood, because the treatment might cause complications, such as secondary malignancy years after cure of leukemia. Public health measures that control some diseases, such as cholera, can alter the likelihood of the remaining etiologies of the clinical problems that the prevented disease would have caused, in this example acute diarrhea.
The palpitations study was published in 1996, and the text states the study period was eight months of 1991. In this instance, you know of no new developments likely to cause a change in the spectrum or probabilities of disease in patients with palpitations.
Resolution of the Scenario
Using the structure outlined in Table 1, your ‘leading hypothesis’ is that acute anxiety is causing your patient’s palpitations. You offer the patient a more in-depth discussion of his psychosocial situation, as the next test to explore this diagnosis (i.e., the pre-test probability is above your test threshold). At the same time, you don’t feel that anxiety is so certain that you can stop considering other disorders (i.e., the pre-test probability is below your threshold for treatment without testing). After reviewing the Weber palpitations study, you decide to include in your ‘active alternatives’ some cardiac arrhythmias (as common, serious and treatable) and hyperthyroidism (less common but serious and treatable), so you arrange testing to exclude these disorders (i.e., these are above your test threshold). Finally, given that none of the 190 study patients had pheochromocytoma, and since your patient has none of the other clinical features of this disorder, you place it into your ‘other hypotheses’ category (i.e., below your test threshold) and decide to hold off on testing for this condition.
We recommend applying these Users’ Guides to identify good evidence on which to base your initial estimates of disease probability for use in differential diagnosis. As you apply this evidence in practice, keep in mind that selecting a patient’s differential diagnosis wisely includes not only considering how likely various disorders are, but also considering how serious are the various diseases if left undiagnosed and untreated, and how much other clinical actions, like treatment or public health measures to reduce disease spread, could help the patient or the community.
References
1. Weber BE, Kapoor WN. Evaluation and outcomes of patients with palpitations. Am J Med 1996; 100: 138-148.
2. Sox HC, Blatt MA, Higgins MC, Marton KI. Medical Decision Making Butterworth, Boston, 1988.
3. Barondess JA, Carpenter CCJ, Eds. Differential Diagnosis, Philadelphia, Lea & Febiger, 1994.
4. Glass RD. Diagnosis: A Brief Introduction, Melbourne, Oxford University Press, 1996.
5. Engel GL. The need for a new medical model: a challenge for biomedicine. Science 1977; 196: 129-136.
6. Laupacis A, Wells G, Richardson WS, Tugwell P, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: V. How to use an article about prognosis. JAMA 1994; 272: 234 - 237.
7. Guyatt GH, Sackett DL, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? JAMA 1993; 270: 2598-2601.
8. Guyatt GH, Sackett DL, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: II. How to use an article about therapy or prevention: B. What are the results and will they help me in caring for my patients? JAMA 1994; 271: 59-63.
9. Dans AL, Dans LF, Guyatt GH, Richardson S, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. JAMA 1998; 279: 545 - 549.
10. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA 1994; 271: 389 - 311.
11. Jaeschke R, Guyatt GH, Sackett DL, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients? JAMA 1994; 271: 703 -707.
12. Pauker SG, Kassirer JP. The threshold approach to clinical decision making. N Engl J Med 1980; 302: 1109 - 1117.
13. Tversky A, Kahneman D. Judgment under uncertainty: heuristics and biases. Science 1974; 185: 1124-1131.
14. Dawson NV, Arkes HR. Systematic errors in medical decision making: judgment limitations. J Gen Intern Med 1987; 2: 183-187.
15. Kassirer JP, Kopelman RI. Cognitive errors in diagnosis: instantiation, classification, and consequences. Am J Med 1989; 86: 433 - 441.
16. Richardson WS. “Where do pre-test probabilities come from?” [EBM Note] Evidence Based Medicine (in press).
17. Guyatt GH, Patterson C, Ali M, Singer J, Levine M, Turpie I, Meyer R. Diagnosis of iron deficiency anemia in the elderly Am J Med 1990; 88: 205 - 209.
18.Kroenke K. Symptoms and science: the frontiers of primary care research [Editorial]. J Gen Intern Med 1997; 12: 509 - 510.
19. Fletcher RH, Fletcher SW, Wagner EH. Clinical Epidemiology: The Essentials, 3/e, Williams & Wilkins, Baltimore, 1996, p. 61-64.
20. Sox HC, Hickam DH, Marton KI, Moses L, Skeff KM, Sox CH, Neal A. Using the patient’s history to estimate the probability of coronary artery disease: a comparison of primary care and referral practices. Am J Med 1990; 89: 7 - 14.
21. Katz DA, Bates DW, Rittenberg E, Onderdonk A, Sands K, Barefoot LA, Snydman D. Predicting Clostridium difficile stool cytotoxin results in hospitalized patients with diarrhea. J Gen Intern Med 1997; 12: 57 - 62.
22. von Reyn CF, Levy BS, Arbeit RD, Friedland G, Crumpacker CS. Infective endocarditis: an analysis based on strict case definitions. Ann Intern Med 1981; 94: 505 - 517.
23. Durack DT, Lukes AS, Bright DK, and the Duke Endocarditis Service. New criteria for diagnosis of infective endocarditis: Utilization of specific echocardiographic findings. Am J Med 1994; 96: 200 - 209.
24. Kroenke K, Lucas CA, Rosenberg ML, Scherokman B, Herbers JE, Wehrle PA, Boggi JO. Causes of persistent dizziness: a prospective study of 100 patients in ambulatory care. Ann Intern Med 1992; 117: 898-904.
25. Kapoor WN. Evaluation and outcome of patients with syncope. Medicine 1990; 69: 160 - 175.
26. Pratter MR, Bartter T, Akers S, et al. An algorithmic approach to chronic cough. Ann Intern Med 1993; 119: 977 - 983.
27. Benbadis SR, Sila CA, Cristea RL. Mental status changes and stroke. J Gen Intern Med 1994; 9: 485 - 487.
28. Ghali JK, Kadakia S, Cooper R, Ferlinz J. Precipitating factors leading to decompensation of heart failure: traits among urban blacks. Arch Intern Med 1988; 148: 2013 - 2016.
29. Katsarkas A. Dizziness in aging - a retrospective study of 1194 cases. Otolaryngol Head Neck Surg 1994; 110: 296 - 301.
30. Altman DG. Confidence Intervals [Appendix], p. 227 - 234, in, Sackett DL, Richardson WS, Rosenberg WMC, Haynes RB, Eds. Evidence-Based Medicine: How To Practice and Teach EBM, New York, Churchill Livingstone, 1997.
31. Hanley JA, Lippman-Hand A. If nothing goes wrong, is everything all right? interpreting zero numerators. JAMA 1983; 249: 1743 - 1745.
32. Newman TB. If almost nothing goes wrong, is almost everything all right? interpreting small numerators. JAMA 1995; 274: 1013.
33. Glasziou P, Guyatt GH, Dans AL, Dans LF, Straus SE, Sackett DL. Applying the results of trials and systematic reviews to individual patients [Editorial]. ACP Journal Club 1998 Nov-Dec; 129: A15 - A16.
34. Barquet N, Domingo P. Smallpox: the triumph over the most terrible of the ministers of death. Ann Intern Med 1997; 127; 635 - 642.
35. Petersdorf RG, Beeson PB. Fever of unexplained origin: report on 100 cases. Medicine 1961; 40: 1-30.
36. Larson EB, Featherstone HJ, Petersdorf RG. Fever of undetermined origin: diagnosis and follow-up of 105 cases, 1970-1980. Medicine 1982; 61: 269-292.
37. Knockaert DC, Vanneste LJ, Vanneste SB, Bobbaers HJ. Fever of unknown origin in the 1980s: an update of the diagnostic spectrum. Arch Intern Med 1992; 152: 51-55.
© 2001 Evidence-Based Medicine Informatics Project
