Integrating Research Evidence with the Care of the Individual Patient
Finlay A. McAlister, Sharon E. Straus, Gordon H. Guyatt, R. Brian Haynes, for the Evidence Based Medicine Working Group
Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (2000;283(21):2829-2836). Copyright 2000, American Medical Association.
- Clinical Scenario
- The Search
- Introduction
- Determining the Applicability of the Evidence to an Individual Patient
- Incorporating Patient Values and Preferences
- Resolution of Clinical Scenario
- References
Clinical Scenario
You are the attending physician on an internal medicine which, one night, admits two patients with strokes (a 65 year old woman - Patient A - and a 65 year old man - Patient B). On examination, both have mild weakness of the right arm and left carotid bruits. Patient A has a history of hypertension and an admission blood pressure of 200/110 mm Hg; neither patient has other relevant medical history or physical examination findings. Aware that carotid bruits are not highly specific for identifying carotid artery stenosis, you send both patients for doppler ultrasonography. [1] Since your radiology department, in a recent audit, demonstrated that their ultrasonographic interpretations are highly correlated with angiographic results [2] you feel confident in their findings that both patients have moderate stenoses (50-69% by NASCET criteria) with no irregularity or ulceration of the plaque surface. [3] Aware of the recent flurry of literature concerning surgical versus medical therapy for patients with symptomatic carotid stenoses, you decide to review the literature to guide your management of these patients. You formulate the question: "In a patient with a mild stroke and moderate ipsilateral carotid stenosis, would a carotid endarterectomy (compared to best medical therapy) reduce the likelihood of subsequent severe stroke or death?"
The Search
A systematic review of randomized trials comparing carotid endarterectomy with standard medical therapy (aspirin in your practice setting) in patients with recent, mild strokes would provide the best evidence addressing your question. You check the database of systematic reviews maintained by the Cochrane Collaboration (The Cochrane Library, Issue 1, 1999. Oxford: Update Software), but find no relevant reviews. You therefore use the advanced screen search of PubMed for the next level of evidence, a single randomized trial. You choose the medical subject headings stroke and carotid endarterectomy and include randomized controlled trial as the publication type. The search identifies 117 articles of which 1 looks most relevant to your question. [4]
Investigators in this trial randomized 2267 patients with moderate carotid stenosis (less than 70%) and ipsilateral transient ischemic attacks or nondisabling stroke within 180 days to carotid endarterectomy or medical care alone. After five years of follow-up, significantly fewer patients in the carotid endarterectomy arm (vs. the medical care arm) had suffered a recurrent disabling stroke (5.3% vs. 10.3%, 49% relative risk reduction {RRR} [95% CI 14% to 83%]) or death (13% vs. 14.9%, 13% RRR [95% CI - 18% to 44%]). The size of the treatment effect was such that 20 patients (95% CI 12 to 70) would have to undergo carotid endarterectomy to prevent 1 more disabling stroke than with medical therapy alone. Although encouraged by these results, you are concerned about the wide confidence intervals, the potential for perioperative complications (1.4% excess risk of disabling stroke or death within the first month of surgery), and question how to apply the results to your patients.
Introduction
While randomized trials provide the most valid estimates of the true effects (both beneficial and harmful) of an intervention, they necessarily report average treatment effects. Whether these results are derived from a homogeneous group of high-risk, highly responsive patients (as in efficacy trials) or a heterogeneous group of "all-comers" (as in effectiveness trials), [5] clinicians must decide how to extrapolate them to individual patients. In this article, we will build upon previous Users’ Guides [6] [7] [8] [9] that assessed the validity and applicability of therapeutic studies to outline a framework that clinicians might use to integrate research results (whether from single trials or systematic reviews) with patient values to determine the optimal care for an individual patient.
Determining the Applicability of the Evidence to an Individual Patient
Previous Users’ Guides and other articles have dealt extensively with these issues. [7] [8] [9] [10] We will not repeat all of the key principles here, but will emphasize that differences between study participants and patients in real-world practice tend to be quantitative (differences in degree of risk of the outcome or responsiveness to therapy) rather than qualitative (no risk, or adverse response to therapy). [8] [10] These variations are often unimportant (for example, angiotensin converting enzyme inhibitors exhibit similar beneficial effects in patients with systolic congestive heart failure regardless of etiology, severity of symptoms, age or sex) [11] or easily remediable (drug dosages can be adjusted based on individual patient responsiveness).
Restricting efficacious therapies to "ideal patients" may result in significant harm to those excluded. For example, while beta-blockers are prescribed to only a minority of patients with acute myocardial infarction, myocardial infarction patients with concomitant conditions that might lead clinicians to withhold treatment (such as peripheral vascular disease, diabetes mellitus, heart failure, or chronic obstructive pulmonary disease) derive substantial survival benefits from beta blocker therapy. [12] This message is a consistent theme emerging from cardiovascular outcomes research. [13]
One key element in extrapolating the results of the carotid endarterectomy trial is local surgical expertise, since the net benefits in the trial were highly sensitive to perioperative complication rates. In fact, the benefits from carotid endarterectomy in this trial (expressed as relative risk reduction [RRR] in disabling stroke) would be reduced by 20% for each 2% absolute increase in the rate of perioperative stroke and death. [14] Moreover, surgical teams whose complication rates and operative volumes would have rendered them ineligible for the trial do most endarterectomies [15] Thus, as pointed out by others, "caution should be exercised in drawing conclusions about the effectiveness of carotid endarterectomy in the general population on the basis of trials of clinical efficacy conducted at highly selected facilities." [15]
Individualizing Treatment Decisions
The process of individualizing research evidence to the care of a particular patient incorporates two components: determining the likelihood that treatment will prevent the target event (at the expense of adverse events) in that patient and incorporating that patient’s values. We will now consider both of these steps in some depth.
Determining the Benefit: Risk Ratio in an Individual Patient
Although we can summarize the results of randomized trials with binary outcomes in a number of ways, the number of patients one would need to treat to prevent one additional event (NNT) [16] has gained widespread acceptance as one clinically relevant format. [17] [18] The NNT is the inverse of the difference in absolute event rates between the experimental and control arms and, thus reflects baseline risk as well as treatment effect. [17] For example, the NNT to prevent one disabling stroke in patients with moderate carotid artery stenosis is 20. [Table 1] [4]
Table 1: Summary of risks and benefits of therapy
|
Analogous to the NNT, the number needed to harm (NNH) is an expression of the number of patients who would need to receive an intervention to cause one additional adverse event. The NNH is the inverse of the absolute difference in adverse event rates between the experimental and control arms. For example, a meta-analysis of 51 studies of carotid endarterectomy in patients with symptomatic carotid stenosis found that the absolute peri-operative mortality rate was 1.6% higher with endarterectomy than medical treatment: this translates into an NNH to cause one additional death in the peri-operative period with carotid endarterectomy of 63, compared with withholding surgery. [Table 1] [19]
While one can easily calculate NNTs when investigators report event rates and relative risks, difficulties arise when investigators report only odds ratios (OR). Since the OR is not always an accurate estimate of the RR (particularly as disease incidence increases above 10%) [20], the clinician must employ standard formulae [18] to derive the NNT or NNH from the OR. [Table 2] [Table 3]
Table 2: Deriving the NNT from the odds ratio
Adapted from reference [18].
The formula for determining the NNT is:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Table 3: Deriving the NNH from the odds ratio
Adapted from reference [18]. The formula for
determining the NNH is:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The average NNT (or NNH) reported in a trial or systematic review may not be directly applicable to an individual patient (due to differences in baseline risk and/or RRR across subgroups), and the clinician is faced with three questions in extrapolating to their patient:
- Is my patient’s RRR likely to be different from the group average?
- What is my patient’s baseline risk of the target event?
- What is my patient’s risk of harm from the treatment?
While we often assume that RRR are constant across the limited range of susceptibilities normally encountered in clinical practice [21] [22] [23], recently published studies have demonstrated that while this is often the case [24] [25] [26] [27] [28] [29], it may not always be. [30] [31] [32] [33] [34] Thus, the clinician must carefully scrutinize the reports of trials or systematic reviews for information on the relative treatment effects in different subgroups. Other papers have outlined criteria for evaluating subgroup analyses and we will not reiterate the issues in this paper. [23] Suffice to say that in those situations where the RRR does appear to differ across subgroups, the clinician should employ the RRR from the subgroup most like their patient.
Returning to our clinical scenario, the RRR in stroke with carotid endarterectomy does differ by degree of stenosis and pre-surgical symptom status. [14] As our patients have symptomatic stenoses of 50-69%, it would be inappropriate to extrapolate the results from a trial of symptomatic patients with high-grade stenoses (>70%) [35] or a trial of asymptomatic patients with moderate stenoses [36] to their situation. However, it is entirely appropriate to extrapolate from the previously identified study [4] which enrolled symptomatic patients with similar degrees of stenoses as our patients.
We will now outline two approaches to addressing the latter two questions, our patient’s risk of adverse events without treatment, and our patient’s risk of harm with therapy. [21]
Approach 1: Generation of Patient-Specific Baseline Risks
Recognizing that patients are rarely identical to the average study patient, clinicians can derive estimates of the patient’s baseline risk from various sources. First, if the study reports risk in various subgroups, they can use the baseline risk for the subgroup most like their patient. However, most trials are not large enough to allow the generation of precise estimates of baseline risk in various patient subgroups and one may have to search for systematic reviews (particularly those including individual patient data) [37] to glean useful information. For example, the AF investigators pooled the individual patient data from all of the randomized trials testing antithrombotic therapy in non-valvular atrial fibrillation and were able to provide estimates of prognosis for patients in clinically important subgroups.[24]
Second, as an extension of the subgroup approach, one can use clinical prediction guides to quantitate an individual patient’s potential for benefit (and harm) from therapy. [32] [38] [39] Returning to our example, a prognostic model that could identify patients with carotid stenosis most likely to benefit from endarterectomy would be very useful. Such a model would need to incorporate the risk of stroke without surgery (and thus the potential benefit from surgery) with the risk of stroke or other adverse outcomes from surgery. Using the European Carotid Surgery Trial database [40], investigators have developed a preliminary version of just such a model. [41] However, our enthusiasm for applying this clinical prediction guide should be tempered until it has been prospectively validated in a different group of patients (and preferably with different clinicians). [38]
Third, one could derive an estimate of their patient’s baseline risk from published papers (preferably population-based cohort studies) [42] that describe the prognosis of similar (untreated) patients. For example, analysis of the Malmo Stroke Registry demonstrated that in the three years after a stroke, patients have a 6% risk of recurrent nonfatal stroke and a 43% risk of death; these risks were even higher in older patients or those with diabetes mellitus or cardiac disease. [43]
Analogous to the estimation of patient-specific baseline risk, clinicians can use these same sources of information to determine an individual patient’s likelihood of harm from treatment. For example, a systematic review of 36 studies relating the risk of peri-operative complications from carotid endarterectomy to various pre-operative clinical characteristics revealed that women were at higher risk than men (odds ratio 1.44 [95% CI 1.14 to 1.83], absolute rate 5.2%). [44]
The final step in generating a patient-specific NNT (or NNH) involves the formula: NNT=1/(PEER x RRR) (where PEER= the patient’s estimated event rate, or baseline risk). [21] Given the three year risk of recurrent disabling stroke in diabetic patients from the Malmo Stroke Registry (8.4%) and the 49% RRR expected with carotid endarterectomy, the patient-specific NNT in a 65 year old diabetic with ipsilateral carotid stenosis and a minor stroke would be calculated as: NNT=1/(0.084 x 0.49)=24. Clinicians who know a patient’s baseline risk and RRR can also call on a nomogram to calculate the NNT. [45]
Approach 2: Clinical Judgment
Alternately, one can use the study NNT and NNH directly to generate patient-specific estimates. This method involves only two steps and is less time-consuming than the previous method (as, depending on the experience of the clinician, it may not require detailed literature review).
First, the clinician estimates the patient’s risk of the outcome event relative to that of the average control patient in the study and converts this risk to a decimal fraction (= ‘ft’). [46] Thus, patients judged to be at less risk than those in the trials will be assigned an ft < 1 and those thought to be at greater risk will be assigned an ft > 1. There are several sources that a clinician could use to obtain a value for ‘ft’. The best estimate would come from a systematic review of all available data about the prognosis of similar patients; individual studies about prognosis would provide the next best estimates. Alternatively, she could use her clinical expertise in assigning a value to ‘ft’. While this may appear to be overly subjective, preliminary data suggests that experienced clinicians may be accurate in estimating relative differences in baseline risk (ie. ft) between patients (far exceeding our abilities to judge absolute risks). [47]
Second, the clinician calculates the patient-specific NNT by dividing the average NNT by ‘ft’. Thus, if the clinician felt that patient A was at one-fifth the risk of the average patient in the trial (based on the reduced baseline risk for women demonstrated in the subgroup analyses reported by the investigators) [4], her patient specific NNT for the prevention of one disabling stroke would be 100 (20/0.2).
In addition to considering the benefits from therapy, the clinician needs to consider a patient’s risk of adverse events from any intervention. Patients A and B need to be informed that carotid endarterectomy does carry with it a risk of peri-operative death. To individualize your patient’s risk of death, you can use the ‘f’ method just described. For example, patient A may be assumed to be at twice the risk (fh = 2) of peri-operative death as patients in the control group of the study because of her gender, hypertension, and the fact that she has left-sided carotid artery stenosis. [4] [44] You can adjust the NNH using ‘fh’, assuming the relative risk increase is constant across the spectrum of susceptibilities (an assumption which, as we’ve noted for RRR, may or may not hold depending on the particular therapy being considered). Thus, patient A’s NNH is estimated to be 32 (63/2).
Incorporating Patient Values and Preferences
We have determined the risks of benefit and harm for the individual, but we must still incorporate patient values into the decision-making process. As outlined in a previous Users’ Guide, [9] systematically-constructed decision analyses and practice guidelines that include an explicit statement of values can be used to integrate the evidence on benefit/harm with patient values to reach treatment recommendations or establish threshold NNTs. [9] [48] Although this situation would be ideal, such evidence is often not available (we could not identify a relevant decision analysis for our scenario). Moreover, as there is often substantial variation in values between individuals, [49] [50] [51] decision analyses which rely on group averages for values may not always be applicable to a particular patient, although close examination of the utility sensitivity analyses can often provide some guidance. [52] [53] [54]
While active patient involvement in decision making can improve outcomes and reported quality of life, and possibly reduce health care expenditures, [55] [56] [57] [58] [59] [60] [61] the initial step in this process is to determine the extent to which your patient wants to be involved with decision-making (recognizing that this may vary with each clinical decision).
How Much Do Patients Want to Participate?
There are 3 main elements to clinical decision making: the disclosure of information (about the risks and benefits of therapeutic alternatives); the exploration of the patient’s values about both the therapy and the potential health outcomes; and, the actual decision. Each patient varies in their desired level of involvement with these steps and clinicians may not accurately gauge the degree to which an individual patient wants to be involved. [62] [63] [64] [65] [66] [67] Some patients may want all available information provided to them and to make the decision themselves with the clinician’s role being that of information provider. Other patients may want all the information provided but may want the clinician to make the final decision. Still others may want to collaborate with their clinician in the process. These differences emphasize the need for clinicians to accurately assess patient preferences for information, discussion and decision-making, and tailor their approach to the individual.
Regardless of whether the clinician, the patient, or the partnership will make the decision, clinicians must explore patients’ values about the therapy and the potential health outcomes. You can elicit your patient’s values in informal ways during exploratory discussions with him/her or by more formal (and time-consuming) methods such as the time-tradeoff, standard gamble or rating scale techniques. [68]
Decision Aids
If your patient’s goal is shared decision making, there are several models for providing shared decision-making support. First, formal clinical decision analysis, incorporating that patient’s likelihood of the outcome events with their own values for each health state, could be used to guide the decision. Performing a clinical decision analysis for each patient would be too time-consuming for the busy clinician, and this approach therefore currently relies on finding an existing decision analysis. While this is the case, either our patient’s values must approximate those in the analysis, or the decision analysis must provide information about the impact of variation in patient values. Computer models available at the bedside may broaden the scope of decision analysis applicability, and permit wider use with individual patients. [69]
Second, investigators have developed numerical methods of presenting information to patients that incorporate calculated patient values though these methods haven’t been fully tested. [39] [70] Third, clinicians can utilize "decision aids" that present descriptive and probabilistic information about the disease, treatment options, and potential outcomes. [71] [72] [73] Most commonly, these decision aids present the outcome data in terms of the percentage of people with a certain condition who do well without intervention compared to the percentage who do well with intervention. While each of these methods has considerable merit, they sometimes fall short in terms of comprehensibility, applicability, and efficiency for use on busy clinical services.
The Likelihood of Being Helped or Harmed
One method of expressing information to patients that incorporates their values, can be applied to any clinical decision, and which preliminary evidence suggests may be useful on busy clinical services is the likelihood of being helped versus harmed. [74] The first step in this method is the exploration of patient values about taking the treatment (relative to not taking it) and the severity of adverse events that might be caused by the treatment (relative to the severity of the target event that we hope to avoid with the treatment). To answer these questions, patients are provided with brief descriptions of both the target event we’d like to prevent and the potential adverse event from the treatment. [Table 4]
Table 4: Sample descriptions of stroke and death
|
Following the review of the description of the target event, the clinician presents the patient with a rating scale (anchored at 0 [=death] and 1 [=full health]) and asks her to place a mark where she would consider the value of the target event.
During your discussions with Patient A, you discover that she is a fiercely independent newspaper journalist who lives alone and previously cared for her father after he suffered a disabling stroke. She believes that a disabling stroke is almost as bad as immediate death and assigns it a value of 0.025. Similarly, you give your patient the description of the adverse event that could result from the therapy (death within 30 days of surgery) and ask her to assess this using the rating scale (she assigned a value of 0.15 since death may not necessarily be immediate). Using the two ratings, you could infer that she believes a disabling stroke to be six times worse than death within the next month (0.15/0.025). This exercise should be repeated on another occasion to confirm that her values are stable.
In contrast, during your conversation with Patient B, you find that he is a former truck driver who recently retired to the country with his wife so that he could be near his daughter and grandson. When you explore his values, he decides that death is 8 times worse than having a disabling stroke.
How can you now incorporate your individual patient’s values into the description of therapy? The average patient with a hemispheric stroke has a 10.3% chance of having a disabling stroke over 5 years, [Table 1] but this can be decreased for patients with ipsilateral moderate carotid stenosis to 5.3% with carotid endarterectomy. [4] The average NNT for such patients is 20. The absolute risk increase for death for patients having carotid endarterectomy is 1.6% [19], which translates to an average NNH of 63 (1/0.016).
To calculate the likelihood of being helped versus harmed (LHH), 1/NNT (=ARR) and 1/NNH (=ARI) are combined into an aggregate ratio. For both patients, the first approximation of the LHH is given by: LHH = (1/NNT) : (1/NNH) = (1/20) : (1/63) = 3 to 1 in favor of surgery. As a first approximation, both patients can be told that ‘carotid endarterectomy is three times as likely to help you as harm you’.
However, this first approximation ignores both patients’ unique individual risks of, and values for, stroke and perioperative death. You can particularize the LHH for each patient using the ‘f’ factors we described previously. As discussed above, women have a lower risk of stroke and the ‘ft’ for Patient A can be estimated at approximately 0.2. [4] This study (and a systematic review of other studies) [44] found that women, patients with left sided carotid disease, and patients with a history of hypertension have increased risks of perioperative deaths (relative risks ranging from 1.4 to 2.3). Thus, Patient A is at an increased risk of death from surgery (fh = 2). Her risk-adjusted LHH is: LHHA = (1/NNT) x ft : (1/NNH) x fh = (1/20) x 0.2 : (1/63) x 2 = 3 to 1 in favor of medical therapy . Similarly, the LHH for Patient B can be particularized for his unique risks. Men had a greater risk of stroke in the trial[4] and you can estimate from the reported subgroup analyses that Patient B’s ft is approximately 1.25. Patient B also has left-sided carotid disease, suggesting that his risk of perioperative death is increased (fh = 2). His risk-adjusted LHH is: LHHB = (1/20) x 1.25 : (1/63) x 2 = 2 to 1 in favor of surgery.
These risk-adjusted LHHs still ignore each patient’s values. Patient A ranked a disabling stroke as 6 times worse than death and this number (the ‘s’ or ‘severity’ factor) can be used to adjust the LHH as follows: LHHA= (1/NNT) x ft x s: (1/NNH) x fh = (1/20) x 0.2 x 6 : (1/63) x 2 = 2 to 1 in favor of surgery. Thus, incorporating Patient A’s values and unique risks of benefit and harm, she is twice as likely to be helped as harmed by surgery. On the other hand, Patient B stated that death was 8 times worse than a stroke and incorporating this into his LHH you calculate: LHHB = (1/20) x 1.25 : (1/63) x 2 x 8 = 4 to 1 in favor of medical therapy.
These two cases illustrate how to incorporate your patient’s values into the decision-making process. At present, this process is time consuming and inexact, and we don’t know how much difference it makes to patients or their clinical outcomes, so this approach is best considered as a logical and feasible, but untested, model. If you are unsure of your patient’s ‘f’ or if there is some uncertainty around your patient’s estimate of values, you could do a sensitivity analysis (inserting different values for these variables into the above equation to see how this is reflected in the LHH). We’ve described a simple formulation for the LHH (ignoring other outcomes from carotid endarterectomy and the risks of the diagnostic workup) [75], but this could be modified for more complex situations.
Resolution of Clinical Scenario
Before making a final decision with your patient, you need to determine what the perioperative complication rate is in your own practice setting. Assuming that local surgical expertise is sufficient to apply the study results, and using Patient A’s individual risk of benefit and harm from surgery, adjusted for her values, carotid endarterectomy appears to be more likely to help her than to harm her. In contrast, using Patient B’s expected event rates and individual values, medical therapy would seem the favored management plan.
Acknowledgements
FAM is a Population Health Investigator of the Alberta Heritage Foundation for Medical Research. The authors thank Dr. Peter Rothwell for providing unpublished information on reference [41].
References
1. Sauve JS, Laupacis A, Ostbye T, Feagan B, Sackett DL. Does this patient have a clinically important carotid bruit? JAMA 1993;270:2843-5.
2. Eliasziw M, Rankin RN, Fox AJ, Haynes RB, Barnett HJ. Accuracy and prognostic consequences of ultrasonography in identifying severe carotid artery stenosis. Stroke 1995;26:1747-52.
3. Eliasziw M, Streifler JY, Fox AJ, et al. Significance of plaque ulceration in symptomatic patients with high-grade carotid stenosis. Stroke 1994;25:304-8.
4. Barnett HJM, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, et al. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. N Engl J Med 1998;339:1415-25.
5. Yusuf S, Held P, Teo KK. Selection of patients for randomized controlled trials: implications of wide or narrow eligibility criteria. Stat Med 1990;9:73-86.
6. Guyatt GH, Sackett DL, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ Guides to the Medical Literature II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA 1993;270:2598-2601.
7. Guyatt GH, Sackett DL, Cook DJ, for the Evidence-Based Medicine Working Group. Users’ Guides to the Medical Literature II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA 1994;271:59-63.
8. Dans AL, Dans LF, Guyatt GH, Richardson S, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature. XIV. How to decide on the applicability of clinical trial results to your patient. JAMA 1998;279:545-9.
9. Guyatt GH, Sinclair J, Cook DJ, Glasziou P, for the Evidence-Based Medicine Working Group.. Users’ guides to the medical literature. XVI. How to use a treatment recommendation. JAMA 1999;281:1836-43.
10. Glasziou P, Guyatt GH, Dans AL, Dans LF, Straus S, Sackett DL. Applying the results of trials and systematic reviews to individual patients.[EBM Note] Evidence-Based Medicine 1998 Nov-Dec;3:165-6.
11. Garg R, Yusuf S, for the Collaborative Group on ACE Inhibitor Trials. Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure. JAMA 1995;273:1450-6.
12. Gottlieb SS, McCarter RJ, Vogel RA. Effect of beta-blockade on mortality among high-risk and low-risk patients after myocardial infarction. N Engl J Med 1998;339:489-97.
13. McAlister FA, Taylor L, Teo KK, Tsuyuki RT, Ackman ML, Yim R, et al. The treatment and prevention of coronary heart disease in Canada: do older patients receive efficacious therapies? J Am Geriatr Soc 1999;(in press).
14. Chassin MR. Appropriate use of carotid endarterectomy. N Engl J Med 1998;339:1468-71.
15. Tu JV, Hannan EL, Anderson GM, Iron K, Wu K, Vranizan K, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med 1998;339:1441-7.
16. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988;318:1728-33.
17. Sackett DL, Haynes RB. Summarising the effects of therapy: a new table and some more terms.[EBM Note] Evidence-Based Medicine 1997 Jul-Aug;2:103-4.
18. McQuay HJ, Moore RA. Using numerical results from systematic reviews in clinical practice. Ann Intern Med 1997;126:712-20.
19. Rothwell PM, Slattery J, Warlow CP. A systematic review of the risks of stroke and death due to endarterectomy for symptomatic carotid stenosis. Stroke 1996;27:260-5.
20. Sackett DL, Deeks JJ, Altman DG. Down with odds ratios![EBM Note] Evidence-Based Medicine. 1996 Sep-Oct;1:164-6.
21. Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB. Evidence-based Medicine. How to practice and teach EBM. London: Churchill-Livingstone, 1999.
22. Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA 1991;266:93-8.
23. Oxman AD, Guyatt GH. A consumer’s guide to subgroup analyses. Ann Intern Med 1992;116:78-84.
24. Atrial Fibrillation Investigators. Risk factors for stroke and efficacy of antithrombotic therapy in atrial fibrillation. Analysis of pooled data from five randomized controlled trials. Arch Intern Med 1994;154:1449-57.
25. Scandinavian Simvastatin Survival Study Group. Baseline serum cholesterol and treatment effect in the Scandinavian Simvastatin Survival Study (4S). Lancet 1995;345:1274-5.
26. ACE Inibitor Myocardial Infarction Collaborative Group. Indications for ACE Inhibitors in the early treatment of acute myocardial infarction. Systematic overview of individual data from 100 000 patients in randomized trials. Circulation 1998;97:2202-12.
27. Fibrinolytic Therapy Trialists’ Collaborative Group. Indications for fibrinolytic therapy in suspected acute myocardial infarction: collaborative overview of early mortality and major morbidity results from all randomised trials of more than 1000 patients. Lancet 1994;343:311-22.
28. Yusuf S, Zucker D, Peduzzi P, Fisher LD, Takaro T, Kennedy JW, et al. Effect of coronary artery bypass graft surgery on survival: overview of 10-year results from randomised trials by the Coronary Artery Bypass Graft Surgery Trialists Collaboration. Lancet 1994;344:563-70.
29. Collins R, Peto R, MacMahon S, Hebert P, Fiebach NH, Eberlein KA, et al. Blood pressure, stroke, and coronary heart disease. Part 2, short-term reductions in blood pressure: overview of randomised drug trials in their epidemiological context. Lancet 1990;335:827-38.
30. Davey Smith G, Egger M. Who benefits from medical interventions? Treating low risk patients can be a high risk strategy. BMJ 1994;308:72-4.
31. Sharp SJ, Thompson SG, Altman DG. The relation between treatment benefit and underlying risk in meta-analysis. BMJ 1996;313:735-8.
32. Rothwell PM. Can overall results of clinical trials be applied to all patients? Lancet 1995;345:1616-9.
33. Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med 1998;17:1923-42.
34. Ioannidis JP, Lau J. Heterogeneity of the baseline risk within patient populations of clinical trials: a proposed evaluation algorithm. Am J Epidemiol 1998;148:1117-26.
35. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med 1991;325:445-53.
36. The Executive Committee for the Asymptomatic Carotid Atherosclerosis Study. Endarterectomy for asymptomatic carotid artery stenosis. JAMA 1995;273:1421-8.
37. Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data. Stat Med 1995;14:2057-79.
38. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules. A review and suggested modifications of methodological standards. JAMA 1997;277:488-94.
39. Glasziou PP, Irwig LM. An evidence based approach to individualising treatment. BMJ 1995;311:1356-9.
40. The European Carotid Surgery Trialists’ Collaborative Group. Randomised trial of endarterectomy for recently symptomatic carotid stenosis: final results of the MRC European Carotid Surgery Trial (ECST). Lancet 1998;351:1379-87.
41. Rothwell PM, Warlow CP. Analysis of treatment effects in individual patients: prediction of benefit from carotid endarterectomy. Lancet 1999;(in press).
42. Laupacis A, Wells G, Richardson WS, Tugwell P, for the Evidence-Based Medicine Working Group. User’s guides to the medical literature. V. How to use an article about prognosis. JAMA 1994;272:234-7.
43. Elneihoum AM, Goransson M, Falke P, Janzon L. Three-year survival and recurrence after stroke in Malmo, Sweden. An analysis of stroke registry data. Stroke 1998;29:2114-7.
44. Rothwell PM, Slattery J, Warlow CP. Clinical and angiographic predictors of stroke and death from carotid endarterectomy: systematic review. BMJ 1997;315:1571-7.
45. Chatellier G, Zapletal E, Lemaitre D, Menard J, Degoulet P. The number needed to treat: a clinically useful nomogram in its proper context. BMJ 1996;312:426-9.
46. Cook RJ, Sackett DL. The number needed to treat: a clinically useful measure of treatment effect. BMJ 1995;310:452-4.
47. Grover SA, Lowensteyn I, Esrey KL, et al. Do doctors accurately assess coronary risk in their patients? Preliminary results of the coronary health assessment study. BMJ 1995;310:975-8.
48. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature. IX. A method for grading health care recommendations. JAMA 1995;274:1800-4.
49. Solomon NA, Glick HA, Russo CJ, Lee J, Schulman KA. Patient preferences for stroke outcomes. Stroke 1994;25:1721-5.
50. Nease RF, Kneeland T, O’Connor GT et al. Variation in patient utilities for outcome of the management of chronic stable angina. Implications for clinical practice guidelines. JAMA 1995;273:1185-90.
51. Samsa GP, Matchar DB, Goldstein L, Bonito A, Duncan PW, Lipscomb J, et al. Utilities for major stroke: results from a survey of preferences among persons at increased risk for stroke. Am Heart J 1998;136:703-13.
52. Sculpher M. The cost-effectiveness of preference-based treatment allocation: the case of hysterectomy versus endometrial resection in the treatment of menorrhagia. Health Econ 1998;7:129-42.
53. Llewellyn-Thomas HA. Investigating patients’ preferences for different treatment options. Can J Nurs Res 1997;29:45-64.
54. Richardson WS, Detsky AS, for the Evidence-Based Medicine Working Group. Users’ Guides to the medical literature. VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients? JAMA 1995;273:1610-3.
55. Szabo E, Moody H, Hamilton T, Ang C, Kovithavongs C, Kjellstrand C. Choice of treatment improves quality of life: a study of patients undergoing dialysis. Arch Intern Med 1997;157:1352-6.
56. Greenfield S, Kaplan SH, Ware JE Jr, Yano EM, Frank HJL. Patients’ participation in medical care: effects on blood sugar control and quality of life in diabetes. J Gen Intern Med 1988;3:448-57.
57. Kaplan SH, Greenfield S, Ware JE Jr. Assessing the effects of physician-patient interactions on the outcomes of chronic disease. Med Care 1989;27(Suppl 3):S110-127.
58. Schulman BA. Active patient orientation and outcomes in hypertension treatment. Application of a socio-organizational perspective. Med Care 1979;17:267-80.
59. Stewart MA. Effective physician-patient communication and health outcomes: a review. CMAJ 1995;152:1423-33.
60. Vickery DM, Golaszewski TJ, Wright EC, Kalmer H. The effect of self-care interventions on the use of medical service within a Medicare population. Med Care 1988;26:580-88.
61. Gage BF, Cardinalli AB, Owens DK. Cost-effectiveness of preference-based antithrombotic therapy for patients with nonvalvular atrial fibrillation. Stroke 1998;29:1083-91.
62. Strull WM, Lo B, Charles G. Do patients want to participate in medical decision making? JAMA 1984;252:2990-4.
63. Degner LF, Kristjanson LJ, Bowman D et al. Information needs and decisional preferences in women with breast cancer. JAMA 1997;277:1485-92.
64. Stiggelbout A, Kiebert GM. A role for the sick role. CMAJ 1997;157:383-9.
65. Rothenbacher D, Lutz MP, Porzsolt F. Treatment decisions in palliative cancer care: patients’ preferences for involvement and doctors’ knowledge about it. Eur J Cancer 1997’33:1184-9.
66. Mazur DJ, Hickam DH. Patients’ preferences for risk disclosure and role I decision making for invasive medical procedures. J Gen Intern Med 1997;12:114-7.
67. Margalith I, Shapiro A. Anxiety and patient participation in clinical decision-making: the case of patients with ureteral calculi. Soc Sci Med 1997;45:419-27.
68. Torrance GW. Measurement of health state utilities for economic appraisal. A review. J Health Econ 1986;5:1-30.
69. Lilford RJ, Pauker SG, Braunholtz A, Chard J. Decision analysis and the implementation of research findings. BMJ 1998;317:405-9.
70. Riegelman R, Schroth WS. Adjusting the number needed to treat. Incorporating adjustments for the utility and timing of benefits and harms. Med Decis Making 1993;13:247-52.
71. Llewellyn-Thomas HA, McGreal MJ, Thiel EC et al. Patients’ willingness to enter clinical trials: Measuring the association with perceived benefit and decision making preference. Soc Sci Med 1991;32:35-42.
72. Levine MN, Gafni A, Markham B, MacFarlane D. A bedside decision instrument to elicit a patient’s preference concerning adjuvant chemotherapy for breast cancer. Ann Intern Med 1992;117:53-8.
73. O’Connor AM. Consumer/patient decision support in the new millenium: where should our research take us? Can J Nurs Res 1997;29:7-12.
74. Straus SE, McQuay HJM, Moore RA, Sackett DL. A proposed patient-centred method of delivering information about the risks and benefits of therapy: the likelihood of being helped versus harmed. Lancet (submitted)
75. Bain DJ, Fergie N, Quin RO, Greene M. Role of arteriography in the selection of patients for carotid endarterectomy. Br J Surg 1998;85:768-70.
© 2001 Evidence-Based Medicine Informatics Project
