Centre for Health Evidence: Home » Users' Guides to EBP
|
Centre for Health Evidence: Home » Users' Guides to EBP |
Finlay A. McAlister, Sharon E. Straus, Gordon H. Guyatt, R. Brian Haynes, for the Evidence Based Medicine Working Group
Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (2000;283(21):2829-2836). Copyright 2000, American Medical Association.
You are the attending physician on an internal medicine which, one night, admits two patients with strokes (a 65 year old woman - Patient A - and a 65 year old man - Patient B). On examination, both have mild weakness of the right arm and left carotid bruits. Patient A has a history of hypertension and an admission blood pressure of 200/110 mm Hg; neither patient has other relevant medical history or physical examination findings. Aware that carotid bruits are not highly specific for identifying carotid artery stenosis, you send both patients for doppler ultrasonography. [1] Since your radiology department, in a recent audit, demonstrated that their ultrasonographic interpretations are highly correlated with angiographic results [2] you feel confident in their findings that both patients have moderate stenoses (50-69% by NASCET criteria) with no irregularity or ulceration of the plaque surface. [3] Aware of the recent flurry of literature concerning surgical versus medical therapy for patients with symptomatic carotid stenoses, you decide to review the literature to guide your management of these patients. You formulate the question: "In a patient with a mild stroke and moderate ipsilateral carotid stenosis, would a carotid endarterectomy (compared to best medical therapy) reduce the likelihood of subsequent severe stroke or death?"
A systematic review of randomized trials comparing carotid endarterectomy with standard medical therapy (aspirin in your practice setting) in patients with recent, mild strokes would provide the best evidence addressing your question. You check the database of systematic reviews maintained by the Cochrane Collaboration (The Cochrane Library, Issue 1, 1999. Oxford: Update Software), but find no relevant reviews. You therefore use the advanced screen search of PubMed for the next level of evidence, a single randomized trial. You choose the medical subject headings stroke and carotid endarterectomy and include randomized controlled trial as the publication type. The search identifies 117 articles of which 1 looks most relevant to your question. [4]
Investigators in this trial randomized 2267 patients with moderate carotid stenosis (less than 70%) and ipsilateral transient ischemic attacks or nondisabling stroke within 180 days to carotid endarterectomy or medical care alone. After five years of follow-up, significantly fewer patients in the carotid endarterectomy arm (vs. the medical care arm) had suffered a recurrent disabling stroke (5.3% vs. 10.3%, 49% relative risk reduction {RRR} [95% CI 14% to 83%]) or death (13% vs. 14.9%, 13% RRR [95% CI - 18% to 44%]). The size of the treatment effect was such that 20 patients (95% CI 12 to 70) would have to undergo carotid endarterectomy to prevent 1 more disabling stroke than with medical therapy alone. Although encouraged by these results, you are concerned about the wide confidence intervals, the potential for perioperative complications (1.4% excess risk of disabling stroke or death within the first month of surgery), and question how to apply the results to your patients.
While randomized trials provide the most valid estimates of the true effects (both beneficial and harmful) of an intervention, they necessarily report average treatment effects. Whether these results are derived from a homogeneous group of high-risk, highly responsive patients (as in efficacy trials) or a heterogeneous group of "all-comers" (as in effectiveness trials), [5] clinicians must decide how to extrapolate them to individual patients. In this article, we will build upon previous Users’ Guides [6] [7] [8] [9] that assessed the validity and applicability of therapeutic studies to outline a framework that clinicians might use to integrate research results (whether from single trials or systematic reviews) with patient values to determine the optimal care for an individual patient.
Previous Users’ Guides and other articles have dealt extensively with these issues. [7] [8] [9] [10] We will not repeat all of the key principles here, but will emphasize that differences between study participants and patients in real-world practice tend to be quantitative (differences in degree of risk of the outcome or responsiveness to therapy) rather than qualitative (no risk, or adverse response to therapy). [8] [10] These variations are often unimportant (for example, angiotensin converting enzyme inhibitors exhibit similar beneficial effects in patients with systolic congestive heart failure regardless of etiology, severity of symptoms, age or sex) [11] or easily remediable (drug dosages can be adjusted based on individual patient responsiveness).
Restricting efficacious therapies to "ideal patients" may result in significant harm to those excluded. For example, while beta-blockers are prescribed to only a minority of patients with acute myocardial infarction, myocardial infarction patients with concomitant conditions that might lead clinicians to withhold treatment (such as peripheral vascular disease, diabetes mellitus, heart failure, or chronic obstructive pulmonary disease) derive substantial survival benefits from beta blocker therapy. [12] This message is a consistent theme emerging from cardiovascular outcomes research. [13]
One key element in extrapolating the results of the carotid endarterectomy trial is local surgical expertise, since the net benefits in the trial were highly sensitive to perioperative complication rates. In fact, the benefits from carotid endarterectomy in this trial (expressed as relative risk reduction [RRR] in disabling stroke) would be reduced by 20% for each 2% absolute increase in the rate of perioperative stroke and death. [14] Moreover, surgical teams whose complication rates and operative volumes would have rendered them ineligible for the trial do most endarterectomies [15] Thus, as pointed out by others, "caution should be exercised in drawing conclusions about the effectiveness of carotid endarterectomy in the general population on the basis of trials of clinical efficacy conducted at highly selected facilities." [15]
The process of individualizing research evidence to the care of a particular patient incorporates two components: determining the likelihood that treatment will prevent the target event (at the expense of adverse events) in that patient and incorporating that patient’s values. We will now consider both of these steps in some depth.
Although we can summarize the results of randomized trials with binary outcomes in a number of ways, the number of patients one would need to treat to prevent one additional event (NNT) [16] has gained widespread acceptance as one clinically relevant format. [17] [18] The NNT is the inverse of the difference in absolute event rates between the experimental and control arms and, thus reflects baseline risk as well as treatment effect. [17] For example, the NNT to prevent one disabling stroke in patients with moderate carotid artery stenosis is 20. [Table 1] [4]
Table 1: Summary of risks and benefits of therapy
|
Analogous to the NNT, the number needed to harm (NNH) is an expression of the number of patients who would need to receive an intervention to cause one additional adverse event. The NNH is the inverse of the absolute difference in adverse event rates between the experimental and control arms. For example, a meta-analysis of 51 studies of carotid endarterectomy in patients with symptomatic carotid stenosis found that the absolute peri-operative mortality rate was 1.6% higher with endarterectomy than medical treatment: this translates into an NNH to cause one additional death in the peri-operative period with carotid endarterectomy of 63, compared with withholding surgery. [Table 1] [19]
While one can easily calculate NNTs when investigators report event rates and relative risks, difficulties arise when investigators report only odds ratios (OR). Since the OR is not always an accurate estimate of the RR (particularly as disease incidence increases above 10%) [20], the clinician must employ standard formulae [18] to derive the NNT or NNH from the OR. [Table 2] [Table 3]
Table 2: Deriving the NNT from the odds ratio
Adapted from reference [18].
The formula for determining the NNT is:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Table 3: Deriving the NNH from the odds ratio
Adapted from reference [18]. The formula for
determining the NNH is:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
The average NNT (or NNH) reported in a trial or systematic review may not be directly applicable to an individual patient (due to differences in baseline risk and/or RRR across subgroups), and the clinician is faced with three questions in extrapolating to their patient:
While we often assume that RRR are constant across the limited range of susceptibilities normally encountered in clinical practice [21] [22] [23], recently published studies have demonstrated that while this is often the case [24] [25] [26] [27] [28] [29], it may not always be. [30] [31] [32] [33] [34] Thus, the clinician must carefully scrutinize the reports of trials or systematic reviews for information on the relative treatment effects in different subgroups. Other papers have outlined criteria for evaluating subgroup analyses and we will not reiterate the issues in this paper. [23] Suffice to say that in those situations where the RRR does appear to differ across subgroups, the clinician should employ the RRR from the subgroup most like their patient.
Returning to our clinical scenario, the RRR in stroke with carotid endarterectomy does differ by degree of stenosis and pre-surgical symptom status. [14] As our patients have symptomatic stenoses of 50-69%, it would be inappropriate to extrapolate the results from a trial of symptomatic patients with high-grade stenoses (>70%) [35] or a trial of asymptomatic patients with moderate stenoses [36] to their situation. However, it is entirely appropriate to extrapolate from the previously identified study [4] which enrolled symptomatic patients with similar degrees of stenoses as our patients.
We will now outline two approaches to addressing the latter two questions, our patient’s risk of adverse events without treatment, and our patient’s risk of harm with therapy. [21]
Recognizing that patients are rarely identical to the average study patient, clinicians can derive estimates of the patient’s baseline risk from various sources. First, if the study reports risk in various subgroups, they can use the baseline risk for the subgroup most like their patient. However, most trials are not large enough to allow the generation of precise estimates of baseline risk in various patient subgroups and one may have to search for systematic reviews (particularly those including individual patient data) [37] to glean useful information. For example, the AF investigators pooled the individual patient data from all of the randomized trials testing antithrombotic therapy in non-valvular atrial fibrillation and were able to provide estimates of prognosis for patients in clinically important subgroups.[24]
Second, as an extension of the subgroup approach, one can use clinical prediction guides to quantitate an individual patient’s potential for benefit (and harm) from therapy. [32] [38] [39] Returning to our example, a prognostic model that could identify patients with carotid stenosis most likely to benefit from endarterectomy would be very useful. Such a model would need to incorporate the risk of stroke without surgery (and thus the potential benefit from surgery) with the risk of stroke or other adverse outcomes from surgery. Using the European Carotid Surgery Trial database [40], investigators have developed a preliminary version of just such a model. [41] However, our enthusiasm for applying this clinical prediction guide should be tempered until it has been prospectively validated in a different group of patients (and preferably with different clinicians). [38]
Third, one could derive an estimate of their patient’s baseline risk from published papers (preferably population-based cohort studies) [42] that describe the prognosis of similar (untreated) patients. For example, analysis of the Malmo Stroke Registry demonstrated that in the three years after a stroke, patients have a 6% risk of recurrent nonfatal stroke and a 43% risk of death; these risks were even higher in older patients or those with diabetes mellitus or cardiac disease. [43]
Analogous to the estimation of patient-specific baseline risk, clinicians can use these same sources of information to determine an individual patient’s likelihood of harm from treatment. For example, a systematic review of 36 studies relating the risk of peri-operative complications from carotid endarterectomy to various pre-operative clinical characteristics revealed that women were at higher risk than men (odds ratio 1.44 [95% CI 1.14 to 1.83], absolute rate 5.2%). [44]
The final step in generating a patient-specific NNT (or NNH) involves the formula: NNT=1/(PEER x RRR) (where PEER= the patient’s estimated event rate, or baseline risk). [21] Given the three year risk of recurrent disabling stroke in diabetic patients from the Malmo Stroke Registry (8.4%) and the 49% RRR expected with carotid endarterectomy, the patient-specific NNT in a 65 year old diabetic with ipsilateral carotid stenosis and a minor stroke would be calculated as: NNT=1/(0.084 x 0.49)=24. Clinicians who know a patient’s baseline risk and RRR can also call on a nomogram to calculate the NNT. [45]
Alternately, one can use the study NNT and NNH directly to generate patient-specific estimates. This method involves only two steps and is less time-consuming than the previous method (as, depending on the experience of the clinician, it may not require detailed literature review).
First, the clinician estimates the patient’s risk of the outcome event relative to that of the average control patient in the study and converts this risk to a decimal fraction (= ‘ft’). [46] Thus, patients judged to be at less risk than those in the trials will be assigned an ft < 1 and those thought to be at greater risk will be assigned an ft > 1. There are several sources that a clinician could use to obtain a value for ‘ft’. The best estimate would come from a systematic review of all available data about the prognosis of similar patients; individual studies about prognosis would provide the next best estimates. Alternatively, she could use her clinical expertise in assigning a value to ‘ft’. While this may appear to be overly subjective, preliminary data suggests that experienced clinicians may be accurate in estimating relative differences in baseline risk (ie. ft) between patients (far exceeding our abilities to judge absolute risks). [47]
Second, the clinician calculates the patient-specific NNT by dividing the average NNT by ‘ft’. Thus, if the clinician felt that patient A was at one-fifth the risk of the average patient in the trial (based on the reduced baseline risk for women demonstrated in the subgroup analyses reported by the investigators) [4], her patient specific NNT for the prevention of one disabling stroke would be 100 (20/0.2).
In addition to considering the benefits from therapy, the clinician needs to consider a patient’s risk of adverse events from any intervention. Patients A and B need to be informed that carotid endarterectomy does carry with it a risk of peri-operative death. To individualize your patient’s risk of death, you can use the ‘f’ method just described. For example, patient A may be assumed to be at twice the risk (fh = 2) of peri-operative death as patients in the control group of the study because of her gender, hypertension, and the fact that she has left-sided carotid artery stenosis. [4] [44] You can adjust the NNH using ‘fh’, assuming the relative risk increase is constant across the spectrum of susceptibilities (an assumption which, as we’ve noted for RRR, may or may not hold depending on the particular therapy being considered). Thus, patient A’s NNH is estimated to be 32 (63/2).
We have determined the risks of benefit and harm for the individual, but we must still incorporate patient values into the decision-making process. As outlined in a previous Users’ Guide, [9] systematically-constructed decision analyses and practice guidelines that include an explicit statement of values can be used to integrate the evidence on benefit/harm with patient values to reach treatment recommendations or establish threshold NNTs. [9] [48] Although this situation would be ideal, such evidence is often not available (we could not identify a relevant decision analysis for our scenario). Moreover, as there is often substantial variation in values between individuals, [49] [50] [51] decision analyses which rely on group averages for values may not always be applicable to a particular patient, although close examination of the utility sensitivity analyses can often provide some guidance. [52] [53] [54]
While active patient involvement in decision making can improve outcomes and reported quality of life, and possibly reduce health care expenditures, [55] [56] [57] [58] [59] [60] [61] the initial step in this process is to determine the extent to which your patient wants to be involved with decision-making (recognizing that this may vary with each clinical decision).
There are 3 main elements to clinical decision making: the disclosure of information (about the risks and benefits of therapeutic alternatives); the exploration of the patient’s values about both the therapy and the potential health outcomes; and, the actual decision. Each patient varies in their desired level of involvement with these steps and clinicians may not accurately gauge the degree to which an individual patient wants to be involved. [62] [63] [64] [65] [66] [67] Some patients may want all available information provided to them and to make the decision themselves with the clinician’s role being that of information provider. Other patients may want all the information provided but may want the clinician to make the final decision. Still others may want to collaborate with their clinician in the process. These differences emphasize the need for clinicians to accurately assess patient preferences for information, discussion and decision-making, and tailor their approach to the individual.
Regardless of whether the clinician, the patient, or the partnership will make the decision, clinicians must explore patients’ values about the therapy and the potential health outcomes. You can elicit your patient’s values in informal ways during exploratory discussions with him/her or by more formal (and time-consuming) methods such as the time-tradeoff, standard gamble or rating scale techniques. [68]
If your patient’s goal is shared decision making, there are several models for providing shared decision-making support. First, formal clinical decision analysis, incorporating that patient’s likelihood of the outcome events with their own values for each health state, could be used to guide the decision. Performing a clinical decision analysis for each patient would be too time-consuming for the busy clinician, and this approach therefore currently relies on finding an existing decision analysis. While this is the case, either our patient’s values must approximate those in the analysis, or the decision analysis must provide information about the impact of variation in patient values. Computer models available at the bedside may broaden the scope of decision analysis applicability, and permit wider use with individual patients. [69]
Second, investigators have developed numerical methods of presenting information to patients that incorporate calculated patient values though these methods haven’t been fully tested. [39] [70] Third, clinicians can utilize "decision aids" that present descriptive and probabilistic information about the disease, treatment options, and potential outcomes. [71] [72] [73] Most commonly, these decision aids present the outcome data in terms of the percentage of people with a certain condition who do well without intervention compared to the percentage who do well with intervention. While each of these methods has considerable merit, they sometimes fall short in terms of comprehensibility, applicability, and efficiency for use on busy clinical services.
One method of expressing information to patients that incorporates their values, can be applied to any clinical decision, and which preliminary evidence suggests may be useful on busy clinical services is the likelihood of being helped versus harmed. [74] The first step in this method is the exploration of patient values about taking the treatment (relative to not taking it) and the severity of adverse events that might be caused by the treatment (relative to the severity of the target event that we hope to avoid with the treatment). To answer these questions, patients are provided with brief descriptions of both the target event we’d like to prevent and the potential adverse event from the treatment. [Table 4]
Table 4: Sample descriptions of stroke and death
|
Following the review of the description of the target event, the clinician presents the patient with a rating scale (anchored at 0 [=death] and 1 [=full health]) and asks her to place a mark where she would consider the value of the target event.
During your discussions with Patient A, you discover that she is a fiercely independent newspaper journalist who lives alone and previously cared for her father after he suffered a disabling stroke. She believes that a disabling stroke is almost as bad as immediate death and assigns it a value of 0.025. Similarly, you give your patient the description of the adverse event that could result from the therapy (death within 30 days of surgery) and ask her to assess this using the rating scale (she assigned a value of 0.15 since death may not necessarily be immediate). Using the two ratings, you could infer that she believes a disabling stroke to be six times worse than death within the next month (0.15/0.025). This exercise should be repeated on another occasion to confirm that her values are stable.
In contrast, during your conversation with Patient B, you find that he is a former truck driver who recently retired to the country with his wife so that he could be near his daughter and grandson. When you explore his values, he decides that death is 8 times worse than having a disabling stroke.
How can you now incorporate your individual patient’s values into the description of therapy? The average patient with a hemispheric stroke has a 10.3% chance of having a disabling stroke over 5 years, [Table 1] but this can be decreased for patients with ipsilateral moderate carotid stenosis to 5.3% with carotid endarterectomy. [4] The average NNT for such patients is 20. The absolute risk increase for death for patients having carotid endarterectomy is 1.6% [19], which translates to an average NNH of 63 (1/0.016).
To calculate the likelihood of being helped versus harmed (LHH), 1/NNT (=ARR) and 1/NNH (=ARI) are combined into an aggregate ratio. For both patients, the first approximation of the LHH is given by: LHH = (1/NNT) : (1/NNH) = (1/20) : (1/63) = 3 to 1 in favor of surgery. As a first approximation, both patients can be told that ‘carotid endarterectomy is three times as likely to help you as harm you’.
However, this first approximation ignores both patients’ unique individual risks of, and values for, stroke and perioperative death. You can particularize the LHH for each patient using the ‘f’ factors we described previously. As discussed above, women have a lower risk of stroke and the ‘ft’ for Patient A can be estimated at approximately 0.2. [4] This study (and a systematic review of other studies) [44] found that women, patients with left sided carotid disease, and patients with a history of hypertension have increased risks of perioperative deaths (relative risks ranging from 1.4 to 2.3). Thus, Patient A is at an increased risk of death from surgery (fh = 2). Her risk-adjusted LHH is: LHHA = (1/NNT) x ft : (1/NNH) x fh = (1/20) x 0.2 : (1/63) x 2 = 3 to 1 in favor of medical therapy . Similarly, the LHH for Patient B can be particularized for his unique risks. Men had a greater risk of stroke in the trial[4] and you can estimate from the reported subgroup analyses that Patient B’s ft is approximately 1.25. Patient B also has left-sided carotid disease, suggesting that his risk of perioperative death is increased (fh = 2). His risk-adjusted LHH is: LHHB = (1/20) x 1.25 : (1/63) x 2 = 2 to 1 in favor of surgery.
These risk-adjusted LHHs still ignore each patient’s values. Patient A ranked a disabling stroke as 6 times worse than death and this number (the ‘s’ or ‘severity’ factor) can be used to adjust the LHH as follows: LHHA= (1/NNT) x ft x s: (1/NNH) x fh = (1/20) x 0.2 x 6 : (1/63) x 2 = 2 to 1 in favor of surgery. Thus, incorporating Patient A’s values and unique risks of benefit and harm, she is twice as likely to be helped as harmed by surgery. On the other hand, Patient B stated that death was 8 times worse than a stroke and incorporating this into his LHH you calculate: LHHB = (1/20) x 1.25 : (1/63) x 2 x 8 = 4 to 1 in favor of medical therapy.
These two cases illustrate how to incorporate your patient’s values into the decision-making process. At present, this process is time consuming and inexact, and we don’t know how much difference it makes to patients or their clinical outcomes, so this approach is best considered as a logical and feasible, but untested, model. If you are unsure of your patient’s ‘f’ or if there is some uncertainty around your patient’s estimate of values, you could do a sensitivity analysis (inserting different values for these variables into the above equation to see how this is reflected in the LHH). We’ve described a simple formulation for the LHH (ignoring other outcomes from carotid endarterectomy and the risks of the diagnostic workup) [75], but this could be modified for more complex situations.
Before making a final decision with your patient, you need to determine what the perioperative complication rate is in your own practice setting. Assuming that local surgical expertise is sufficient to apply the study results, and using Patient A’s individual risk of benefit and harm from surgery, adjusted for her values, carotid endarterectomy appears to be more likely to help her than to harm her. In contrast, using Patient B’s expected event rates and individual values, medical therapy would seem the favored management plan.
FAM is a Population Health Investigator of the Alberta Heritage Foundation for Medical Research. The authors thank Dr. Peter Rothwell for providing unpublished information on reference [41].
© 2001 Evidence-Based Medicine Informatics Project
© 2001 Centre for Health Evidence.
Home.
Users' Guides to EBP.
Webmaster.
Disclaimer.