Centre for Health Evidence: Home » Users' Guides to EBP

Applying Clinical Trial Results.
Part A: How to Use an Article Measuring the Effect of an Intervention on Surrogate End Points

Heiner C. Bucher, Gordon H. Guyatt, Deborah J. Cook, Anne Holbrook, Finlay A. McAlister, for the Evidence-Based Medicine Working Group

Based on the Users' Guides to Evidence-based Medicine and reproduced with permission from JAMA. (1999;282(8):771-778). Copyright 1999, American Medical Association.

• Clinical Scenario
• Introduction
• Guides for the evaluation of surrogate end point trials
• Are the results valid?
• What were the results?
• Will the results help me in caring for my patients?
• Resolution of the Scenario
• Conclusion
• Acknowledgements
• References

Clinical Scenario

You are a physician seeing a 62-year-old woman with postmenopausal osteoporosis. Her bone mineral density as measured by dual-energy X-ray absorptiometry is 2.5 standard deviations below the mean value in premenopausal women and, though she does not suffer from back pain, a spinal radiograph shows an old vertebral fracture. Although she has not yet experienced problems as a result of her vertebral fracture, she is disturbed by the prospect that she may end up like her mother whose osteoporotic fractures have resulted in severe, chronic, back pain. The patient suffers from reflux esophagitis and a past endoscopy has revealed non-specific gastritis. A specialist had prescribed alendronate which the patient had to stop after several weeks because of dyspepsia. She has searched the Web, and discovered a new drug, raloxifene, and wonders whether this drug might be an alternative. You know that this drug has been licensed for the prevention of postmenopausal osteoporosis. You promise to examine the literature and to get back to her.

Using Medline you identify a study of raloxifene for the treatment of osteoporosis demonstrating an effect on bone mineral density [1]. You are wondering whether this warrants administration to lower your patient's risk of osteoporotic fracture.

Introduction

Ideally, clinicians making treatment decisions should always refer to methodologically strong clinical trials examining the impact of therapy on clinically important outcomes. By clinically important outcomes we mean outcomes that are important to patients such as health-related quality of life, morbid end points such as stroke or myocardial infarction, or death. Often, however, conducting these trials requires such large sample size, or long patient follow-up, that their feasibility becomes questionable. Substituting surrogate end points for the target event allows conduct of shorter and smaller trials, thus offering an apparent solution to the dilemma.

A surrogate end point may be defined as 'a laboratory measurement or a physical sign used as a substitute for a clinically meaningful end point that measures directly how a patient feels, functions or survives' [2]. Surrogate end points include physiologic variables (such as bone mineral density as a surrogate for long-bone fractures, blood pressure for stroke, LDL cholesterol levels for myocardial infarction, and CD4 cell count for AIDS and AIDS-related mortality) or measures of subclinical disease (such as degree of atherosclerosis on coronary angiography).

The use of surrogate end points is indispensable for drug evaluation in phase II and early phase III trials geared to establishing a drug's promise of benefit. In many countries, companies may obtain drug approval by demonstrating a positive impact on surrogate end points. The use of surrogate end points for regulatory purposes reflects drug approval decisions that regulators must make in the face of public health exigencies.

Reliance on surrogate end points may be beneficial or harmful. On the one hand, use of the surrogate end point may lead to the rapid and appropriate dissemination of new treatments. For example, the Food and Drug Administration's (FDA) decision to approve new antiretroviral drugs based on information from trials using surrogate end points recognized the enormous need for effective therapies for patients with HIV infection. Subsequently, several of these drugs have proved effective in randomized trials focusing on clinically important outcomes [3] [4] [5] [6].

On the other hand, reliance on surrogate endpoints may lead to excess morbidity and mortality. For example, while dihydropyridine calcium channel blockers are efficacious in lowering blood pressure, their effects on clinically important outcomes such as stroke, myocardial infarction, or death are less certain. While at least one of these agents has been shown to be beneficial in the treatment of hypertension [7], preliminary evidence from comparative trials of other dihydropyridines versus antihypertensive drugs from other classes suggests that they may be less beneficial than diuretics or angiotensin converting enzyme inhibitors [8] [9] [10] [11].

How are clinicians to distinguish between these two situations? A surrogate outcome will be reliable only if there is a validated causal connection between change in surrogate and change in the clinically important outcome, and if the surrogate fully captures all of the effects of treatment on that outcome. In this Users' Guide, we build on previous discussions of how one can establish a causal relationship [12] and present an approach to critical appraisal of studies using surrogate end points and application of their results to managing individual patients.

As our discussion will make evident, the clinician needs to assess far more than a single study to make the decision about the adequacy of a surrogate. Evaluation may require a comprehensive review of observational studies of the relation between the surrogate and the target, and of some or all of the randomized trials that have evaluated treatment impact on both the surrogate and the target. While most clinicians would hesitate to conduct such an investigation, our guidelines will allow them to evaluate experts', or pharmaceutical industry's, arguments for prescribing treatments on the basis of their effect on surrogate end points.

Guides for the evaluation of surrogate end point trials

In this guide, we follow the frame work of previous articles in the series [13] and ask three sorts of questions: Are the results valid? (In particular, is the surrogate end point a valid substitute for clinically important outcome?); What were the results?; and Will the results help me in caring for my patients? [Table 1]

Table 1: Users' guide for a surrogate end point trial Are the results valid? Necessary but not sufficient: Is there a strong, independent, consistent association between the surrogate end point and the clinical end point? Is there evidence from randomized trials in other drug classes that improvement in the surrogate end point has consistently lead to improvement in the target outcome?* Is there evidence from randomized trials in the same drug class that improvement in the surrogate end point has consistently lead to improvement in the target outcome?* * Answers to one or both of these questions must be yes for surrogate trial to be an adequate guide for clinical action. What were the results? How large, precise, and lasting was the treatment effect? Effect must be large, precise, and lasting to consider a surrogate trial as possible basis for offering patients the intervention Will the results help me in caring for my patients? Are the likely treatment benefits worth the potential harms and costs? Offer intervention on basis of surrogate data only if patient's risk of the target outcome high, patient places a high value on avoiding the target outcome, and no satisfactory alternative therapies

I. Are the results valid?

1. Is there a strong, independent, consistent association between the surrogate end point and the clinical end point?

To provide a valid substitute for an important target outcome, the surrogate must be associated or correlated with that target. In general, researchers choose surrogate end points because they have found a correlation between a surrogate and a target outcome in observational studies, and their understanding of the biology makes it plausible that changes in the surrogate will invariably lead to changes in the important outcome. The stronger the association, the more likely the causal link between the surrogate and the target. Many biologically plausible surrogates are only weakly associated with clinically important outcomes. For example, measures of respiratory function in patients with chronic lung disease, or conventional exercise tests in patients with heart and lung disease, are only weakly correlated with capacity to undertake activities of daily living [14] [15]. When correlations are low, the surrogate is likely to be a poor substitute for the target outcome.

In addition to the strength of the association, one's confidence in the validity of the association depends on whether it is consistent across different studies and after adjustment for known confounders. For example, ecologic studies such as the Seven Countries Study [16] suggested a strong correlation between serum cholesterol levels and coronary heart disease mortality even after adjusting for other predictors such as age, smoking, and systolic blood pressure. Subsequent cohort studies confirmed this association and suggested that long-term reductions in serum cholesterol of 0.6 mmol/L would lower the risk of coronary heart disease by approximately 30% [17].

Similarly, cohort studies have consistently revealed that a single measurement of plasma viral load predicts the subsequent risk of AIDS or death in HIV infected patients [18] [19] [20] [21] [22] [23]. For example, in one study the proportion of patients that progressed to AIDS after 5 years in the lowest through the highest quartiles of viral load was 8, 26, 49, and 62%, respectively [23]. Moreover, this association retained its predictive power after adjustment for other potential predictors such as CD4 cell count [18] [19] [20] [21] [22].

Returning to the scenario, you are wondering if you can substitute bone mineral density for fractures or health-related quality of life in considering whether to recommend raloxifene. A large cohort study investigated risk factors for hip fracture [24]. Postmenopausal women with a calcaneal bone density in the highest third had a hip fracture rate of 9.4 per 1000 woman-years while women in the middle and lowest third had a fracture rate per 1000 women year of 14.7 and 27.3, respectively. Furthermore, after considering other risk factors for osteoporotic hip fractures including maternal history of hip fracture, previous fractures from any site, poor self rated health, use of long acting benzodiazepines, impaired visual function, and reduced physical activity, bone mineral density continued to predict the risk of hip fracture [24].

These findings are consistent across studies looking at the association between bone density and fracture risk [25] [26]. Thus, bone mineral density is a moderately strong, independent predictor of fracture, and meets our first criterion for an acceptable surrogate end point.

While meeting this first criterion is necessary, it is not sufficient to support reliance on a surrogate outcome. As we will emphasize below [Table 1], before offering an intervention on the basis of effects on a surrogate outcome, the clinician should note a consistent relation between surrogate and target in randomized trials; the effect of the intervention on the surrogate must be large, precise and lasting; and the benefit-risk tradeoff must be clear.

2. Is there evidence from randomized trials in other drug classes that improvement in the surrogate end point has consistently lead to improvement in the target outcome?

Given the possibility of effects unrelated to the surrogate end point, pathophysiologic studies, ecological studies, and cohort studies are insufficient to establish that the link between surrogate and clinically important outcomes is ironclad. Surrogate end points can only be considered validated when their relationship with the clinically important outcome has been firmly established in long-term randomized trials showing that modification of the surrogate is associated with concomitant modifications in the target outcome of interest. For example, although ventricular ectopic beats are associated with adverse prognosis in patients with myocardial infarction [27] and class I antiarrhythmic agents effectively suppress ventricular arrhythmias in animals and humans [28], these drugs have proved to an increase mortality when evaluated in randomized trials [29]. In this case, reliance on the surrogate end point of suppression of non-lethal arrhythmias lead to the deaths of tens of thousands of patients [30].

The treatment of heart failure provides another instructive example. Trials of angiotensin converting enzyme (ACE) inhibitors in heart failure have demonstrated parallel increases in exercise capacity [31] [32] [33] [34] and a decrease in mortality [35], suggesting clinicians may be able to rely on exercise capacity as a valid surrogate. Milrinone [36] and epoprostol [37] have both demonstrated improved exercise tolerance in patients with symptomatic heart failure. However, when these drugs were evaluated in randomized controlled trials both showed an increase in cardiovascular mortality which in one instance was statistically significant [38], and in the second case led to the early termination of the study [39]. Thus, exercise tolerance is inconsistent in predicting improved mortality and is therefore an unsatisfactory substitute. Other suggested surrogate end points in heart failure have included ejection fraction, heart rate variability, and markers of autonomic function [40]. The dopaminergic agent ibopamine positively influences all three surrogate end points, and yet a randomized trial demonstrated that the drug increases mortality in heart failure [41].

CD4 cell count is an example of a surrogate end point which has been validated in randomized trials. A number of trials comparing different classes of antiretroviral therapies have demonstrated that patients randomized to more potent drug regimens had higher CD4 cell counts and were less likely to progress to AIDS or death [6] [42]. While there is no guarantee that the next trial using a different class of drugs will show the same pattern, these results greatly strengthen our inference that if therapy for HIV infection increases CD4 count, a reduction in AIDS-related mortality will result.

Returning to our scenario, trials of etidronate [43] [44] and alendronate [45] for the prevention of osteoporotic fractures in postmenopausal women have shown parallel increases in bone mineral density and reduced incidence of new vertebral fractures. This would suggest that clinicians might rely on bone density to evaluate new drugs in osteoporosis making the assumption that if they saw increases in bone density, decreases in fractures would follow.

However, another secondary prevention trial in post-menopausal women using sodium fluoride showed divergent results [46]. Although sodium fluoride increased bone mineral density at the lumbar spine by 35% over 5 years, more vertebral and non-vertebral fractures occurred in the intervention group than in the placebo group (163 and 72 in 101 women with sodium fluoride versus 136 and 24 in 101 women with placebo). In another, randomized trial, fluoride again showed a large increase in bone density without any change in fracture rate [47]. Inferences on the basis of unchanged bone density may also be problematic: a study of calcium and vitamin D in the elderly showed virtually no change in bone density, but a reduction in fracture risk of approximately 50% [48]. Thus, increase in bone mineral density as a surrogate end point has shown an inconsistent relation to osteoporotic fractures.

3. Is there evidence from randomized trials in the same drug class that improvement in the surrogate end point has consistently lead to improvement in the target outcome?

Clinicians are in a stronger position relying on surrogate end points if the new drug they are considering is from a class of drugs in which the relation between changes in the surrogate and changes in the target has been verified in randomized trials. For instance, thiazide diuretics and beta blockers have both been shown to reduce blood pressure and clinically important outcomes such as stroke in hypertensive patients. Thus, we would be much more comfortable relying on reduction in blood pressure to justify administering a new beta blocker or thiazide diuretic than to justify offering a novel antihypertensive agent from another class [49]. As alluded to in the Introduction, four trials have now shown that calcium channel blockers are less efficacious than thiazides or angiotensin converting enzyme inhibitors in preventing hard clinical endpoints despite exerting similar degrees of blood pressure lowering [8] [9] [10] [11].

We will consider the example of cholesterol reduction as a surrogate for cardiovascular outcomes such as myocardial infarction and death in Part B of this User's Guide [50]. Briefly, several large trials of primary and secondary prevention of coronary heart disease with statins have consistently shown that these drugs reduce cardiovascular outcomes [51]. We could therefore make a strong case that a new statin with a similar LDL-cholesterol lowering potency will also reduce clinically important outcomes. However, we would be very reluctant to generalize to another class of lipid- lowering agents since trials of one such class (the fibrates) have shown that these drugs reduce the incidence of myocardial infarction but increase the risk of mortality from other causes (with no impact on overall mortality) [51] [52] [53].

This criterion is complicated by the variable definitions of drug class. A manufacturer of a drug related to a class of agents with a consistent positive association between modification of a surrogate end point and modification of the target (such as a beta blocker) will naturally argue for a broad definition of class. Manufacturers of agents that are related to drugs with known or suspected adverse effects on target events (clofibrate, or some calcium antagonists) are likely to argue, on the other hand, that the chemical or physiological connection is not sufficiently close to consider the new drug to be in the same class as the harmful agent. Part B will address these issues more fully [50].

Returning to the scenario, we have established that because of the inconsistent relation between increase in bone mineral density and fracture reduction we would be reluctant to offer patients a new anti-osteoporotic agent solely on the basis of evidence of its effect on the surrogate end point. Raloxifene, the drug we are considering for our patient, is a nonsteroidal benzothiophene, a selective estrogen-receptor modulator and thus, represents a new class of drugs for the prevention of osteoporosis related bone fractures. Thus, it is likely that the mechanisms of action will be considerably different from bisphosphonates and an assumption that similar reductions in loss of bone density will lead to parallel reductions in clinical fractures is not warranted. In Table 2, we apply our validity criteria to a number of controversial examples of the use of surrogate end points.

Table 2: Selected examples of applied validity criteria for the critical evaluation of studies using surrogate end points Types of intervention (References) Surrogate end point End point Criterion 1 (References) Criterion 2 (References) Criterion 3 (References)       Is there a strong, independent, consistent association between the surrogate end point and the clinical end point? Is there evidence from randomized trials in other drug classes that improvement in the surrogate end point has consistently lead to improvement in the target outcome? Is there evidence from randomized trials in the same drug class that improvement in the surrogate end point has consistently lead to improvement in the target outcome? Nonsteroidal benzothiophene Raloxifene 1 Bone mineral density Osteoporotic fractures yes 24, 25, 26 no 46, 47 no 1, 63 Proteinase inhibitor‡ Nelfinavir 64 Reverse transcriptase inhibitor Abacavir 65 HIV-1 viral plasma load AIDS or death yes 18, 19, 20, 21, 22 yes 18, 19, 20, 21, 22 yes 66, 67, 68 yes 67, 68, 69 yes 6, 42 yes 66, 69 Proteinase inhibitor‡ Nelfinavir 64 Reverse transcriptase inhibitor Abacavir 65 CD4 cell count AIDS or death yes 18, 19, 20, 21, 22 yes 18, 19, 20, 21, 22 yes 66, 67, 68 yes 67, 68, 69 yes 6, 42 yes 66, 69 Antihypertensive drugs Amlodipine 70 New Thiazid diuretic Blood pressure reduction Stroke, MI, cardiovascular mortality yes 71, 72 yes 71, 72 yes 73 yes 73 no 74 yes 73 Antilipidemic drugs Atorvastatin 75, 76 Bezafibrate 77, 78 Cholesterol, or LDL-cholesterol reduction MI, death from MI yes 16, 17 yes 16, 17 no 51 no 51 yes 51 no 51 ‡ in combination therapy with two reverse transcriptase inhibitors MI: myocardial infarction

II. What were the results?

1. How large, precise, and lasting was the treatment effect?

We are interested not only in whether an intervention alters a surrogate end point, but also in the magnitude, precision, and duration of the effect. If an intervention shows large reductions in the surrogate end point, the 95% confidence intervals around those large reductions are narrow, and the effect persists over a sufficiently long period, our confidence that the target outcome will be favourably effected increases. Positive effects that are smaller, with wider confidence intervals, and shorter duration of follow up, leave us less confident.

We have already cited evidence suggesting that CD4 counts may be an acceptable surrogate for mortality in patients with HIV infection. A randomized controlled trial of immediate versus delayed zidovudine therapy in HIV-infected asymptomatic individuals declared a positive result for immediate therapy, largely on the basis of a greater proportion of treated patients with CD4 cell counts above 350 per mm³ at a median follow-up of 1.7 years 54. Subsequently, the Concorde study addressed the same question in a randomized trial with a median follow-up of 3.3 years 55. The Concorde investigators found a continuous decline in CD4 cells in both treated and control groups, but the median difference of 30 cells per mm3 in favour of treated patients at study termination was statistically significant. However, the study showed no effect of zidovudine in terms of reduced progression to AIDS or death. The median CD4 cell difference was insufficient to impact on clinically important outcomes. The Concorde authors made the following conclusion: The small but highly significant and persistent difference in CD4 count between the groups was not translated into a significant clinical benefit and "called into question the uncritical use of CD4 cell counts as a surrogate endpoint". Had the Concorde analysis that showed significantly shorter times to reach a CD4 count of 350 per mm³ in the control group been regarded as fundamental, the trial might have been stopped early with a false positive result.

Returning to our scenario, the trial of raloxifene in osteoporotic women demonstrated that after two years of treatment raloxifene-treated patients in the group receiving the highest dose showed an increase in bone mineral density at the lumbar spine of 2.2%(SE 0.3) compared to a slight decrease in the control group 0.8% (SE 0.3). This difference in change over time was statistically significant (p < 0.03). Ideally, the investigators would have provided us with a confidence interval around the 3.0% difference in percentage change in bone mineral density in the treatment and control groups. As we will illustrate when we consider weighing benefits and harms, the magnitude of the effect on the surrogate may (or may not) help us estimate the size of possible impact on the target outcome.

III. Will the results help me in caring for my patients?

The questions clinicians should ask themselves in applying the results are the same ones we have suggested for any issue of therapy or prevention [56] and elaborated on in our Users' Guide regarding applicability [57]. These three questions have to do with whether the results can be applied to your patient's care, whether all important outcomes were considered, and whether the likely benefits are worth the down sides of treatment.

"Can the results be applied to my patient's care" refers to the extent to which the patient before you is similar to those who participated in the published studies under consideration, and the extent to which the therapy, and the associated technologies for monitoring and responding to complications, are available in your setting. "Were all important outcomes considered" relates to the focus of this Users' guide, and all the issues we have raised thus far: was the primary outcome really the one in which patients will be interested? This second criterion also draws issues of adverse intervention effects to our attention. Applying the third criterion, judging whether the benefits are worth the down sides of treatment, presents particular challenges when investigators have focused on surrogate end points, and we will discuss this criterion in some detail.

1. Are the likely treatment benefits worth the potential harms and costs?

To know whether to offer a treatment to their patients, clinicians must be able to estimate the magnitude of the likely benefit. When the data available are limited to the effect on a surrogate end point, estimating the extent to which treatment will reduce clinically important outcomes becomes a challenge.

One approach is to extrapolate from one or more randomized trials assessing a related intervention in a similar patient population that provides both surrogate end point and clinical outcome data. For example, until recently there was very little long-term data on the efficacy of lovastatin in reducing clinically important outcomes. However, one could extrapolate from short-term dose efficacy studies assessing the surrogate endpoint of cholesterol lowering. Thus, since 40 mg lovastatin produced a similar degree of LDL cholesterol lowering as 40 mg pravastatin (31% reduction vs. 34% reduction) in the CURVES Study [58], one could theorize that lovastatin would have similar long-term benefits to pravastatin. Subsequently, the AFCAPS/TexCAPS Trial (a five year trial assessing the efficacy of lovastatin in the primary prevention of ischemic heart disease) [59] did confirm that this agent had a benefit profile similar to pravastatin (as determined by the five year long, primary prevention WOSCOPS Trial) [60]: the relative risk reductions (and 95% confidence intervals) for myocardial infarction were 40% (17% to 57%) and 31% (17% to 43%) respectively. However, this approach is likely to be seriously flawed when one is extrapolating from trials of another class of drugs.

Returning to our scenario, to estimate the magnitude of the fracture reduction we might expect with raloxifene (where we only have surrogate end point data), we could (recognizing the limitations of this approach pointed out above) examine the results of randomized controlled trials of alendronate (a drug from a different class in which we have data on the same surrogate end point as well as clinical end points such as fracture reduction). While alendronate appears to improve vertebral bone density by 7.5% over two years (versus control) [45], raloxifene is associated with only a 3.0% improvement over the same time frame. A systematic overview of the alendronate trials [61] reported a 29% reduction in relative risk of nonvertebral fracture over a period of two years. Only one trial looked at symptomatic vertebral fractures in women with decreased bone density and an existing vertebral fracture [62]. This study demonstrated a relative risk reduction of 55% with alendronate and suggested that our patient's risk over three years of a nonvertebral fracture would be approximately 15%; symptomatic vertebral fracture would be about 5%. Given the relative risk reductions with alendronate, one would need to treat approximately 25 women to prevent a non-vertebral fracture and 40 to prevent a symptomatic vertebral fracture over a three year period.

Since the improvement in bone mineral density with raloxifene is at best 50% of the effect of alendronate, we would anticipate a considerably lower reduction in fracture risk with raloxifene. However, interim analysis of an ongoing raloxifene trial [63] reported a 46% relative risk reduction with this therapy (despite less increase in bone mineral density than seen with the alendronate trials). This serves to emphasize the dangers of extrapolating results across classes when it is uncertain that the effects on clinically important outcomes are mediated in the same fashion by the two comparison drugs.

In deciding whether the likely magnitude of the treatment effect warrants offering our patients the intervention, clinicians must consider not only the uncertainty associated with that estimate, but the trade off with potential toxicity and costs of therapy. In addition, clinicians must ponder the consequences of not treating, and the available management alternatives. The deadly and usually relentless progression of HIV infection, and the paucity of alternative therapies, has contributed to the readiness of patients, clinicians, and regulatory agencies to accept evidence from surrogate end points in instituting novel therapies in HIV-infected patients. In osteoporosis, where the consequences of the condition are less immediately devastating, and a variety of agents are available, the case for relying on surrogate end points is far less compelling.

Resolution of the Scenario

We have found a strong, consistent, independent and biologically plausible association between bone mineral density and vertebral and nonvertebral fractures. Randomized trials, however, have failed to show a consistent association between increased bone density and reduced fracture across all drug classes.

Because our patient is at substantial risk of fracture over the short term, the NNT to prevent both nonvertebral and vertebral fractures is moderate, as is the absolute benefit she might expect. Moreover, she is interested in longer-term fracture prevention, and her risk will grow over time. One might offer her alternative interventions, including hormone replacement therapy, calcium and vitamin D, bisphosphonates, or calcitonin. While there is evidence from randomized trials supporting the use of bisphosphonates or calcitonin to decrease osteoporotic fractures, randomized trial data showing fracture reduction in populations similar to our patient with the other agents is lacking. Our patient is very concerned about her long term risk. Raloxifene was well tolerated during this two year trial but no information is available about long term side effects. Our judgment is that a number of options (including a trial of etidronate, offering hormone replacement therapy, calcium and vitamin D, calcitonin, or suggesting only a balanced diet and exercise) would be reasonable. Data indicating a reduction in fracture rate with raloxifene, which a preliminary report suggest may soon be available [63], would greatly strengthen the case for including raloxifene as one of the options.

Conclusion

When we use surrogate end points to make inferences about expected benefit we are making assumptions regarding the link between the surrogate end point and the target outcome. We have outlined criteria clinicians can use to decide when these assumptions might be appropriate. Even if a surrogate end point meets all of these criteria, inferences about a treatment benefit may still prove misleading. Thus, treatment recommendations based on surrogate outcome effects can never be strong. Furthermore, difficulties in estimating the magnitude of effects on clinically important end points compromises economic analysis examining the cost-effectiveness of alternative management strategies.

These considerations emphasize that waiting for randomized trials investigating the effect of the intervention on outcomes of unequivocal importance to patients is the only ironclad solution to the surrogate outcome dilemma. However, when patients' risk of serious morbidity or mortality are high, this "wait and see" strategy may pose problems for many patients and their physicians.

We encourage clinicians to critically question therapeutic interventions in which the only proof of efficacy is from surrogate end point data. When the surrogate end point meets all our validity criteria, the effect of the intervention on the surrogate end point is large, the patient's risk of the target outcome is high, the patient places a high value on avoiding the target outcome, and there are no satisfactory alternative therapies, clinicians can recommend therapy on the basis of randomized trials evaluating only surrogate end points. In other situations, clinicians must carefully consider the known side effects and cost of therapy, and the possibility of unanticipated adverse effects, before recommending an intervention solely on the basis of surrogate end point data.

Acknowledgements

We are grateful to Cliff Rosen for helpful comments concerning the scenario and the associated discussion. Deborah Maddock provided invaluable co-ordination for the EBM Working Group in the development of this manuscript.

References

1. Delmas PD, Bjarnason NH, Mitlak BH, Ravoux AC, Shah AS, Huster WJ, et al. Effects of raloxifene on bone mineral density, serum cholesterol concentrations, and uterine endometrium in postmenopausal women. N Engl J Med 1997; 337:1641-1647.

2. Temple RJ. A regulatory authority's opinion about surrogate endpoints. In: Nimmo WS, Tucker GT, editors. Clinical Measurement in Drug Evaluation. New York: J Wiley, 1995:

3. Hammer SM, Katzenstein DA, Hughes MD, Gundacker H, Schooley RT, Haubrich RH, et al. A trial comparing nucleoside monotherapy with combination therapy in HIV-infected adults with CD4 cell counts from 200 to 500 per cubic millimeter. Aids Clinical Trials Group Study 175 Study Team. N Engl J Med 1996; 335:1081-1090.

4. Delta Coordinating Committee. Delta: A randomised double-blind controlled trial comparing combinations of zidovudine plus didanosine or zalcitabine with zidovudine alone in HIV?infected individuals. Delta Coordinating Committee. Lancet 1996;348:283-291.

5. Saravolatz LD, Winslow DL, Collins G, Hodges JS, Pettinelli C, Stein DS, et al. Zidovudine alone or in combination with didanosine or zalcitabine in HIV-infected patients with the acquired immunodeficiency syndrome or fewer than 200 CD4 cells per cubic millimeter. Investigators for the Terry Beirn Community Programs for clinical research on AIDS. N Engl J Med 1996; 335:1099-1106.

6. Hammer SM, Squires KE, Hughes MD, Grimes JM, Demeter LM, Currier JS, et al. A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less. Aids Clinical Trials Group 320 Study Team. N Engl J Med 1997; 337:725-733.

7. Staessen JA, Fagard R, Thijs L, Celis H, Arabidze GG, Birkenhager WH, et al. Randomised double-blind comparison of placebo and active treatment for older patients with isolated systolic hypertension. The systolic hypertension in Europe (SYST-EUR) trial investigators. Lancet 1997; 350:757-764.

8. Psaty BM, Siscovick DS, Weiss NS, Koepsell TD, Rosendaal FR, Lin D, et al. Hypertension and outcomes research. From clinical trials to clinical epidemiology. Am J Hypertens 1996; 9:178-183.

9. Borhani NO, Mercuri M, Borhani PA, Buckalew VM, Canossa-Terris M, Carr AA, et al. Final outcome results of the multicenter isradipine diuretic atherosclerosis study (MIDAS). A randomized controlled trial. JAMA 1996; 276:785-791.

10. Tatti P, Pahor M, Byington RP, Di Mauro P, Guarisco R, Strollo G, et al. Outcome results of the fosinopril versus amlodipine cardiovascular events randomized trial (FACET) in patients with hypertension and NIDDM. Diabetes Care 1998; 21:597-603.

11. Estacio RO, Jeffers BW, Hiatt WR, Biggerstaff SL, Gifford N, Schrier RW. The effect of nisoldipine as compared with enalapril on cardiovascular outcomes in patients with non-insulin-dependent diabetes and hypertension. N Engl J Med 1998; 338:645-652.

12. Anonymous. How to read clinical journals: IV. To determine etiology or causation. Can Med Assoc J 1981; 124:985-990.

13. Oxman AD, Sackett DL, Guyatt GH. Users' guides to the medical literature. I. How to get started. The Evidence-based Medicine Working Group. JAMA 1993; 270:2093-2095.

14. Guyatt GH, Thompson PJ, Berman LB, Sullivan MJ, Townsend M, Jones NL, et al. How should we measure function in patients with chronic heart and lung disease?. J Chronic Dis 1985; 38:517-524.

15. Mahler DA, Weinberg DH, Wells CK, Feinstein AR. The measurement of dyspnea, contents, interobserver agreement, and physiologic correlates of two new clinical indexes. Chest 1984; 85:751-758.

16. Verschuren WM, Jacobs DR, Bloemberg BP, Kromhout D, Menotti A, Aravanis C, et al. Serum total cholesterol and long-term coronary heart disease mortality in different cultures. Twenty-five-year follow-up of the seven countries study. JAMA 1995;274:131-136.

17. Law MR, Wald NJ, Thompson SG. By how much and how quickly does reduction in serum cholesterol concentration lower risk of ischaemic heart disease? BMJ 1994;308:367-372.

18. Mellors JW, Rinaldo CR, Jr., Gupta P, White RM, Todd JA, Kingsley LA. Prognosis in HIV-1 infection predicted by the quantity of virus in plasma. Science 1996;272:1167-1170.

19. Mellors JW, Kingsley LA, Rinaldo CR, Jr., Todd JA, Hoo BS, Kokka RP, et al. Quantitation of HIV-1 RNA in plasma predicts outcome after seroconversion. Ann Intern Med 1995; 122:573-579.

20. Ruiz L, Romeu J, Clotet B, Balague M, Cabrera C, Sirera G, et al. Quantitative HIV-1 RNA as a marker of clinical stability and survival in a cohort of 302 patients with a mean CD4 cell count of 300 x 10(6)/l. AIDS 1996; 10:F39-44.

21. O'Brien TR, Blattner WA, Waters D, Eyster E, Hilgartner MW, Cohen AR, et al. Serum HIV-1 RNA levels and time to development of Aids in the Multicenter Hemophilia Cohort Study. JAMA 1996; 276:105-110.

22. Yerly S, Perneger TV, Hirschel B, Dubuis O, Matter L, Malinverni R, et al. A critical assessment of the prognostic value of HIV-1 RNA levels and CD4+ cell counts in HIV-infected patients. The Swiss HIV Cohort Study. Arch Intern Med 1998;158:247-252.

23. Ho DD. Viral counts in HIV infection. Science 1996; 272:1124-1125.

24. Cummings SR, Nevitt MC, Browner WS, Stone K, Fox KM, Ensrud KE, et al. Risk factors for hip fracture in white women. Study of Osteoporotic Fractures Research Group. N Engl J Med 1995; 332:767-773.

25. Marshall D, Johnell O, Wedel H. Meta-analysis of how well measures of bone mineral density predict occurrence of osteoporotic fractures. BMJ 1996; 312:1254-1259.

26. Huang C, Ross PD, Wasnich RD. Short-term and long-term fracture prediction by bone mass measurements: a prospective study. J Bone Miner Res 1998; 13:107-113.

27. Bigger JT, Jr., Fleiss JL, Kleiger R, Miller JP, Rolnitzky LM. The relationships among ventricular arrhythmias, left ventricular dysfunction, and mortality in the 2 years after myocardial infarction. Circulation 1984; 69:250-258.

28. McAlister FA, Teo KK. Antiarrhythmic therapies for the prevention of sudden cardiac death. Drugs 1997; 54:235-252.

29. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. N Engl J Med 1991; 324:781-788.

30. Moore TJ. Simon and Shuster, editor. Deadly Medicine. New York: 1995.

31. Drexler H, Banhardt U, Meinertz T, Wollschlager H, Lehmann M, Just H. Contrasting peripheral short-term and long-term effects of converting enzyme inhibition in patients with congestive heart failure. A double-blind, placebo-controlled trial. Circulation 1989;79:491-502.

32. Lewis GR. Comparison of lisinopril versus placebo for congestive heart failure. Am J Cardiol 1989; 63:12D-16D.

33. Giles TD, Fisher MB, Rush JE. Lisinopril and captopril in the treatment of heart failure in older patients. Comparison of a long- and short-acting angiotensin-converting enzyme inhibitor. Am J Med 1988; 85:44-47.

34. Riegger GA. Effects of quinapril on exercise tolerance in patients with mild to moderate heart failure. Eur Heart J 1991; 12:705-711.

35. Garg R, Yusuf S. Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure. Collaborative Group on ACE Inhibitor Trials. JAMA 1995; 273:1450-1456.

36. Di Bianco R, Shabetai R, Kostuk W, Moran J, Schlant RC, Wright R. A comparison of oral milrinone, digoxin, and their combination in the treatment of patients with chronic heart failure. N Engl J Med 1989; 320:677-683.

37. Sueta CA, Gheorghiade M, Adams KF, Jr., Bourge RC, Murali S, Uretsky BF, et al. Safety and efficacy of epoprostenol in patients with severe congestive heart failure. Epoprostenol Multicenter Research Group. Am J Cardiol 1995; 75:34A-43A.

38. Packer M, Carver JR, Rodeheffer RJ, Ivanhoe RJ, Di Bianco R, Zeldis SM, et al. Effect of oral milrinone on mortality in severe chronic heart failure. The Promise Study Research Group. N Engl J Med 1991; 325:1468-1475.

39. Califf RM, Adams KF, McKenna WJ, Gheorghiade M, Uretsky BF, McNulty SE, et al. A randomized controlled trial of epoprostenol therapy for severe congestive heart failure: The Flolan International Randomized Survival Trial (FIRST). Am Heart J 1997;134:44-54.

40. Yee KM, Struthers AD. Can drug effects on mortality in heart failure be predicted by any surrogate measure? Eur Heart J 1997; 18:1860-1864.

41. Hampton JR, van Veldhuisen DJ, Kleber FX, Cowley AJ, Ardia A, Block P, et al. Randomised study of effect of ibopamine on survival in patients with advanced severe heart failure. Second prospective randomised study of ibopamine on mortality and efficacy (PRIME II) investigators. Lancet 1997; 349:971-977.

42. Cameron DW, Heath-Chiozzi M, Danner S, Cohen C, Kravcik S, Maurath C, et al. Randomised placebo-controlled trial of ritonavir in advanced HIV- 1 disease. The Advanced HIV Disease Ritonavir Study Group. Lancet 1997; 351:543-549.

43. Watts NB, Harris ST, Genant HK, Wasnich RD, Miller PD, Jackson RD, et al. Intermittent cyclical etidronate treatment of postmenopausal osteoporosis. N Engl J Med 1990; 323:73-79.

44. Storm T, Thamsborg G, Steiniche T, Genant HK, Sorensen OH. Effect of intermittent cyclical etidronate therapy on bone mass and fracture rate in women with postmenopausal osteoporosis. N Engl J Med 1990; 322:1265-1271.

45. Liberman UA, Weiss SR, Broll J, Minne HW, Quan H, Bell NH, et al. Effect of oral alendronate on bone mineral density and the incidence of fractures in postmenopausal osteoporosis. The Alendronate Phase III Osteoporosis Treatment Study Group. N Engl J Med 1995; 333:1437-1443.

46. Riggs BL, Hodgson SF, O'Fallon WM, Chao EY, Wahner HW, Muhs JM, et al. Effect of fluoride treatment on the fracture rate in postmenopausal women with osteoporosis. N Engl J Med 1990; 322:802-809.

47. Meunier PJ, Sebert J-L, Reginster J-Y, Briancon D, Appelboom T, Netter P, et al. Fluoride salts are no better at preventing new vertebral fractures than calcium-vitamin D in postmenopausal osteoporosis: The FAVO Study. Osteoporosis Int 1998; 8:4-12.

48. Dawson-Hughes B, Harris SS, Krall EA, Dallal GE. Effect of calcium and vitamin D supplementation on bone density in men and women 65 years of age and older. N Engl J Med 1997; 337:670-676.

49. McAlister FA, Straus S, Sackett DL. Randomized clinical trials of antihypertensive drugs: All that glitter is not gold. Can Med Assoc J 1998; 159:488-490.

50. McAlister FA, Laupacis A, Wells GA, Sackett DL. User's guide to the medical literature XIX: Applying clinical trial results. Part B. Guidelines for determining whether a drug is exerting (more than) a class effect. Evidence-based Medicine Working Group. JAMA 1999; In press.

51. Bucher HC, Griffith LE, Guyatt GH. Systematic review on risk and benefit of different cholesterol lowering interventions. Arterioscler Thromb Vasc Biol 1998; In press.

52. Muldoon MF, Manuck SB, Matthews KA. Lowering cholesterol concentration and mortality: a quantitative review of primary prevention trials. BMJ 1990; 301:309-314.

53. Smith GD, Song F, Sheldon TA. Cholesterol lowering and mortality: the importance of considering initial level of risk. BMJ 1993; 306:1367-1373.

54. Cooper DA, Gatell JM, Kroon S, Clumeck N, Millard J, Goebel FD, et al. Zidovudine in persons with asymptomatic HIV infection and CD4+ cell counts greater than 400 per cubic millimeter. The European- Australian Collaborative Group. N Engl J Med 1993;329:297-303.

55. Concorde Coordinating Committee. Concorde: MRC/ANRS randomised double-blind controlled trial of immediate and deferred zidovudine in symptom-free HIV infection. Concorde Coordinating Committee. Lancet 1994; 343:871-881.

56. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-based Medicine Working Group. JAMA 1993; 270:2598-2601.

57. Dans AL, Dans LF, Guyatt GH, Richardson S. Users' guides to the medical literature: XIV. How to decide on the applicability of clinical trial results to your patient. Evidence-based Medicine Working Group. JAMA 1998; 279:545-549.

58. Jones P, Kafonek S, Laurora I, Hunninghake D. Comparative dose efficacy study of atorvastatin versus simvastatin, pravastatin, lovastatin, and fluvastatin in patients with hypercholesterolemia (the CURVES study). Am J Cardiol 1998; 81:582-587.

59. Downs JR, Clearfield M, Weis S, Whitney E, Shapiro DR, Beere PA, et al. Primary prevention of acute coronary events with lovastatin in men and women with average cholesterol levels: Results of AFCAPS/TEXCAPS. Air Force/Texas Coronary Atherosclerosis Prevention Study. JAMA 1998; 279:1615-1622.

60. Shepherd J, Cobbe SM, Ford I, Isles CG, Lorimer AR, Macfarlaine PW, et al. Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia. N Engl J Med 1995; 333:1301-1307.

61. Karpf DB, Shapiro DR, Seeman E, Ensrud KE, Johnston CC, Jr., Adami S, et al. Prevention of nonvertebral fractures by alendronate. A meta-analysis. Alendronate Osteoporosis Treatment Study Groups. JAMA 1997; 277:1159-1164.

62. Black DM, Cummings SR, Karpf DB, Cauley JA, Thompson DE, Nevitt MC, et al. Randomised trial of effect of alendronate on risk of fracture in women with existing vertebral fractures. Fracture Intervention Trial Research Group. Lancet 1996;348:1535-1541.

63. Ettinger B, Black D, Cummings S, Genant H, Gloor C, Lips P, et al. Raloxifene reduces the risk of incident vertebral fractures: 24 month interim analyses. Osteoporosis Int 1998;8 (Suppl. 3):11

64. Clendeneninn N, Quart B, Anderson R, Knowles M, Chang Y. Analysis of long-term virologic data from the Viracept (Nelfinavir) 511 Protocol using 3 HIV-RNA assays. 5th Conference on Retroviruses and Opportunistic Infections 1998; Chicago: Abstract.

65. Fischl M, Greenberg S, Clumeck N, Peters B, Rubio R, Pobiner B, et al. Safety and activity of abacavir (1592, ABC) with 3TC/ZDV in antiretroviral naive subjects. 12th World AIDS Conference, Geneva, June 28-July 3 1998; Abstract.

66. Brun-Vezinet F, Boucher C, Loveday C, Descamps D, Fauveau V, Izopet J, et al. HIV-1 viral load, phenotype, and resistance in a subset of drug- naive participants from the Delta trial. The national virology groups. Delta virology working group and coordinating committee. Lancet 1997; 350:983-990.

67. Montaner JS, Reiss P, Cooper D, Vella S, Harris M, Conway B, et al. A randomized, double-blind trial comparing combinations of nevirapine, didanosine, and zidovudine for HIV-infected patients: the INCAS trial. Italy, the Netherlands, Canada and Australia study. JAMA 1998; 279:930-937.

68. CEASAR Coordinating Committee. Randomised trial of addition of lamivudine or lamivudine plus loviride to zidovudine- containing regimens for patients with HIV- 1 infection: the CAESAR trial. Lancet 1997; 349:1413-1421.

69. Katzenstein DA, Hammer SM, Hughes MD, Gundacker H, Jackson JB, Fiscus S, et al. The relation of virologic and immunologic markers to clinical outcomes after nucleoside therapy in HIV-infected adults with 200 to 500 CD4 cells per cubic millimeter. Aids Clinical Trials Group Study 175 Virology Study Team. N Engl J Med 1996;335:1091-1098.

70. Julius S. Amlodipine in hypertension: An overview of the clinical dossier. J Cardiovasc Pharmacol 1988; 12 Suppl 7:S27-33.

71. Collins R, Peto R, MacMahon S, Hebert P, Fiebach NH, Eberlein KA, et al. Blood pressure, stroke, and coronary heart disease. Part 2, short-term reductions in blood pressure: Overview of randomised drug trials in their epidemiological context. Lancet 1990; 335:827-838.

72. MacMahon S, Peto R, Cutler J, Collins R, Sorlie P, Neaton J, et al. Blood pressure, stroke, and coronary heart disease. Part 1, prolonged differences in blood pressure: Prospective observational studies corrected for the regression dilution bias. Lancet 1990;335:765-774.

73. Psaty BM, Smith NL, Siscovick DS, Koepsell TD, Weiss NS, Heckbert SR, et al. Health outcomes associated with antihypertensive therapies used as first-line agents. A systematic review and meta-analysis. JAMA 1997; 277:739-745.

74. Furberg CD, Psaty BM, Meyer JV. Nifedipine. Dose-related increase in mortality in patients with coronary heart disease. Circulation 1995; 92:1326-1331.

75. Heinonen TM, Stein E, Weiss SR, McKenney JM, Davidson M, Shurzinske L, et al. The lipid-lowering effects of atorvastatin, a new HMG-coA reductase inhibitor: Results of a randomized, double-masked study. Clin Ther 1996; 18:853-863.

76. Bakker-Arkema RG, Davidson MH, Goldstein RJ, Davignon J, Isaacsohn JL, Weiss SR, et al. Efficacy and safety of a new HMG-coA reductase inhibitor, atorvastatin, in patients with hypertriglyceridemia. JAMA 1996; 275:128-133.

77. Winocour PH, Durrington PN, Bhatagnar D, Ishola M, Mackness M, Arrol S, et al. The effect of bezafibrate on very low density lipoprotein (VLDL), intermediate density lipoprotein (IDL), and low density lipoprotein (LDL) composition in type 1 diabetes associated with hypercholesterolaemia or combined hyperlipidaemia. Atherosclerosis 1992; 93:83-94.

78. Jones IR, Swai A, Taylor R, Miller M, Laker MF, Alberti KG. Lowering of plasma glucose concentrations with bezafibrate in patients with moderately controlled NIDDM. Diabetes Care 1990; 13:855-863.

© 2001 Evidence-Based Medicine Informatics Project

© 2001 Centre for Health Evidence.
Home. Users' Guides to EBP. Webmaster. Disclaimer.