Editorial Cost-effectiveness: How do we avoid making a difficult science incomprehensible?
Ophthalmic Epidemiology 0928-6586/04/US$ 22.00 Ophthalmic Epidemiology – 2004, Vol. 11, No. 5, pp. 331– 335 DOI: 10.1080/09286580490904529 © 2004 Taylor & Francis Ltd.
Kevin D. Frick1 Steven M. Kymes2 Gretchen A. Jacobson1
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
Accepted 18 September 2004 1
Department of Health Policy and Management, Johns Hopkins Bloomberg School of Health, Baltimore, Maryland, USA 2 Department of Ophthalmology and Visual Sciences, Washington University, St. Louis, Missouri, USA
We have all had the experience. You start a new job, be it academic or corporate. You walk into a presentation by a colleague. Before the third slide comes up on the screen, you find yourself totally lost in a morass of method, terminology, and tradition beyond your comprehension. You know the speaker is brilliant and you have full confidence in the conclusions; however, you find it impossible to follow the logic that takes the work from beginning to end. Readers of Ophthalmic Epidemiology may be sympathetic to this dilemma. Although the journal has published cost-effectiveness analyses and provided a forum for discussion of the framework for this work,1–4 the minutia of cost-effectiveness research methods remain foreign (and, perhaps incomprehensible) to many clinicians, epidemiologists, statisticians, and others. This editorial describes several key points about cost-effectiveness methods and presentation and discusses the use of the term ‘cost-effectiveness’ to facilitate a more critical evaluation of this research by all readers of the journal. Some perceive economic evaluation to be a ‘soft science’ that requires little more than use of a hand calculator to sum the costs and benefits of an intervention. This perception does not consider that conducting a valid and defensible cost-effectiveness study requires knowledge of epidemiological and clinical aspects of the condition in question as well as aspects of economics, psychology, and decision sciences. In addition, the presentation of cost-effectiveness results requires a sensitivity to and understanding of the ethical and political considerations that affect policy adoption and implementation. Costeffectiveness investigators’ end product is not a decision on how to allocate resources but a tool for clinical and political decision makers who decide the use of society’s resources. While not all cost-effectiveness Cost-effectiveness: how do we avoid making a difficult science
Correspondence and reprint requests to: Kevin D. Frick Johns Hopkins Bloomberg School of Public Health Department of Health Policy and Management 624 N. Broadway, Room 606, Baltimore, MD 21212 USA Fax: 410-955-0470 E-mail:
[email protected]
331
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
researchers have graduate training in all the areas discussed above, readers should understand that conducting high-quality costeffectiveness research and using it in practice require collaboration with investigators in these disciplines. In cost-effectiveness research, as with other fields of research, standards must be considered. A panel convened by the United States Public Health Service published recommendations in 1996,5 and the World Health Organization published its own recommendations for economic evaluation in 2003.6 Separately the International Society for Pharmacoeconomics and Outcomes Research has compiled a list of requirements for analyses that various international governments and payers have developed (http://www.ispor.org/PEguidelines/index.asp). The plethora of cost-effectiveness ‘cookbooks’ may make costeffectiveness analysis seem as simple as picking the correct guidelines. However, just as in designing clinical studies expert judgment must be used to address issues ranging from study design to outcomes assessment, so it is in cost-effectiveness research. These decisions only begin with appropriate data on the quantities and prices of resources consumed. Other issues that the theory underlying economic evaluation points to addressing include but are not limited to the very definition of ‘price,’ to the definition of benefits, to the techniques for dealing with the passage of time. Obviously, a single editorial cannot cover all methodological issues. Those who wish to become better acquainted with these concepts should consider taking a short course on the topic or consulting texts describing the framework for cost-effectiveness analyses.1,6–9 Here, we briefly consider two points that are fundamental to ensuring comprehensibility of the methods: the presentation of the results and the evaluation of the precision of the cost-effectiveness estimate. The results of a cost-effectiveness analysis should be presented in two ways – numerically and graphically.A numerical interpretation is useful for comparing a new treatment or intervention to alternatives that include the current practice. These results are interpreted as the amount of money (‘cost’) that must be spent for each additional positive outcome that is achieved (‘effectiveness’). In the study by Trevithick et al. in this issue of Ophthalmic Epidemiology, the result is in dollars per quality adjusted life year (QALY) gained. Costeffectiveness analyses using this measure are referred to as ‘cost-utility’ analyses, reflecting the way in which QALY values are obtained. Other cost-effectiveness analyses in ophthalmic care might focus on cost per case of amblyopia detected or cost per case of blindness avoided. However, a cost per QALY study is most informative. In theory, QALYs can be used to compare treatments that affect both morbidity and mortality and to compare diverse interventions for a variety of conditions that affect morbidity and mortality in different ways. When comparing cost-effectiveness ratios using QALYs, a lower incremental cost-effectiveness ratio (ICER) indicates that less spending for the same gain (i.e., one QALY), thus implementing the intervention with a lower ICER represents a better use of resources. The intervention with a higher ICER should be implemented only if resources remain after the lower cost option is implemented and 332
K.D. Frick et al.
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
there are not other alternatives yielding more QALYs for which the extra resources could be used. Since it is rare (particularly in the United States) for decision makers to know precisely the quantity of resources available to implement health programs, ICERs are often compared with cost/QALY thresholds to determine if an intervention is relatively cost-effective. The levels of these thresholds are an object of considerable controversy among health economists but most would agree that in industrialized nations $100,000/QALY is a reasonable threshold.10,11 While the numerical presentation can illustrate whether a new treatment is expensive in comparison with other treatments, ratios are inappropriate when one alternative is both less expensive and more effective than another (i.e., ‘dominant’). In such cases, presenting a dollar saved per QALY gained or a dollar saved per case of blindness avoided is meaningless and often confusing. A graphical presentation can resolve the problem by showing directly that one alternative dominates another (Figure 1). The authors of the cost-effectiveness study in this issue wrote about the cost saved per QALY saved (gained), which is the consequence of dominance, and this is uninformative. Whether more dollars saved per QALY gained or fewer dollars saved per QALY gained should be preferred is not unambiguous. The ambiguity can be illustrated with an example in which the conclusion that ‘more dollars saved per QALY gained is preferable’ is misleading. Suppose that there were three options – not treated, Age-related Eye Disease Study antioxidant supplementation (AREDS), and a hypothetical third option. The ‘extra’ dollars in this case are actually savings. Given the data represented in the graph, the dollars saved per QALY gained would be higher when comparing the hypothetical option to no treatment than when comparing AREDS to no treatment. In fact, any point below the line segment connecting no treatment and AREDS but still representing a QALY gain would have the same characteristic. However, AREDS clearly saves more money and leads to more QALYs gained than the hypothetical option. Examples could be constructed in which AREDS saves more dollars per
Fig. 1. Potential misinterpretation of incremental cost-effectiveness ratios when one alternative dominates all others.
Extra QALYs in Comparison with Not Treating
60000
AREDS
50000 40000 Hypothetical
30000 20000 10000 0
-80,000,000
-30,000,000
Untreated 20,000,000
70,000,000
"Extra" Dollars in Comparison with Not Treating
Cost-effectiveness: how do we avoid making a difficult science
333
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
QALY gained and dominates the alternative. In contrast to a numerical ratio, the graph does not allow for a misinterpretation of what is dominated. This would be critical when comparing more than the two alternatives in the paper. Finally, while the results of economic evaluation are often presented in a deterministic manner, it should always be recognized that this is not the case. The ICER is only an estimate, and the precision of the ICER is a function of the assumptions and inputs of the analysis. Most who work in the medical sciences are accustomed to interpreting the precision of estimates such as odds ratios and prevalence in the context of confidence intervals. Under certain conditions, a confidence interval may be described around an ICER,12 but more commonly, the ICER’s variability is considered using sensitivity analyses. In conducting sensitivity analyses the researcher poses the question, ‘Would the costeffectiveness decision change if an assumption regarding the value of an input changed?’ The ICER is recalculated after changing the values of key parameters. If the cost-effectiveness decision remains unchanged (i.e., the value of the ICER does not cross the costeffectiveness threshold), then the cost-effectiveness of the intervention is said to be ‘robust’ to changes in the assumptions on which the analysis is based. The readers and decision makers can be confident that the decision implied by the initial analysis is the appropriate, costeffectiveness–based, policy decision. If this is not the case, then the analysis is said to be ‘sensitive’ to changes in the parameters that resulted in crossing the cost-effectiveness threshold. In this alternative case, users of the evaluation – clinicians and health policy makers – may need more information to clarify the value of the parameter to which the results are sensitive before policy implementation. However, there may be sufficient information about the most likely value of the parameter to make a policy recommendation, albeit one made with less confidence than with a robust analysis. In conclusion, those reporting on cost-effectiveness analyses need to consider the complexity of the methods and the background of their readers and make every effort to communicate the methods, results, and implications clearly and concisely. Clear and concise reporting helps those who are not experts to understand the science of costeffectiveness and is ultimately to the benefit of all. This is not to suggest that an extended and complete discussion of methods, results, and implications is unnecessary for critical review, but those writing on costeffectiveness should recall the paraphrased cliché: In telling the story of the forest one need not discuss each individual tree. References 1 Frick KD, West SK. The SAFE strategy for trachoma control: planning a cost-effectiveness analysis of the antibiotic component and beyond. Ophthalmic Epidemiol. 2001;8(4):205–214. 2 Frick KD, Keuffel EL, Bowman RJ. Epidemiological, demographic,
334
K.D. Frick et al.
and economic analyses: measurement of the value of trichiasis surgery in The Gambia. Ophthalmic Epidemiol. 2001;8(2–3): 191–201. 3 Brown GC, Brown MM, Sharma S, Busbee B, Brown H. Incremental cost-effectiveness of laser therapy for visual loss secondary to branch
4
5
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
6
7
8
retinal vein occlusion. Ophthalmic Epidemiol. 2002;9(1):1–10. Lertsumitkul S, La Nauze J. Cost-minimization analysis of capsulotomy after ECCE and IOL implantation. Ophthalmic Epidemiol. 1996;3(2):77–83. Gold M, Franks P, Erickson P. Assessing the health of the nation. The predictive validity of a preference-based measure and selfrated health. Medical Care. 1996; 34(2):163–177. Tan-Torres Edejer T, Baltussen R, Adam T, Hutubessy R, Acharya A, Evans DB, Murray CJL, eds. Making Choices in Health: WHO Guide to Cost-Effectiveness Analysis. Geneva, Switzerland: World Health Organization, 2003. Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine (ed 1). New York: Oxford University Press, 1996; 425. Drummond MF, O’Brien B, Stoddart GL, Torrance GW. Methods for the
9
10
11
12
Economic Evaluation of Health Care Programmes (ed 2). Oxford: Oxford University Press, 1997; 305. Haddix AC, Teutsch SM, Corso PS, eds. Prevention Effectiveness: A Guide to Decision Analysis and Economic Evaluation (ed 2). New York, NY: Oxford University Press, 2002. Ubel PA, Hirth RA, Chernew ME, Fendrick AM. What is the price of life and why doesn’t it increase with the rate of inflation? Arch Intern Med. 2003;163:1637–41. Gillick MR. Medicare coverage for technological innovations – time for new criteria? N Engl J Med. 2004; 350(21):2199–2203. Polsky D, Glick HA, Willke RJ, Schulman KA. Confidence intervals for cost-effectiveness ratios: a comparision of four methods. Health Economics. 1997;6:243–252.
Cost-effectiveness: how do we avoid making a difficult science
335
Ophthalmic Epidemiol Downloaded from informahealthcare.com by Washington University Library For personal use only.
