Journal of Clinical Epidemiology 62 (2009) 1279e1283
A network meta-analysis combined direct and indirect comparisons between glaucoma drugs to rank effectiveness in lowering intraocular pressure Rikkert van der Valka,b, Carroll A.B. Webersa, Thomas Lumleyc, Fred Hendriksea, Martin H. Prinsb, Jan S.A.G. Schoutena,* a
Department of Ophthalmology, Maastricht University Hospital, Maastricht, The Netherlands b Department of Epidemiology, Maastricht University, Maastricht, The Netherlands c Department of Biostatistics, University of Washington, Seattle, WA, USA Accepted 26 April 2008
Abstract Objective: It is difficult to rank treatments according to their effect size when several treatments are available and not all treatments have been compared directly. The purpose of this study was to show a new statistical technique (network meta-analysis) to address this problem and to rank glaucoma drugs according to their intraocular pressure (IOP)-reducing effect. Study Design and Setting: Network meta-analysis of randomized controlled trials was used to combine direct and indirect estimates of the effect of eight drugs and placebo from 28 randomized controlled trials in patients with primary open-angle glaucoma or ocular hypertension patients, 6,841 for the peak effect and 6,953 patients for the trough effect. Results: All drugs differ from placebo in lowering IOP. At the peak, the rank order from high to low in terms of the mean IOP reduction reached is bimatoprost, travoprost and latanoprost, brimonidine, timolol, dorzolamide, betaxolol, brinzolamide. At the trough, this rank order is bimatoprost, latanoprost, travoprost, timolol, betaxolol, dorzolamide, brinzolamide, brimonidine. The results based on direct or indirect estimates were similar. This ranking differed from the ranking based on the mean IOP change from baseline of all arms including the study drug from all randomized controlled trials. Conclusions: A network meta-analysis can be used to combine direct and indirect treatment effects in a formal way. Applied to glaucoma medications, it shows that there is a rank order in treatment effects on IOP. Ó 2009 Elsevier Inc. All rights reserved. Keywords: Network; Meta-analysis; Glaucoma; Drugs; Intraocular pressure; Ocular hypertension
1. Introduction It is difficult to choose the most effective treatment if more than two treatments exist for the same disease. A meta-analysis is helpful if more than one randomized controlled trial (RCT) has been conducted, but it will only show the difference in effect between two treatments that have been compared directly head-to-head in RCTs. A meta-analysis cannot be conducted if treatments have not been compared directly. The value of a meta-analysis is also limited if more than two treatments exist. It is most likely that head-to-head comparisons of two treatments are made in one trial and other comparisons made in No grant was received for this study. * Corresponding author. Department of Ophthalmology, Maastricht University Hospital, PO box 5800, 6202 AZ Maastricht, The Netherlands. Tel. þ31-0-43-387-53-44; fax: þ31-0-43-387-53-43. E-mail address:
[email protected] (J.S.A.G. Schouten). 0895-4356/09/$ e see front matter Ó 2009 Elsevier Inc. All rights reserved. doi: 10.1016/j.jclinepi.2008.04.012
another trial. Therefore, it is unlikely that all treatments are compared directly in the one trial, especially if more than three treatments exist. However, treatments can be compared indirectly. For example, a study has shown that latanoprost is more effective than timolol in lowering intraocular pressure (IOP), and another study has shown that timolol is more effective than dorzolamide in lowering IOP [1e3]. In such a case, it is likely that latanoprost is more effective than dorzolamide. It is more complicated if several RCTs have been conducted for several treatments and direct as well as indirect comparisons are possible. A method that combines the direct and indirect comparisons is of value and its need has been recognized [4]. The statistical technique of a network meta-analysis makes it possible to make this desired comparison in a formal statistical way [5]. The method has been applied for antihypertensive drugs by Psaty et al. and Elliott et al. [6,7]. Hence, we used a network meta-analysis to compare and rank all commonly
1280
R. van der Valk et al. / Journal of Clinical Epidemiology 62 (2009) 1279e1283
combining direct and indirect comparisons. Several trials have been conducted to compare two drugs directly. However, no trial has been conducted to compare dorzolamide and travoprost directly. How can we rank these drugs to each other? One could use the indirect comparisons. For example, dorzolamide is compared directly with timolol, and timolol is compared directly with travoprost. The results from these trials show that timolol is more effective than dorzolamide, and travoprost is more effective than timolol. The obvious conclusion from these indirect comparisons is that travoprost is more effective than dorzolamide. However, there are other indirect comparisons that can be used to compare travoprost and dorzolamide. Dorzolamide is compared directly with latanoprost, and latanoprost is compared directly with travoprost. Both pathways of indirect comparisons, through timolol and through latanoprost should be combined. An estimate of the amount of agreement between these indirect estimates (and also direct comparisons in other examples) is needed to obtain one estimate and confidence interval (CI). When there is discrepancy between the indirect estimates, such that through one pathway, one would conclude that travoprost is more effective than dorzolamide, and through the other pathway that dorzolamide is more effective than travoprost, one would be less confident about the combined estimate of effect. There are three explanations for such a discrepancy. The studies are underpowered such that, in fact, the estimates really are consistent. There can also be heterogeneity. Lastly, the estimates of the effect are reliable but they disagree. In the first two cases, the heterogeneity can be analyzed using traditional meta-analytic methods. In the last case, there is a new form of uncertainty that comes about when a network of direct and indirect
What is new? It is difficult for a clinician to decide which treatment is most effective if more than two treatments exist. Few randomized trials are conducted to compare more than three treatments. A traditional meta-analysis usually only compares two treatments directly with each other. Another method is therefore needed to compare the effectiveness of treatments. A network meta-analysis uses direct and indirect comparisons of treatments from RCTs to analyze the hierarchy in treatment effects and tests for consistency of the relations of the network. Applied to glaucoma drugs, it showed a clinically useful hierarchy in drugs according to their effect on intraocular pressure.
used glaucoma drugs and show the application of this method.
2. Methods Figure 1 gives all the combinations of glaucoma drugs that have been compared in a trial. It gives the total number of trials per combination of two drugs. An example can be extracted from this Fig. 1 to show the principle of
betaxolol
travoprost
2
bimatoprost
4 2
3
2
brimonidine
3 1
timolol 1 4 5
2
2
brinzolamide
1
placebo
1
2 1
dorzolamide 1
latanoprost Fig. 1. The network of all direct and indirect comparisons of all commonly used glaucoma drugs with the numbers showing the number of direct comparisons between two drugs.
R. van der Valk et al. / Journal of Clinical Epidemiology 62 (2009) 1279e1283
comparisons is made. This latter form of uncertainty is called incoherence [5]. If the incoherence is large, combining the results is not an appropriate way to analyze the data, as is also the case for heterogeneity in a traditional meta-analysis. A more sensible approach would then be to search for sources of heterogeneity. If the incoherence is small, one could combine the data and report a summary estimate of effect and 95% CIs. In Fig. 1, there are multiple paths from one drug to another drug. A method is needed to determine the weights for each path and to compute a CI taking into account the overlap between paths. One solution to this problem is to write a hierarchical model containing components for sampling variability, treatment heterogeneity, and incoherence, and to apply maximum likelihood to this model [5]. The network meta-analysis is based on a linear mixed-effects model for differences between treatments. This model incorporates heterogeneity between multiple trials of the same pair of treatments and, in addition, adds a random effect for each treatment pair. Let Yijk be the reported treatment effect comparing treatments i and j in trial k and s2ijk be the reported variance of Yijk. The model is Yijk 5 mi mj þ eijk þ xij, where mi represents the effect of treatment i; eijk | N(0, a(b þ sijk)2) is a residual incorporating sampling error with heterogeneity specified by parameters a and b, and xij | N(0, u2) is an inconsistency effect specific to a pair of treatments, rather than a trial. The treatment-pair effect xij allows indirect comparisons to be inconsistent with direct comparisons of the same treatments. The standard deviation, u, of the treatment-pair effect is termed the ‘‘incoherence’’ of the network. The incoherence summarizes the difference between direct and indirect comparisons. It is the standard deviation u of the random effects that is needed to represent inconsistencies between comparisons. Information about this parameter comes from sums of residuals around closed loops in the network and it is not identifiable in the absence of these loops. By using a linear mixed model, the incoherence random effect is incorporated in inferences in a similar way to the heterogeneity estimate in a standard random-effect metaanalysis [8]. When the incoherence is small or moderate, it is used to increase the standard error of the estimated treatment differences and to reduce the weight given to indirect comparisons. When the incoherence is sufficiently large, combining the trials may not be appropriate. The incoherence is a standard deviation; its units are the same as the units of the treatment effects, in this case, percent reduction. An incoherence of 1% would mean that biases of the order of 1% could be expected from inconsistencies. In this meta-analysis, incoherence above about 0.25% would have been too large for us to be comfortable with the results. A maximum likelihood or restricted maximum likelihood estimation in this model is possible using software for linear mixed models. In the study by Lumley, the statistical details for estimation in R and S-plus (R Foundation
1281
for Statistical Computing, Vienna, Austria) are described using the lme() function [5]. The data of a recently published meta-analysis of RCTs on the IOP-lowering effects of all commonly used glaucoma drugs were used for this network meta-analysis [9]. The studies were selected as objective as possible according to strict criteria as used for systematic reviews. This is described in detail in a previous publication [9]. Peak and trough effects on IOP were noted. Peak and trough moments for each medication were as advised by the American Academy of Ophthalmology [10]. The relative change in IOP from baseline was calculated and combined using the network meta-analysis [5]. The differences in IOP-lowering effects of all commonly used monotherapy strategies were compared with 0.5% timolol twice daily in patients diagnosed with primary open-angle glaucoma or ocular hypertension. We combined randomized clinical trial data from 28 trials that included 6,953 patients at peak and 6,841 patients at trough who were randomized to nine monotherapy treatment strategies [9]. In total, 36 direct comparisons for peak and 34 for trough were available. Depending on the drug, the indirect comparisons used information from one to five trials or pairs of trial arms. Direct and indirect estimations of IOP reductions were also analyzed separately. As an estimate of variability, the incoherence is presented. In addition, the ranking was calculated based on the mean percent IOP change from baseline. This mean percent change was based on all the arms including the study drug. The arms of this study drug could have come from several RCTs including comparisons with different drugs. It is, therefore, different from the analysis that includes only direct estimates, because the latter includes only the arms that are involved in the comparison of two drugs.
3. Results The network meta-analysis revealed a rank order of the glaucoma drugs according to their IOP-lowering effect. The estimates for incoherence in this network meta-analysis were small: 0.0002% for peak and 0.0001% for trough. Analyzing the data ignoring the possibility of incoherence gave similar results. Here, we present the data taking incoherence into account. The rank order in relative IOP-lowering effects at peak and trough are presented in Table 1. A change of 5% corresponds with a change of about 1 mm Hg. The results show that all drugs statistically significantly differ from placebo in lowering IOP. Bimatoprost, travoprost, and latanoprost reduce IOP statistically significantly more than timolol (mean difference, 95% CI) by 6% (3e9%) to 8% (6e11%) at peak, and by 3% (1e5%) to 6% (4e8%) at trough. Timolol reduces IOP statistically significantly more than betaxolol, dorzolamide, and brinzolamide by 5e7% (3e10%) at trough, and by 4e7% (2e13%) at peak. At trough, timolol reduces IOP
33 31 31 27 23 22 17 25 5
(35 (32 (33 (29 (25 (24 (19 (28 (10
to to to to to to to to to
31) 29) 29) 25) 22) 20) 15) 22) 0)
1 3 2 4 6 7 8 5 9
statistically significantly more than brimonidine: 7% (4e13%), and at peak, no difference was found (Table 1). The difference in IOP-lowering effect between two drugs based on the direct comparison was the same or almost the same as compared with the results obtained with the indirect comparison of the same two drugs (data not shown). The results of the previously published meta-analyses show the relative change from baseline for every drug as well as the ranking based on this amount of change (Table 1). This shows that the ranking based on the network meta-analysis differs from the ranking based solely on the relative change from baseline of all the arms including the study drug.
(4e9) (2e7) (4e13) (5 to 2) (15e22) Abbreviations: IOP, intraocular pressure; CI, confidence interval.
1 2 3 4 5 6 7 8 9
28 29 28 26 20 17 17 18 5
(29 to 27) (32 to 25) (30 to 26) (28 to 25) (23 to 17) (19 to 15) (19 to 15) (21 to 14) (9 to 1)
2 1 2 4 5 7 7 6 9
8 6 6 0 7 4 8 1 18
(11 to 6) (9 to 3) (9 to 4)
1 3 2 5 7 6 8 4 9
4. Discussion
Bimatoprost 6 (8 to 4) Travoprost 3 (5 to 1) Latanoprost 3 (4 to 1) Timolol 0 Betaxolol 5 (3e8) Dorzolamide 6 (3e8) Brinzolamide 6 (3e9) Brimonidine 7 (4e10) Placebo 15 (13e17)
Drug
Trough, relative IOP difference with timolol (%) (95% CI)
Ranking according to relative difference Relative IOP change with timolol from baseline at at trough trough (%) (95% CI)
Ranking according to relative change Peak, relative IOP from baseline difference with timolol at trough (%) (95% CI)
Ranking according to relative difference Relative IOP change with timolol from baseline at at peak peak (%) (95% CI)
Ranking according to relative change from baseline at peak
R. van der Valk et al. / Journal of Clinical Epidemiology 62 (2009) 1279e1283
Table 1 Relative (%) change in trough and peak IOP reached by the most commonly used glaucoma drugs and placebo compared with timolol calculated by network meta-analysis, the relative mean IOP change from baseline, and the ranking according to the previous column in the table.
1282
In this study, we used a new methodology to compare all available drugs for glaucoma to assess a rank order of efficacy. This method has been used successfully to compare blood pressureelowering drugs and has not been applied in ophthalmology [6,7]. Traditional reviews of efficacy could give a biased assessment of the published results, because no formal procedure to select and summarize trials is conducted. Therefore, a traditional meta-analysis is used. However, a traditional meta-analysis cannot rank the drugs according to their efficacy but only report the efficacy of drugs that have been compared directly. The clinician is left with a meta-analysis in which not all RCTs are included and which only shows pairwise comparisons. Selecting the best intervention becomes cumbersome or could be biased if only published meta-analyses are used. A systematic review of all RCTs in a domain is needed for a network meta-analysis, because it cannot be based on a selection of interventions if it is to aid the clinician in deciding on the best therapy. Moreover, a network meta-analysis is a method to quantify a rank order, even if no direct comparison between two interventions has been conducted. We conducted a systematic review on the IOP-lowering effect of the glaucoma drugs. The results have been published elsewhere [9]. Applied to glaucoma drugs, we showed with a network meta-analysis that there is a rank order in IOPreducing effect of glaucoma drugs. The ranking presented is based on the differences in IOP reduction with timolol. If one of the other drugs had been used as reference, the ranking would have been the same. Timolol was chosen as the reference, because it was the gold standard for treatment of primary open-angle glaucoma and ocular hypertension before newer drugs were introduced and it also was the reference in many of the earliest RCTs of these new drugs. Even today, it is a commonly used drug and the drug with which fixed combined preparations of all other new drugs have been made. A ranking of drugs can be based on our previous study in which the weighted change in IOP from baseline was
R. van der Valk et al. / Journal of Clinical Epidemiology 62 (2009) 1279e1283
calculated for every drug based on all the arms that included this drug, as is shown in Table 1. It shows that the ranking can be different from the results obtained with a network meta-analysis. In this case, the ranking is especially different for brimonidine. Groups of drugs like the hypotensive lipids (bimatoprost, travoprost, and latanoprost) rank higher compared with timolol in all rankings, and the carbo-anhydrase inhibitors (dorzolamide and brinzolamide) always rank lower than timolol as does the selective beta-blocker betaxolol. The ranking based on the weighted change from baseline includes all the arms of the study drug to calculate this weighted change. This is different from the analysis that includes only the direct comparisons, which only includes the studies with the drug, and its direct comparison with another drug or the combined analysis of direct and indirect comparisons with a different type of weighing as described earlier. The network analysis that included only direct estimates gave similar results as the network analysis that included the direct and indirect comparisons. Moreover, the incoherence was small. The results could therefore be combined. If there had been sufficient heterogeneity between the direct and indirect comparisons, the results should not be combined. This principle also underlies the test for heterogeneity in a traditional meta-analysis. It is of relevance to notice that the indirect comparison is not based on the random distribution of patients between two treatments. Differences in effect, or better ‘‘change,’’ could therefore be attributed also to differences in study population or design issues. However, as noted, the heterogeneity was small and the analysis with and without the indirect comparisons gave similar results. There are two important features of a meta-analysis approach that uses indirect comparisons. The first is that it preserves the randomization; the second is that it allows for the possibility that indirect comparisons may not be valid. Approaches lacking the first feature are not acceptable; those lacking the second feature are relatively undesirable. Network meta-analysis was the first method to explicitly assess inconsistency between direct and indirect comparisons and incorporate it in the meta-analysis. It clearly preserves randomization, as the only inputs to the computations are randomized direct comparisons. Network metaanalysis is also computationally straightforward. It has two disadvantages: the treatment of trials with more than two arms is somewhat ad hoc, and it does not offer much assistance in diagnosing which trials are responsible for any inconsistencies. The most important alternative to network meta-analysis is a family of Bayesian techniques for evidence synthesis. The most developed of these is by Lu and Ades [11]. Their approach selects a set of trials from the network that are modeled by random ‘‘inconsistency factors’’ in a hierarchical Bayesian model. This approach handles multiarm trials more elegantly than network meta-analysis and provides more guidance in tracking down the trials that are responsible for
1283
any inconsistency. The computational workload is not large, but setting up the Bayesian model is relatively complicated. Lu and Ades note that ‘‘an inconsistency model must be programmed very carefully,’’ and that they have been unable to produce a general formula or mechanical method for some parts of the setup. When there are no loops in the network of trials, for example, when all treatments are compared only with placebo, assessment of consistency is not possible and there is no disadvantage in using simpler methods that assume the validity of indirect comparisons. The methods described by Hasselblad and Lu and Ades are good references for these techniques [12,13]. In conclusion, this network meta-analysis is a formal and explicit method that makes optimal use of all available data, because it uses all available trials and direct as well as indirect estimates of the treatment effect. Moreover, it tests the amount of heterogeneity between direct and indirect comparisons. Applied to all commonly used glaucoma drugs, it shows that there is a rank order in glaucoma medications.
References [1] Netland PA, Landry T, Sullivan EK, Andrew R, Silver L, Weiner A, et al. Travoprost compared with latanoprost and timolol in patients with open-angle glaucoma or ocular hypertension. Am J Ophthalmol 2001;132:472e84. [2] Boyle JE, Ghosh K, Gieser DK, Adamson IA. Dorzolamide-Timolol Study Group. A randomized trial comparing the dorzolamide-timolol combination given twice daily to monotherapy with timolol and dorzolamide. Ophthalmology 1998;105:1945e51. [3] Strahlman E, Tipping R, Vogel RInternational Dorzolamide Study Group. A double-masked, randomized 1-year study comparing dorzolamide (Trusopt), timolol, and betaxolol. Arch Ophthalmol 1995;113: 1009e16. [4] Caldwell DM, Ades AE, Higgins JP. Simultaneous comparison of multiple treatments: combining direct and indirect evidence. BMJ 2005;331:897e900. [5] Lumley T. Network meta-analysis for indirect treatment comparisons. Stat Med 2002;21:2313e24. [6] Psaty BM, Lumley T, Furberg CD, Schellenbaum G, Pahor M, Alderman MH, et al. Health outcomes associated with various antihypertensive therapies used as first-line agents: a network meta-analysis. JAMA 2003;289:2534e44. [7] Elliott WJ, Meyer PM. Incident diabetes in clinical trials of antihypertensive drugs: a network meta-analysis. Lancet 2007;369:201e7. [8] Berlin JA, Laird NM, Sacks HS, Chalmers TC. A comparison of statistical methods for combining event rates from clinical trials. Stat Med 1989;8:141e51. [9] van der Valk R, Webers CAB, Schouten JSAG, Zegers MP, Hendrikse F, Prins MH. Intraocular pressure-lowering effects of all commonly used glaucoma drugs, a meta-analysis of randomized clinical trials. Ophthalmology 2005;112:1177e85. [10] Liesegang TJ, Skuta GL, Cantor LB, editors. Basic and clinical science course. 2003e2004 ed. San Francisco: American Academy of Ophthalmology; 2003. Section 10 Glaucoma. [11] Lu G, Ades AE. Assessing evidence inconsistency in mixed treatment comparisons. J Am Stat Assoc 2006;101:447e59. [12] Hasselblad V. Meta-analysis of multi-treatment studies. Med Decis Making 1998;18:37e43. [13] Lu G and Ades AE. Combination of direct and indirect evidence in mixed treatment comparisons. Stat Med 2004;23:3105e3124.
