Commentary (wileyonlinelibrary.com) DOI: 10.1002/sim.4227
Published online in Wiley Online Library
Comment on ‘Multivariate meta-analysis: Potential and promise’ Larry V. Hedges∗ † This article thoughtfully articulates the advantages that multivariate meta-analysis has compared to its univariate counterpart. I would add to the advantages of multivariate meta-analysis that it provides the facility to study the multivariate structure of the distribution of effect parameters, which is sometimes of interest. In social research the covariance structure of the random effects is itself a topic of research interest. For example, do particular implementations of treatments that produce larger effects on one outcome suffer from compensatory loss of effect on other outcomes. An example of this kind in health sciences is the work that I did with Tom Philipson to investigate the covariation of treatment effects (measured as log-odds ratios) of two different ways of measuring the outcomes (subject evaluation and conventional evaluation) across a set of clinical trials [1]. Some years ago, I was part of an effort to better understand data synthesis methods used in areas of science outside of statistics and biostatistics [2]. In the physical sciences the purpose of critical reviews is less to summarize estimates from previous studies than to determine values of physical constants (where the studies may or may not be providing reasonable estimates). We studied major efforts to systematically review data on thermodynamic constants. In this area practitioners noted that the first cardinal rule is to be extremely critical of the experimental data. But ‘The second cardinal rule in the evaluation of data is to work with closely related groups or chemical families of substances and never, if possible, with data for only a single substance. From here on, the evaluation process reduces to all the usual tools of the analyst . . . to test the reliability and internal consistency of the data’ [3], p. 116. Such critical reviews used ensembles of related parameters and estimates of these parameters across studies to both lend strength to one another and to identify physical impossibilities in the ensembles of study results (e.g., when physical theory dictated monotonicity or bounds that were not observed). This perspective has much to offer even in areas of science where the theory is less well established such as medicine and the social sciences, where the analogy is not to work with single outcomes, but with groups of related outcomes whenever possible. I believe that multivariate methods of meta-analysis provide a useful framework in which to formalize some of these ideas, by embedding estimates of ensembles of parameters in a multivariate structure.
References 1. Philipson T, Hedges LV. Subject evaluation in social experiments. Econometrica 1998; 66:381--408. 2. Draper D, Gaver DP, Goel PK, Greenhouse JB, Hedges LV, Morris CN, Tucker JR, Waternaux C. Combining Information: Statistical Issues and Opportunities for Research. American Statistical Association: Washington, DC, 1993. 3. Zwolinski BJ, Chao J. Critically evaluated tables of thermodynamic data. In Thermochemistry and Thermodynamics, Skinner HA (ed.). MTP International Review of Science: Physical Chemistry, Series one, vol. 10. Butterworth: London, 1972.
Copyright © 2011 John Wiley & Sons, Ltd.
Statist. Med. 2011, 30 2499
2499
Department of Statistics, Northwestern University, Evanston, IL, U.S.A. ∗ Correspondence to: Larry V. Hedges, Department of Statistics, Northwestern University, Evanston, IL, U.S.A. † E-mail:
[email protected]
