Summary
Multiple imputation of missing variance measures can be expected to
become a standard feature to increase the quality and trustworthiness of
future meta-analyses, as advocated by Gurevitch et al. and Nakagawa et
al. Our results clearly show that complete-case and unweighted analyses,
although frequently applied, can potentially lead to deviation in the
grand means and thus biased conclusions and should therefore be replaced
with or (at least) compared to the results of multiple imputation
analyses. The same imputation methods might also be applied re-evaluate
the robustness of already published meta-analyses.
With our simulation study, we aim to raise more awareness on the problem
of incompletely reported study results and their frequent omission in
ecological meta-analyses. Our results discourage the use of
complete-case, unweighted and sample-size weighted meta-analyses since
all three options could result in deviation of the grand means and
confidence intervals. Even in the absence of valid predictors for the
imputation of missing SDs or SSs, their imputation has the advantage of
including all incompletely reported effect sizes while at the same time
preserving the weights of the reported ones.
In summary, our study provides compelling evidence that future
meta-analyses would benefit from a routine application of imputation
algorithms to fill unreported SDs and SSs in order to increase both, the
amount of synthesized effect sizes and the validity of the derived grand
mean estimates. The provided R-script number three could thereby be used
to quickly assess to what degree the results of one’s own meta-analysis
might be affected by the different options to treat missing SDs and SSs.