Fig. 1. Results of our systematic review on ecological meta-analyses and their treatment of missing variances and sample sizes in primary studies summarized by 505 ecological meta-analyses that were published until March 23, 2018 (cf. Supplement S1 and S2, Supporting Information).
Fig. 2. Workflow to compare the effects of the multiple imputation of deleted standard deviations (SDs) and sample sizes (SSs) with a meta-analysis of a complete dataset. (1) We deleted between 10% and 90% of the SDs, SSs or both in the control and treatment groups of an artificial dataset. (2) The deleted values were imputed (in green) via multiple imputations (100 times), all done with the same imputation method. (3) Each of the 100 datasets was analysed with a separate linear mixed-effects meta-analyses (imputed values in green). (4) The resulting 100 grand means, and confidence intervals were averaged according to Rubin’s rules42 in order to obtain single estimates. (5) These estimates were compared to the results of an analysis of the complete data set (i.e. without missing values).
Fig. 3. Effects of imputing SDs and SSs that arem issing c ompletely a t r andom (MCAR) on the grand mean (coloured line) and confidence interval (shaded area) with respect to the results of fully informed weighted meta-analyses. Rows shows results for the 14 methods to treat missing values (c.f. Table 1). Columns show result for the log response ratio, Hedges’d and Fisher’s z effect sizes with 10% (top) up to 90% (bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed. Each panel shows the deviation of the grand mean and its approximated 95% confidence interval (divided by two for better visibility) from the results obtained with a fully informed weighted meta-analysis. Deviations to the right indicate lower values and deviations to the right indicate higher values.
Fig. 4. Effects of imputing SDs and SSs that arem issing a t r andom (MAR) on the grand mean (coloured line) and confidence interval (shaded area) with respect to the results of fully informed weighted meta-analyses. Rows shows results for the 14 methods to treat missing values (c.f. Table 1). Columns show result for the log response ratio, Hedges’ d and Fisher’sz effect sizes with 10% (top) up to 90% (bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed. Each panel shows the deviation of the grand mean and its approximated 95% confidence interval (divided by two for better visibility) from the results obtained with a fully informed weighted meta-analysis. Deviations to the right indicate lower values and deviations to the right indicate higher values.
Fig. 5. Effects of imputing SDs and SSs that arem issing n ot a t r andom (MNAR) on the grand mean (coloured line) and confidence interval (shaded area) with respect to the results of fully informed weighted meta-analyses. Rows shows results for the 14 methods to treat missing values (c.f. Table 1). Columns show result for the log response ratio, Hedges’ d and Fisher’s z effect sizes with 10% (top) up to 90% (bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed. Each panel shows the deviation of the grand mean and its approximated 95% confidence interval (divided by two for better visibility) from the results obtained with a fully informed weighted meta-analysis. Deviations to the right indicate lower values and deviations to the right indicate higher values.
Fig. 6. Effects of imputing SDs and SSs that arecor related with effect sizes and m issingc ompletely a t r andom (corMCAR ) on the grand mean (coloured line) and confidence interval (shaded area) with respect to the results of fully informed weighted meta-analyses. Rows shows results for the 14 methods to treat missing values (c.f. Table 1). Columns show result for the log response ratio, Hedges’d and Fisher’s z effect sizes with 10% (top) up to 90% (bottom) of standard deviations (SDs) and/or sample sizes (SSs) removed. Each panel shows the deviation of the grand mean and its approximated 95% confidence interval (divided by two for better visibility) from the results obtained with a fully informed weighted meta-analysis. Deviations to the right indicate lower values and deviations to the right indicate higher values.