Test and effect size details

Indrajeet Patil

2020-01-10

This vignette provides a go-to summary for which test is carried out for each function included in the package and what effect size it returns. Additionally, there are also recommendations on how to interpret those effect sizes.

Note that the following recommendations on how to interpret the effect sizes are just suggestions and there is nothing universal about them. The interpretation of any effect size measures is always going to be relative to the discipline, the specific data, and the aims of the analyst. Here the guidelines are given for small, medium, and large effects and references should shed more information on the baseline discipline with respect to which these guidelines were recommended. This is important because what might be considered a small effect in psychology might be large for some other field like public health.

expr_t_onesample

parametric

Test: One-sample t-test
Effect size: Cohen’s d, Hedge’s g

Effect size Small Medium Large Range
Cohen’s d 0 – < 0.20 0.20 – < 0.50 ≥ 0.80 [0,1]
Hedge’s g 0 – < 0.20 0.20 – < 0.50 ≥ 0.80 [0,1]

non-parametric

Test: One-sample Wilcoxon Signed-rank Test
Effect size: r ( = \(Z/\sqrt(N_{obs})\))

Effect size Small Medium Large Range
r 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

robust

Test: One-sample percentile bootstrap test
Effect size: robust location measure

expr_t_parametric

within-subjects design

parametric

Test: Student’s dependent samples t-test
Effect size: Cohen’s d, Hedge’s g

Effect size Small Medium Large Range
Cohen’s d 0.20 0.50 0.80 [0,1]
Hedge’s g 0.20 0.50 0.80 [0,1]

non-parametric

Test: Wilcoxon signed-rank test
Effect size: r ( = \(Z/\sqrt(N_{pairs})\))

Effect size Small Medium Large Range
r 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

robust

Test: Yuen’s dependent sample trimmed means t-test
Effect size: Explanatory measure of effect size (\(\xi\))

Effect size Small Medium Large Range
\(\xi\) 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

Reference: https://CRAN.R-project.org/package=WRS2/vignettes/WRS2.pdf

between-subjects design

parametric

Test: Student’s and Welch’s independent samples t-test
Effect size: Cohen’s d, Hedge’s g

Effect size Small Medium Large Range
Cohen’s d 0.20 0.50 0.80 [0,1]
Hedge’s g 0.20 0.50 0.80 [0,1]

non-parametric

Test: Two-sample Mann–Whitney U Test
Effect size: r ( = \(Z/\sqrt(N_{obs})\))

Effect size Small Medium Large Range
r 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

Reference: https://rcompanion.org/handbook/F_04.html

robust

Test: Yuen’s independent sample trimmed means t-test
Effect size: Explanatory measure of effect size (\(\xi\))

Effect size Small Medium Large Range
\(\xi\) 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

Reference: https://CRAN.R-project.org/package=WRS2/vignettes/WRS2.pdf

expr_anova_parametric

within-subjects design

parametric

Test: Fisher’s repeated measures one-way ANOVA
Effect size: \(\eta^2_p\), \(\omega^2\)

Effect size Small Medium Large Range
\(\omega^2\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]
\(\eta^2_p\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]

Reference:

non-parametric

Test: Friedman’s rank sum test
Effect size: Kendall’s W

In the following table, k is the number of treatments, groups, or things being rated.

k Small Medium Large Range
k = 3 < 0.10 0.10 – < 0.30 ≥ 0.30 [0,1]
k = 5 < 0.10 0.10 – < 0.25 ≥ 0.25 [0,1]
k = 7 < 0.10 0.10 – < 0.20 ≥ 0.20 [0,1]
k = 9 < 0.10 0.10 – < 0.20 ≥ 0.20 [0,1]

robust

Test: Heteroscedastic one-way repeated measures ANOVA for trimmed means
Effect size: Not available

between-subjects design

parametric

Test: Fisher’s or Welch’s one-way ANOVA
Effect size: \(\eta^2\), \(\eta^2_p\), \(\omega^2\), \(\omega^2_p\)

Effect size Small Medium Large Range
\(\eta^2\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]
\(\omega^2\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]
\(\eta^2_p\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]
\(\omega^2_p\) 0.01 – < 0.06 0.06 – < 0.14 ≥ 0.14 [0,1]

Reference:

non-parametric

Test: Kruskal–Wallis test
Effect size: \(\epsilon^2\)

Effect size Small Medium Large Range
\(\epsilon^2\) 0.01 – < 0.08 0.08 – < 0.26 ≥ 0.26 [0,1]

Reference: https://rcompanion.org/handbook/F_08.html

robust

Test: Heteroscedastic one-way ANOVA for trimmed means
Effect size: Explanatory measure of effect size (\(\xi\))

Effect size Small Medium Large Range
\(\xi\) 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]

Reference: https://CRAN.R-project.org/package=WRS2/vignettes/WRS2.pdf

expr_contingency_tab

association test - unpaired

Test: Pearson’s \(\chi^2\)-squared test
Effect size: Cramér’s V

In the following table, k is the minimum number of categories in either rows or columns.

k Small Medium Large Range
k = 2 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [0,1]
k = 3 0.07 – < 0.20 0.20 – < 0.35 ≥ 0.35 [0,1]
k = 4 0.06 – < 0.17 0.17 – < 0.29 ≥ 0.29 [0,1]

Reference: https://rcompanion.org/handbook/H_10.html

association test - paired

Test: McNemar’s test
Effect size: Cohen’s g

Effect size Small Medium Large Range
Cohen’s g 0.05 – < 0.15 0.15 – < 0.25 ≥ 0.25 [0,1]

Reference: https://rcompanion.org/handbook/H_05.html

goodness-of-fit test

Test: Pearson’s \(\chi^2\)-squared goodness-of-fit test
Effect size: Cramér’s V

In the following table, k is the number of categories.

k Small Medium Large Range
k = 2 0.100 – < 0.300 0.300 – < 0.500 ≥ 0.500 [0,1]
k = 3 0.071 – < 0.212 0.212 – < 0.354 ≥ 0.354 [0,1]
k = 4 0.058 – < 0.173 0.173 – < 0.289 ≥ 0.289 [0,1]
k = 5 0.050 – < 0.150 0.150 – < 0.250 ≥ 0.250 [0,1]
k = 6 0.045 – < 0.134 0.134 – < 0.224 ≥ 0.224 [0,1]
k = 7 0.043 – < 0.130 0.130 – < 0.217 ≥ 0.217 [0,1]
k = 8 0.042 – < 0.127 0.127 – < 0.212 ≥ 0.212 [0,1]
k = 9 0.042 – < 0.125 0.125 – < 0.209 ≥ 0.209 [0,1]
k = 10 0.041 – < 0.124 0.124 – < 0.207 ≥ 0.207 [0,1]

Reference: https://rcompanion.org/handbook/H_03.html

expr_corr_test

parametric

Test: Pearson product-moment correlation coefficient
Effect size: Pearson’s correlation coefficient (r)

Effect size Small Medium Large Range
Pearson’s r 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [-1,1]

non-parametric

Test: Spearman’s rank correlation coefficient
Effect size: Spearman’s rank correlation coefficient (\(\rho\))

Effect size Small Medium Large Range
Spearman’s \(\rho\) 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [-1,1]

robust

Test: Percentage bend correlation coefficient
Effect size: Percentage bend correlation coefficient (\(\rho_{pb}\))

Effect size Small Medium Large Range
\(\rho_{pb}\) 0.10 – < 0.30 0.30 – < 0.50 ≥ 0.50 [-1,1]

Suggestions

If you find any bugs or have any suggestions/remarks, please file an issue on GitHub: https://github.com/IndrajeetPatil/ggstatsplot/issues

Session Information

For details, see- https://indrajeetpatil.github.io/ggstatsplot/articles/web_only/session_info.html