Test and effect size details

Indrajeet Patil

2021-05-29

Introduction

Here a go-to summary about statistical test carried out and the returned effect size for each function is provided. This should be useful if one needs to find out more information about how an argument is resolved in the underlying package or if one wishes to browse the source code. So, for example, if you want to know more about how one-way (between-subjects) ANOVA, you can run ?stats::oneway.test in your R console.

Abbreviations used: CI = Confidence Interval

Bird’s-eye view summary

The table below summarizes all the different types of analyses currently supported in this package-

Description Parametric Non-parametric Robust Bayesian
Between group/condition comparisons ✅ ✅ ✅ ✅
Within group/condition comparisons ✅ ✅ ✅ ✅
Distribution of a numeric variable ✅ ✅ ✅ ✅
Correlation between two variables ✅ ✅ ✅ ✅
Association between categorical variables ✅ ✅ ❌ ✅
Equal proportions for categorical variable levels ✅ ✅ ❌ ✅
Random-effects meta-analysis ✅ ❌ ✅ ✅

Summary of Bayesian analysis

Analysis Hypothesis testing Estimation
(one/two-sample) t-test ✅ ✅
one-way ANOVA ✅ ✅
correlation ✅ ✅
(one/two-way) contingency table ✅ ✅
random-effects meta-analysis ✅ ✅

Summary of tests and effect sizes

Here a go-to summary about statistical test carried out and the returned effect size for each function is provided. This should be useful if one needs to find out more information about how an argument is resolved in the underlying package or if one wishes to browse the source code. So, for example, if you want to know more about how one-way (between-subjects) ANOVA, you can run ?stats::oneway.test in your R console.

two_sample_test + oneway_anova

No. of groups: 2 => two_sample_test
No. of groups: > 2 => oneway_anova

between-subjects

Hypothesis testing

Type No. of groups Test Function used
Parametric > 2 Fisher’s or Welch’s one-way ANOVA stats::oneway.test
Non-parametric > 2 Kruskal–Wallis one-way ANOVA stats::kruskal.test
Robust > 2 Heteroscedastic one-way ANOVA for trimmed means WRS2::t1way
Bayes Factor > 2 Fisher’s ANOVA BayesFactor::anovaBF
Parametric 2 Student’s or Welch’s t-test stats::t.test
Non-parametric 2 Mann–Whitney U test stats::wilcox.test
Robust 2 Yuen’s test for trimmed means WRS2::yuen
Bayesian 2 Student’s t-test BayesFactor::ttestBF

Effect size estimation

Type No. of groups Effect size CI? Function used
Parametric > 2 \(\eta_{p}^2\), \(\omega_{p}^2\) ✅ effectsize::omega_squared, effectsize::eta_squared
Non-parametric > 2 \(\epsilon_{ordinal}^2\) ✅ effectsize::rank_epsilon_squared
Robust > 2 \(\xi\) (Explanatory measure of effect size) ✅ WRS2::t1way
Bayes Factor > 2 \(R_{Bayesian}^2\) ✅ performance::r2_bayes
Parametric 2 Cohen’s d, Hedge’s g ✅ effectsize::cohens_d, effectsize::hedges_g
Non-parametric 2 r (rank-biserial correlation) ✅ effectsize::rank_biserial
Robust 2 \(\delta_{R}^{AKP}\) (Algina-Keselman-Penfield robust standardized difference) ✅ WRS2::akp.effect
Bayesian 2 \(\delta_{posterior}\) ✅ bayestestR::describe_posterior

within-subjects

Hypothesis testing

Type No. of groups Test Function used
Parametric > 2 One-way repeated measures ANOVA afex::aov_ez
Non-parametric > 2 Friedman rank sum test stats::friedman.test
Robust > 2 Heteroscedastic one-way repeated measures ANOVA for trimmed means WRS2::rmanova
Bayes Factor > 2 One-way repeated measures ANOVA BayesFactor::anovaBF
Parametric 2 Student’s t-test stats::t.test
Non-parametric 2 Wilcoxon signed-rank test stats::wilcox.test
Robust 2 Yuen’s test on trimmed means for dependent samples WRS2::yuend
Bayesian 2 Student’s t-test BayesFactor::ttestBF

Effect size estimation

Type No. of groups Effect size CI? Function used
Parametric > 2 \(\eta_{p}^2\), \(\omega_{p}^2\) ✅ effectsize::omega_squared, effectsize::eta_squared
Non-parametric > 2 \(W_{Kendall}\) (Kendall’s coefficient of concordance) ✅ effectsize::kendalls_w
Robust > 2 \(\delta_{R-avg}^{AKP}\) (Algina-Keselman-Penfield robust standardized difference average) ✅ WRS2::wmcpAKP
Bayes Factor > 2 \(R_{Bayesian}^2\) ✅ performance::r2_bayes
Parametric 2 Cohen’s d, Hedge’s g ✅ effectsize::cohens_d, effectsize::hedges_g
Non-parametric 2 r (rank-biserial correlation) ✅ effectsize::rank_biserial
Robust 2 \(\delta_{R}^{AKP}\) (Algina-Keselman-Penfield robust standardized difference) ✅ WRS2::wmcpAKP
Bayesian 2 \(\delta_{posterior}\) ✅ bayestestR::describe_posterior

one_sample_test

Hypothesis testing

Type Test Function used
Parametric One-sample Student’s t-test stats::t.test
Non-parametric One-sample Wilcoxon test stats::wilcox.test
Robust Bootstrap-t method for one-sample test trimcibt (custom)
Bayesian One-sample Student’s t-test BayesFactor::ttestBF

Effect size estimation

Type Effect size CI? Function used
Parametric Cohen’s d, Hedge’s g ✅ effectsize::cohens_d, effectsize::hedges_g
Non-parametric r (rank-biserial correlation) ✅ effectsize::rank_biserial
Robust trimmed mean ✅ trimcibt (custom)
Bayes Factor \(\delta_{posterior}\) ✅ bayestestR::describe_posterior

corr_test

Hypothesis testing and Effect size estimation

Type Test CI? Function used
Parametric Pearson’s correlation coefficient ✅ correlation::correlation
Non-parametric Spearman’s rank correlation coefficient ✅ correlation::correlation
Robust Winsorized Pearson correlation coefficient ✅ correlation::correlation
Bayesian Pearson’s correlation coefficient ✅ correlation::correlation

contingency_table

two-way table

Hypothesis testing

Type Design Test Function used
Parametric/Non-parametric Unpaired Pearson’s \(\chi^2\) test stats::chisq.test
Bayesian Unpaired Bayesian Pearson’s \(\chi^2\) test BayesFactor::contingencyTableBF
Parametric/Non-parametric Paired McNemar’s \(\chi^2\) test stats::mcnemar.test
Bayesian Paired ❌ ❌

Effect size estimation

Type Design Effect size CI? Function used
Parametric/Non-parametric Unpaired Cramer’s \(V\) ✅ effectsize::cramers_v
Bayesian Unpaired Cramer’s \(V\) ✅ effectsize::cramers_v
Parametric/Non-parametric Paired Cohen’s \(g\) ✅ effectsize::cohens_g
Bayesian Paired ❌ ❌ ❌

one-way table

Hypothesis testing

Type Test Function used
Parametric/Non-parametric Goodness of fit \(\chi^2\) test stats::chisq.test
Bayesian Bayesian Goodness of fit \(\chi^2\) test (custom)

Effect size estimation

Type Effect size CI? Function used
Parametric/Non-parametric Cramer’s \(V\) ✅ bayestestR::describe_posterior
Bayesian ❌ ❌ ❌

meta_analysis

Hypothesis testing and Effect size estimation

Type Test Effect size CI? Function used
Parametric Meta-analysis via random-effects models \(\beta\) ✅ metafor::metafor
Robust Meta-analysis via robust random-effects models \(\beta\) ✅ metaplus::metaplus
Bayes Meta-analysis via Bayesian random-effects models \(\beta\) ✅ metaBMA::meta_random

Effect size interpretation

See effectsize’s interpretation functions to check different rules/conventions to interpret effect sizes:

https://easystats.github.io/effectsize/reference/index.html#section-interpretation

Dataframe as output

Although the primary focus of this package is to get expressions containing statistical results, one can also use it to extract dataframes containing these details.

For a more detailed summary of these dataframe: https://indrajeetpatil.github.io/statsExpressions//articles/web_only/dataframe_outputs.html

References

Suggestions

If you find any bugs or have any suggestions/remarks, please file an issue on GitHub: https://github.com/IndrajeetPatil/ggstatsplot/issues