Cliff's Delta Effect Size for Ordinal / Non-parametric Comparisons

Compute Cliff's delta, a non-parametric effect size for the difference between two groups. It is the standardized version of the Mann-Whitney statistic and estimates the probability that a randomly drawn value from one group exceeds a randomly drawn value from the other, minus the reverse probability: \(\delta = (\#\{x > y\} - \#\{x < y\}) / (n_1 n_2)\). It ranges from -1 to 1 and, unlike the rank-biserial r, is unaffected by ties beyond their contribution to the counts.

See the Datanovia tutorial Wilcoxon Test in R for a worked walkthrough.

cliff_delta(
  data,
  formula,
  comparisons = NULL,
  ref.group = NULL,
  ci = FALSE,
  conf.level = 0.95,
  ci.type = "perc",
  nboot = 1000,
  ...,
  boot.parallel = getOption("boot.parallel", "no"),
  boot.ncpus = getOption("boot.ncpus", 1L)
)

Arguments

data

a data frame containing the variables in the formula.

formula

a formula of the form x ~ group where x is a numeric variable and group is a factor with two or more levels.

comparisons

A list of length-2 vectors specifying the groups of interest to be compared. For example to compare groups "A" vs "B" and "B" vs "C", the argument is as follow: comparisons = list(c("A", "B"), c("B", "C"))

ref.group

a character string specifying the reference group. If specified, for a given grouping variable, each of the group levels will be compared to the reference group (i.e. control group).

If ref.group = "all", pairwise two sample tests are performed for comparing each grouping variable levels against all (i.e. basemean).

ci

if TRUE, a percentile bootstrap confidence interval is computed and added as the columns conf.low and conf.high, as for cohens_d() and wilcox_effsize().

conf.level

The level for the confidence interval.

ci.type

The type of confidence interval to use. Can be any of "norm", "basic", "perc", or "bca". Passed to boot::boot.ci.

nboot

The number of replications to use for bootstrap.

...

other arguments; accepted for interface compatibility with cohens_d() and wilcox_effsize() but not used (Cliff's delta has no test backend to forward them to). paired is rejected: the statistic is defined for two independent samples only.

boot.parallel

The type of parallel operation to be used when computing the bootstrap confidence interval. Allowed values are "no" (default), "multicore" and "snow". Passed to boot(). Defaults to getOption("boot.parallel", "no"), so it can also be set globally with options(boot.parallel = "multicore"). Only used when ci = TRUE.

boot.ncpus

Integer. The number of processes to be used in the parallel bootstrap. Defaults to getOption("boot.ncpus", 1L). Note that boot.parallel has no effect unless boot.ncpus > 1. Only used when ci = TRUE.

Value

a tibble with one row per comparison and the columns .y., group1, group2, effsize (Cliff's delta), n1, n2 and magnitude; conf.low / conf.high are added when ci = TRUE.

Details

The magnitude thresholds are those of Romano et al. (2006): |delta| < 0.147 "negligible", < 0.33 "small", < 0.474 "medium", otherwise "large". Cliff's delta is algebraically identical to the rank-biserial correlation, so the point estimate equals effectsize::rank_biserial().

References

Cliff, N. (1993). Dominance statistics: Ordinal analyses to answer ordinal questions. Psychological Bulletin, 114(3), 494-509.

Romano, J., Kromrey, J. D., Coraggio, J., & Skowronek, J. (2006). Appropriate statistics for ordinal level data. Annual meeting of the Florida Association of Institutional Research.

See also

The Datanovia tutorial: Wilcoxon Test in R.

Examples

# Two-samples Cliff's delta
ToothGrowth %>% cliff_delta(len ~ supp)
#> # A tibble: 1 × 7
#>   .y.   group1 group2 effsize    n1    n2 magnitude
#> * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
#> 1 len   OJ     VC       0.279    30    30 small    

# Pairwise comparisons
ToothGrowth %>% cliff_delta(len ~ dose)
#> # A tibble: 3 × 7
#>   .y.   group1 group2 effsize    n1    n2 magnitude
#> * <chr> <chr>  <chr>    <dbl> <int> <int> <ord>    
#> 1 len   0.5    1       -0.832    20    20 large    
#> 2 len   0.5    2       -0.992    20    20 large    
#> 3 len   1      2       -0.695    20    20 large    

# Grouped data
ToothGrowth %>%
  dplyr::group_by(supp) %>%
  cliff_delta(len ~ dose)
#> # A tibble: 6 × 8
#>   .y.   group1 group2 effsize supp     n1    n2 magnitude
#> * <chr> <chr>  <chr>    <dbl> <fct> <int> <int> <ord>    
#> 1 len   0.5    1        -0.85 OJ       10    10 large    
#> 2 len   0.5    2        -1    OJ       10    10 large    
#> 3 len   1      2        -0.47 OJ       10    10 medium   
#> 4 len   0.5    1        -1    VC       10    10 large    
#> 5 len   0.5    2        -1    VC       10    10 large    
#> 6 len   1      2        -0.94 VC       10    10 large