Nemenyi Post-Hoc Test for Friedman Rank Sums

Source: R/friedman_nemenyi_test.R

Performs the Nemenyi (Wilcoxon-Nemenyi-McDonald-Thompson) all-pairs post-hoc test for a two-way balanced complete block design, following a significant Friedman rank sum test. The treatment rank sums are compared pairwise and the test statistic is referred to the studentized range distribution, which already accounts for the multiplicity of the all-pairs comparisons (so, as for tukey_hsd(), there is no separate p-value adjustment step). It should only be used as a post-hoc procedure when the Friedman test is itself significant.

The Nemenyi test is the rank-based, repeated-measures analogue of Tukey's HSD. Unlike friedman_conover_test() (the Durbin-Conover test), it does not borrow the residual rank variance and is therefore more conservative.

friedman_nemenyi_test(data, formula, detailed = FALSE)

Arguments

data

a data.frame containing the variables in the formula.

formula

a formula of the form a ~ b | c, where a (numeric) is the dependent variable name; b is the within-subjects factor variable (the treatment); and c is the column name containing the individuals/subjects (block) identifier. Should be unique per individual.

detailed

logical value. If TRUE, returns the rank-sum estimate and the test method in the output.

Value

return a data frame with some of the following columns:

  • .y.: the y (outcome) variable used in the test.

  • group1,group2: the compared treatments in the pairwise tests.

  • n1,n2: the number of blocks (subjects) contributing to each treatment.

  • estimate: the rank-sum difference.

  • estimate1, estimate2: the rank sums of the two treatments, respectively.

  • statistic: the studentized-range test statistic.

  • p.adj: the p-value (already adjusted for multiple comparisons via the studentized range distribution).

  • method: the statistical test used to compare groups.

  • p.adj.signif: the significance level of the adjusted p-values.

The returned object has an attribute called args, which is a list holding the test arguments.

Details

For a balanced complete block design with \(b\) blocks and \(k\) treatments, the observations within each block are ranked. Let \(R_j\) be the sum of the within-block ranks for treatment \(j\). The pairwise statistic for treatments \(i\) and \(j\) is $$q_{ij} = \frac{|R_i - R_j|}{\sqrt{b\,k\,(k+1)/12}}$$ and the p-value is obtained from the studentized range distribution with \(k\) groups and infinite degrees of freedom.

References

Nemenyi, P. (1963) Distribution-free Multiple Comparisons. PhD Thesis, Princeton University.

Hollander, M., Wolfe, D. A. (1973) Nonparametric Statistical Methods. Wiley.

Examples

# A balanced complete block design: 3 treatments measured on 6 subjects
df <- data.frame(
  id        = factor(rep(1:6, 3)),
  treatment = factor(rep(c("A", "B", "C"), each = 6)),
  score     = c(4, 6, 3, 5, 4, 5,    7, 8, 6, 7, 9, 6,    6, 9, 7, 8, 8, 9)
)

# Omnibus Friedman test
df %>% friedman_test(score ~ treatment | id)
#> # A tibble: 1 × 6
#>   .y.       n statistic    df       p method       
#> * <chr> <int>     <dbl> <dbl>   <dbl> <chr>        
#> 1 score     6      9.33     2 0.00940 Friedman test

# Nemenyi all-pairs post-hoc
df %>% friedman_nemenyi_test(score ~ treatment | id)
#> # A tibble: 3 × 8
#>   .y.   group1 group2    n1    n2 statistic  p.adj p.adj.signif
#> * <chr> <chr>  <chr>  <int> <int>     <dbl>  <dbl> <chr>       
#> 1 score A      B          6     6     3.27  0.0545 ns          
#> 2 score A      C          6     6     4.08  0.0109 *           
#> 3 score B      C          6     6     0.816 0.832  ns