blog/kruskal-wallis-test-nonparametric-version-anova/

[giscus[bot]]

blog/kruskal-wallis-test-nonparametric-version-anova/

Learn how to perform the Kruskal-Wallis test in R (the nonparametric version of the ANOVA) to compare 3 groups or more under the non-normality assumption

https://statsandr.com/blog/kruskal-wallis-test-nonparametric-version-anova/

[Linophy]

Great post. Thank you for showing me how to do Kruskal-Wallis and relevant post-hoc tests using R: I happen to have to use Mann-Whitney and Kruskal-Wallis tests routinely in data analysis, so this post is especially convenient for transitioning into solely using statistics & R tools.

[AntoineSoetewey]

Glad you find it useful @Linophy !

[drfurtado]

Excellent!

[AntoineSoetewey]

Thanks!

[yunier-olivera]

Nice post. It clarified an old doubt I had. Thanks!

Can I ask you a question?:
Do you know a nonparametric alternative for when the groups are not independent?

[AntoineSoetewey]

Hello Yunier,

It depends on the number of groups:

• If there are two groups, you can use the Wilcoxon test for paired samples.
• If there are three or more groups, you can use the Friedman test.

Hope this helps.

Regards,
Antoine

[yunier-olivera]

It is what I needed. Thanks a lot!

[kforner]

Great, thanks.

[reghais]

Thanks a lot!

[AntoineSoetewey]

You’re welcome!

[saeedraeisi]

Good Day,

I grouped a column based on the frequencies through k_means. is it correct?
I used One-way ANOVA, and Two-way ANOVA. Since in both the the data were not normal (even after using Box-cox Transformation in some of the tests normally was rejected), then I used NON-Parametic method for One-Way but I could not find alternative method for Two-way.
would you help me?

[AntoineSoetewey]

Hello,

I grouped a column based on the frequencies through k_means. is it correct?

Sorry I don't understand your question. Can you rephrase it?

then I used NON-Parametic method for One-Way but I could not find alternative method for Two-way

The Scheirer–Ray–Hare test is a nonparametric test used for a two-way ANOVA.

Regards,
Antoine

[saeedraeisi]

Two variables of my dataset are "ID number" and "Frequencies", I wanted to group "ID number" based on the frequencies, what should i do?
I used k_means it gave me 3 groups.
Thank you.

[AntoineSoetewey]

Yes you can use k-means clustering in that case. You could also use hierarchical clustering (see a tutorial here).

Or if you know how many groups you want, you could also simply:

• split your observations equally among the number of desired groups,
• or divide the range of observations by the number of desired groups (as if you would build a histogram).

Regards,
Antoine

[saeedraeisi]

Hello,

I want to predict a numeric variable according to 2 categorical variables with less than 5 levels, what is the best ML algorithm?

[AntoineSoetewey]

You could use a multiple linear regression.

[pourghaedi]

Hi, for comparing the samples between two groups, I have 18 observation in each group. Also, there are 6 variables in each group and the variables are correlated to each other. I know that MANOVA in such cases is the relevant statistical test but in my case with such a very small dataset , what is the non-parametric test equivalent to MANOVA?

[AntoineSoetewey]

Hello,

Thanks for your question.

I’m not familiar with non-parametric alternatives to MANOVA. However, here is what I found:

• You could use it in combination with bootstrapping or permutation tests to tackle the violation of normality.
• There exists a multivariate Kruskal-Wallis test, see here and here.
• I also found this PhD thesis: “The results of this dissertation showed that the proposed nonparametric kernel-based methods have greater power than the counterpart parametric methods, i.e. MANOVA, in detecting differences between groups in multivariate settings when the underlying distribution of the data is not normal.”.

Hope this helps.

Regards,
Antoine

[lara2504]

Very useful post! But I still have one question: when do you need to use the Dunn's test and when the Pairwise Wilcoxon test?

[AntoineSoetewey]

Good question @lara2504 !

In the case of the Kruskal–Wallis test, it is often preferable to use the Dunn test over the pairwise Wilcoxon test.

See the last section "Optional technical note: Schwenk dice and pairwise Wilcoxon–Mann–Whitney tests" of this document for a brief discussion.

Hope this helps.

Regards,
Antoine

[lara2504]

That helps, thank you very much!

[miguelkllee]

Hi, thanks for your detailed and clear explanation. I have a question, I want to perform a post-hoc analysis on a dataset with 16 groups. Instead of comparing all groups, I want to compare with a reference group (similar to Dunnett’s test for ANOVA). I wonder if there is an equivalent version of post-hoc analysis for the Kruskal-Wallis test. Thank you.

[AntoineSoetewey]

Dear @miguelkllee,

You can use the Steel test.

See more information in:

• Steel, R. G. D. (1959) A Multiple Comparison Rank Sum Test: Treatments Versus Control, Biometrics 15, 560–572.
• https://search.r-project.org/CRAN/refmans/PMCMRplus/html/steelsKSampleTest.html

Hope this helps.

Regards,
Antoine

[miguelkllee]

Thank you.

[Lindiida]

Very helpful post, thank you! I did the Kruskal-Wallis test on my data which gave a significant result (P = < 0.05). However, Dunn's post hoc test returned no significant difference between any group. Can you explain why I first get a significant result but then in the multiple comparisons the differences are ns?
Thank you!

[AntoineSoetewey]

Dear @Lindiida,

It is indeed possible that the results are contradictory in some cases.

You'll notice that it may also happen with ANOVA, not just with Kruskal-Wallis. Some of the reasons are, among others:

• Kruskal-Wallis (ANOVA) and post-hoc tests examine different questions and they are based on different methodologies, so there is not always a direct relationship in the sense that a significant Kruskal-Wallis (ANOVA) does not guarantee a significant post-hoc test.
• It is a good practice to control the overall alpha level, but Dunn's post-hoc test is quite conservative. Having contradictory results are even more likely if you have a large number of levels because the more comparisons you have, the more your p-values will get adjusted.
• A weakly significant Kruskal-Wallis (ANOVA) and/or small number of observations in each group.

The best thing you can do is to report that the Kruskal-Wallis test was significant and that Dunn was not. Most people will understand that Dunn's test is conservative and that methodologies are not the same, leading to contradictory conclusions. If needed, you could also:

• choose a less conservative post-hoc test or a less conservative p-values adjustment method,
• increase sample size, or
• decrease the number of pairwise comparisons by comparing groups only with a "control" group via the Steel’s test (which is, intuitively, similar to the Dunnett's test after an ANOVA).

Hope this helps.

Regards,
Antoine

[Lindiida]

Thank you Antoine, very comprehensive answer! Is using the uncorrected Dunn's test after Kruskal-Wallis more "wrong" than using the corrected one and just stating what you said "report that the Kruskal-Wallis test was significant and that Dunn was not"?

[AntoineSoetewey]

Hello @Lindiida,

There is a substantial increase in type I error after multiple testing, in particular with a large number of comparisons. For example, given a significance level of 0.05, the probability of observing at least one significant result just due to chance is 14% with as low as 3 comparisons, 64% with 20 comparisons, and almost certain (99.4%) with 100 comparisons (see more details on the computations in this section).

So I believe it is essential to address the multiple testing issue by adjusting the p-values. Therefore, for me it is more “wrong” to report unadjusted p-values from Dunn’s test than to report that Kruskal-Wallis is significant while Dunn is not.

If needed and depending on the field, you could choose a less conservative p-value adjustment method, but I would not choose no adjustment at all.

Hope this helps.

Regards,
Antoine

[Lindiida]

Thanks, this helps!

[miguelkllee]

Hi Antoine,

Thanks for your reply to @Lindiida post, it is really helpful.

I have an additional question. Could you please provide some examples or links on how to conduct the task:

decrease the number of pairwise comparisons by comparing groups only with a "control" group (similar to the Dunnett's test after an ANOVA). ?

Thank you.

Best wishes,
Kang-Lung

[AntoineSoetewey]

Dear Kang-Lung,

The intuition is to do a test only if it is really needed, by comparing groups only with respect to a reference/control group for example.

Suppose this example. We have 3 groups:

• Patients who received a placebo (the reference group)
• Patients who received drug A
• Patients who received drug B

Instead of comparing the following groups:

1. placebo vs. A
2. placebo vs. B
3. A vs. B

We could actually be interested only in comparing:

1. placebo vs. A
2. placebo vs. B

If we are interested in testing whether a new drug (A or B) is better than the placebo, but we are not interested in testing whether drug A and B are different.

This decreases the number of comparisons (from 3 to 2), leading to smaller adjustments to the p-values. This will not make a big difference with a small number of groups, but with a large number of groups it may make a difference.

This is exactly the idea behind the Dunnett’s test after an ANOVA, and the same principle can be applied after a Kruskal-Wallis test via the Steel’s test (see my answer to your previous comment for references).

Hope this helps.

Regards,
Antoine

[Linophy]

I would agree with what Antonie wrote. Actually, in my field of plant genetics we routinely use Krustal-Wallis test for non-parametric data; and Bonferroni correction is applied. Though one might say it is simply to apply Bonferroni correction - because you just divide the global α-level by number of comparing groups, I find it more challenging than other correction approaches. One would have to critically think about the biological meaning of every comparison prior to their inclusion in the test, and thus the division factor will only be decided after one has thought thoroughly from a biology standpoint. On the other hand, it is exactly because of this challenge that I prefer Bonferroni correction approach to the others, so as to figure out results that are actually biologically meaningful.

[AntoineSoetewey]

Thanks for your input @Linophy!

Bonferroni correction is indeed one of the simplest correction method, but still quite useful and which requires some thinking (especially in biology where it is common to have many many groups).

I’m glad to read how it is used in practice in your field.

Regards,
Antoine

[cschier]

Hello, thank you for this clear post. One thing I am still unsure about: It is stated that the Kruskal-Wallis test is the equivalent of a one-sided ANOVA. But I cannot find any option to specify the direction of the effect. How do I know what direction hypothesis I am testing during the test. Is it true that Kruskal-Wallis is automatically one-sided? Or is it generally two-way? If so, how can I change it and specifically mention the direction I want to test (similar to the "alternative="less"/"greater"" in the Mann-Whitney-U test? Looking forward to your answer. Best, Clara

[AntoineSoetewey]

Dear @cschier,

• The Kruskal-Wallis test is the non-parametric equivalent of a one-way ANOVA
• The Kruskal-Wallis test is automatically two-sided

See my reply below for more details.

Regards,
Antoine

[Linophy]

If I am not mistaken, the Kruskal-wallis test is by definition a one-way non-parametric test. For two-way counterpart, I'm thinking of Friedman test. Antoine, please help to check my points too ( +:

[AntoineSoetewey]

Dear @Linophy,

You right; the Kruskal-wallis test is by definition a one-way non-parametric test.

However, the Friedman test is the non-parametric equivalent of the repeated measures ANOVA (so one-way, but with paired samples instead of independent samples).

Regards,
Antoine

[Linophy]

Oh, I kind of missed the other point mentioned by Clara. I don't think the "one-way" means "one-sided" (i.e. less than OR greater). By your way of "wording", the Kruskal-wallis test might be considered "two-sided", because it would show whether there is difference in medians among compared groups.

[cschier]

Thank you Linophy! I think then it was indeed my confusion between one-way and one-sided! Thank you so much for clarifying. :)

[AntoineSoetewey]

Dear @Linophy and @cschier,

Thanks for your questions and your answers.

Just to be clear, one-sided and one-way are indeed two different concepts :

• One-way means "one factor", as opposed to two-way. A one-way ANOVA means that there is one grouping variable (for example when you want to compare a quantitative variable between three groups), whereas a two-way ANOVA means there are 2 grouping variables (for instance if you want to compare a quantitative variable between three groups and two genders).
• A one-sided is used when you want to specify a direction in the test (as you mentioned, less than or greater), whereas a two-sided test is used if you just want to test whether the two groups are different or not.

So a Kruskal-Wallis test is the non-parametric equivalent of a one-way ANOVA. In fact, ANOVA and Kruskal-Wallis tests can both be considered as two-sided tests, because we test whether the groups (usually > 2) are different or not in terms of the quantitative variable. We cannot use the one-sided alternative with an ANOVA or Kruskal-Wallis test. If you can choose the alternative "less" or "greater" with a Mann-Whitney test, it is because there are only 2 groups to compare.

Note that you can use the one-sided alternative with the post-hoc tests if you wish to specify a direction.

Hope this helps.

Regards,
Antoine

[hellentruong]

Hi Antoine, Thank you for your detailed post. May I have a question? I have difficulty when show results after comparing 5 groups, Kruskal Wallis test for checking the general differences, and Dunn test for checking the difference between two groups. How to show the results in a clear and understandable way. Thank you

[AntoineSoetewey]

Hello @hellentruong,

I believe this plot from the {ggstatsplot} package is a great way to present your results: https://statsandr.com/blog/kruskal-wallis-test-nonparametric-version-anova/#combination-of-statistical-results-and-plot

Otherwise, I recommend reporting at least the p-values of the Kruskal-Wallis test and the pairwise comparisons.

Hope this helps.

Regards,
Antoine

[hellentruong]

Thank you for your response. With 5 groups and pairwise comparision, I have 10 p values and I'' still struggling how to present this results effectively without confusing. Should I make another table about p values when comparing each 2 groups? Thanks in advance

[AntoineSoetewey]

Sure you can always present a table that lists all the pairwise comparisons between each group if you prefer! In this case, include the adjusted p-values and you can even highlight the comparisons which are significant.

[hellentruong]

Thank you for your advice