blog/student-s-t-test-in-r-and-by-hand-how-to-compare-two-groups-under-different-scenarios/

[utterances-bot]

Student's t-test in R and by hand: how to compare two groups under different scenarios - Stats and R

Learn how to apply the Student's t-test by hand and in R in order to compare two independent or paired samples with known or unknown variances

https://statsandr.com/blog/student-s-t-test-in-r-and-by-hand-how-to-compare-two-groups-under-different-scenarios/

[AntoineSoetewey]

Comment written by carmenanadb on June 03, 2020 12:36:30:

Thanks! What a perfect summary!

[AntoineSoetewey]

Comment written by carmenanadb on June 03, 2020 12:36:30:

Thanks! What a perfect summary!

Comment written by Antoine Soetewey on June 03, 2020 12:57:32:

Glad you liked it!

[AntoineSoetewey]

Comment written by Ffigures on October 20, 2020 12:12:50:

You relived my headache, thanks for clearing the concepts.

[AntoineSoetewey]

Comment written by Ffigures on October 20, 2020 12:12:50:

You relived my headache, thanks for clearing the concepts.

Comment written by Antoine Soetewey on October 20, 2020 12:25:51:

Haha you're welcome!

[tanthiamhuat]

Indeed, very good and organized information, thank Antoine for that summarized concise explanations.

[AntoineSoetewey]

Indeed, very good and organized information, thank Antoine for that summarized concise explanations.

You're welcome!

[rwd-gb]

Hi Antoine,

Thanks for creating such a helpful website, its awesome. However, I'm needing some additional advice running a T-test on two means.

I'm completing a study focused on a sustainable finance initiative and investor behaviour. This means I have two groups of investors - signatories (who have signed up to this initiative) and non-signatories (normal investment companies).

I need to complete a t-test of the mean USD invested each year by these two groups. Unfortunately, I am unsure how to write this in R as I have only ever done very basic t-tests. The group is designated in a column using 0 and 1.

Any advice on how to lay this out in code would be incredibly helpful!

[AntoineSoetewey]

Hello Rory,

Thanks for your comment.

See my code here for your t-test given the structure of your dataset. I would like to emphasise the fact that there are some outliers (especially in the signatory == 1 group).

This causes problems for your t-test results because the means and variances are sensitive to outliers. I advise to either:

• use the non-parametric version (the Wilcoxon test),
• use a transformation on your data (log or Box-cox for example),
• or remove outliers (but be careful if you choose this option)

Following your question I added the assumption of no significant outliers in this section.

Hope this helps.

Regards,
Antoine

[rwd-gb]

Hi Antoine, Wow, I did not expect such a comprehensive answer with that much code. That's very generous of you. I will take heed of your advice, the data I sent you is a tiny fraction of what I have - around 12,000 investors, but you are correct there will still be outliers in that. The main purpose of this first t test is to show that the two datasets are very different. Then I will match the signatories to other control investors, re-run the t test and show they are more balanced. I appreciate your help - and let me know what your thoughts are regarding my email about tutoring. Kind regards, Rory

[AntoineSoetewey]

Indeed I forgot that it was a subset of your data!

For your information, with such a large sample size, you are likely to reject the null hypothesis of equal means even with a tiny difference between the two groups. (Remember that the sample size is taken into consideration in the t-test.)

Good luck with your thesis!

[rwd-gb]

Hi Antoine, I ran the code on my full dataset and got these results... Do you have a minute to explain the meaning of a -14 t value to me? I was under the impression I would see a positive number as this is what I have been reading about online. [image: image.png] Best, Rory

[AntoineSoetewey]

The numerator of the test statistic is (sample mean 1 - sample mean 2) - (population mean 1 - population mean 2).

Under your null hypothesis, (population mean 1 - population mean 2) = 0, so it becomes (sample mean 1 - sample mean 2).

Depending on what you set as group 1 or 2, your t value is positive or negative.

You often see this value being positive because we usually take the absolute value, and then compare it to a critical value (found in the tables).

So if your t value is 14 or -14, it doesn't make a difference (for a two-sided test at least). In both cases, it's very extreme so you reject the null hypothesis of the two groups being equal in terms of total USD spent.

If this is still unclear, see this article about hypothesis testing by hand.

Regards,
Antoine

[rwd-gb]

Hi Antoine, Thanks very much for your help, I really appreciate it. I assume its also sensible to include the fact that the very low p value helps with the rejection of the null? Regards, Rory

[AntoineSoetewey]

The p-value is the probability to observe your sample or something more extreme if the null hypothesis were true.

So the more extreme (i.e., the further from 0) the test statistic, the lower the p-value. With a t-stat = -14, you must have a p-value very close to 0.

And remember that when the p-value is smaller than the significance level (usually alpha = 0.05), you reject the null hypothesis. In your case the null hypothesis is rejected (even at a low significance level).

To reply to your question: you should of course include the p-value in your results. It's even better to include it rather than the t-stat, since the p-value is computed based on the t-stat and more importantly it gives more "information".

See this note about p-value and significance level if needed.

Regards,
Antoine

[gmacar]

We will see that for some versions of the t-test, there is no default function built in R (at least to my knowledge, do not hesitate to let me know if I’m mistaken).

IMO, in the first scenario, you can use BSDA::z.test()

By the way, great article! 👍

[AntoineSoetewey]

I wasn't aware of this function, thanks for your input!

I added a note about the function in scenario 1, and also in scenario 1 of the one sample t-test.

[ytalp17]

A sincere thank you from me to you, Antoine, for this masterpiece! It helped me a lot to conceptualize t-test in general.

Nevertheless, I might have a small objection: Given the homoscedasticity assumption of the t-test, Scenario 3 (Unequal and unknown variance) does not seem likely to happen at all. My best guess would be you did it for the sake of pedagogical reasons, unless I miss something.

[ytalp17]

Also, another question that just popped into my mind; Why do T-test have an assumption of homoscedasticity, if we have a formula for unequal variances?

[ytalp17]

I just realized that I confused population variance and sample variance with each other. :)
Anyway, thank you!

[AntoineSoetewey]

So no more questions @ytalp17 ?

[ytalp17]

Why does T-test have an assumption of homoscedasticity?

[AntoineSoetewey]

Parametric tests requires homoscedasticity because they are sensitive to any dissimilarities (the formula and the underlying distribution rely on the fact that population variances are the same in both groups). Therefore, using the t-test instead of the Welch's t-test leads to biased and skewed test results.

[tanmayghonge]

Hi Antoine,

Question regarding the normality assumption of samples.

Under Assumptions --> Normality --> Second Bullet Point: it is mentioned that for large samples (n>=30) normality of data is not required because 'sample means' are well approximate by a normal distribution. I understand the implications of central limit theorem here, however, for small samples (n < 30), the normality condition is applicable to 'samples' not 'sample means'. Why?

[AntoineSoetewey]

Dear @tanmayghonge,

You can see it as follows (in the context of a two-sample t-test but it can be applied to a one-sample t-test and a t-test for paired samples as well):

• It all boils down to the fact that the t-test is used to compare groups, and the means become representative of these groups when data in the populations follow a normal distribution.
• We can never be sure that data in the populations follow a normal distribution (since we don’t have access to the whole populations).
• This means that, by default, you can consider that the normality condition is always applicable to the samples.
• And it is simply that this normality condition can be relaxed (= doesn’t have to be reached) for large samples, thanks to the central limit theorem.
• Since the central limit theorem doesn’t apply to small samples, the normality condition still have to be verified for small samples.

Hope this helps.

Regards,
Antoine