Running Independent and Repeated T-tests
You run an experiment where you matched subjects based on their political leanings (liberals) and you test to see if you can use fear in the ad to make them vote like conservatives. You measure them on a Likert scale of 1-10 [10 = extremely likely] on how likely they are to vote for the conservative candidate. You are worried about how well you matched your subjects, so you want to run you t-test as both an independent and repeated/matched types. Use α = .05, two-tailed.
(Show all steps for partial credit).
Neutral Ad | Fear Ad | Difference score | |
1 | 8 | 7 | |
3 | 3 | 0 | |
2 | 7 | 5 | |
3 | 4 | 1 | |
1 | 8 | 7 | |
Mean | 2 | 6 | 4 |
SS | 4 | 22 | 44 |
Paired and Independent Sample T-test
1. Looking at your findings on the paired and independent sample t-tests, why are your conclusions different about the differences between group 1 and 2?
Paired sample t-test compares the difference in the likeliness for the two ads based on each individual whereas in the independent samples t-test, the comparison is made in the means of all the individuals in likeliness for the two ads. Thus, even though likeliness to vote was significantly lower for neutral ad than fear ad in independent samples t-test, there was no significant difference found in overall difference in likeliness in two ads in paired t-test.
2. How might you (the researcher) have prevented this issue?
This could have been solved by avoiding the issues for both kinds of tests. The sample size could have been larger to reduce the statistical error. Measures can also be taken to mitigate carry-over and order effects of the repeated measures test by randomizing the order of treatments.
3. Calculate Cohen’s dand r2for both the matched-samples and independent means t-tests.
4. What information do we learn from Cohen’s d and r2? Explain what they each tell us.
The value of cohen’s d for repeated samples t-test of 1.205 means that the means of likelihood for two ads differ by more than 1.2 standard deviations and the value of 0.645 for R-square tells us that about 64.5% of the variation in the difference is explained by the two ads.
The value of cohen’s d for independent samples t-test of 2.2 means that the the difference in means of likelihood for two ads is more than 2.2 standard deviations and the value of 0.606 for R-square tells us that about 60.6% of the variation in the mean difference is explained by the two ads.
PART B–– Theory Questions
1. What does it mean to reject the null hypothesis?
The rejection of null hypothesis typically means that we have accepted the claim for which the analysis was performed. This will conclude that the results are statistically significant.
2. Explain two ways a researcher might increase their statistical power?
• Increasing the sample size
• Using a higher significance level
3. Cohen’s d is theoretically immune to sample size. However, how might sample size indirectly affect Cohen’s d?Hint: is it likely to be larger or smaller with small samples?
It increases with larger sample size and thus, it is likely to be smaller with small samples.
4. Paired sample t-tests have potential disadvantages. What are those disadvantages? Explain two of them.
• Since the subjects/individuals used are the same in the paired sample t-tests, there is a loss of degrees of freedom resulting from same sample.
• There can be problems resulting in the repeated measures design where certain subjects may behave favorable in one experiment than in other, resulting in carry over effects.
5. Which t-test (repeated-measures or independent means) is the most powerful and why?
Repeated measures t-test is usually considered to be most powerful because in the repeated measures t-test, each individual acts their own control, resulting in nullifying any individual differences which results in smaller error term. This smaller error term in paired t-test makes it more powerful.
6. How can you know the Type I error rate of a study?
The probability of making a type 1 error in a study is usually the alpha level. If the p-value of the study falls below this value, we reject the null hypothesis of the study.
7. Explain what is meant when we write p< .05. Hint: your explanation should include the concepts of probability, the study, and the study’s hypotheses.
When we write p<0.05, this means that the p-value is smaller than 0.05. Thus, at 5% alpha level, the decision criteria states that reject the null hypothesis if p-value is smaller than 0.05, else we fail to reject the null hypothesis. Thus, when p<0.05, at 5% alpha level, we will rject the null hypothesis concluding the results to be statistically significant which means that the results are supporting study’s claim.
T-test Methods
During this section, you learned three different types of t-tests. For each of the hypothetical studies below, indicate which t-test would be the MOST APPROPRIATE for the study.
1-Sample t-test Repeated-Measures t-test Independent Means t-test
1. A researcher administers the Beck Depression Instrument (BDI) to a sample of undergraduates. She is interested in whether students are significantly different from the midpoint of the scale.
One-sample t-test
2. An ad agency wants to test which of two versions of their advertisements for a new brand of alcohol is preferred by college students. They’re afraid that people who see both of the ads may be unable to judge any differences objectively.
Independent Means t-test
3. A statistician at a brewing company wants to test the quality of a new batch of stout against the data he has for all batches of stout made by the company.
Independent Means t-test
4. An undergraduate statistics student wants to test whether they are happier on days they are in class versus days they are not. They measure their happiness on Saturday when they are not in class and on Tuesday when they are in class for an entire semester.
Repeated Measures t-test
5. A psychologist is interested in whether people who have survived a natural disaster are more empathetic of victims of natural disasters than people who have not experienced a natural disaster.
Independent Means t-test
6. A psychologist is interested in testing whether people use the same language with their friends as they do with their family. Specifically, she is interested in the frequency with which people use curse words. She records her participants’ conversations using their smartphones and counts the number of times each participant curses in front of their friends and then in front their parents.
Repeated Measures t-test
PART D: Fill in the decision table with the words: Type I Error, Type II Error,Correct Decision
True State | |||
H0 is TRUE | H0 is FALSE | ||
Your Decision | Fail to Reject H0 | Correct | Type 2 Error |
Reject H0 | Type 1 Error | Correct |