{"id":109,"date":"2024-08-21T12:03:15","date_gmt":"2024-08-21T12:03:15","guid":{"rendered":"http:\/\/localhost\/researchmethods\/?post_type=content&p=109"},"modified":"2024-09-12T13:55:23","modified_gmt":"2024-09-12T13:55:23","slug":"chapter-17-testing-for-differences-between-two-samples","status":"publish","type":"content","link":"https:\/\/staging.routledgelearning.com\/researchmethods\/student-resources\/chapter-17-testing-for-differences-between-two-samples\/","title":{"rendered":"Chapter 17 – Testing for differences between two samples"},"content":{"rendered":"\n
\n
\n\tHome<\/a>\n<\/span>\u203a<\/span>\n\tStudent Resources<\/a>\n<\/span>\u203a<\/span>\n\tChapter 17 \u2013 Testing for differences between two samples\n<\/span><\/div>\n\n

Chapter 17 – Testing for differences between two samples<\/h1>\n\n\n

This chapter introduces statistical tests for assessing the significance of differences between two samples and also introduces calculations of effect size and power.<\/p>\n<\/div>\n<\/div>\n\n\n\n

\n
\n
\n

Exercises<\/h2>\n\n\n\n

Here are the data sets, in SPSS and in MS Excel, for the results that are calculated by hand in this chapter of the book. The related t<\/em>, unrelated t<\/em> and single sample t<\/em> data sets are all contained in the Excel file t<\/em> test data sheets.xls. The files in SPSS are unrelated t<\/em> sleep data.sav, related t<\/em> imagery data.sav and single sample t<\/em> test data.sav.<\/p>\n\n\n\n

Exercise 17.1<\/h3>\n\n\n\n
\n
Download – Parametric test data sets<\/a><\/div>\n<\/div>\n\n\n\n

The data files for the non-parametric tests are linked below. The excel file nonparametric test data.xls contains the data for the Mann-Whitney, Wilcoxon and Sign test calculations. The SPSS files are, respectively, mannwhitney stereotype data.sav, wilcoxon module ratings data.sav and sign test therapy data.sav.<\/p>\n\n\n\n

\n
Download – Non-parametric data sets<\/a><\/div>\n<\/div>\n\n\n\n

t<\/em><\/strong> tests on further data sets<\/strong><\/p>\n\n\n\n

\n
Download – Scenario data sets<\/a><\/div>\n<\/div>\n\n\n\n

Data sets are provided here that correspond with the three research designs described below. Your first task is to identify which type of t test should be performed on the data for each design: choose from:   <\/p>\n\n\n\n

Unrelated t<\/em> test       Related t<\/em> test          Single sample t<\/em> test<\/strong><\/p>\n\n\n\n

Scenario 1: Participants are asked to solve one set of anagrams in a noisy room and then solve an equivalent set in a quiet room. The prediction is that participants will perform worse in the noisy room. Data are given in seconds.<\/p>\n\n\n\n

Show answer<\/summary>\n

related t<\/em> test<\/p>\n\n\n\n

Reason:   Repeated measures design. Each participant performed in both conditions.<\/p>\n<\/details>\n\n\n\n

Scenario 2: A sample of children is selected from a \u2018free\u2019 school where the educational policy is radically different from the norm and where students are allowed to attend classes when they like and are also involved in deciding what lessons will be provided by staff. It is suspected their IQ scores may be lower than the average.<\/p>\n\n\n\n

Show answer<\/summary>\n

single sample t<\/em> test<\/p>\n\n\n\n

We have only one group, but we can compare their mean with the standardized mean for IQ of 100.<\/p>\n<\/details>\n\n\n\n

Scenario 3: One group of participants is asked to complete a scale concerning attitudes to people with disabilities. A second group of children is shown a film about the experiences of people with disabilities and then asked to complete the attitude scale a week later. The research is trying to show that changes in attitude last beyond the limits of the typical short-term laboratory experiment.<\/p>\n\n\n\n

Show answer<\/summary>\n

Unrelated t<\/em> test<\/p>\n\n\n\n

Two separate groups each with a mean on the same DV. Independent samples design.<\/p>\n<\/details>\n\n\n\n

Now conduct the appropriate test on each data set and give a full report of the result including: t<\/em> value, df<\/em>, p<\/em> value (either exact or in the \u2018p<\/em> less than \u2026\u2019 format), 95% confidence limits for the mean difference and effect size.<\/p>\n\n\n\n

Show answer<\/summary>\n

Scenario 1 (related t<\/em>)<\/strong>
The mean time to solve anagrams in the noisy room (M = 193.45 secs, SD = 43.16) was higher than the mean time for the quiet room (M = 178.25, SD = 24.52) resulting in a mean difference of 15.2 seconds. This difference was not significant, t<\/em> (19) = 1.558, p<\/em> = .136. The mean difference (95% CI: -5.22 to 35.62) was small (Cohen\u2019s d = 0.35).<\/p>\n\n\n\n

Scenario 2 (single sample t<\/em>)<\/strong>
The \u2018free\u2019 school children had a lower mean than the standard average IQ of 100 (M = 97.7, SD = 9.5). This difference, however, was not significant, t<\/em> (24) = 1.22, p<\/em> = .236. The difference between the sample mean and the population mean was small (2.32, 95% CI: -6.26 to 1.62, Cohen\u2019s d<\/em> = 0.243).<\/p>\n\n\n\n

Note<\/strong> that SPSS uses the sample SD to calculate Cohen\u2019s d<\/em>.<\/p>\n\n\n\n

Scenario 3 (unrelated t<\/em>)<\/strong>
The film group produced a higher mean attitude score (M = 25.65, SD = 5.25) than the control group (M = 22.2, SD = 4.76). The difference between means was significant, t<\/em> (38) = 2.18, p<\/em> = .036. The difference between means (difference = 3.45, 95% CI: 0.24 to 6.66) was moderate (Cohen\u2019s d<\/em> = 0.69).<\/p>\n\n\n\n

Note<\/strong> Effect size is calculated using \n \n d<\/mi>\n =<\/mo>\n \n \n \n \n x<\/mi>\n \u203e<\/mo>\n <\/mover>\n 1<\/mn>\n <\/msub>\n \u2212<\/mo>\n \n \n x<\/mi>\n \u203e<\/mo>\n <\/mover>\n 2<\/mn>\n <\/msub>\n <\/mrow>\n s<\/mi>\n <\/mfrac>\n <\/mrow>\n<\/math>where s is the mean standard deviation for the two groups ( sample sizes are equal).<\/p>\n<\/details>\n\n\n\n

Exercise 17.2<\/h3>\n\n\n\n

Non-parametric tests on the scenario data sets<\/strong><\/p>\n\n\n\n

Select below the appropriate non-parametric tests that can be used on the Scenario 1 and 3 data from the t<\/em> test exercises. In one scenario more than one appropriate test can be selected.<\/p>\n\n\n\n

Scenario 1<\/strong><\/p>\n\n\n\n

(Anagrams in noisy and quiet rooms)<\/p>\n\n\n\n

Wilcoxon          Mann-Whitney             Sign test<\/p>\n\n\n\n

Show answer<\/summary>\n

Wilcoxon and Sign test, but Wilcoxon is a lot more powerful.<\/p>\n<\/details>\n\n\n\n

Scenario 3<\/strong><\/p>\n\n\n\n

(Control and film groups\u2019 attitudes towards disabled people)<\/p>\n\n\n\n

Wilcoxon          Mann-Whitney             Sign test<\/p>\n\n\n\n

Show answer<\/summary>\n

Mann-Whitney<\/p>\n<\/details>\n\n\n\n

Now conduct the appropriate test on each data set and give a full report of the result including: T<\/em> or U<\/em>, appropriate N<\/em> values, p<\/em> value (either exact or in the \u2018p<\/em> less than \u2026\u2019 format) and effect size.<\/p>\n\n\n\n

Show answer<\/summary>\n

Scenario 1: (Wilcoxon)<\/strong><\/p>\n\n\n\n

The differences between time taken to solve anagram in the noisy room and time taken in the quiet room were ranked according to size for each participant. A Wilcoxon T<\/em> analysis on the difference ranks showed a rank total of 139 where noisy room times were higher than quiet room times and a rank total of 71 where quiet room times were higher. Hence, quiet rooms times were generally lower than noisy room times but this difference was not significant, T <\/em>(N<\/em> = 20) = 71, p<\/em> = .204. The estimated effect size was small to moderate, r<\/em> = 0.2.<\/p>\n\n\n\n

Scenario 1: (Sign test)<\/strong><\/p>\n\n\n\n

For each participant the difference between noisy room and quiet room time was found and the sign of this difference recorded. The 13 cases where quiet room score was less than noisy room score were contrasted with the 7 cases where the difference was in the opposite direction using a sign test analysis. The difference was found not to be significant with S<\/em> (N<\/em> = 20) = 7, p<\/em> = .263.<\/p>\n\n\n\n

Scenario 3: Mann-Whitney<\/strong><\/p>\n\n\n\n

The children\u2019s disability attitude scores were ranked as one group. The rank total for the control group was 339.5 whereas the total for the film trained group was 480.5. Using a Mann-Whitney analysis significance was very nearly achieved with U<\/em> (N<\/em> = 40)= 129.5, p<\/em> = .056. The effect size was moderate, r<\/em> = 0.3<\/p>\n\n\n\n

NOTE: <\/strong>If using later versions of SPSS and not the \u2018Legacy dialogs\u2019 you\u2019ll find the higher<\/em> value of U<\/em> is reported. Just to check. SPSS gives U<\/em> = 270.5.  Total ranks = N<\/em>1<\/sub> X N<\/em>2<\/sub> = 400. Subtracting 270.5 from 400 gives 129.5 which is the answer given here. It is conventional to use the lower value and to check this in tables which usually use lower values.<\/p>\n<\/details>\n\n\n\n

Exercise 17.3<\/h3>\n\n\n\n

Have a go at this short quiz to test your understanding and identify any gaps in your knowledge.<\/p>\n\n\n\n

\n