Significance Tests - Intro
What are significance tests for?
- Significance tests are used to see just how far an IV caused a DV in an experiment, as opposed to it being due to chance
- They work on probability, with 1 = absolutely will happen, 0 = absolutely won't happen
- Experimental - the IV affected the DV, and a significant difference was seen
- Null - the IV had no effect on the DV, and any difference was due to chance
1 of 4
Significance Tests - Levels of Significance
Which level of significance should we use?
- <50% or <0.5 - NO
- This would give too much margin for error, leading us to reject the null when we had really ought to accept it
- The results were due to chance after all
- This is called a Type 1 error
- <1% or <0.01 - NO
- This is too strict: we might accept the null when we should have rejected it
- The results were due to the IV after all
- This is called a Type 2 error
- <5% or <0.05 - YES
- This is ideal for social science
- A 5% probability that it was due to chance - we can live with
2 of 4
Levels of Data
- When data is put into categories or just named
- E.g. favourite subject at school, did you finish the race (yes/no)
- When data is ranked
- You can line them up but the difference between them might not be the same
- E.g. positions in a race, strongly agree/disagree
- Has equal measurements on a measurement scale, e.g. temperature: there is the same difference between 4 and 6 degrees as between 10 and 12 degrees
- No true zero - zero doesn't mean nothing. 0 degrees doesn't mean no temperature and 0 on an intelligence test doesn't mean you have no intelligence
- Has a true zero, e.g. weight and distance: 0 kgs really is no weight
3 of 4
Choosing a Statistical Test
Is it nominal data?
- Yes - use CHI-SQUARE
- No - move on
Is it a correlation?
- Yes - use SPEARMAN'S RHO
- No - move on
Is it a test of difference between two sets of scores?
- Yes - use WILCOXON T or MANN-WHITNEY U
Which one do you use?
- Repeated Measures - WILCOXON T
- Independent Measures - MANN-WHITNEY U
4 of 4