1. What is the issue with multiple comparisons?
- If you do lots of tests on a single dataset/multiple studies some will be signif only by chance. e.g probability of getting a posi result x 13 experiments...likely that one will be significant
- A post hoc definition of success. Not defining predictions before the experiment
- Bad data are not included in the study and people being reluctant to publish negative results/replications
1 of 18
Other questions in this quiz
2. Which of these is NOT a general flaw in PSI experiments?
- File Drawer Problem
- Multiple comparison
- Texas Sharpshooter Fallacy
- Post-hoc statistical tests
- Planned contrast statistical tests
- Stopping criteria
3. Why was PEARs result highly significant?
- Experimentor bias manipulated the results to make the effect larger than it was
- Although the result was significant, the experiment was done under uncontrolled conditions and so is not generalisable to the general population
- Significant was high because n was high, not because of a large effect. Effect driven by a few in thousand events.
- Significant was high because n was small, not because of a large effect. Effect driven by a few in thousand events.
4. Which of these was NOT an issue with Bem et al (2011)?
- All results near threshold
- Correlational measures
- Data peeking and file drawer problem
- Unequal variances
- Eneven sample sizes
- 1 tail t-tests
5. What is a reason for paying more attention to P VALUES over effect sizes?
- If you have controlled your experiment even small effects are interesting
- Because small experimental biases can give rise to a high p if n is high, not a'real result'