Hypothesis tests applied to means, paired sample t-test, unpaired samples t-test.

- Prevent unreasonable conclusions from data

- Use samples to infer about populations

- Calculate probability that the effects in the data could have arisen by chance.

- Allow a decision if chance or 'real effect'.

- 'Real effects' have below-threshold probability of arising by chance.

- Significance threshold in Psychology usually 0.05, or 0.01 for larger studies.

- Several tests can be employed for data analysis.

- Best test depends on research question and type of data.

   - for nominal data, binomial or chi-square tests

   - for continuous data, it depends on research questions - relationships between variable (correlation). Differences between groups (t-test)

   - Parametric methods for normally distributed data.

   - Non-parametric methods for non-normal data.

- Parametric statistics, such as the t-test, assume that:

     - The data we have collected are normally distributed.

- If that is not the case, or we have reason to believe it is not the case, we should use non-parametric tests.

- If we're comparing two groups of data, the groups must have similar variances.

- The data must be of internal or ratio level of measurement.

- Non-parametric tests do not make any seriously restrictive assumptions about the data we have collected, and are just as good as parametric ones, although sometimes have a little less power than their parametric equivalent.

- The likelihood of finding significant effects with a given n, effect size and variance.

- Non-parametric test usually less sensitive and thus yield less power with the same dataset.

- Often need larger sample size to compare.

Tests of differences between groups

-          Mann-Whitney

o   T test for independent samples

-          Wilicoxon’s matched-pairs Signed-ranks test

o   T test for matched samples

-          Kruskal-Wallis One way anova

o   Anova for between subject designs

-          Friedman’s rank test for k samples

o   Anova for repeated measures designs.

A significant problem with the t-test is that we typically accept significance with each t-test of 95% (alpha=0.05).  For multiple tests these accumulate and hence reduce the validity of the results.

ANalysis Of VAriance (ANOVA) overcomes these problems by using a single test to detect significant differences between the treatments as a whole.

ANOVA assumes parametric data.

-          Tests of differences between groups

o   T-tests

§  2 Related samples

§  2 unrelated samples

-          Analysis of Variance (ANOVA)

o   K related or unrelated samples

-          Typical application: Within-subjects design with two conditions (repeated measures design)

-          Example: happiness scores before and after psychotherapy.

-          Each subject contributes two scores (which is why the Before and After samples are related)

-          Research question: does experimental manipulation affect scores on the dependent variable?

-          Null hypothesis: experimental manipulation does not affect scores on the dependent variable

-          T-test for related samples can be used to test this null hypothesis

-          Outline:

o   Compute difference score for each subject

o   Test null hypothesis that mean difference score = 0

-          Instead of analysing the two sets of X scores separately, the data are reduced to one set of difference scores D

-          H₀:

o   m₁ = m₂

o   m₁ - m₂ = 0

o   m_D = 0

-          Via student t distribution

-          Critical parameter: df (degrees of freedom)

-          Approaches normal distribution as df gets larger

-          Degrees of freedom (df) = n – 1 = 4

-          At t = -1.6, df = 4, two-tailed, we get p = 0.18

-          Or use critical value (two-tailed test, a = 0.05): 2.776

-          t value less than critical value or p > 0.05:

-          null hypothesis cannot be rejected

-          Conclusion: no significant effect of the treatment

-          T = -1.622, df = 4

-          T distribution function at that t has p = 0.09

-          Because hypothesis is two-tailed, total p = 0.18

-          Typical application: between-subjects (independent samples) experiment with two conditions

-          Central question: is there enough evidence to conclude that the two samples were drawn from different populations?

a)       The test leads to a rejection of the null hypothesis

-          “The results showed that younger subjects recalled a mean of 19.3 words, whereas the older subjects recalled a mean of 13.0 words.  The standard deviations were 2.67 and 2.91, respectively. A t-test showed that these means were significantly different, t(18) = 5.02, p < 0.001.”

b)      The test does not lead to a rejection of the null hypothesis:

-          “The results showed that younger subjects recalled a mean of 13.3 words, whereas the older subjects recalled a mean of 12.0 words, with standard deviations of 1.77 and 2.24, respectively. A t-test revealed no significant difference (t(11) = 0.142) between the two participant groups.”