Research Methods: Reliability and Validity

Reliability

If you want the tests to meaningful, the tests needs to be reliable and valid
Reliable tests give consistent results
A reliable test carried out in the a same circumstances on the same participants should always give the same results
Ways of assessing reliability:
- Test-retest = involves administering the same test or questionaire to the same person on different occasions = test results the same each time they are administered= reliable / there must be sufficient time between the test and the retest to ensure the P cannot recall their answers to the questions to the test but not so long that their attitudes, opinions or abilities may have changed = scores correlated = positive correlations means the test was reliable
- Inter-observer reliability = a way on removing the observers subjectivity and bias into the research = involves small-scale trail run (pilot study) of the observation in order to check that observers are applying behavioural categories in the same way, or it may be reported at the end of a study to show that data collected was reliable = observers need to watch the same event, but record their data independently > data collected by the observers should be correlated to assess its reliability

1 of 4

Questionaires = when comparing the two sets of data (using the retest method) > correlation needs to exceed +.80 > low test-retest reliability means the test requires some of the items to be deselected or rewritten such as the questions asked may be complex or ambiguous = use open questions instead
Interviews = Ensuring reliability is to use the same interviewer each time = need to make sure the interviewer is not asking leading or ambiguous questions = use structured interviews where the interviewer's behaviour is more controlled by the fixed questions
Experiments = reliable because of the strict control over the procedure = e.g the instructions given to the p's and the condition they are tested in = precise replication of a particular method rather than the findings = however what could affect this is if the conditions were slightly different each time they were tested
Observations = make sure behavioural categories have been properly operationalised and that they are measurable and self-evident > categories should not overlap e.g. hugging and cuddling > all possible behaviours should be covered on the checklist

2 of 4

Validity refers to the method producing a result that is genuine and legitmate and represents what is actually 'out there' in the real world
Includes whether the researcher has measured what they intended to measure
Internal validity = whether the effects observed in an experiment are due to the manipulation of the independent variable and not some other factor = demand characteristics threaten this = refer to Milgram's study where p's may have been playing along with the experimental situation
External validity = generalising findings from one setting to other settings - in particular to 'everyday life' = the actual setting of the experiement does not make the findings more realistic = its what we are actually testing in regards to the DV and the IV
Temporal validity = whether findings or concepts within a theory, hold true over time = Asch's study suggested that high rates of conformity was a product of a high conformist era
Assessing validity = eyeballing to check or using concurrent validity = this involves matching the results to another recognised and well-established test = close agreement of the two sets of data would indicate that the new test has high concurrent validity and exceeds +.80 correlation

3 of 4

Experimental research = Using control groups means the researcher is better able to assess whether changes in the DV were to the effect of the IV = Lombroso did not have a control group
- Standardise procedures to minimise the impact of participants reactivity and investigator effects
- Single-blind and double-blind procedures is designed to achieve the same aim = resude demand characteristics = single blind procedures the p's are not aware of the aims of the study = in double blind - a thir party conducts the investigation without knowing its main purpose to reduce demand characteristics and investigator effects
Questionaires = lie scale within the questions in order to assess the consistency of a respondent's response and to control for the effects of social desirability bias = ensuring anonymous
Observations = high ecological validity because there is minimal intervention by the researcher = using covert observations = behaviour is more natural and authentic
Qualitative methods = has better ecological validity because case studies and unstructed interviews is better able to reflect the P's reality = using triangulation increases validity by using different sources of evidence =

4 of 4