Issues of Reliability and Validity

Reliability refers to consistency of measuring an instrument. (e.g. a questionnare or scale). Reliability of such questionnares or scales can be measured in terms of whether two independent assesors give similar diagnosis (inter-rater reliability) or whether tests used to deliver these diagnosis are consistent over time (test-retest reliability).

The publicaition of DSM-III in 1980 was designed to provide a more reliable system for classifying psychiatric disorders. In a review of the success of DSM-III, Carson (1991) claimed that DSM-III had fixed the problem of inter-rater reliability once and for all. Psychatrists now had a reliable classification system, so this should have led to much greater agreement over who did, or did not, have schizophrenia.

Cognitive Screening Tests such as RBANS are important in the diagnosis of schizophrenia as they measure the degree of neuropsychological impairment. Wilks et al (2003) administered two alternate forms of the test to schizophrenic patients over intervals varying from 1-134 days. The test retest reliability (correlation of scores across two test periods) was at 8.4.

Validity refers to the extent that a diagnosis represents something that is real and distinct from other disorders and the extent that a classification system such as ICD or DSM measures what it claims to measure. Reliability and validity are inextricably linked because a diagnosis cannot be valid if it is not reliable.

Comorbidity is an important issue for the validity of diagnosis of mental illness. It refers to the extent that two or more conditions co-occur. Psychiatric Comorbidites are common among patients with schizophrenia. These include substance abuse, anxiety and symptoms of depression. Buckley et al (2009) estimate that comorbid depression occurs in 50% of patients, and 47% of patients also have a lifetime diagnosis of comorbid substance abuse. Such comorbidity creates difficulties in the diagnosis of a disorder and also in deciding what treatment to have.

Klosterkotter et al (1994) assessed 489 admissons to psychiatric unit in Germany to determine whether positive or negative symptoms were more valid for a diagnosis of schizophrenia. They found that positive symptoms were more useful for diagnosis than were negative symptoms.

Peope diagnosed as schizophrenic rarely share the same symptoms, nor is there evidence that they share the same outcomes. The prognosis for patients varies with about 20% recovering their previous level of functioning. 10% achieving signifcant and lasting improvement with intermittent relapses (Bentall et al, 1988). A diagnosis of schizophrenia, therefore, has little predicitve validity - some people never appear to recover from the disorder, but many do.