8) The Empire of Medical Statistics

After WWII, the provision of health care expanded enormously across western societies. But, the private and public bodies that mediated such care were interested in minimising the cost of this expansion as governments were worried that healthcare was a bottomless pit where endless resources could be provided, but demand would never be satisifed. So, they increased the efficiency of distribution and the uses of medical treatment, e.g. by giving treatments to those who needed them most, and using the most effective treatments.

Epidemiology is the study of the prevalence of any health condition. Epidemiology greatly empowered the field of medical statistics and transformed the understanding of causation.

Up until the late 18th century, the medical market was relatively unregulated, so from the early 19th century, physicians sought to regulate the provision of medical care, but their position was relatively weak. The growing working classes were also organising into trade unions which covered the costs of members' medical care, but following the model set by the Poor Law, only if such care was provided by physicians. Medical insurance was relatively cheap, but was still a little too expensive for the working classes. These early forms of medical insurance also didn't cover the costs of care for workers' families which left dependent women and children, and the poorer workers exposed.

Even in the late 19th century, medical professions enjoyed little control over the provision of medical care. But, the German and British state also became increasingly involved in securing health and productivity of the nation by seeking to expand access to professional medical care. The National Health Insurance allowed people to pay some money in which their employer and the government would match which would then be paid to the person when sick. It was very difficult for women to get this, however.

Significantly, as these different providers of medical insurance, like friendly societies, expanded coverage, they increasingly entered into contracts with physicians to control the cost and manner of treatment. They wished the patient to undergo tried and tested treatments, but these varied enormously between physicians. The reliability of any therapeutic drugs was also only guaranteed by either the attending physician's judgement or the manufacturers' assurances. So, insurance companies started paying physicians a certain amount to accept responsibility for the treatment of those within the societies. Doctors hated it. Poor Law authorities also provided something similar for paupers.

In the US, as elsewhere, the official regulation of the market in patent medicines and standardisation of medical education went some way to resolving these problems, but much still depended on physicians' judgement and manufacturers' assurances. Unsurprisingly, the medical profession was unsympathetic, if not opposed, to insurance schemes as they undermined the professional autonomy of the physician. This explains why the British medical profession, especially consultants, resisted the introduction of the NHS in 1948, and the American medical profession continues to resist the introduction of anything similar.

Relatively few could afford the costs of the policies offered by the nascent insurance industry in the early 19th century. These early forms of life insurance were also viewed as morally dubious bets against an unknown, but divinely ordained future.

As insurance schemes grew larger and more complex, the taming of such chances became increasingly important to maintain the viability of Health Insurance. There were a few cases where the firms had become bankrupted which meant the customers had lost their money. The government couldn't ignore this and so tried to help.

The sanitarian statistical surveys, undertaken by people like Edwin Chadwick, provided insurers with a model of how to tame chance by compiling comprehensive actuarial statistics concerning the chance of disease and death under different circumstances. They also helped to lessen the money lost through the loss of labour that followed death. Insurers sought to expand the number of people surveyed in this manner, aiming to increase the reliability of their actuarial tables. Insurers also hired physicians to screen applicants by comparing family histories and actuarial tables, which allowed them to collect the right amount of money for treatment the person may be likely to need which minimised the risk of insuring a particular applicant.

Increasing sample sizes created problems. For example, sampling large numbers of patients raised the possibility that the physcian might not ask exactly the same question (problem of coordination). Physicans not associated with the insurers might also be reluctant to participate in statistical surveys, so hindering the expansion of the sampled population (non-compliance).

So the problem was how to reach more reliable assessments with small samples, which were far easier to organise and police. Between 1910 and 1920, Ronald Fisher argued that chance wasn't the enemy of certainty, but the tool necessary to establish certainty. He introduced the idea that statistical surveys shouldn't aim for the contrast of large and uniformed samples, but for the contrast between randomly selected individuals (randomisation). These samples moved away from certainty, relying on likelihood instead, which was still very helpful.

The technical answer to the political problem of coordination became particularly important during, and immediately after, WWII when antibiotics were first introduced. Streptomycin was particularly expensive to produce, and it was very important to quickly assess its effectiveness and the best way of using this precious drug. Statisticians, like Arthur Bradford Hill, pioneered the teaching of Fisher's statistical approach in medical schools and the use of randomised control trials to answer these latter problems. Hill selected a small group of patients with TB and then randomly distributed treatment with streptomycin. While some treated patients died, they could conclude that streptomycin was a very effective treatment for TB.

Hill and his collaborator, Richard Doll, are most famous for claiming that smoking is the cause of lung cancer. As the British state, especially after the National Insurance Act, collected increasing volumes of statistics on illness and death in Britain, a number of British physicians were alarmed by an increasing incidence of lung cancer, but couldn't explain why. In the 1930s, some clinicians suspected that the increasing consumption of cigarettes might be the cause, but couldn't find any physiological evidence to support this. In 1946, Doll and Hill were charged with establishing the cause of lung cancer. Using the same statistical methods Hill used in the trial of streptomycin, they concluded in 1950 that smoking was the cause.

Their conclusion was received with great scepticism as many smokers didn't die of lung cancer, and many who died of lung cancer didn't smoke. There was also no evidence that tobacco contained carcinogenic chemicals that couldn't be found elsewhere and in greater concentration. But, individual cases weren't important to Doll and Hill as they were interested in diagnosing the cause of disease at the level of the population (shift from individual to collective). They eventually retreated from saying smoking caused cancer to say it was a very important contributory factor instead.

By 1962, the NHS was beginning to mount national campaigns against smoking. Similar campaigns were also carried out in countries without national health insurance schemes, like the USA, by charitable groups. However, the greater effect in the US was achieved by insurance companies increasing premiums on smokers.