Thursday, March 26, 2009

Homeopathy Is Antiscience (Part 2)

"Who you gonna believe, me or your own eyes?" - Chico Marx

In the first post in this series I argued that homeopathy has a very low probability of effectiveness, because it ignores the Avogadro limit, it contradicts the dose-response relationship, and it uses "remedies" so ridiculous they are beyond satire. However, this does not mean that homeopathy is impossible: a low a priori probability could still, in theory, be nudged upwards by the accumulation of sufficiently convincing evidence. Using Bayesian mathematics, the probability can be recalculated after each study, giving a new prior probability for the next study which will be higher after a positive result and lower after a negative result. Eventually, given enough evidence, the prior probability could become high enough that even the most hardened sceptic would be forced to acknowledge the reality of the phenomenon. Think of Newton struggling with his theory of gravitation: he hated the idea of action at a distance, that one body could influence another instantaneously across empty space, yet the exquisite precision of his calculations gradually forced him to accept that it was indeed possible. So let's try to apply the same wisdom to homeopathy. Just how strong is the evidence?

The evidence for homeopathy can be grouped into four categories: in increasing order of credibility, these are individual reports, customer satisfaction surveys, experimental studies and meta-analyses. Of these, the first two are not generally considered very convincing, for several important reasons that will be explained. Let's start with individual reports...

Individual reports (aka anecdotes)

It is very easy to find individual accounts of the power of homeopathy. Pro-homeopathy sites are plastered with testimonials, such as this incredible story from classical homeopath George Vithoulkas:

A case in a coma for three months after an aorta transplant and a rejection process

The doctors became desperate.. ..the man had one or two days to live.. ..homeopathy could do something for the old man.. ..suspend all allopathic medication.. seven days his consciousness returned and in twelve days from the beginning of the homeopathic treatment he asked us to take him home and we did.. month later the man was so well that the only thing that was left was a swelling of his ankles.

Virtually every sceptical article attracts dozens of responses describing how homeopathy had miraculous effects for this or that illness, written by homeopaths and by homeopathy users. They range from the mundane to the incredible. Here's a mundane one that appeared this week on DC's Improbable Science:

My hayfever was so bad before I started taking pollen that I could at times be practically unconscious. I could never go for a walk in the park without first being dosed up with massive amounts anti-histamine. When I first bought the homeopathic remedy I knew nothing about medicine; I simply bought it because I was having an attack and couldn’t get any anti-histamine. I remember wondering how something that made me ill would make me better, but as I had no alternative I bought it and it helped.. .. I can now go out walking in the park now without any anti-histamine. I would definitely say that was a cure.

Here's another recent one, from a Guardian CIF thread:

I went to the Royal Homeopathic Hospital yesterday on referral from my GP. I've never been in a more welcoming and clean hospital. My God they actually had plants in pots there unlike the sterile NHS wars where any natural life is deemed a potential health hazard (by evidence-based standards of course).

I spoke to a homeopathic doctor (who was medically trained as well) and we went into the reasons why homeopathy is so attacked by the establishment. It's really sad to see that it is ignorance and big business conspiring, and they face a threat of having funding taken away.

I got more out of that visit than several visits to my GPs, who barely had time to speak to me and could only offer me a steroid spray or inhaler for my wheezing cough. They even tried to prevent me from my right to be referred to a homeopathist. They failed. Long may homeopathy and alternative healing that springs from a verifiable methodology continue.

People who post such stories seem genuinely surprised that their readers are not immediately converted to the homeopathic cause. To them, their experience feels so powerful that it trumps any scientific study: they simply "know" it works. They seem to regard any expression of scepticism as an accusation that they are lying, or as evidence that the sceptic is just part of a Big Pharma conspiracy (this attitude was beautifully spoofed in The Onion). In fact, most sceptics (myself included) are happy to accept that the vast majority of CAM users are genuine people who really believe their chosen therapies work. Even among practitioners there are probably many honest, kindly people: not every homeopath is a venal, lying con-artist.

The problem is this: there are many, many ways by which people can come to believe that a CAM treatment provides a benefit. Sceptics reject anecdotes simply because these other ways are all far more probable explanations for the benefit than the idea that improbable therapies like homeopathy actually work. It is worth spelling this out in detail because so many people fail to grasp this point. Let's have a look at the possibilities...

Why bogus therapies often seem to work

The list that follows is largely based on the late Barry Beyerstein's famous article Why Bogus Therapies Often Seem to Work, which everybody should read. My version adds a couple more points and unpacks the placebo effect to make 12 different explanations. If you can think of any more, please let me know and I will add them to the list.

  1. The patient was not really ill in the first place. Many CAMs are simply providing reassurance for the worried well.

  2. The illness was time-limited, so the patient would have recovered anyway. Colds, as the old joke has it, normally last a week, but with the right treatment can be cured in as little as seven days.

  3. Because the patient has been feeling ill, they have taken time off work and cut down on their other activities. This rest has helped them recover.

  4. The original diagnosis was wrong. The apparently intractable condition that has been "cured" was actually a less serious, time-limited illness.

  5. The illness is episodic or cyclical, so if the patient usually takes their CAM when it is at its worst, the CAM will appear to work because of regression to the mean.

  6. CAM therapists may encourage patients to change other lifestyle factors such as eating better, exercising more, sleeping more, drinking less and so on, and it may be these factors that make the difference.

  7. Patients may take a range of treatments, including conventional ones, but attribute any improvement to the CAM. There is a nice example of this in John Diamond's book Snake Oil, in which a woman tells him that Gerson therapy has cured her cancer. On being pressed for more details, she admits that she has had chemotherapy too, but is convinced that it was the Gerson regime (carrot juice and coffee enemas) that made the difference.

  8. Maybe talking to a therapist has improved their mood, so that they feel more positive about their condition. In this view, CAM is really just a branch of counselling. CAM consultations often last an hour or more, and involve talking about all aspects of the patient's life. It is not surprising that this can help people feel better.

  9. There is an unconscious, conditioned placebo effect: they experience a learned response to the rituals of treatment. There is a good chapter on this in Barker Bausell's book Snake Oil Science, in which he describes experimental studies on animals and humans, showing that (for example) mice given repeated injections of an immunosuppressant continue to show the same suppression even after the injections are changed to a saline placebo.

  10. There is an unconscious, evolved placebo effect. In Dylan Evans's book Placebo he argues that pain and inflammation after injury serve an adaptive purpose, to encourage us to immobilise the injury site, and to seek help. Once this "acute phase response" has done its job, the pain and inflammation are no longer needed and can decline.

  11. There is a conscious placebo effect: they believe they will improve so they do. This effect of belief is what most people mean when they talk about "the placebo effect", and it is a wonderful thing that should be studied in more detail. Ben Goldacre's Bad Science explains that oval placebos work better than round ones, two placebos work better than one, coloured placebos work better than white ones, branded placebos work better than those in plain packaging and so on.

  12. There is an investment effect: people who have given up time and money for a treatment are highly motivated for it to work, and so convince themselves that it has. The power of this effect is generally underappreciated, but is well-known to psychologists. For example, in this classic paper by Edward Desi, among students carrying out a tedious task for either a large reward or a small one, it was found that the ones getting the smaller reward reported having enjoyed the task more.

So when a sceptic dismisses an anecdote, they are not merely being stubborn or short-sighted. They are thinking about all these possibilities, comparing the probabilities, and applying Occam's Razor.

In the next post I will look at the next type of evidence: customer satisfaction surveys.


Neuroskeptic said...

Thanks for this clear & thorough discussion. I particularly like your breaking-down of what is so often lazily called "the placebo effect" into the various possible ways in which people might "feel better"...

But I'm going to have to object to one thing (I'd feel lost writing a comment if I didn't)

"The evidence for homeopathy can be grouped into four categories: in increasing order of credibility, these are individual reports, customer satisfaction surveys, experimental studies and meta-analyses."

I don't think meta-analyses are necessarily better than experimental studies. I would place much more faith in a single, very good, study, than in a meta-analysis which includes that good study along with a dozen crap ones - and therefore aggregates all of their crapness.

Mike Eslea said...

@ Neuroskeptic

Ah, but there's a lot more to meta-analysis than just aggregating. A proper meta-analysis (like the famous Shang
et al 2005
in The Lancet) includes study quality as a factor, which makes it very clear whether the good studies give different results from the crap ones...

Mike Eslea said...

Forget that last link - it's behind a paywall. This

looks like it goes to the full paper.

More on meta-analysis when I get around to it...

canonical_view said...

Nice - I really like your explanation of the various effects by which CAM/placebos may exert their effects :)

Neuroskeptic said...

Ok, true. But still, if a crap study gets any "weight", I'd still say that's a bad thing. Although you could argue that even bad evidence is still some evidence. I suppose it's a matter of opinion...