Do Anti-Depressants work?

In the middle of 2013 I had a nasty bout of depression and was prescribed anti-depressant drugs. Although undiagnosed, I think I may have suffered low level depression for a few years, but had avoided anti-depressants and indeed other treatment for a couple of reasons:

  • I am a man, and men are bad at looking after their own health.
  • The stigma around mental health. It’s tough to face it and do something about it. Consider how you react to these two statements “I broke my leg and took 6 months to recover”, and ” I broke my mind and took 6 months to recover”.
  • The opinion of people influential in my life at that time. My GP friend Michael presented a statistic that anti-depressants were only marginally better that placebos (75% versus 70%) in treating depression. I was also in a close relationship with a person who held an “all drugs are bad”, anti-western medicine mentality. At the time I lacked the confidence to make health choices that were right for me.

Combined, these factors cost me 18 months of rocky mental health.

When my health collapsed the mental health care professionals recommend the combination of anti-depressants and counselling with a psychologist or psychiatrist. The good news is that this treatment, combined with a lot of hard work, and putting positive, supportive, relationships around me, is working. I came off the bottom quite quickly (a few months), and have continued to improve. I am currently weaning myself off the anti-depressants, and life is good, and getting better, as I “re-wire” my thought process.

That’s the difficult, personal bit out of the way. Lets talk about anti-depressants and science.

Did Anti-deps help me?

Due to Michael’s statistic above (anti-deps only 5% better than placebo) I was left with lingering doubts about anti-depressants. Could I be fooling myself, using something that didn’t work? This was too much for the scientist in me, so I felt compelled to check the evidence myself!

Now, the fact that I “got better” is not good enough. I may have improved from the counselling alone. Or through the “natural history” of disease, just like we automatically heal in 1-2 weeks from a common cold.

The health care professionals I worked with are confident anti-depressants function as advertised, based on their training and years of experience. This has some weight, but the causes and effects in mental health are complex. Professionals can hold mistaken beliefs. Indeed a wise professional will adapt as medical science advances and new therapies are replaced by old. They are not immune to unconscious bias. So the views of professionals, even based on years of experience, is not proof.

Trust Me. I’m a Doctor

I am a “Dr”, but not a medical one. I have a PhD in Electronic Engineering. I don’t know much about medicine, but I do know something about research. In a PhD you create a tiny piece of new knowledge, something human kind didn’t know before. It’s hard, and takes years, and even then the “contribution” you make is usually minor and left to gather dust on a shelf in a university library.

But you do learn how to find out what is real and what is not. How to separate facts from bullshit. You learn about scientific rigour. You do that by performing “research and disappointment” for four years, and finding out just how wrong you can be so many times before finally you get to to core of something real. You learn that what you want to believe, that your opinion, means nothing when it gets tested against the laws of nature.

So with the help of Michael and a great (and very funny) book on how medical trials work called Snake Oil Science, I did a little research of my own.

Drilling into a few studies

What I was looking for were “quality” studies, which have been carefully designed to sort out what’s true from what’s not. So my approach was to look into a few studies that supported the negative hypothesis. Get beyond the headlines.

One high quality study with the widely presented conclusion “anti-deps useless for mild and moderate depression” was (JAMA 2010). This paper and it’s conclusion has been debunked here. Briefly, they used the results from 3 studies of just one SSRI (Paxil) and used that under-representation to draw impossibly broad conclusions.

Ben Goldacre is campaigning against publication bias. This is the tendency for journals only to publish positive results. This is a real problem and I support Ben’s work. Unfortunately, it also feeds alt-med conspiracy theories about big pharma.

Ben has a great TED Talk on the problem of publication bias in drug trials. To lend credibility he cites a journal paper (NEJM 358 Turner). Ben presents numbers from this paper that suggest anti-depressants don’t work, due to selective publishing of only positive trials.

Here a couple of frames from Ben’s TED talk (at the 7:30 mark). Big pharma supplied the FDA with these results to get their nasty western meds approved:

However here are the real results with all trials included:

Looks like a damning case against anti-deps, and big pharma. Nope. I took the simple step of reading the paper, rather than accepting the argument from authority that comes from a physician quoting a journal paper, in A TED talk. Here is a direct quote from the paper Ben cited:

“We wish to clarify that non-significance in a single trial does not necessarily indicate lack of efficacy. Each drug, when subjected to meta-analysis, was shown to be superior to placebo. On the other hand, the true magnitude of each drug’s superiority to placebo was less than a diligent literature review would indicate.”

Just to summarise: Every drug. Superior to a placebo. This means they work.

The paper continues. By averaging all the data the overall mean effect size over all studies (published and not, all drugs) was 32% over a placebo. That’s actually quite positive.

So while Ben’s argument of publication bias is valid, his dramatic implication that anti-deps don’t work is wrong, at least from this study.

Yes publication bias is a big problem and needs to be addressed. However science is at work, self correcting, and it’s good to see guys like Ben working on it. It’s a classic trick used by alt-med as well: just quote good results, and ignore the results that show the alt-med therapies to be ineffective. This is Bad Science.

However this doesn’t discredit science, and shouldn’t make us abandon high quality trials and fall back on even poorer science like anecdotes and personal experience.

Breathless Headlines

This article from CBC News. No references to clinical studies, some leading questions, and a few personal opinions. So it’s just a hypothesis – but no more that that. A lack of understanding of the chemical functionality of a drug doesn’t invalidate it’s use. This isn’t the first time an effective drug’s function wasn’t well understood. For example Paracetamol isn’t completely understood even today.

As usual, a little digging reveals a very different slant that’s makes the CBC article look misleading. The author of the book is quoted in Wikipedia:

“Whitaker acknowledges that psychiatric medications do sometimes work but believes that they must be used in a ‘selective, cautious manner’. It should be understood that they’re not fixing any chemical imbalances. And honestly, they should be used on a short-term basis.”

I am attracted to the short term approach, and it is the approach suggested by the mental health care professionals that has helped me. Like a bandage or cast, anti-deps can support one while other mental health repairs are going on.

In contrast, the CBC article (first para):

“But people are questioning whether these drugs are the appropriate treatment for depression, and if they could even be causing harm.”

Poor journalism and cherry picking.

My Conclusions

My little investigation is by no means comprehensive. However the high quality journal papers I’ve studied so far support the hypothesis that anti-deps work and debunk the “anti-depressants are not effective compared to placebo” argument to my satisfaction.

I would like to read more studies of the combination of psycho-therapy and SSRIs – if anyone has any references to high quality journal papers on these subjects please let me know. The mental health nurse that treated me last year suggested recovery was about “40% SSRIs + 60% therapy”. I can visualise this treatment as a couple of normal distribution curves overlapping, with the means added together to be your mental health.

Medicine and Engineering

I was initially aghast at some of the crappy science even I can pick up in these “journal” papers. “This would never happen in engineering” I thought. However I bet some similar tricks are at play. There are pressures to “publish, patent” etc that would encourage bad science there too. For example signal processing papers rarely publish their source code, so it’s very hard to reproduce a competing algorithm. All you have is a few of the core equations. If I make a bug while simulating a competitors algorithm, it gives me the “right” answer – Oh look mine is better!

In my research: Some people using Codec 2 say it sounds bad and doesn’t work well for HF comms. Other people are saying it’s great and much better than the legacy analog technology. Huh? Well, I could average them out in a meta study and say “it’s about the same as analog”. Or use my internal bias and self esteem to simply conclude Codec 2 is awesome.

But what I am actually doing is saying “Hmm, that’s interesting – why can two groups of sensible people have the opposite results? Lets look into that”. Turns out different microphones make Codec 2 behave in different ways. This is leading me to investigate the effect of the input speech filtering. So through this apparent conflict we are learning more and improving Codec 2. What an awesome result!

I suspect it’s the same with anti-deps. Other factors are at play and we need better study design. Frustrating – we all want definitive answers. But no one said Science was easy. Just that it’s self correcting.

That’s why IFL Science.

Email to an Anti-Vaxer

I recently had an email conversation with an anti-vaxer. I enjoy critical thinking exercises like this, and thought my response might be useful to publish.

The anti-vaxers key points (I’ve paraphrased here as it was a private email) were:

“An Australian TV program A Current Affair (ACA) screened a segment around a family who was awarded a $10M settlement after their baby had a serious adverse reaction to a vaccination. The ACA article claimed 200 other similar cases, and that a cover up by professionals was involved. The Anti-vaxer knows Doctors, Nurses, and Microbiologists who share the same anti-vaxer views, and considers the list of ingredients in vaccines to be horrible.”

Here is my response:

I researched the case you mentioned and there is indeed evidence (Link) that the poor child sustained brain damage as a result of that vaccination procedure. It’s food for thought:

  1. If true, that’s 1 serious adverse affect out of perhaps 10M vaccinations in this country (my guess) last year. I understand there are often adverse but minor reactions (e.g. fever). It also appears the authorities dropped the ball on that vaccine. However it has been noticed, and corrected. That’s why I love science.

    When was the last time an anti-vaxer said “Oh, you know what – I was wrong”.

  2. Mumps (alone) very badly hurts 2 children out of 1000 (Ref).
  3. If you get in a car your risk of death is 5 in 100,000 per year (Ref). So when your drive to your anti-vaxer meeting you have a 500 times greater risk of stone cold death than a possible risk of injury from vaccination.
  4. “A Current Affair” is not known for reputable science journalism. That doesn’t mean their claims are not true, but it would be prudent to look further rather than accept a tabloid TV story just because it fits your world view. Do you have any evidence for the claims of the other 200 cases? If it’s truly a criminal cover up involving harm to a child, are the Police investigating? Why not?
  5. Dr/Nurses having an opinion is called an “argument from authority”. Their view has some validity, and is a good source of a hypothesis (i.e. an untested idea or theory). But it doesn’t prevail over evidence. It’s not a fact – merely an opinion. The gold standard for evidence is peer reviewed journal papers. If you really want to know – go Google a few of them. I was surprised to find the medical ones quite easy to read.
  6. There are horrible substances in nature (like animal faeces spread on my vegetables to fertilise them, or nasty bacteria in milk). Doesn’t make vegetables bad for me. Oh wait.
  7. However it’s a lot of work to read up on all this, and we seem to get one health “risk” shoved at us after another from friends (“They say…….”), Facebook, and the media.

    So I have a very quick test I use to filter claims. Do you personally know anyone who has sustained any permanent damage from a vaccine? Ever been to a funeral where some one has died from a vaccine?

    David’s very simple test of what’s really a health risk and what’s not – “who do you know who has died from this?”. I know many people have died in a car accident, or from suicide, heart disease, old age, and cancer. So they are real risks for me.

  8. These graphs from the CDC say it all. Infectious disease. Nailed. Deaths halved. Life spans doubled. IFL vaccinations and anti-biotics!

    – David