In the middle of 2013 I had a nasty bout of depression and was prescribed anti-depressant drugs. Although undiagnosed, I think I may have suffered low level depression for a few years, but had avoided anti-depressants and indeed other treatment for a couple of reasons:
- I am a man, and men are bad at looking after their own health.
- The stigma around mental health. It’s tough to face it and do something about it. Consider how you react to these two statements “I broke my leg and took 6 months to recover”, and ” I broke my mind and took 6 months to recover”.
- The opinion of people influential in my life at that time. My GP friend Michael presented a statistic that anti-depressants were only marginally better that placebos (75% versus 70%) in treating depression. I was also in a close relationship with a person who held an “all drugs are bad”, anti-western medicine mentality. At the time I lacked the confidence to make health choices that were right for me.
Combined, these factors cost me 18 months of rocky mental health.
When my health collapsed the mental health care professionals recommend the combination of anti-depressants and counselling with a psychologist or psychiatrist. The good news is that this treatment, combined with a lot of hard work, and putting positive, supportive, relationships around me, is working. I came off the bottom quite quickly (a few months), and have continued to improve. I am currently weaning myself off the anti-depressants, and life is good, and getting better, as I “re-wire” my thought process.
That’s the difficult, personal bit out of the way. Lets talk about anti-depressants and science.
Did Anti-deps help me?
Due to Michael’s statistic above (anti-deps only 5% better than placebo) I was left with lingering doubts about anti-depressants. Could I be fooling myself, using something that didn’t work? This was too much for the scientist in me, so I felt compelled to check the evidence myself!
Now, the fact that I “got better” is not good enough. I may have improved from the counselling alone. Or through the “natural history” of disease, just like we automatically heal in 1-2 weeks from a common cold.
The health care professionals I worked with are confident anti-depressants function as advertised, based on their training and years of experience. This has some weight, but the causes and effects in mental health are complex. Professionals can hold mistaken beliefs. Indeed a wise professional will adapt as medical science advances and new therapies are replaced by old. They are not immune to unconscious bias. So the views of professionals, even based on years of experience, is not proof.
Trust Me. I’m a Doctor
I am a “Dr”, but not a medical one. I have a PhD in Electronic Engineering. I don’t know much about medicine, but I do know something about research. In a PhD you create a tiny piece of new knowledge, something human kind didn’t know before. It’s hard, and takes years, and even then the “contribution” you make is usually minor and left to gather dust on a shelf in a university library.
But you do learn how to find out what is real and what is not. How to separate facts from bullshit. You learn about scientific rigour. You do that by performing “research and disappointment” for four years, and finding out just how wrong you can be so many times before finally you get to to core of something real. You learn that what you want to believe, that your opinion, means nothing when it gets tested against the laws of nature.
So with the help of Michael and a great (and very funny) book on how medical trials work called Snake Oil Science, I did a little research of my own.
Drilling into a few studies
What I was looking for were “quality” studies, which have been carefully designed to sort out what’s true from what’s not. So my approach was to look into a few studies that supported the negative hypothesis. Get beyond the headlines.
One high quality study with the widely presented conclusion “anti-deps useless for mild and moderate depression” was (JAMA 2010). This paper and it’s conclusion has been debunked here. Briefly, they used the results from 3 studies of just one SSRI (Paxil) and used that under-representation to draw impossibly broad conclusions.
Ben Goldacre is campaigning against publication bias. This is the tendency for journals only to publish positive results. This is a real problem and I support Ben’s work. Unfortunately, it also feeds alt-med conspiracy theories about big pharma.
Ben has a great TED Talk on the problem of publication bias in drug trials. To lend credibility he cites a journal paper (NEJM 358 Turner). Ben presents numbers from this paper that suggest anti-depressants don’t work, due to selective publishing of only positive trials.
Here a couple of frames from Ben’s TED talk (at the 7:30 mark). Big pharma supplied the FDA with these results to get their nasty western meds approved:
However here are the real results with all trials included:
Looks like a damning case against anti-deps, and big pharma. Nope. I took the simple step of reading the paper, rather than accepting the argument from authority that comes from a physician quoting a journal paper, in A TED talk. Here is a direct quote from the paper Ben cited:
“We wish to clarify that non-significance in a single trial does not necessarily indicate lack of efficacy. Each drug, when subjected to meta-analysis, was shown to be superior to placebo. On the other hand, the true magnitude of each drug’s superiority to placebo was less than a diligent literature review would indicate.”
Just to summarise: Every drug. Superior to a placebo. This means they work.
The paper continues. By averaging all the data the overall mean effect size over all studies (published and not, all drugs) was 32% over a placebo. That’s actually quite positive.
So while Ben’s argument of publication bias is valid, his dramatic implication that anti-deps don’t work is wrong, at least from this study.
Yes publication bias is a big problem and needs to be addressed. However science is at work, self correcting, and it’s good to see guys like Ben working on it. It’s a classic trick used by alt-med as well: just quote good results, and ignore the results that show the alt-med therapies to be ineffective. This is Bad Science.
However this doesn’t discredit science, and shouldn’t make us abandon high quality trials and fall back on even poorer science like anecdotes and personal experience.
This article from CBC News. No references to clinical studies, some leading questions, and a few personal opinions. So it’s just a hypothesis – but no more that that. A lack of understanding of the chemical functionality of a drug doesn’t invalidate it’s use. This isn’t the first time an effective drug’s function wasn’t well understood. For example Paracetamol isn’t completely understood even today.
As usual, a little digging reveals a very different slant that’s makes the CBC article look misleading. The author of the book is quoted in Wikipedia:
“Whitaker acknowledges that psychiatric medications do sometimes work but believes that they must be used in a ‘selective, cautious manner’. It should be understood that they’re not fixing any chemical imbalances. And honestly, they should be used on a short-term basis.”
I am attracted to the short term approach, and it is the approach suggested by the mental health care professionals that has helped me. Like a bandage or cast, anti-deps can support one while other mental health repairs are going on.
In contrast, the CBC article (first para):
“But people are questioning whether these drugs are the appropriate treatment for depression, and if they could even be causing harm.”
Poor journalism and cherry picking.
My little investigation is by no means comprehensive. However the high quality journal papers I’ve studied so far support the hypothesis that anti-deps work and debunk the “anti-depressants are not effective compared to placebo” argument to my satisfaction.
I would like to read more studies of the combination of psycho-therapy and SSRIs – if anyone has any references to high quality journal papers on these subjects please let me know. The mental health nurse that treated me last year suggested recovery was about “40% SSRIs + 60% therapy”. I can visualise this treatment as a couple of normal distribution curves overlapping, with the means added together to be your mental health.
Medicine and Engineering
I was initially aghast at some of the crappy science even I can pick up in these “journal” papers. “This would never happen in engineering” I thought. However I bet some similar tricks are at play. There are pressures to “publish, patent” etc that would encourage bad science there too. For example signal processing papers rarely publish their source code, so it’s very hard to reproduce a competing algorithm. All you have is a few of the core equations. If I make a bug while simulating a competitors algorithm, it gives me the “right” answer – Oh look mine is better!
In my research: Some people using Codec 2 say it sounds bad and doesn’t work well for HF comms. Other people are saying it’s great and much better than the legacy analog technology. Huh? Well, I could average them out in a meta study and say “it’s about the same as analog”. Or use my internal bias and self esteem to simply conclude Codec 2 is awesome.
But what I am actually doing is saying “Hmm, that’s interesting – why can two groups of sensible people have the opposite results? Lets look into that”. Turns out different microphones make Codec 2 behave in different ways. This is leading me to investigate the effect of the input speech filtering. So through this apparent conflict we are learning more and improving Codec 2. What an awesome result!
I suspect it’s the same with anti-deps. Other factors are at play and we need better study design. Frustrating – we all want definitive answers. But no one said Science was easy. Just that it’s self correcting.
That’s why IFL Science.