Evidence Schmevidence #25

The more I do work in the healthcare sector, the more sceptical I get about ‘clinical evidence’.

I started my career in the aerospace so I understand the idea of evidence. When aerospace engineers get things wrong, aeroplanes fall out of the sky, and that’s never a good thing. Evidence is what keeps planes flying.

A couple of year’s ago, after a particularly frustrating experience with a part of the UK NHS that will remain nameless for the moment, we did a few calculations to translate the current level of safety performance of the NHS into aerospace terms. The numbers came out a bit scary. So we did them again. And then again from a different direction. Expressed in mortality rates, the NHS, we concluded, is currently equivalent to a shade under 2000 plane crashes per day. Over UK air-space.

When we shared the data with this part of the NHS, asking them to check over our numbers to see where we were going wrong, surprise, surprise, we never heard from them again.

Clinical Evidence matters came to a head again a couple of months ago. This time with a different part of the NHS. This time in a workshop setting. We were talking about Complex Systems Theory. And someone – inevitably I now see – asked the clinical evidence question. Where’s your evidence that treating healthcare as a complex system is a better way of doing things?

For a few moments, I didn’t know what to say. Fortunately, the question came just before a break, so I muttered a no-doubt inane answer and tried to move on.

Over the break a made a new slide. Here’s a copy of it:

schmevidence 1

After the break, when everyone was back in the room, I put it up on the screen and asked for a hands-up vote. Everyone sat there, paralysed. ‘Any thoughts?’ I probed. Nothing.

I’d kind of anticipated the reaction, so, after I’d let the tension build a bit more, I advanced the presentation to a new version of the slide. This is it:

schmevidence 2

Now answering the question had become easy. Within a minute we had a collective answer: 10% A, 90% C.

It’s a fine line sometimes. And it’s difficult to know which side we’re on.

Is it better to treat a system as a system or not as a system? There’s a clue in the question, right?

Is it better to treat – say – a headache as a system or with a pill?

Is it better to deal with crime as a system or by longer prison sentences?

Is it better to deal with education as a system or by re-introduction of grammar schools?

Is it better, post the Brexit vote, to have experts or not to have experts?

At which point on the line do we make the transition from tautology to ‘we really don’t know, so we need to go gather some actual evidence’?

I have some sympathy with those that, per Michael Gove’s epoch-making statement, ‘have had enough of experts’. But the alternative is not to say, ‘oh, in that case, let’s listen to dumb, stupid people instead’, it’s to ask the question, ‘experts in what?’

Wherever we all might individually draw the tautology line, given the choice of treating a system as a system or not a system, there’s really only one sensible answer. And a good part of the answer to the ‘experts in what?’ question, therefore, ought to be, ‘experts in systems’.

So why then did 90% of the people in my workshop ignore the obvious? People say and do things for two reasons; the good one and the real one. Apparent absence of clinical evidence is a good reason for denying the need to treat systems as systems. The real reason, of course, is that the 90% of people that voted ‘C’ in my workshop simply didn’t understand what a system was, and so used clinical evidence as their get-out-of-jail-free’ card.

To me, anyone that doesn’t understand systems, probably shouldn’t be working within one, but that, unfortunately, would mean no-one could go to work anymore. Everything in life is a system. Life is systems. So, assuming we have to have people working within systems that don’t understand systems, that doesn’t also mean we can or should allow managers the same privilege. And there’s the problem in a nutshell. Not just in the NHS, but in all walks of life, 90% of managers or leaders have no idea what a system is. So what we end up with are a million and one ‘fixes’ that backfire: headache medications that lead to addiction and long-term digestive tract injury; harsher prison sentences that increase crime-rates; education standards initiatives that increasingly make students into dysfunctional members of society; homeless shelters that perpetuate homelessness; food-aid programmes that increase starvation.

When it comes to politics my main rule is anyone that wants to be a politician, shouldn’t be allowed to become one. My second rule is, whoever’s left over, is only allowed to become a politician once they’ve graduated Systems Theory class. My new rule, as of two months ago, is that what applies to politicians also applies to managers and leaders.