The necessary discomfort of soft intelligence

It’s comforting to have hard facts and figures so that we can feel like we know exactly what went wrong and what went right. When the data in question speak to patient safety, to the need to prevent harm to patients and their families, the urge to find the answers in ‘the facts’ may be given greater. But, as my colleagues and I suggest in an open-access paper published this month, this might be exactly the time when it’s most important to look beyond hard data to what we call ‘soft intelligence’: stories, narratives, complaints, worries and comments that can’t be reduced to numbers but nevertheless can offer valuable insights into hazards, vulnerabilities and risks in healthcare systems.

A new approach to data

Calls for more use of ‘soft intelligence’ have been made by academic and policy commentators alike in the wake of recent healthcare failings. The Berwick, Francis and Kirkup reports all made similar points about the limitations of hard data and the need for complementary use of alternative sources of knowledge. In his report on deaths at Morecambe Bay maternity unit, Dr Bill Kirkup highlighted the limitations of “high-level metrics” in identifying problems. Sir Robert Francis went further in his second inquiry into the quality of care at Mid Staffordshire, suggesting that the numbers could offer false comfort: “statistics and reports were preferred to patient experience data.” This led the panel of experts led by Don Berwick to call for a broader approach to understanding what goes on at the ‘sharp end’ of care:

“Leaders need first-hand knowledge of the reality of the system at the front line, and they need to learn directly from and remain connected with those for whom they are responsible. [… They must] hear the patient voice at every level, even when that voice is a whisper.”

But how exactly how clinicians and healthcare managers might go about obtaining and making use of soft intelligence remains less clear.

The study

The study, which I worked on with Lorna McKee and Mary Dixon-Woods, involved 107 in-depth qualitative interviews with senior leaders involved in quality and safety in the NHS, and resulted in an interesting set of findings that suggested that managers and clinicians are already well aware of the problems that can be caused by reliance on hard data alone—but are still searching for the best ways to make use of soft intelligence.

Participants were unanimously dedicated to the improvement of patient safety, and seemed to be very aware of the benefits of using soft data to highlight problems in quality of care. They also recognised the need for interpretive work when dealing with soft data, to turn data into ‘intelligence’ to prevent its becoming an overwhelming blur of noise. They did this by, for example,

• aggregating soft data: looking for trends or and interpreting these as evidence of importance
• triangulating soft data with hard data to add nuance, or to indicate specific aspects of the hard data that might be particularly interesting or useful
• instrumentalising soft data: using them as a rhetorical device to add emotional force to an argument for change, usually premised on quantitative data

The danger of systematising soft data

These are all valid approaches, with much to commend them. But they’re not perfect. Reliability or ‘repeatability’ is not the same thing as validity: sometimes it’s the lone voice that is saying the most important things. Systematising data collection tends to emphasise majority views: as others have found, it can even alter or corrupt the original information contained within the data. Put bluntly, if we treat soft data in the same way as quantitative data, we could lose their greatest benefit: a way to access the unknown and unexpected.

Some of our interviewees were finding creative ways of overcoming this tendency. They were exploring intriguing approaches that tried to exploit the ‘problem sensing’ potential of soft data, through things like listening clinics, open days, staff and patient forums, and similar opportunities for dialogue. Occasions like this enabled senior leaders to explore soft data, understand aberrant opinions, and gain access to worries and concerns raised spontaneously by patients and colleagues.

Soft intelligence and ‘chronic unease’

What this points towards is the need for what writers in other safety-critical industries have called ‘chronic unease’: a consistent sense of concern and willingness to ask difficult questions that can offer a defence against complacency or undue comfort in apparently positive metrics. As Peter Davies, the CQC’s Regional Advisor in Primary Care for Northern England put it:

“The paper highlights the need for radical discomfort and to consider, even when things are going smoothly, what weeds are growing in an otherwise well-tended garden.”

As others have pointed out, this means making the best use we can of the resources we have that can highlight areas for concern, interrogating much more fully the rich data available in, for example, incident reports. But it’s also about a particular mindset, taking the time to question received wisdom and having the temerity to speak up about problems even when others think everything is fine.

There is a clear desire in the NHS for a proactive approach to patient safety. Achieving this may sometimes require sacrificing the security and satisfaction of apparently definitive statistics for the complexity and uncertainty of soft intelligence, however discomforting it may feel.