It is easy to overlook the fact that over 90% of medical treatments are not backed by strong evidence. People can find it frustrating – even infuriating – when a review concludes that the evidence for a treatment is too weak to say whether it helps or harms.
This has been the case with the NHS England’s recent decision to restrict new prescriptions of cross-sex hormones for 16- and 17-year-olds.
The struggle to base clinical decisions on solid evidence is not new, nor is it unique to gender medicine. Archie Cochrane, a pioneering Scottish researcher, awarded obstetrics and gynaecology a wooden spoon in 1979 for the worst use of scientific evidence in clinical practice – a damning verdict that prompted the field to overhaul how it evaluated and applied research. It led to the first evidence-based textbook, a global movement and an online library.
Other medical fields have also struggled to meet this challenge, often through no fault of their own. Paediatrics, for example, faces a difficult balancing act when trying to produce clear, reliable studies.
To understand the complexities involved, look no further than your medicine cabinet. Paracetamol (acetaminophen) is considered the pain relief and antifever medicine of choice for infants and children. Weight-adjusted doses are scaled down safely from adult quantities, making it a versatile and trusted option across all age groups.
Aspirin, by contrast, occupies a more cautionary position. Its use in children and adolescents – particularly for viral illness, such as influenza or chickenpox – carries a well-documented risk of Reye’s syndrome – a rare but potentially fatal condition. Authorities actively advise against prescribing aspirin to anyone under 16, unless there is a specific clinical reason to do so.
These differences show that doctors cannot treat children as if they are just small adults. Evidence in children’s medicine is built up slowly. It includes treatments that work for all ages, some just for children, some with weak evidence, and some that cannot be fully studied for ethical or legal reasons.
Patrick Thomas/Shutterstock.com
In 2023, my colleagues and I at the University of Sheffield synthesised evidence on child and adolescent obesity to inform World Health Organization guidelines. While evidence on obesity treatments is generally plentiful, we faced challenges identifying published experiences of children regarding medical treatments.
Data for adolescents was limited and the experience of children of ten or under was entirely lacking. Without evidence, policymakers avoid “risky” options. But without policy support, researchers have little reason to study them.
How a verdict is reached
Looking beyond the complete absence of evidence, how does a health organisation decide evidence is “too weak”? Rather than a snap judgment, their verdict usually factors in four related concerns, each one lowering confidence a little further.
The first, most obvious reason is that studies may not have been designed or carried out very well. If parents know which children received the real medicine and which received the dummy version during a cough syrup trial, they may consciously or unconsciously report that the treatment looked better — or worse — than it really was. This design flaw makes it difficult to trust its conclusions.
Much of what medicine thinks it knows about treatments in children comes from observational data — records of what happened to patients in real-world clinical settings. Although valuable, these studies carry a trap. Children who receive a particular treatment are rarely typical. A rule of thumb is to ask whether a comparison is fair: were children who received the treatment genuinely similar to those who didn’t? If that question can’t be answered clearly, the finding deserves healthy scepticism.
A second concern arises when different studies asking the same question arrive at different conclusions. It is not enough to trust the majority verdict or the larger studies. It takes time to build a picture for each age group one study at a time — gathering enough to answer the question for an “average” child, if such a child ever exists.
Third, evidence may not match the question being asked. In the early 2000s, antidepressants were prescribed to children and teenagers with depression, largely based on evidence from adult studies. Close examination revealed that children taking some antidepressants showed higher rates of suicidal thoughts than those on a dummy pill.
Finally, studies need sizeable numbers of participants to narrow down uncertainty. Small studies of these antidepressants found that they appeared to reduce suicidal thinking. However, the true benefit of antidepressants lay somewhere between substantial and negligible – undermining confidence in study findings. Larger studies were needed.
Regulators in the US and the UK faced a dilemma: act on uncertain evidence or wait for better data while children continued to receive a potentially harmful treatment. Decisions still needed to be made. The regulators could not truly know when they decided to withdraw some of the antidepressants whether they had ultimately saved lives or denied young people much-needed treatment.
Thankfully, the evidence base in medicine, including paediatrics, is continually improving. Obsolete treatments are squeezed out of the health system, uncertainties about established treatments are reduced and new treatments are evaluated. A verdict for now is not a verdict forever. Identifying the causes of uncertainty helps direct attention to where future tipping points lie.
Andrew Booth receives funding from the World Health Organization and the National Institute for Health Research (UK). He is a co-convenor of the Cochrane Qualitative and Implementation Methods Group.
