Oncologists often use the phrase “clinically meaningful benefit” to describe the effect of an experimental treatment. But is the benefit meaningful for patients? A new paper suggests that benefit claims in journal articles often fall short.
Figure: Added months of survival deemed clinically meaningful (Dreicer et al.)Jessica Dreicer, a third-year internal medicine resident in the OHSU School of Medicine, and coauthors including OHSU assistant professor Vinay Prasad, M.D., M.P.H., searched articles published in 2014 and 2015 to find 53 claims of meaningful benefit in randomized cancer studies.
They compared the outcome of each trial with the thresholds of clinically meaningful benefit proposed by the American Society of Clinical Oncology and the European Society of Medical Oncology. (In terms of overall survival, for example, the ASCO guidelines suggest that new drugs should result in a relative increase in median overall survival of at least 20 percent, or 2.5 to 6 months.)
Claims of a “clinically meaningful” benefit based on overall survival did not meet the ASCO and ESMO guidelines 60 to 70 percent of the time. Claims based on progression-free survival did not stack up 30 percent of the time.
“Given that the ASCO and ESMO thresholds are modest, we believe real world usage that falls short of this is setting the bar too low for our patients,” concluded Dreicer, Prasad and co-authors. “Future research should explore what magnitudes of benefit patients consider meaningful benefit, and whether these might serve as an externally valid metric for professional societies.”
Claims of a “clinically meaningful” survival benefit did not meet ASCO and ESMO guidelines 60 to 70 percent of the time.
‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾‾
Part of the problem is the increasing pervasiveness of clinical trials designed to detect effect sizes that are too small to be clinically meaningful. In a review of 226 randomized clinical trials of systemic cancer therapies, researchers found that less than a third were designed to detect or exclude a difference between treatment arms that could meet ESMO criteria for meaningful clinical benefit.
The authors, including study design and author interpretations of ‘benefit’ have evolved since the 1970s. Median sample size increased from 100 in the 70s and 80s, to 446 in the 90s, to 722 in the years 2005–2009. And while the effect size remained stable over time, authors today are substantially more likely to strongly endorse the experimental arm of the trial as the new standard of care, they reported.
Overstating the benefits of a new cancer therapy is no trivial matter. It undermines the credibility of journals. It confuses decisions about how to allocate funding for medical care. And it harms patients who pursue treatments based on exaggerated expectations of benefit without a realistic understanding of the risk of adverse effects and the personal financial toll.
∞
Further reading:
Clinically meaningful benefit: real world use compared against the American and European guidelines by Jessica J. Dreicer, Sham Mailankody, Farhad Fahkrejahani and Vinay Prasad. Blood Cancer Journal (December 14, 2017)
A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS) by Nathan I. Cherny and others. Annals of Oncology (2015)
American Society of Clinical Oncology Perspective: Raising the Bar for Clinical Trials by Defining Clinically Meaningful Outcomes by Lee M. Ellis and others. Journal of Clinical Oncology (2014)
Do Contemporary Randomized Controlled Trials Meet ESMO Thresholds for Meaningful Clinical Benefit? by J. C. Del Paggio and others. Annals of Oncology (2017)
