There’s a big problem with the way we study new drugs to treat cancer: Most clinical trials report that new treatments are more effective than they turn out to be.

These pharmaceutical-sponsored randomized Phase III trials, which evaluate how new cancer treatments compare to standard care, often find benefits—but then subsequent studies report that these drugs don’t improve the chances that a person with that cancer will survive longer.

In the latest study, published in the September 2021 issue of JNCCN—Official Journal of the National Comprehensive Cancer Network, researchers analyzed 362 industry-sponsored Phase III randomized trials in oncology from 2008 to 2017.

They found that 58% reported false positive results.

 

“A false positive result means that a clinical trial concludes that a drug has efficacy when the drug actually does not prolong survival in a clinically meaningful way,” explains lead researcher Changyu Shen, PhD, an associate professor at Harvard Medical School at the time this study was conducted. “The main negative effects for cancer patients are that false positive results may lead to approval of drugs with insufficient benefit in prolonging survival, thus exposing patients to adverse effects of drugs that are unlikely to produce meaningful health gains.”

What’s more, when someone with cancer winds up taking an ineffective drug, they may miss out on the opportunity to try something that is effective. “Patients could also miss the optimal window of intervention when they could receive other treatments with meaningful benefits,” says Shen. Finally, there’s the cost. “The drug could also impose a hefty financial burden on patients with only limited survival benefit.”

To remedy the problem, Shen and colleagues advise a more stringent standard before drug trials even proceed to Phase III.

A quick primer: Clinical trials, that is, studies that involve humans, usually proceed through three phases. In a Phase I trial, the goal is to discover the highest dose of the new treatment that can be given safely without causing severe side effects; it is not primarily about whether the drug works. If found safe, the drug then proceeds to Phase II, which evaluates whether the new drug appears to have a benefit, such as tumor shrinkage, or a longer period before a tumor starts growing again, or longer survival. If Phase II results are positive, the drug then goes on to a Phase III trial, in which it is tested against the best-known current standard of care to see whether it is more effective than what is already being used and to compare side effects.

The issue is the green light to go from Phase II to III. To Shen and colleagues, too many experimental drugs proceed to Phase III. Currently, a drug’s effectiveness needs to pass certain statistical thresholds to make the cut. But those standards need to change, the researchers argue, because they’re allowing too many ineffective drugs to move along.

Tightening the scientific standards, Shen admits, may mean a less robust drug pipeline. “A more stringent approach in Phase III trials could result in fewer oncology medications moving to Phase III trials,” he says. But that’s actually a good thing for many patients, he argues. “Ideally, we would like to allow medications with clinically meaningful benefit in prolonging survival to continue to move to Phase III and stop those medications without meaningful benefit using data collected in Phase II.”

The issue is related to another controversy—the use by the Food and Drug Administration (FDA) of surrogate endpoints, which are markers estimated to be associated with better outcomes, as a basis for accelerated emergency FDA approval for cancer drugs. The FDA generally requires further study to determine whether the approved drugs improve survival before granting full approval. Recently, due to those post-approval studies, the FDA has withdrawn approval for certain immunotherapy drugs for specific cancers because those studies found no actual benefit over standard care. “We did not study the quality of surrogate endpoints in our study,” says Shen. “But I think this is an important question that needs more research to answer.” 

How can a person in treatment for cancer use this information to get better care? One way is to question your oncologist not just about the potential benefits and side effects of a possible new treatment but about how confident researchers are in the results—what their level of certainty is. “Central to the challenge facing a patient,” Shen says, “is the uncertainty of risk/benefit of a drug.”

His advice: “Patients should seek information on the range of survival benefits instead of just an estimate. For example, ‘12 months survival benefit’ is not sufficiently informative without some uncertainty measure. Such a number is particularly fragile when faced with questions such as ‘How sure are you about it?’”

Asking for a range helps. “‘The survival benefit is somewhere between six and 16 months,’ adds more information," he says. “It means that with confidence, the survival benefit is at least six months and no more than 16 months for the average patient.”

The study is titled, “Underperformance of Contemporary Phase III Oncology Trials and Strategies for Improvement.”

See also, “FDA Reconsiders Accelerated Approvals of Immunotherapy Drugs” and “Asking The Right Questions.”