Nature Medicine today retracted a striking paper suggesting lung cancer patients who received immunotherapy in the morning instead of after 3 p.m. were half as likely to see their tumors grow and survived for nearly 1 year longer.
The clinical trial, published in February, got extensive media coverage and excited some physicians. But it immediately attracted scrutiny. On PubPeer and social media, researchers said the large increase in survival seemed almost too good to be true—it rivaled the benefit observed in previous trials in patients who received immunotherapy in addition to chemotherapy, compared with chemotherapy alone. That “something as mundane as clock time” could work as well as adding a powerful drug “would reshape oncology practice overnight” if true, internist Anil Makam of the University of California San Francisco wrote in his newsletter.
He and other researchers pointed out other red flags. For example, none of the 210 Chinese patients had dropped out of the trial because of side effects—extremely uncommon for an oncology trial. Discrepancies in the trial records showed the study design and outcome measures—which are supposed to be locked before the study starts—had changed during the study. And unusual gaps in treatment times and oddities in patients’ follow-up suggested participants hadn’t really been randomly assigned to the two treatment groups.
Besides, the treatment consisted of antibodies whose levels in the blood slowly decline over weeks, so there weren’t “a lot of good biological plausible hypotheses” for why giving it in the morning would make a difference, cancer researcher and oncologist Sumanta Pal told STAT at the time.
Those concerns prompted Nature’s editors to add an “alert” note to the paper days after its publication. After reviewing additional data provided by the authors, the journal has now retracted the paper. The editors cited the problems others had noticed as well as discrepancies between the Chinese and English versions of the protocol, “unexpected data patterns,” and evidence that most patients had been randomized on the day of treatment, which is uncommon. Because of these and other concerns, the editors “no longer have confidence in the integrity of the results,” a retraction note says.
The last author on the paper, Yongchang Zhang of the Hunan Cancer Hospital, wrote in an email to Science that an internal review confirmed there were several problems with the study. “We acknowledge these shortcomings and sincerely apologize for any inconvenience caused to the Journal and its readers,” wrote Zhang, who said he spoke for all five corresponding editors on the paper. He did not explain how the problems arose.
The retraction is a reminder of the importance of “resisting the temptation to fall for the hype and jump to changing practice” when studies haven’t been properly scrutinized and replicated, Paolo Tarantino, a medical oncologist at Dana-Farber Cancer Institute, writes in an email to Science. (He was among the first to note the paper’s inconsistencies.)
Quite a few earlier studies had hinted at a benefit of giving immunotherapy in the morning. A meta-analysis of 29 studies, published in JAMA in May, suggested doing so was associated with better outcomes for several solid tumors.
But almost all of those studies were retrospective, meaning they looked at patients treated in the past. Such studies are prone to bias, however: There may be reasons why patients with higher chances of survival get treatment earlier in the day, such as their access to health care or their overall health. The now-retracted study was the only one included in the meta-analysis that used a randomized, controlled design, which reduces the risk of bias.
Meanwhile, additional prospective studies the past few months have cast doubt on the supposed benefits of morning cancer treatments. A study presented in March at the European Lung Cancer Congress that drew on data from eight international randomized trials involving more than 3000 lung cancer patients concluded the time of day when immunotherapy is administered makes no difference for survival. Another study, presented this spring at the American Society of Clinical Oncology’s annual meeting, found no association between morning treatment and a survival benefit in breast cancer patients.
More study of the phenomenon is needed, the authors of the latter study wrote in an email to Science. The effects of timing may depend on the type of tumor, the exact treatment, and on individual patients’ characteristics, say trial co-leader Valentina Guarneri of the University of Padova and colleagues. “The question may be less ‘What is the best time to treat?’ and more ‘For whom, if anyone, does timing matter?’” they add. Several prospective studies are currently exploring such questions.
The retraction of this paper from a high-impact journal should also prompt a broader reflection on the current state of peer review, in which unpaid reviewers struggle to keep up with a steady rise in submitted papers, Tarantino says. Journals need to improve the process, he adds, especially for “high-impact clinical studies that may influence patient care.”