'Not statistically significant:' researchers worldwide react with disappointment over Montreal COVID treatment study

Doctor defends COVID-19 drug study Dr. Jean-Claude Tardif explains why he claimed he'd found a drug effective against COVID, when other researchers say it's too soon to know.

Montreal — Many in the international science community reacted with disappointment and not a little frustration on Wednesday, after finally seeing statistical data from a study on the effectiveness of  medication colchicine.

“We can confirm that the primary outcome was not statistically significant,” said Dr. Emily McDonald a scientist at the MUHC’s Research Institute.

The anti-inflammatory drug, now used to treat gout was touted by the Montreal Heart Institute just days ago in a press release, as an effective oral medication for non-hospitalized COVID-19 patients. In a Friday press release, the MHI researchers claimed the drug reduced the risk of death or hospitalization for COVID-19 by 21 per cent.

But while the press release called it “a major scientific discovery,” there were no meaningful statistics provided so scientists could understand the real implications for patients. Nor had the study been peer reviewed and published in a scientific journal.

On Wednesday, a copy of a science manuscript was published on a preprint server for health sciences for everyone to see - some commenting on social media that they were not the results they’d been “sold.”

“It is the sense that the press release perhaps oversold the results of the trial, for instance, [the reduction of] deaths was not significant, now that we have the final numbers, but that was something the press release seemed to suggest,” said Dr. McDonald.

STUDY TAKEAWAYS

Investigators observed two major side effects in the trial data: diarrhea and also “a greater number of blood clots in a group that received the treatment - and this was statistically significant,” said the MUHC specialist.

She says that is a risk factor that needs to be considered and evaluated further.

The average age of participants was 50, so older people in their 70s and 80s were not well represented even though “these people may be likely to have more toxicity from the medication” according to McDonald.

Another limitation - more than 90 per cent of the population in the study was caucasian, so there isn’t much information about the effects of the drug in different ethnicities.

SCIENCE BY PRESS RELEASE

Releasing results of any clinical trial uniquely through a press release before the science has been independently evaluated is “really a symptom of the pandemic,” McDonald says.

But with scientists desperate for some positive news about treating COVID-19, that the facts didn’t live up to the triumphant media announcement is clearly irksome to some.

Especially since the principal investigator Dr. Jean-Claude Tardif explained in the online manuscript that the study was halted for logistical reasons but also because “the perceived need to disseminate the study results rapidly in view of the current state of the pandemic.”

McDonald said she believes researchers are operating in uncharted territory and that rushing out results quickly is a symptom of the pandemic.

“Researchers are really trying to find that balance between getting results out to the public to prevent unnecessary deaths, along with the necessary scientific scrutiny we require, to have confidence in the results.

The path forward now many are suggesting is to wait for an additional, confirmatory trial to see if colchicine shows any promise as a treatment for COVID-19 at all.

CAUTION URGED

Even before the manuscript and more complete data were posted online, Quebec’s College of Physicians, the health ministry and scientists urged doctors on Monday, to proceed with great caution in response to the positive press release.

“The results look good...and there is reason to believe it will have an effect, but there’s so much more of the story we’re missing,” said McDonald at the time.

But while it would be great news for people around the world if colchicine, an inexpensive, generic and available medication is indeed beneficial, the chief complaint was that no meaningful numbers were communicated in the press release to help scientists understand if that is true.

Compounding the concern for some, in an interview with The Canadian Press the following day Dr. Jean-Claude Tardif, the lead investigator and director of the MHI, said, "any doctor, tomorrow, who reads this can definitely decide to prescribe if he wants."

But McDonald and other colleagues said it’s just too early to make those claims with such conviction since the study hasn’t yet been peer-reviewed and published by a scientific journal.

“I’m not, ‘oh, I don’t buy it’,” Ashish Jha, dean of the Brown University School of Public Health in Rhode Island, was quoted in a story on the MHI announcement, published on the health and science site STAT.

“It’s possible. There’s enough plausibility here,” said Jha. “This could be a real finding, and if it is, that would be great. But this press release doesn’t get us there.”

There’s also confusion about what the press release did say. “Did it reach statistical significance? We're not quite sure," said McDonald, since the press release also suggests some results “‘approached’ statistical significance.”

‘ENCOURAGING’, BUT ‘PRELIMINARY’: DOCTORS ASKED TO BE PRUDENT

Quebec's College of Physicians and the Order of Pharmacists sent out a memo on Monday, asking their members to steer clear of colchicine for now.

The warning appears to be a response to a sentence in the MHI press release saying patients at risk of complications can be treated with colchicine “as soon as the diagnosis of COVID-19 is confirmed by a PCR test.”

However, the College of Physicians asked doctors “to act with great prudence if they were considering prescribing colchicine to patients who have COVID-19 until they receive clear directives.”

Pharmacists will not be permitted to dispense colchicine if the drug is prescribed for someone as a precaution, in the event they eventually test positive for COVID-19.

The orders’ statement also called the results of the clinical trial “encouraging” but “preliminary,” because the data haven’t been formally analyzed and published.

They noted that Quebec’s Health Ministry and the National Institute for Excellence in Health and Social Services were involved in the discussion.

Some doctors came to those same conclusions, the day the scant details were made public.

One family doctor in Montreal, who preferred to remain anonymous, told CTV News by email that “he would not be comfortable prescribing colchicine for COVID-19 at this time.”

SEEKING SAFETY INFORMATION

How is colchicine used now?

• An anti-inflammatory medication used to treat gout, and a rare illness called familial mediterranean fever

• Used off-label to treat pericarditis, an inflammation of the sac around the heart

When a drug is used in a new way, however, it also has to be evaluated for new harm, according to McDonald.

“We knew how to prescribe colchicine for gout yesterday, but that doesn’t mean we know how to prescribe it for COVID today.”

And while colchicine usually causes only mild side effects including diarrhea and nausea, McDonald says it can in rare instances be toxic and even fatal if a person is prescribed too high of a dose.

It can also be contraindicated for someone with certain kidney problems and can interact with common drugs like certain antibiotics or blood pressure medications.

“So, we need to wait for proper results [in a published study] before we prescribe it widely. Even a drug with a few side effects becomes more problematic when given to a very large population,” said McDonald.

She cites the hydroxychloroquine study at the R-I-MUHC that she co-authored as an example. Like colchicine, it was a repurposed medication and safe for other uses but was discovered to cause cardiac arrhythmia when used to treat patients with COVID-19.

“There are just so many steps in the scientific process before we can say this is a medication that is going to make a major difference in the world and in the pandemic.”

McDonald says she thinks once a manuscript is submitted, it will take about two to three weeks to get some answers. Science journals and reviewers have cut out the red tape, to respond to the urgency of the pandemic.

NUMBERS, NUMBERS, NUMBERS

Participants in Canada, the United States, Europe, South America and South Africa took part in the large randomized, controlled colchicine study.

The researchers provided four percentage figures to the public last week, to highlight the various reductions in deaths, hospitalizations, and the need for mechanical ventilation.

But those numbers don’t offer enough insight, says McDonald, who is also an internist at the MUHC.

“I think if you are going to release a number you should probably give the upper and low number around it - that’s the confidence interval. So, these numbers were released without the statistics around them,” and that means scientists can’t interpret the results, she said.

And seeing more “absolute” numbers would be more helpful than “relative” numbers, McDonald said.

“You can imagine reducing two complications to one complication is very different from reducing 200 to 100.

If all you show is a relative decrease of 50 per cent, that’s missing information that would help us know whether it’s important to patients.”

LISTEN: What's more effective in fighting COVID: investing in treatment or vaccines? Dr. Mitch Shulman weighs in on CJAD 800 Radio