Coronavirus vaccine trials will not tell us if a jab saves lives, expert stresses

Ongoing coronavirus vaccine trials cannot prove a jab could save lives, one expert has stressed.

An effective immunisation programme has long been hailed as a route back to life as we knew it.

Hopes were raised in July when scientists from the University of Oxford found a vaccine candidate induced “strong antibody and T-cell immune responses up to day 56 of the ongoing trial”.

Antibodies and T-cells make up part of the immune system, helping to prevent an infection from taking hold.

Russia’s controversial vaccine candidate also brought about an immune response within 21 days, however, some experts later called the results “highly unlikely”.

Read more: The risk factors for long COVID

Writing in The BMJ, the journal’s associate editor Dr Peter Doshi stressed vaccine trials are not set up to show a jab reduces the risk of hospitalisation, intensive care admission or death.

Another expert called Dr Doshi’s comments “questionable”, but added “a number of the facts are correct”.

Dr Peter Hotez from the Baylor College of Medicine in Houston has said: “Ideally, you want an antiviral vaccine to do two things.

“First, reduce the likelihood you will get severely ill and go to hospital, and two, prevent infection and therefore interrupt disease transmission.”

While Dr Doshi agrees, he has argued “current [coronavirus] trials are not actually set up to prove either”.

Several coronavirus jab candidates are in phase 3 of clinical development. At an advanced stage, significant results mean the vaccine may be considered for approval by the US Food & Drug Administration (FDA) or the European Medicines Agency.

Read more: Nurses describe working amid pandemic

“None of the trials currently underway are designed to detect a reduction in any serious outcome such as hospitalisations, intensive care use or deaths,” wrote Dr Doshi.

“Nor are the vaccines being studied to determine whether they can interrupt transmission of the virus.”

This echoes concerns voiced after the Oxford scientists released their vaccine results.

The research was hailed as “promising”, “encouraging” and “extremely positive”, however, some also pointed out an immune response may not translate to protection against complications when the coronavirus is encountered outside of a laboratory.

Watch: Can you catch coronavirus twice?

Not all clinical trials have released details on the participants they are analysing.

Dr Doshi claims those we know of are evaluating mild coronavirus cases.

Honing in on the pharmaceutical giant Moderna, Dr Doshi noted how the firm’s executives have listed the rate of hospitalisation as a “key secondary endpoint” of its coronavirus vaccine trial.

Dr Tal Zaks, Moderna’s chief medical officer, later told The BMJ the trial “lacks adequate statistical power to assess that endpoint”.

A lack of statistical power typically means the number of participants is too small or the trial’s duration too short to accurately gauge whether a jab influences a particular outcome.

Early research suggests four out of five coronavirus cases are mild.

Even trials with more than 30,000 patients would therefore have relatively few severe disease incidences, noted Dr Doshi.

Read more: Circuit breaker needed to stem UK’s second coronavirus wave

“Hospitalisations and deaths from COVID-19 [the disease caused by the coronavirus] are simply too uncommon in the population being studied for an effective vaccine to demonstrate statistically significant differences in a trial of 30,000 people,” he wrote.

“The same is true regarding whether it can save lives or prevent transmission: the trials are not designed to find out.”

Dr Zaks told The BMJ Moderna’s trial would need to expand vastly in order to collect hospitalisation data.

“Neither of these I think are acceptable in the current public need for knowing expeditiously that a vaccine works,” he said.

“Would I like to know this [vaccine] prevents mortality? Sure, because I believe it does.

“I just don’t think it’s feasible within the timeframe [of the trial]. Too many would die waiting for the results before we ever knew that.”

Dr Doshi pointed out the Moderna trial also fails to assess whether a vaccine prevents a coronavirus-positive individual from passing the infection on.

Dr Zaks added: “Our trial will not demonstrate prevention of transmission because in order to do that you have to swab people twice a week for very long periods, and that becomes operationally untenable.”

Watch: What is long COVID?

Like fellow pharmaceutical giants Pfizer and Johnson and Johnson, Moderna’s trial is assessing whether its vaccine leads to a minimum 30% relative risk reduction in participants developing laboratory-confirmed coronavirus, which is consistent with FDA and international guidance.

Relative risk reduction describes the extent a treatment lowers the odds of developing a bad outcome compared with the control group, who did not have the therapy.

Dr Zaks added flu vaccines tend to protect against severe disease more effectively than mild symptoms.

“To Moderna, it’s the same for COVID-19,” wrote Dr Doshi. “If their vaccine is shown to reduce symptomatic COVID-19, they will feel confident it also protects against serious outcomes.”

The ongoing coronavirus vaccine trials have also tended to enrol relatively healthy participants, despite the elderly and already ill being at greater risk of infection complications.

“If the frail elderly are not enrolled into vaccine trials in sufficient numbers to determine whether there is a reduction in cases in this population, there can be little basis for assuming any benefit against hospitalisation or mortality,” wrote Dr Doshi.

As the pandemic continues to unfold, Dr Doshi stressed now is the time to question how these trials are being carried out.

“The COVID-19 vaccine trials may not have been designed with our input, but it is not too late to have our say and adjust their course,” he wrote.

“With stakes this high, we need all eyes on deck.”

Professor Stephen Evans from the London School of Hygiene & Tropical Medicine agreed with some of Dr Doshi’s points, but added implementing this strict trial criteria may be easier said than done.

“These comments in the BMJ are questionable, although a number of their facts are correct,” said Professor Evans.

“Trials of vaccines to show benefit in death or serious hospital outcomes would need to be absolutely massive (hundreds of thousands) and probably never complete because people would be reluctant to participate if the vaccine had already shown efficacy against infection.

“Unlike other surrogates, infection is a necessary step along the path to death or other serious outcomes, and so is likely to be a good surrogate [to study in a trial].

“Certainly some trials have recruited people aged over 70, though there are probably relatively few [trials], if any, above 85, so [only] some vaccines will know if they work as well in the elderly.

He added: “My view is it would be best in future after new vaccines are introduced following initial trials that show efficacy against infection, for us to then continue with cluster randomised trials, which could further measure their efficacy and safety.

“Or if not, then very good surveillance, especially for safety, but also for efficacy, must be done. Such follow-up is more possible today than it was in the past with good electronic health records.”