The risks of relying on peer review

A retraction that illustrates, in one observer's words, how journals are “fundamentally ill-equipped to deal with the pandemic.”

When the journal BMC Infectious Diseases published a study by a professor at Michigan State University, claiming that there could be as many as 278,000 deaths caused by COVID vaccines, it had gone through six months of outside review from two different people. You can read even their comments (and the author’s responses), which most journals don’t post. And in the anti-vaccine circles that fixated on that incredibly high number (for context, the CDC has identified just nine deaths caused by vaccines), many repeatedly pointed out that it had been peer-reviewed, that traditional marker of credibility.

But once the study was out in the world, it got a very different kind of peer review — from researchers who immediately saw flaws that seemed to disqualify the whole premise. One medical expert, noting that it was partially based on a survey funded by an anti-vaccine advocate and that its author had written multiple blog posts skeptical of vaccines, called it “antivax propaganda disguised as a survey.”

Now — after two months of being seen and shared millions of times — this peer-reviewed study is in the process of being retracted, as I reported exclusively for The Chronicle of Higher Education (free for anyone to read with an account). No word yet on when it’ll happen, and the author says he stands by his research. Here’s a bit about the study:

In Skidmore’s study, a hired company administered an online survey to an anonymous group of 2,800 people representative of the U.S. population in late 2021, and asked them about their experiences during the pandemic. The study found that those who knew someone who’d had a health problem from Covid were more likely to be vaccinated, while those who knew someone who’d experienced a health problem after being vaccinated were less likely to be vaccinated themselves.

Skidmore then went further and took the number of vaccine-caused deaths that the respondents reported knowing about — 57, according to the study — and used them to estimate the total number of people who had died for the same reason. To flesh out the estimate, he counted deaths reported to a federal database called the Vaccine Adverse Event Reporting System, known as VAERS, and arrived at the figure 278,000.

This methodology for calculating vaccine-induced deaths was rife with problems, observers noted, chiefly that Skidmore did not try to verify whether anyone counted in the death toll actually had been vaccinated, had died, or had died because of the vaccine.

When participants were asked to write about the health problem of the vaccinated person they knew “best,” answers like “death from a heart attack after vaccination by a few weeks,” “they passed away from Covid,” and “cancer retu[r]ned in his body spread all over” all counted as vaccine-caused deaths, according to study materials posted online. In addition, the VAERS database is open to reports from anyone and does not verify any of them. (Skidmore also did not cite the official VAERS website but an anonymously run WordPress site that posts about the need to “defund the CDC.”)

You can read more about this, including why the author objects to the retraction and the journal’s explanation, here.