Most COVID-19 RCT Abstracts Posted to medRxiv Published After 1 Year

One in 5 preprint abstracts of COVID-19 randomized clinical trials (RCTs) found on medRxivremained unpublished for at least a year after they were posted to the server, new research published in JAMA shows.

According to this study’s authors, the unpublished preprints were less complete and more highly spun than those that made it to publication. Spin refers to reporting practices used in research that distort interpretation of results. This can mislead readers and make them view results in a more favorable light.

Although preprints, or research that is not peer reviewed, are becoming increasingly important in medical communication, in rapidly evolving areas like COVID-19, preprints “can influence practice and potentially cause harm,” the researchers wrote.

To better understand timeliness, completeness, and spin in abstracts of all RCTs related to COVID-19, the investigators assessed those posted to medRxivduring the first 2 years of the pandemic.

All included abstracts were posted between January 2020 and December 2021. The researchers then used title and reverse author searches on June 1, 2022, and November 15, 2022, to identify any published counterparts. They also analyzed blinded abstracts in a random order and scored each one on measures of completeness. In addition, “a random sample of 75 of the 161 preprint abstracts and their companion published abstracts, and the 75 preprint abstracts without companion publications, were presented in random order to 1 of 2 reviewers, who scored whether spin was present.”

Instances of spin included statistically significant secondary outcomes that were emphasized in a negative study, along with claims that extended beyond the target population of the trial.

Of the 236 preprints, 77.1% had published counterparts as of November 2022, and of the 182 published counterparts, 92.0% were published within a year of posting to medRxiv. The median (IQR) time to publication was measured at 134 (86-222) days.

A total of 161 published counterparts (68.2%), were included in the analysis. Across 12 of 17 categories, the percentage of abstracts fulfilling each completeness metric was lowest in the unpublished preprint abstracts.

Data also showed:

• For all 4 categories of spin, the percentage with spin was highest in unpublished preprint abstracts (eg, 65.3% with spin in the conclusion section vs 41.3% in published preprint abstracts and 34.7% in published abstracts)
• Unpublished preprint abstracts had a higher median global spin score of 3 (1-6) compared with a score of 1 (0-3) for published preprints and a score of 1 (0-2) for their companion publications
• The blinded side-by-side comparison of preprint abstracts and their companion publications similarly favored the published version for both completeness and lack of spin

“Adoption of COVID-19 treatment protocols based on erroneous preprints suggests potential problems associated with less complete, more highly spun preprint abstracts,” the researchers wrote. Results also revealed any improvements in completeness and spin that come with publication were modest, a finding that indicates the peer-review process and journal editing had little effect in improving reporting quality.

Furthermore, “the data demonstrate that a gap exists between preprint abstracts eventually published and those that remain unpublished in terms of completeness and spin, potentially indicating selection for desirable qualities in the journal review process,” the authors said.

The use of imperfect proxy measures for completeness and spin, and the focus on a simple topic (COVID-19) mark limitations to the study findings’ generalizability.

Reference

Spungen H, Burton J, Schenkel S, and Schriger DL. Completeness and spin of medRxiv preprint and associate published abstracts of COVID-19 randomized clinical trials. JAMA. 2023;329(15):1310-1312.