A new initiative from the watchdogs behind Retraction Watch is taking aim at flawed or faked medical science research to the tune of nearly $1 million.
The Center for Scientific Integrity just launched the Medical Evidence Project, a two-year effort to identify published medical research with a negative effect on health guidelines—and to make sure people actually hear about it.
Equipped with a $900,000 grant from Open Philanthropy and a core team of up to five investigators, the project will use forensic metascience tools to identify issues in scientific articles, and report its findings via Retraction Watch, the foremost site for scientific watchdogging.
“We originally set up the Center for Scientific Integrity as a home for Retraction Watch, but we always hoped we would be able to do more in the research accountability space,” said Ivan Oransky, executive director of the Center and co-founder of Retraction Watch, in a post announcing the grant. “The Medical Evidence Project allows us to support critical analysis and disseminate the findings.”
According to Nature, these flawed and falsified documents are vexing because they skew meta-analyses—reviews that combine the findings from multiple studies to draw more statistically robust conclusions. If one or two bunk studies make it into a meta-analysis, they can tip the scales on health policy.
In 2009, to name one case, a European guideline recommended the use of beta-blockers during non-cardiac surgery, based on turn-of-the-millennium research that was later called into question. Years later, an independent review suggested that the guidance may have contributed to 10,000 deaths per year in the UK.
Led by James Heathers, a science integrity consultant, the team’s plan is to build software tools, chase down leads from anonymous whistleblowers, and pay peer reviewers to check their work. They’re aiming to identify at least 10 flawed meta-analyses a year.
The team is picking its moment wisely. As Gizmodo previously reported, AI-generated junk science is flooding the academic digital ecosystem, showing up in everything from conference proceedings to peer-reviewed journals. A study published in Harvard Kennedy School’s Misinformation Review found that two-thirds of sampled papers retrieved through Google Scholar contained signs of GPT-generated text—some even in mainstream scientific outlets. About 14.5% of those bogus studies focused on health.
That’s particularly alarming because Google Scholar doesn’t distinguish between peer-reviewed studies and preprints, student papers, or other less-rigorous work. And once this kind of bycatch gets pulled into meta-analyses or cited by clinicians, it’s hard to untangle the consequences. “If we cannot trust that the research we read is genuine,” one researcher told Gizmodo, “we risk making decisions based on incorrect information.”
We’ve already seen how nonsense can slip through. In 2021, Springer Nature retracted over 40 papers from its Arabian Journal of Geosciences—studies so incoherent they read like AI-generated Mad Libs. Just last year, the publisher Frontiers had to pull a paper featuring anatomically impossible AI-generated images of rat genitals.
We’ve entered the era of digital fossils, in which AI models trained on web-scraped data are beginning to preserve and propagate nonsense phrases as if they were real scientific terms. For example, earlier this year a group of researchers found a garbled set of words from a 1959 biology paper embedded in the outputs of large language models including OpenAI’s GPT-4o.
In that climate, the Medical Evidence Project’s goal feels more like triage than cleanup. The team is dealing with a deluge of flawed information, hiding in plain sight, and plenty of which can have very real health consequences if taken at face value.
Read the full article here
