Associate Professor in Economics. University Paris-Saclay. #SocialMedia #AI #Misinformation http://www.thomas-renault.com
Thomas Renault
Community-based fact-checking works
Community notes reducethe subsequent spread of misleading posts by 61% and increase the odds that users delete their false posts by 94%
But they appear too late to stop the viral stage of the diffusion making the effect modest.
www.nature.com/articles/s41...
🚨 A few months ago, a paper published in Science reported that “LLM adoption is associated with a large increase in researchers’ scientific output.”
In a comment released today, we show that the reported effects are driven by a methodological issue in the empirical design. arxiv.org/abs/2605.17979
Similar “LLM effects” can be reproduced when replacing “LLM adoption” (detected from abstract content) with random keywords, random treatment assignment, or even publication periods predating the release of ChatGPT.
We discuss the mechanism in detail in the paper.
📄 Paper: arxiv.org/abs/2605.17979
💻 Code and replication materials: github.com/trenault/llm...
Comments and feedback are very welcome.
With @abergeaud.bsky.social and Clément Bosquet
The core issue is that the treatment variable (being classified as an LLM adopter) is mechanically related to researchers’ publication volume: the more papers an author publishes, the more likely they are to be classified as treated.
Hello -- I'm looking for active X Community Notes contributors who want to talk to me for a piece on @indicator.media. Happy to use their pseudonym in the story. Please share with your networks!
Kusumegi et al. (2025) study whether researchers' preprint output rises after adopting large language models (LLMs), dating adoption as the first month in which at least one submitted abstract exceeds...
