Genuinely unsure how this will play out for science.
We have tools to do better science. Make fewer errors. Eliminate menial tasks
But also to produce sloppier science. Make more errors than ever before. And create even more need for AI to help clean up the mess
The case study has already begun.
Seeing how these results are based on 2024 AI models, and based on my experience with agentic AI (such as Claude in VS Code, where it has access to data and the paper, and can iteratively run analyses) it seems clear that in the future AI will assist researchers in reproducibility checks.
Daniel Lakens
1/ Can AI help researchers check whether published social science results actually reproduce? In our new PNAS paper, we tested this directly in the AI Replication Games: 288 researchers, 103 teams, and real replication packages from quantitative social science.
