This is a great study and (one of?) the first of its kind! A note of caution from a human contributor though: given the slow academic publishing process, the findings should primarily be interpreted as a snapshot of AI capabilities (and the capabilities of researchers using AI) in 2024. (1/4)
Joris Frese
1/ Can AI help researchers check whether published social science results actually reproduce? In our new PNAS paper, we tested this directly in the AI Replication Games: 288 researchers, 103 teams, and real replication packages from quantitative social science.
