In a Nature study, Team Scientists @szaszibarnabas.bsky.social, Gideon Nave, Michael Inzlicht and collaborators attempted replications of 274 findings from social and behavioral science journals. 55.1% were successfully replicated.
https://go.nature.com/3Q4YxuW
A large-scale study on the replicability of claims from social and behavioural science journals reports that about half of the results replicate in the same patterns as the original study.
The Iowa Gambling Task is an extreme example of Jingle Fallacy and schmeasurement.
In 100 articles we found 244 different ways of scoring it, 177 were never reused. Correlations between them range -.99 to .99.
At the same time, we show meta-analyses combine these results as if they’re equivalent.
It's ironic to see a discipline care **so much** about unbiasedness (causal inference!) at the level of a single test but then have a research production system and culture that is basically a ferocious bias generation machine. This is not good.
Per protocol analysis strikes again!
Folks, if you randomize but then don‘t analyze some of the people who got randomized (maybe because they didn’t adhere to instructions, maybe because they dropped out), randomization will no longer do all the heavy causal inference lifting.
