For years, we've known that running a standard t-test on cross-validation folds violates sample independence. We wanted to see how widespread this issue actually is.
The result? 97% of the studies used an invalid statistical test. 🧵👇
Our Nature paper on the hashtag#scaling hashtag#behavior and economics of hashtag#machine hashtag#learning predictions in high-dimensional brain scans is out !
Congrats to the whole team.
www.nature.com/articles/s41...
What a fantastic effort. Truly inspiring to see brilliant people dig deeply into these meta scientific issues.
This is the best time to be doing neuroimaging.
Just incredible results from a massive effort— moves the field forward. Bravo!!!
@nichols.bsky.social collaborated with researchers at the National University of Singapore on a recent study published in @nature.com on how longer duration fMRI brain scans reduce costs and improve prediction accuracy for AI models. Read more about the study below 👇
Just dropped in @natcomms.nature.com: we show that re-engaging a thalamic–ventral tegmental circuit with deep brain stimulation can reignite consciousness in patients with severe brain injury. Work led by Aaron Warren, with @andreashorn.org @foxmdphd.bsky.social @ others! tinyurl.com/4kz8j89b
A super important and well designed study. Curious if those who took such interest in the original "BWAS needs impossibly huge n" will pay any attention to it
I'm so proud to see this great paper finally published in @nature.com!
Shaoshi Zhang
This new Yeo Lab tool should immediately and permanently replace sample-size-only power calculations for functional MRI.
www.nature.com/articles/s41...
danilobzdok
Really nice study, and extends some of the ideas developed in this paper pubmed.ncbi.nlm.nih.gov/32673043/
Although the number of participants is important for phenotypic prediction accuracy in brain-wide association studies using functional MRI, scanning for at least 30 min offers the greatest cost effect...
www.nature.com
Mary Elizabeth Sutherland
Jim Thompson
Nico Dosenbach
1/11 Excited to share our @Naturestudy led by @leonooi.bsky.social @csabaorban.bsky.social @shaoshiz.bsky.social
AI performance is known to scale with logarithm of sample size (Kaplan 2020), but in many domains, sample size can be # participants or # measurements...
doi.org/10.1038/s415...
Laurence Hunt
1/11 Excited to share our @Naturestudy led by @leonooi.bsky.social @csabaorban.bsky.social @shaoshiz.bsky.social
AI performance is known to scale with logarithm of sample size (Kaplan 2020), but in many domains, sample size can be # participants or # measurements...
doi.org/10.1038/s415...
In people with severe brain injuries, stimulation restored consciousness by engaging a deep brain circuit for wakefulness—revealing a target that may also guide treatment in stroke and epilepsy.
In a meta-analysis of 210 biomedical AI studies that statistically compared models under cross-validation, 97% used invalid statistical tests.
Here's our new preprint doi.org/10.64898/202... led by @tianchu.bsky.social @hetuli.bsky.social @shaoshiz.bsky.social @nichols.bsky.social 1/N
8/ In contrast to standard power calculations, our results suggest that jointly optimizing sample size and scan time can boost prediction accuracy while cutting costs. For more complex study design, you can check out our calculator: thomasyeolab.github.io/OptimalScanT...
1/11 Excited to share our @Naturestudy led by @leonooi.bsky.social @csabaorban.bsky.social @shaoshiz.bsky.social
AI performance is known to scale with logarithm of sample size (Kaplan 2020), but in many domains, sample size can be # participants or # measurements...
doi.org/10.1038/s415...
1/11 Excited to share our @Naturestudy led by @leonooi.bsky.social @csabaorban.bsky.social @shaoshiz.bsky.social
AI performance is known to scale with logarithm of sample size (Kaplan 2020), but in many domains, sample size can be # participants or # measurements...
doi.org/10.1038/s415...
Thomas Yeo
Thomas Yeo
Thomas Yeo
Thomas Yeo
Thomas Yeo
Thomas Yeo
1/11 Excited to share our @Naturestudy led by @leonooi.bsky.social @csabaorban.bsky.social @shaoshiz.bsky.social
AI performance is known to scale with logarithm of sample size (Kaplan 2020), but in many domains, sample size can be # participants or # measurements...
doi.org/10.1038/s415...
9/ Check out the actual study for many more analyses, e.g., phenotypic variation, scan parameters, signal to noise ratio, etc! doi.org/10.1038/s415...
Thank you to the editor @meharpist.bsky.social and anonymous reviewers for the many helpful suggestions, which greatly improved the study.
Thomas Yeo
Thomas Yeo
doi.org
Although the number of participants is important for phenotypic prediction accuracy in brain-wide association studies using functional MRI, scanning for at least 30 min offers the greatest cost effect...