Here's bonus slides on cross-validation tests, separate from our preprint. Covering:
1. paired (sign-flip) permutation test
2. label-swap permutation test
3. sample-level vs fold-averaged stats
4. a common misapplication of the corrected t-test
5. three bootstrap variants 1/N
Thomas Yeo
In a meta-analysis of 210 biomedical AI studies that statistically compared models under cross-validation, 97% used invalid statistical tests.
Here's our new preprint doi.org/10.64898/202... led by @tianchu.bsky.social @hetuli.bsky.social @shaoshiz.bsky.social @nichols.bsky.social 1/N
