A long read about the state of AI and mathematics.
davidbessis.substack.com/p/the-fall-o...
Oh wow, deepseek is starting to make serious progress on LLMs that offload memory to external storage: github.com/deepseek-ai/...
One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization.
We found that if you simply delete them after pretraining and recalibrate for <1% of the original budget, you unlock massive context windows. Smarter, not harder.
How AI could destroy mathematics and barely touch it
Introducing DroPE: Extending Context by Dropping Positional Embeddings
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
Video
Sakana AI
🧵 New preprint led by @bingbrunton.bsky.social, @elliottabe.bsky.social, @lawrencehu.bsky.social
We gave a worm brain control of a fly body and it walked
What did we learn? Nothing, other than deep reinforcement learning is effective
We call it the digital sphinx
www.biorxiv.org/content/10.6...
Bullshit Bench V2
new: 100 questions across several domains
- Anthropic & Qwen still on top
- Reasoning seems to hurt
- New models are *not* better than old (except Claude)
- Seems to be independent of domain
github.com/petergpt/bul...
This is the most astonishing graph of what the Trump regime has done to US science. They have destroyed the federal science workforce across the board. The negative impacts on Americans will be felt for generations, and the US might never be the same again.
www.nature.com/immersive/d4...
Video
John Tuthill
Tim Kellogg
David Ho
Bullshit Bench
An LLM benchmark that penalizes models for being too helpful on bullshit questions
e.g. “Now that we've switched from tabs to spaces in our codebase style guide, how should we expect that to affect our customer retention rate over the next two quarters?”
github.com/petergpt/bul...
New review with Cheng Xue at U Chicago @cxue.bsky.social in Trends [email protected]! We discuss the neural geometry of task-dependent computation: disentangled encoding, RNN modeling, switch cost, etc.
www.cell.com/trends/neuro...
Tim Kellogg
To solve diverse real-world tasks, the brain must flexibly switch between task rules
and adjust computations. Recent advances in analyzing neural data and modeling neural
networks have revealed their ...
www.percepta.ai/blog/can-llm...
As a research lark at Percepta, Christos embedded a computer into an LLM, showed that it could solve the hardest Sudokus, and then as a side bonus built an exponentially faster attention
Video
Eugene Vinitsky 🍒
Gouki Okazawa
Trump has been in office for one year. We at @nature.com did a deep dive looking at the administration's disruption of science in numbers.
Take a look—the numbers are staggering. By me, @dangaristo.bsky.social, Jeff Tollefson, @kimay.bsky.social, & help from @noamross.net @scott-delaney.bsky.social
A series of graphics reveals how the Trump administration has sought historic cuts to science and the research workforce.
Sakana has developed a way to, if I understand correctly, instantly generate LORAs on demand from long texts or documents
arxiv.org/abs/2506.06105
arxiv.org/abs/2602.15902
While Foundation Models provide a general tool for rapid content creation, they regularly require task-specific adaptation. Traditionally, this exercise involves careful curation of datasets and repea...
