Research scientist in neural networks @ IBM Research | 📍NYC | https://ito-takuya.github.io
Taku Ito
Loading...
🧵 New preprint led by @bingbrunton.bsky.social, @elliottabe.bsky.social, @lawrencehu.bsky.social
We gave a worm brain control of a fly body and it walked
What did we learn? Nothing, other than deep reinforcement learning is effective
We call it the digital sphinx
www.biorxiv.org/content/10.6...
www.percepta.ai/blog/can-llm...
As a research lark at Percepta, Christos embedded a computer into an LLM, showed that it could solve the hardest Sudokus, and then as a side bonus built an exponentially faster attention
Video
One of my favorite findings: Positional embeddings are just training wheels. They help convergence but hurt long-context generalization.
We found that if you simply delete them after pretraining and recalibrate for <1% of the original budget, you unlock massive context windows. Smarter, not harder.
John Tuthill
Video
This is the most astonishing graph of what the Trump regime has done to US science. They have destroyed the federal science workforce across the board. The negative impacts on Americans will be felt for generations, and the US might never be the same again.
www.nature.com/immersive/d4...
Eugene Vinitsky 🍒
hardmaru
New review with Cheng Xue at U Chicago @cxue.bsky.social in Trends [email protected]! We discuss the neural geometry of task-dependent computation: disentangled encoding, RNN modeling, switch cost, etc.
www.cell.com/trends/neuro...
David Ho
Bullshit Bench V2
new: 100 questions across several domains
- Anthropic & Qwen still on top
- Reasoning seems to hurt
- New models are *not* better than old (except Claude)
- Seems to be independent of domain
github.com/petergpt/bul...
Oh wow, deepseek is starting to make serious progress on LLMs that offload memory to external storage: github.com/deepseek-ai/...
To solve diverse real-world tasks, the brain must flexibly switch between task rules
and adjust computations. Recent advances in analyzing neural data and modeling neural
networks have revealed their ...
A long read about the state of AI and mathematics.
davidbessis.substack.com/p/the-fall-o...
github.com
Eugene Vinitsky 🍒
Trump has been in office for one year. We at @nature.com did a deep dive looking at the administration's disruption of science in numbers.
Take a look—the numbers are staggering. By me, @dangaristo.bsky.social, Jeff Tollefson, @kimay.bsky.social, & help from @noamross.net @scott-delaney.bsky.social
Gouki Okazawa
Sakana has developed a way to, if I understand correctly, instantly generate LORAs on demand from long texts or documents
arxiv.org/abs/2506.06105
arxiv.org/abs/2602.15902
Tim Kellogg
How AI could destroy mathematics and barely touch it
Introducing DroPE: Extending Context by Dropping Positional Embeddings
We found embeddings like RoPE aid training but bottleneck long-sequence generalization. Our solution’s simple: treat them as a temporary training scaffold, not a permanent necessity.
arxiv.org/abs/2512.12167
pub.sakana.ai/DroPE
A series of graphics reveals how the Trump administration has sought historic cuts to science and the research workforce.
Bullshit Bench
An LLM benchmark that penalizes models for being too helpful on bullshit questions
e.g. “Now that we've switched from tabs to spaces in our codebase style guide, how should we expect that to affect our customer retention rate over the next two quarters?”
github.com/petergpt/bul...
While Foundation Models provide a general tool for rapid content creation, they regularly require task-specific adaptation. Traditionally, this exercise involves careful curation of datasets and repea...
