Machine Learning ELLIS PhD at Johannes Kepler University Linz and University of Oxford
Lukas Aichberger
Loading...
We unlocked the working memory of LLMs 💥
Reasoning in Memory (RiM) replaces autoregressive "thinking out loud" with fixed memory blocks that form a task-specific workspace for latent reasoning.
The key idea is simple: reasoning should happen inside the LLM, not in its output!
Hot take: I think we just demonstrated the first AI agent computer worm 🤔
When an agent sees a trigger image it's instructed to execute malicious code and then share the image on social media to trigger other users' agents
This is a chance to talk about agent security 👇
Lukas Aichberger
Yarin
4/ Results
RiM consistently improves over direct-answer SFT, outperforms Coconut, and becomes competitive with CoT-style reasoning across model families and scales on math reasoning tasks.
Crucially, it preserves direct-answer inference speed 🚀
2/ Intuition
Established reasoning methods make LLMs externalize their thoughts.
Chain-of-thought generates text token by token.
Coconut replaces text with continuous thoughts, but still generates them step by step.
RiM instead moves reasoning into the LLM's working memory 🧠
5/ Takeaway
LLMs do not always need to externalize their thoughts.
They can learn to reason in working memory instead, decoupling intermediate computation from autoregressive generation 💡
Full paper:
arxiv.org/abs/2605.30343
Huge thanks to @hochreitersepp.bsky.social for the guidance!
3/ Method
Simply adding memory blocks to the context is not enough. The hard part is making the LLM actually use them.
RiM solves this with an efficient two-stage training curriculum that teaches the model to route reasoning through the memory-block representations 🧩
Lukas Aichberger
To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this couples reasoning to auto...
⚠️ Beware: Your AI assistant could be hijacked just by encountering a malicious image online!
Our latest research exposes critical security risks in AI assistants. An attacker can hijack them by simply posting an image on social media and waiting for it to be captured. [1/6] 🧵
