An attempt to reduce the ambiguity and increase interpretability of LLMs by shifting away from the current self-attention block in transformers
arxiv.org/abs/2512.19428
We revisit a basic question in sequence modeling: is explicit self-attention actually necessary for strong performance and reasoning? We argue that standard multi-head attention is best seen as a form...
