Mechanistic understanding of systematic failures in language models is something more research should strive for IMO. This is really interesting work in that vein by @ziling-cheng.bsky.social, highly recommend you check it out.
Andrei Mircea
Do LLMs hallucinate randomly? Not quite.
Our #ACL2025 (Main) paper shows that hallucinations under irrelevant contexts follow a systematic failure mode ā revealing how LLMs generalize using abstract classes + context cues, albeit unreliably.
š Paper: arxiv.org/abs/2505.22630 1/n
