7/8
What this means: Current LLMs may lack architectural foundations for genuine behavioral consistency.
Training on diverse text creates models simulating myriad personalities in superposition. Post-training may be a brittle stabilization attempt.
4/8
Finding 2: Chain-of-thought reasoning INCREASES variability while DECREASING perplexity
Models become more confident yet less consistent. Explanation paradoxically undermines reliability.
6/8
Finding 4: Misaligned personas show increased variability
Antisocial/schizophrenia persona prompts increase inconsistency vs baseline. Behavioral inconsistency itself may serve as a misalignment signal.
5/8
Finding 3: Conversation history cuts both ways
Amplifies instability in smaller models (<50B) but reduces it in larger ones. Multi-turn interactions can progressively degrade behavioral predictability.
8/8
📄 Paper: arxiv.org/abs/2508.04826
💻 Code: github.com/tosatot/PERSIST
Thanks to co-authors: @saskiahelbling.bsky.social, @yjmantilla.bsky.social , Mahmood Hegazy, Alberto Tosato, David John Lemay, Irina Rish, @introspection.bsky.social
2/8
We tested 25 open-source models (1B-685B params) across 2M+ responses to personality questionnaires (BFI, Short Dark Triad), systematically varying question order, paraphrasing, personas, and reasoning modes. ⤵️
1/8
Do LLMs have stable personalities? We ran 2 million tests. (Spoiler: no.) 🧵
Paper accepted at AAAI 2026 - Alignment Track
Safe deployment requires behavioral consistency. We found persistent instability across scales, reasoning modes, and personas. ⤵️
