Survey-style tests developed for humans may not predict how LLMs actually behave.
Our #EACL2026 paper shows they can even be misleading when measuring racism and sexism!
Check out the paper 👇🏼
👋🏼 I'm at #EMNLP2025 presenting "The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for LLMs"
🕑 Thu. Nov 6, 12:30 - 13:30
📍 Findings Session 2, Hall C3
🚨 TADA Speaker Series Spring 2026 schedule is here! 🚨
We've assembled a fantastic lineup of researchers exploring the future of survey research in the age of LLMs.
Mar 18 - May 27, online at 17:00 CEST. Join us!
More info & signup: tada.cool
We are delighted to welcome @marlutz.bsky.social to our lab over the next few months! 🎉
She'll work on the representation of different demographic groups in LLMs.
#NLProc
Marlene Lutz
Very honored to be one out of seven outstanding papers at this years' EMNLP :)
Huge thanks to my amazing collaborators @fatemehc.bsky.social @anamarasovic.bsky.social @boknilev.bsky.social , this would not have been possible without them!
Are you using survey-style questionnaires designed for humans to measure characteristics of LLMs?
In our #EACL2026 paper, we evaluate both the reliability and validity of such tests and found that their scores do not reflect real-world model behavior. In fact, they can be deceptive!
🧵1/3
Marlene Lutz
🚨New paper alert🚨
🤔 Ever wondered how the way you write a persona prompt affects how well an LLM simulates people?
In our #EMNLP2025 paper, we find that using interview-style persona prompts makes LLM social simulations less biased and more aligned with human opinions.
🧵1/7
