Other cool findings:
1. We prove that (RSA)^2 is more expressive than QUD-based RSA.
2. Naively applying RSA to LLMs leads to probability π΄π±π³π¦π’π₯πͺπ―π¨, not π―π’π³π³π°πΈπͺπ―π¨! Are there better ways to use RSA with LLMs?
3. What if we don't know the rhetorical strategies? We develop a clustering algorithm too!
Cesare
What do systematic hallucinations in LLMs tell us about their generalization abilities?
Come to our poster at #ACL2025 on July 29th at 4 PM in Level 0, Halls X4/X5. Would love to chat about interpretability, hallucinations, and reasoning :)
@mcgill-nlp.bsky.social @mila-quebec.bsky.social
Thanks to collaborators David Austin, Pablo Piantanida and Jackie Cheung. We also received some amazing feedback from the @mila-quebec.bsky.social @mcgill-nlp.bsky.social community! And thanks to Jennifer Hu, Justine Kao and Polina Tsvilodub for sharing their datasets.
We test (RSA)^2 on two existing figurative language datasets: hyperbolic number expressions (e.g. βThis kettle costs 1000$β) and ironic utterances about the weather (e.g. βThe weather is amazingβ during a Montreal blizzard). We obtain meaning distributions which are compatible with those of humans!
A new paper accepted in @colmweb.org COLM 2025! I led a group of 3 brilliant students to dive deep into the problem of discrimination in language models. We discovered that models that take racist decisions donβt always have biased thoughts!
What about LLMs? We integrate LLMs within (RSA)^2 and test them on a new dataset, PragMega+. We show that LLMs augmented with (RSA)^2 produce probability distributions which are more aligned with human expectations.
How can we use models of cognition to help LLMs interpret figurative language (irony, hyperbole) in a more human-like manner? Come to our #ACL2025NLP poster on Wednesday at 11AM (exhibit hall - exact location TBA) to find out! @mcgill-nlp.bsky.social @mila-quebec.bsky.social @aclmeeting.bsky.social
Our new paper in #PNAS (bit.ly/4fcWfma) presents a surprising findingβwhen words change meaning, older speakers rapidly adopt the new usage; inter-generational differences are often minor.
w/ Michelle Yang, βͺ@sivareddyg.bsky.socialβ¬ , @msonderegger.bsky.socialβ¬ and @dallascard.bsky.socialβ¬π(1/12)
