With the large influx of submissions and a faster pace of research, reproducibility is more important than ever.
With this reproducibility challenge, we want to put the focus on best practices wrt. baselines🧱, ablations🌈, eval🔎 and generalizability🗺️ of interpretability!
We are delighted to welcome @marlutz.bsky.social to our lab over the next few months! 🎉
She'll work on the representation of different demographic groups in LLMs.
#NLProc
📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge!
A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.
Martin Tutek
📣 Announcing the BlackboxNLP 2026 Reproducibility Challenge!
A new track dedicated to rigorous robustness checks of NLP interpretability work - stress-testing baselines, ablations, generalizability, and evaluation.
📢 The workshop on Insights from negative results will be back at EMNLP'26!
Your most-insightful failures can be submitted in 4 pages by June 25. It's also possible to commit short papers reviewed through ARR.
insights-workshop.github.io/2026/cfp
Ever used a top-ranked LLM that just... felt wrong for you?
You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation?
New paper: "From Feelings to Metrics" 🧵
❗The full paper submission deadline for COLM is ~14 hours from now (11:59pm AOE)!
Please submit your final PDFs on the same page where you uploaded your abstracts. And please use the provided LaTeX templates; do not handwrite your manuscript like this llama is!
Good luck!
✨ it's coming ✨
NEMI 2026 will be lit. It will also be the new BU interp supergroup's debut ball. Come meet us!
MilaNLP Lab
How can generative AI better support human creativity, without limiting it? If you have thoughts, we invite submissions to our ICML workshop on Generative AI, Creativity, and Human-AI Co-Creation
📍 July 2026, Seoul
📄 Submit by: April 24 (AOE)
🔗 Submission link: openreview.net/group?id=ICM...
FYI #ACL2026 has an unusual registration system this year, and probably a lot of people who want to attend will not be able to.
Spots are limited to 3.5k people, and only presenting authors can register during the first phase. Then, *if* there are spots left, others can try to register.
