New paper on LLMs and research methodology: Justify your prompts! direct.mit.edu/coli/article...
#nlproc
@ai2.bsky.social I'm confused by the OLMoTrace paper. The paper says that the trace feature is useful for "Fact checking" (see image), but the Limitations section notes: "The
retrieved documents should not be interpreted as [...] supporting evidence or citations for the LM output."
A contradiction?
Noce paper on how well benchmarks cover what people do at work
arxiv.org/abs/2603.01203
Quote
"these observations suggest that agent benchmarking effort is driven less by alignment with real-world employment structure or economic value, and more by methodological convenience."
Article is in Dutch, but maybe understandable via Google Translate.
TL;DR: AI poetry is used in 3 different ways.
1. To study progress in creative computing;
2. To explore new literary forms and to see how people interpret those;
3. To (deceptively) market AI with appeals to its 'humanity.'
It is somewhat disappointing that the flagship journals of NLP (CL and TACL) only publish PDFs. It would be great if we could somehow move to include HTML publications as well.
More generally, the accessibility of documents in the ACL Anthology is limited. We're building massive technical debt.
📣 Announcing the release of the 🕊️ Annotated Encyclical 🕊️ from the ethics & society folks at @hf.co. Includes citations to relevant academic work.
Very much a WIP. Please add work we haven't added yet!
huggingface.co/spaces/socie...
Not the best idea ever to announce this during the Bluesky blackout..
In a post-CHI blog post, I talk about what I believe technology like the Anthropic Interviewer will mean for qualitative research.
doomscrollingbabel.manoel.xyz/p/qualitativ...
The International conference on Natural Language Generation (INLG) just issued the first Call For Papers: 2026.inlgmeeting.org/calls.html
#nlproc
Our analysis is based on this article by @lakens.bsky.social: online.ucpress.edu/collabra/art...
The original article discusses arguments that are used in empirical research to justify the choice of sample size. We translated those justification strategies to the domain of LLMs.