- All systems will be human evaluated (no downsampling using automatic metrics) and we are preparing a new contrastive humeval protocol
- LLM benchmarking focussed on open-weight models
- Abstract submission has been replaced with a model card poll
All details are at www2.statmt.org/wmt26/transl...
Multimodal context - same as last year, for spoken domain, we provide original video, while for other domains, image can be provided with additional context (such as screenshots or infographics). Purely text-to-text systems can still participate as in the past
Instruction following context in prompts. Systems may disregard them but failing to follow instructions is considered a translation error. You can expect the following phenomena: formal/informal voice, glossaries, structured translation (JSON, HTML, ...), style and expressions (e.g. "yuhuuu", "tbh")
You may participate in up to 20 language pairs out of which we host 9 new ones:
Czech to Vietnamese
Chinese to Japanese (direction reversed)
EN to Armenian
EN to Belarusian
EN to Indonesian
EN to Kazakh
EN to Ladin
EN to Ligurian
EN to Northern Sámi
Ready for our poster today at #COLM2025!
💭This paper has had an interesting journey, come find out and discuss with us! @swetaagrawal.bsky.social @kocmitom.bsky.social
Side note: being a parent in research does have its perks, poster transportation solved ✅