While effective for chess♟️, Elo ratings struggle with LLM evaluation due to volatility and transitivity issues.
New post in collaboration with AI Singapore explores why Elo falls short for AI leaderboards and how we can do better.
Introducing ✨Tiny Aya✨, a family of massively multilingual small language models built to run where people actually are.
Tiny Aya delivers strong multilingual performance in 70+ global languages in a 3.35B parameter model, efficient enough to run locally, even on a phone.
We’re not your average lab. We’re a hybrid research environment dedicated to revolutionizing the ML space.
And we’re hiring a Senior Research Scientist to co-create with us.
If you believe in research as a shared, global effort — this is your chance.
What if the way we verify synthetic code is limiting model performance?
In our latest work we uncover the Verification Ceiling Problem: strict “all tests must pass” rules throw away useful data, while weak tests let errors through.
🌱Very proud of our team's latest release 😊 meet Tiny Aya, a massively multilingual model with 3.35B parameters.
Tech report here: github.com/Cohere-Labs/...
Video
github.com
At its core, @cohereforai.bsky.social is about advancing open science, exploring fundamental research questions, and sharing knowledge broadly
always guided by the pleasure of finding things out.
Excited for what’s next!
I'm excited to share that I'll be stepping into the role of Head of @cohereforai.bsky.social. It's an honor and a responsibility to lead such an extraordinary group of researchers pushing the boundaries of AI research.
Very grateful to @sarahooker.bsky.social for shaping the lab into what it is today, and looking forward to working with Joelle whose guidance will be invaluable as we write our next chapter. Also big thanks to @nickfrosst.bsky.social
and @aidangomez.bsky.social Ivan for their support and trust.
