Translating MMLU is great, but global users of multilingual #LLMs don't care all that much about an LLM's understanding of US Law!
Our new #NLProc work centers multilingual #LLM evaluations toward regional knowledge in 44 languages.
Antoine Bosselut
🚀 Introducing INCLUDE 🌍: A multilingual LLM evaluation benchmark spanning 44 languages!
Contains *newly-collected* data, prioritizing *regional knowledge*.
Setting the stage for truly global AI evaluation.
Ready to see how your model measures up?
#AI #Multilingual #LLM #NLProc
