she/her
incoming @ Blender Lab, UIUC
prev @ McGillNLP & Mila
occasionally live on ckut 90.3 fm :-)
adadtur.github.io
Ada
Loading...
Our paper and code, as well as our full dataset release (with model and human judgments!) are linked below! An immense thank you to all my amazing co-authors!
📄Paper: arxiv.org/abs/2606.05409
💻Code: github.com/AdaDTur/NVRD
💾Data: huggingface.co/datasets/ada...
🌐Demo: adadtur.github.io/nvrd-demo/
Super excited to finally announce my latest research “Would you still call this Dax? Novel Visual References in VLMs and Humans”!
We studied how vision-language models (VLMs) adopt new visual concepts and map them to language compared to humans, and found that…
Ada
Here are some more examples to illustrate the cool phenomena we study, spanning a) different levels of novelty (known, composed, fully novel) and b) different perturbations on the objects.
1) Known object → Style Degradation,
2) Novel object → Texture Shift,
3) Composed object → Part Addition.
1. Humans and models broadly agree that shape/structural perturbations (opposed to e.g. color or texture) change concept identity the most.
2. Models, however, considerably over-generalize relative to humans, accepting mappings that humans opt to reject.
My first last-author paper is out!
If you saw this dog below and someone showed you the second image, would you consider them the same word/concept?
(more examples in Ada's thread)
We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision
Ada
Ada
Ada
Building a VLM can be surprisingly simple: You keep both the LLM and vision encoder frozen, you just train a small MLP that projects into the LLM embedding space as prefixes. That’s it 😮
But how and why does that work? How do visual tokens relate to language, i.e. do they have interpretable NNs?
Super cool project that I really enjoyed being part of! tl;dr - when a human or model encounters new visual stimuli, how closely is it mapped to other, previously encountered concepts? (Come for weird dog-monster, stay for the science 🙂 )
🚨New Paper!🚨 How do reasoning LLMs handle inferences that have no deterministic answer? We find that they diverge from humans in some significant ways, and fail to reflect human uncertainty… 🧵(1/10)
Takeaway: reasoning LLMs are getting better and better on math and code—deterministic reasoning tasks. But we should also evaluate them on open-ended, inherently uncertain everyday reasoning! (9/10)
