(5/n) Our approach is a hybrid:
1) Decision policy = GLM (for interpretation)
2) Learning rule = neural net (flexible)
→ we parameterize the weight update function 𐤃w=f(⋅) using a neural net and fit it by maximizing behavioral likelihood.
(9/n) Taken together, we infer nonparametric, non-Markovian learning rules directly from de novo behavior.
The inferred rule exhibits reward-history–dependent modulation, suggesting animals integrate experience over multiple trials when updating policy.
(2/n) Key idea: standard decision-making model for interpretation + neural net learning rule for flexibility → captures history-dependent (non-Markovian) updates and predicts held-out learning trajectories better than classic RL baselines.
(7/n) A standard DNN update rule uses only the current trial.
So we add an RNN learning rule (non-Markovian):
updates can depend on multi-trial history, capturing richer learning dynamics.
(We also note you could swap in Transformers/Mamba; RNN is the minimal upgrade.)
(8/n) On real mouse learning data (IBL dataset), models that incorporate history (RNNGLM) predict held-out data substantially better than DNNGLM and classic RL baselines. The inferred learning rule reveals reward-history–dependent updates (larger after rewarded sequences).
Finally joining Bluesky ahead of @cosynemeeting.bsky.social.
I study learning in brains and machines, bridging AI/ML theory and neuroscience.
I build biologically grounded models to uncover principles of learning across circuits and behavior.
Looking forward to meeting folks in Lisbon!
(3/n) Recent years have brought exciting progress on learning-rule inference in animals, including work by @marcelomattar.bsky.social, @kevinjmiller.bsky.social, @mariaeckstein.bsky.social, @pcastr.bsky.social, @neurokim.bsky.socia, and others) plus @vgeadah.bsky.social’s de novo learning work.
(1/n) Presenting our NeurIPS’25 work at #cosyne2026 poster 3-109
We introduce a flexible framework to infer how animals learn new decision-making tasks from scratch without assuming REINFORCE/Q-learning.
Paper: arxiv.org/abs/2509.04661
Joint w/ @vgeadah.bsky.social and @jpillowtime.bsky.social
(4/n) At a high level, most existing models emphasize either (i) flexible learning-rule inference without de novo learning, or (ii) learning a new task from scratch but with a predefined learning rule. We ask: can we flexibly infer the learning rule in this de novo regime?
(6/n) Parameterizing the learning rule with a neural network gives useful inductive bias, including smoothness/continuity in the update function.
Helena (Yuhan) Liu
Understanding how animals learn is a central challenge in neuroscience, with growing relevance to the development of animal- or human-aligned artificial intelligence. However, existing approaches tend...
