In a sense, memory may be the dark matter of active vision. Understanding the world through iterative glimpses requires memory. Our results indicate that the visual system already tailors each glimpse to the computational demands of that memory scaffold. /8
The same picture emerges from the neural level: Longer fixations show enhanced thetaāgamma coupling in frontal and hippocampal regions, a hallmark neural signature linked to memory encoding. /7
And indeed, that is what we find. Patches later mentioned in scene descriptions are fixated substantially longer, and patches predicted to be more memorable are fixated longer. /6
From reaction time studies, we know: harder stimuli take longer to process. So perhaps the brain lingers on difficult patches? Surprisingly, we found the opposite. /4
Every day, we make around 200,000 eye movements. Some fixations last over 500 ms, others barely 150 ms. What makes a long fixation different from a short one? /2
Despite decades of research, we have surprisingly little understanding of which brain processes drive this variability. We tackle this by combining MEG and eye tracking across 4,080 natural scenes as well as neural network modelling. /3
Huge thanks to the team!
@carmenamme.bsky.social
@martinhebart.bsky.social
@peterkonig.bsky.social
@timkietzmann.bsky.social
www.nature.com/articles/s41...
#NeuroAI #neuroscience /fin
Easier-to-classify image patches receive longer fixations. Neural dynamics in visual cortex stabilise at the same latency regardless of how long a fixation lasts. This points away from initial visual processing demands and towards something downstream. /5
I totally agree, science is not a simple optimization problem, or at least not in the form conceived by most existing systems. @battleday.bsky.social and I wrote about this recently from a somewhat different perspective (emphasizing the 'problem problem'):
gershmanlab.com/pubs/Battled...