In the W-S lab's first preprint, we describe how genomic language models know something about RNA thermodynamics. Though we think this is cool, things get tricky!
A growing practice for interpreting LMs is to perturb input tokens, often called "Categorical Jacobian": 👇
