It's been a long time since I reconnected with my interpretability past; here's our new analysis of the phenomenon of "detokenization": models reconstruct full word representations ("error") at the last state of the subword-tokenized location ["err", "or"].
