BayesVLM requires estimating Hessians over the image and text proj. layers, where access to the pretraining data (or proxy of this) is needed, but only 10 batches is sufficient. Estimating pseudo-data count and prior precision params is also needed, which is similar to what temp. scaling needs.
