A big optimization I worked on after my last post was making our voxelizer lazy. So, it only updates parts of the scene that changed. This made a huge difference for our performance and allowed us to push for larger level sizes!
Below is a high-level drawing of how it works: