(1) large-scale registration of existing 3D head datasets, and
(2) self-supervised training on vast in-the-wild 2D video datasets using pseudo ground-truth surface normals.
Finally, we show that geometry-aware pretraining on pixel-aligned reconstruction tasks
Matthias Niessner
📢 Intrinsic Image Fusion for Multi-View 3D Material Reconstruction 📢
We combine generative material priors with inverse path tracing: 1) define a parametric texture space 2) fuse monocular predictions across views into consistent textures
Pix2NPHM obtains fast and reliable NPHM reconstructions on real-world data. Inference-time optimization against surface normals and canonical point maps can further increase fidelity.
Key to successful and generalized training of our ViT-based network are:
Face tracking & 3D reconstruction are often limited by the representational capacity of PCA-based face models. By lifting NPHMs to a first-class reconstruction primitive, we enable more accurate geometry, richer expressions, and finer animation control.
Today in our TUM AI - Lecture Series we'll have the amazing Ruiqi Gao, Google DeepMind.
She'll talk about "𝐁𝐮𝐢𝐥𝐝𝐢𝐧𝐠 𝐠𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐰𝐨𝐫𝐥𝐝 𝐦𝐨𝐝𝐞𝐥𝐬: progress and challenges".
Live stream: www.youtube.com/live/CkOSMqw...
7pm GMT+1 / 10am PST (Tue Dec 16th).
significantly outperforms generic visual pretraining (e.g., DINO-style features) in terms of generalization.
🌍https://simongiebenhain.github.io/Pix2NPHM
🎥https://youtu.be/MgpEJC5p1Ts
Great work by Simon Giebenhain, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Zhe Chen.
3) optimize low-dimensional parameters for physically-grounded reconstructions.
The results are relightable PBR textures for 3D scenes: check out the result on a real-world 3D scan from the ScanNet++ dataset!
Matthias Niessner
Matthias Niessner
📢Pix2NPHM: Learning to Regress NPHM Reconstructions From a Single Image📢
We directly regress neural parametric head models (NPHMs) from a single image — fast, stable, and significantly more expressive than classical 3DMMs such as FLAME.
Matthias Niessner
🌍https://peter-kocsis.github.io/IntrinsicImageFusion
🎥https://youtu.be/-Vs3tR1Xl7k
Great work by Peter Kocsis and Lukas Hollein!
