Computer Graphics
TU Braunschweig

N-SfC: Robust and Fast Shape Estimation from Caustic Images

N-SfC: Robust and Fast Shape Estimation from Caustic Images

This paper handles the highly challenging problem of reconstructing the shape of a refracting object from a single image of its resulting caustic.

Due to the ubiquity of transparent refracting objects in everyday life, reconstruction of their shape entails a multitude of practical applications. While we focus our attention on inline shape reconstruction in glass fabrication processes, our methodology could be adapted to scenarios where the limiting factor is a lack of input measurements to constrain the reconstruction problem completely.

The recent Shape from Caustics (SfC) method casts this problem as the inverse of a light propagation simulation for synthesis of the caustic image, that can be solved by a differentiable renderer. However, the inherent complexity of light transport through refracting surfaces currently limits the practical application due to reconstruction speed and robustness. Thus, we introduce Neural-Shape from Caustics (N-SfC), a learning-based extension incorporating two components into the reconstruction pipeline: a denoising module, which both alleviates the light transport simulation cost, and also helps finding a better minimum; and an optimization process based on learned gradient descent, which enables better convergence using fewer iterations.

Extensive experiments demonstrate that we significantly outperform the current state-of-the-art in both computational speed and final surface error.

Presentation Video:


Source Code: Available Soon

Author(s):Marc Kassubeck, Moritz Kappel, Susana Castillo, Marcus Magnor
Published:September 2023
Type:Article in conference proceedings
Book:Proc. Vision, Modeling and Visualization (VMV) (The Eurographics Association)
Presented at:Vision, Modeling and Visualization (VMV) 2023
Project(s): Physical Parameter Estimation from Images 

  title = {N-SfC: Robust and Fast Shape Estimation from Caustic Images},
  author = {Kassubeck, Marc and Kappel, Moritz and Castillo, Susana  and Magnor, Marcus},
  booktitle = {Proc. Vision, Modeling and Visualization ({VMV})},
  organization = {Eurographics},
  isbn = {978-3-03868-232-5},
  doi = {10.2312/vmv.20231224},
  editor = {T. Grosch and M. Guthe},
  pages = {33--41},
  month = {Sep},
  year = {2023}