Logo CG
Monocular Video Augmentation
Logo TU

Abstract

The goal of this project is to augment video data with high-quality 3D geometry, while only using a single camera as input.
As an application of this project, we want to dress a person in a video with artificial clothing.
We reconstruct a 3D human pose from 2D input data. This information can be used to drive a cloth simulation creating a plausible 3D garment for the observed pose. Composing this animated garment into the original video creates the illusion of the person wearing different clothing.
We aim at real-time framerates for this system, allowing for virtual mirror applications.

In cooperation with the Institute of Computer and Network Engineering (www.ida.ing.tu-bs.de) we try to accelerate as much of the vision algorithms using programmable hardware such as FPGAs to make the whole augmentation system real-time capable.

The research project is funded by the German Science Foundation, DFG MA2555/8-1.

Publications

Lorenz Rogge, Thomas Neumann, Markus Wacker, and Marcus Magnor:
"Monocular Pose Reconstruction for an Augmented Reality Clothing System",
in Proc. Vision, Modeling and Visualization (VMV) 2011, pp. 339–346, September 2011.
Part of project "Monocular Video Augmentation".
[pdf] [bib]

In this paper, we present an approach for realizing an augmented reality system for try-on of apparel. The core component of our system is a quick human pose estimation algorithm based on a single camera view only. Being limited to monocular input data, pose reconstruction may be ambiguous. We solve this problem by using a markered suit, though not relying on any specific marker layout. To recover 3D joint angles of the person using the system we use Relevance Vector Machine regression with image descriptors that include neighborhood configurations of visible colored markers and image gradient orientations. This novel combination of image descriptors results in a measurable improvement in reconstruction quality. We initialize and evaluate our algorithm with pose data acquired using a motion capture system. As the final step, we simulate a cloth draped around a virtual character adopting the estimated pose. Composing the original view and the rendered cloth creates the illusion of the user wearing virtual garments.


Line
TU Braunschweig - Fakultät für Mathematik und Informatik - Computer Graphics - Research Projects - Monocular Video Augmentation