Logo CG
Multiple Kinect Studies
Logo TU

Abstract

This project investigates multi-camera setups using Microsoft Kinects. Active structured light from the Kinect is used in several scenarious, including gas flow description, motion capture and free-viewpoint video.

While the ability to capture depth alongside color data (RGB-D) is the starting point of the investigations, the structured light is also used more directly. In order to combine Kinects with passive recording approaches, common calibration with HD cameras is also a topic.

Publications

Yannic Schröder, Kai Berger, and Marcus Magnor:
"Super Resolution for Active Light Sensor Enhancement",
Bachelor thesis, March 2012.
Part of project "Multiple Kinect Studies".
[pdf] [bib]

This thesis provides a new approach for enhancing disparity detection of color-depth-sensors (RGB-D-sensors). The infrared images captured by an RGB-D-sensor are evaluated to detect subtle di erences in the light path of the sensor's laser projection and these di erences are exported as o set vectors. An algorithm is introduced, implemented, and tested, that is able to detect and quantify the light path's di erences. Furthermore, an approach to obtain a reference brightness distribution of an average spot of the RGBD- sensor's projector is presented. The reference brightness distribution is used to identify spot positions with sub-pixel accuracy in infrared images captured with the RGB-D-sensor. The system is employed to detect refractive media, being introduced to the captured scene. It is shown that the presence of refractive media introduced to a scene can be detected.

Kai Berger, Kai Ruhl, Mark Albers, Yannic Schröder, Alexander Scholz, Stefan Guthe, and Marcus Magnor:
"The capturing of turbulent gas flows using multiple Kinects",
in Proc. CDC4CV 2011, November 2011.
ISBN: 978-1-4673-0061-2
Part of project "Multiple Kinect Studies".
[pdf] [bib]

We introduce the Kinect as a tool for capturing gas flows around occluders using objects of different aerodynamic properties. Previous approaches have been invasive or require elaborate setups including large printed sheets of elaborated noise patterns and neat lighting. Our method is easier to set up while still producing good results. We show that three Kinects are sufficient to qualitatively reconstruct nonstationary time varying gas flows in the presence of occluders.

Yannic Schröder, Alexander Scholz, Kai Berger, Kai Ruhl, Stefan Guthe, and Marcus Magnor:
"Multiple Kinect Studies",
Technical Report no. 09-15, ICG, October 2011.
Part of project "Multiple Kinect Studies".
[pdf] [bib]

The Microsoft Kinect is a new color-depth (RGB-D) tracking device designed for home entertainment. With its great performance and low price compared to state-of-the-art depth tracking systems it is the target of a variety of academic research. We investigate the possibilities of using multiple Kinects in the same environment for capturing opaque and translucent objects in motion. Arising issues with the laser subsystems interfering with each other are solved using fast rotating disks, creating a time division multiple access (TDMA) scenario. We also introduce an algorithm to evaluate the quality of captured depth images and compare the quality of depth images created with our multiplexing technique to depth images obtained without multiplexing.

Kai Berger, Kai Ruhl, Christian Brümmer, Yannic Schröder, Alexander Scholz, and Marcus Magnor:
"Markerless Motion Capture using multiple Color-Depth Sensors",
in Proc. Vision, Modeling and Visualization (VMV) 2011, pp. 317–324, October 2011.
Part of project "Multiple Kinect Studies".
[pdf] [bib]

With the advent of the Microsoft Kinect, renewed focus has been put on monocular depth-based Motion Capturing. However, this approach is limited in that an actor has to move facing the camera. Due to the active light nature of the sensor, no more than one device has been used for motion capturing so far. In effect, any pose estimation must fail for poses facing away from the depth camera. Our work investigates on reducing or mitigating the detrimental effects of multiple active light emitters, thereby allowing motion capture from all angles. We systematically evaluate the concurrent use of one to four Kinects, including calibration, error measures and analysis, and present a time-multiplexing approach.

Kai Ruhl, Kai Berger, Christian Lipski, Felix Klose, Yannic Schröder, Alexander Scholz, and Marcus Magnor:
"Integrating multiple depth sensors into the virtual video camera",
Poster at Siggraph, August 2011.
SIGGRAPH '11: ACM SIGGRAPH 2011 Posters
Part of projects "Multiple Kinect Studies", "Virtual Video Camera", and "Who Cares?".
[pdf] [bib]

In this ongoing work, we present our efforts to incorporate Microsoft Kinect depth sensors into a multi camera system for free view-point video. Both the video cameras and the depth sensors are consumer grade. Our free-viewpoint system, the Virtual Video Camera, uses image-based rendering to create novel views between widely spaced (up to 15 degrees) cameras, using dense image correspondences. The introduction of multiple depth sensors into the system allows us to obtain approximate depth information for many pixels, thereby providing a valuable hint for estimating pixel correspondences between cameras.

Related Projects

"Reality CG"

Scope of Reality CG is to pioneer a novel approach to modelling, editing and rendering in computer graphics. Instead of manually creating digital models of virtual worlds, Reality CG will explore new ways to achieve visual realism from the kind of approximate models that can be derived from conventional, real-world imagery as input.


Line
TU Braunschweig - Fakultät für Mathematik und Informatik - Computer Graphics - Research Projects - Multiple Kinect Studies