In an interdisciplinary German-Mexican research project partially funded by German DFG (Deutsche Forschungsgemeinschaft, grants MA 2555/7-1 and 444 MEX-113/25/0-1) and Mexican CONACyT (Consejo Nacional de Ciencia y Tecnología, grants 49447 and UNAM DGAPA-PAPIIT IN108506-2), we are currently evaluating different approaches for automatical reconstruction of plausible three-dimensional models of planetary nebulae. The team comprises astrophysicists working on planetary nebula morphology as well as computer scientists experienced in the field of reconstruction and visualization of astrophysical objects.

Humans have been fascinated by astrophysical phenomena since prehistoric times. But while the measurement and image acquisition devices have enormously evolved by now, many restrictions still apply when capturing astronomical data. The most notable limitation is our confined vantage point in the solar system, disallowing us to observe distant objects from different points of view.

In the "Astrographics" research project, we work on various methods to overcome these limitations using computer vision and computer graphics algorithms. We have, among other things, computed plausible 3D surface data for the moon and complete 2D images of radio galaxies from sparse spatial frequency measurements.

Traditional optic flow algorithms rely on consecutive short-exposure images. In contrast, long-exposed images contain integrated motion information directly in form of motion blur. In this project, we use the additional information provided by a long exposure image to improve robustness and accuracy of motion field estimation. Furthermore, the long exposure image can be used to determine the moment of occlusion for the pixels in any of the short exposure images that are occluded or disoccluded.
This work has been funded by the German Science Foundation, DFG MA2555/4-1

The goal of this project is to augment video data with high-quality 3D geometry, while only using a single camera as input.
As an application of this project, we want to dress a person in a video with artificial clothing.
We reconstruct a 3D human pose from 2D input data. This information can be used to drive a cloth simulation creating a plausible 3D garment for the observed pose. Composing this animated garment into the original video creates the illusion of the person wearing different clothing.
We aim at real-time framerates for this system, allowing for virtual mirror applications.

In cooperation with the Institute of Computer and Network Engineering (www.ida.ing.tu-bs.de) we try to accelerate as much of the vision algorithms using programmable hardware such as FPGAs to make the whole augmentation system real-time capable.

The research project is funded by the German Science Foundation, DFG MA2555/8-1.

We present a system to automatically construct high resolution images from an unordered set of low resolution photos. It consists of an automatic preprocessing step to establish correspondences between any given photos. The user may then choose one image and the algorithm automatically creates a higher resolution result, several octaves larger up to the desired resolution. Our recursive creation scheme allows to transfer specific details at subpixel positions of the original image. It adds plausible details to regions not covered by any of the input images and eases the acquisition for large scale panoramas spanning different resolution levels.

We present a novel multi-view, projective texture mapping technique. While previous multi-view texturing approaches lead to blurring and ghosting artefacts if 3D geometry and/or camera calibration are imprecise, we propose a texturing algorithm that warps (``floats'') projected textures during run-time to preserve crisp, detailed texture appearance. Our GPU implementation achieves interactive to real-time frame rates. The method is very generally applicable and can be used in combination with many image-based rendering methods or projective texturing applications. By using Floating Textures in conjunction with, e.g., visual hull rendering, light field rendering, or free-viewpoint video, improved rendering results are obtained from fewer input images, less accurately calibrated cameras, and coarser 3D geometry proxies. In a nutshell, the notion of Floating Textures is to correct for local texture misalignments by determining the optical flow between projected textures and warping the textures accordingly in the rendered image domain. Both steps, optical flow estimation and multi-texture warping, can be efficiently implemented on graphics hardware to achieve interactive to real-time performance.

We present a method for image interpolation which is able to create high-quality, perceptually convincing transitions between recorded images. By implementing concepts derived from human vision, the problem of a physically correct image interpolation is relaxed to an image interpolation that is perceived as physically correct by human observers. We find that it suffices to focus on exact edge correspondences, homogeneous regions and coherent motion to compute such solutions. In our user study we confirm the visual quality of the proposed image interpolation approach. We show how each aspect of our approach increases the perceived quality of the interpolation results, compare the results obtained by other methods and investigate the achieved quality for different types of scenes.

This project focuses on using electroencephalography (EEG) to analyze the human visual process. Human visual perception is becoming increasingly important in the analyses of rendering methods, animation results, interface design, and visualization techniques. Our work uses EEG data to provide concrete feedback on the perception of rendered videos and images as opposed to user studies that just capture the user's response. Our results so far are very promising. Not only have we been able to detect a reaction to artifacts in the EEG data, but we have also been able to differentiate between artifacts based on the EEG response.

The goal of this research project is to develop and evaluate new approaches to interactive ray tracing. Our main focus are new data structures, representations and algorithms for fast, memory efficient and realistic image synthesis. Our research covers various topics from basic research for faster intersection tests to acceleration data structures, compression, image-based techniques and real-time global illumination.

Multi-view video camera setups record many images that capture nearly the same scene at nearly the same instant in time. Neighboring images in a multi-video setup restrict the solution space between two images: correspondences between one pair of images must be in accordance with the correspondences to the neighboring images.
The concept of accordance or consistency for correspondences between three neighboring images can be employed in the estimation of dense optical flow and in the matching of sparse features between three images.

This work has been partially funded by the German Science Foundation, DFG MA2555/4-2.

This project investigates multi-camera setups using Microsoft Kinects. Active structured light from the Kinect is used in several scenarious, including gas flow description, motion capture and free-viewpoint video.

While the ability to capture depth alongside color data (RGB-D) is the starting point of the investigations, the structured light is also used more directly. In order to combine Kinects with passive recording approaches, common calibration with HD cameras is also a topic.

Scope of Reality CG is to pioneer a novel approach to modelling, editing and rendering in computer graphics. Instead of manually creating digital models of virtual worlds, Reality CG will explore new ways to achieve visual realism from the kind of approximate models that can be derived from conventional, real-world imagery as input.

The Virtual Video Camera research project is aimed to provide algorithms for rendering free-viewpoint video from asynchronous camcorder captures. We want to record our multi-video data without the need of specialized hardware or intrusive setup procedures (e.g., waving calibration patterns).

Goal of this project is to assess the quality of rendered videos
and especially detect those frames that contain visible artifacts,
e.g. ghosting, blurring or popping.

Official music video "Who Cares" by Symbiz Sound; the first major production using our Virtual Video Camera.

Dubstep, spray cans, brush and paint join forces and unite with the latest digital production techniques. All imagery depicts live action graffiti and performance. Camera motion added in post production using the Virtual Video Camera.

Goal of this research project is to develop and evaluate a fundamentally new approach to exhaustively search for, and interactively characterize any non-random mutual relationship between attribute dimensions in general data sets. To be able to systematically consider all possible attribute combinations, we propose to apply image analysis to visualization results in order to automatically pre-select only those attribute combinations featuring non-random relationships. To characterize the found information and to build mathematical descriptions, we rely on interactive visual inspection and visualization-assisted interactive information modeling. This way, we intend to discover and explicitly characterize all information implicitly represented in unbiased sets of multi-dimensional data points.

The TU Braunschweig participated in the DARPA Urban Challenge 2007, its autonomous vehicle 'Caroline' was among the finalists. The Computer Graphics lab provided the real-time vision algorithms for that task.

Caroline's computer vision system consists of two separate systems. The first is a monocular color segmentation based system that classifies the ground in front of the car as drivable, undrivable or unknown. It assists in situations where the drivable terrain and the surrounding area (e.g. grass, concrete or shrubs) differ in color and it deals with man-made artifacts such as lane markings as well as bad lighting and weather conditions. The second vision system is a multi-view lane detection that identifies the different kinds of lanes described by DARPA, such as broken and continuous as well as white and yellow lane markings. Using four high-resolution color cameras and state-of-the-art graphics hardware, it detects its own lane and the two adjacent lanes to the left and right with a field of view of 175 degrees at up to 35 meters. The output of the lane detection algorithm is directly processed by the artificial intelligence.