Immersive Digital Reality
Abstract
Motivated by the advent of mass-market head-mounted immersive displays, we set out to pioneer the technology needed to experience recordings of the real world with the sense of full immersion as provided by VR goggles.
Immersive Digital Reality
a DFG Reinhart Koselleck Project
Project Summary
Motivated by the advent of mass-market head-mounted immersive displays, we set out to pioneer the technology needed to experience recordings of the real world with the sense of full immersion as provided by VR goggles. To achieve this goal, a number of interdisciplinary, tightly interrelated challenges from video processing, computer graphics, computer vision, and applied visual perception need to be addressed concertedly. By importing the real world into immersive displays, we want to lay the foundations for the way we may watch movies in the future, leaving fixed-viewpoint, limited field-of-view screens behind for a completely immersive, collective experience.
Researchers
Alumni
Job Openings
We are always looking for excellent researchers. Want to join the project?
Talks
| January 12, 2018 | Keynote presentation at VR Walkthrough Technology Day, TU Tampere, Finland (presentation video) |
| April 20, 2017 | Invited talk at Stanford Computer Graphics Lab (GCafe), Stanford University, USA |
| January 23, 2017 | Invited talk at University of Konstanz/SFB TRR 161: "Visual Computing - Bridging Real and Digital Domain" |
Events
| June 30-July 3, 2019 | Real VR - Importing the Real World into Immersive VR and Optimizing the Perceptual Experience of Head-Mounted Displays, Dagstuhl Seminar 19272 |
| June 7-8, 2017 | Symposium on Visual Computing and Perception (SVCP),TU Braunschweig |
In the News
| May 4, 2016 | Article in the local Braunschweiger Zeitung (in German). |
Publications
Gaze and Motion-aware Real-Time Dome Projection System
in Proc. IEEE Virtual Reality (VR) Workshop, to appear.
PerGraVAR
Comparing Unobtrusive Gaze Guiding Stimuli in Head-mounted Displays
in Proc. IEEE International Conference on Image Processing (ICIP), IEEE, October 2018.
Comparison of Unobtrusive Visual Guidance Methods in an Immersive Dome Environment
in ACM Transactions on Applied Perception, vol. 15, no. 4, ACM, October 2018.
Low Cost Setup for High Resolution Multiview Panorama Recording and Registration
in Proc. European Signal Processing Conference (EUSIPCO), September 2018.
On the Delay Performance of Browser-based Interactive TCP Free-viewpoint Streaming
in Proc. IFIP Networking 2018 Conference (NETWORKING 2018), IEEE, pp. 1-9, July 2018.
ICG Real-Time Stereo Dome Projection System
Technical Report no. 1, Computer Graphics Lab, TU Braunschweig, May 2018.
Gaze Guidance in Immersive Environments
Poster @ IEEE Virtual Reality 2018, March 2018.
Automatic Upright Alignment of Multi-View Spherical Panoramas
Poster @ European Conference on Visual Media Production 2017, December 2017.
Best Student Poster Award
Subtle Gaze Guidance for Immersive Environments
in Proc. ACM Symposium on Applied Perception (SAP), ACM, September 2017.
Comparative analysis of three different modalities for perception of artifacts in videos
in ACM Transactions on Applied Perception, vol. 14, no. 4, pp. 1-12, September 2017.
Perception-driven Accelerated Rendering
in Computer Graphics Forum (Proc. of Eurographics EG), vol. 36, no. 2, The Eurographics Association and John Wiley & Sons Ltd., pp. 611-643, April 2017.
Gaze Visualization for Immersive Video
in Burch, Michael and Chuang, Lewis and Fisher, Brian and Schmidt, Albrecht and Weiskopf, Daniel (Eds.): Eye Tracking and Visualization, Springer, ISBN 978-3319470238, pp. 57-71, March 2017.
Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering
Poster @ German Conference on Pattern Recognition 2016, September 2016.
Gaze-contingent Computational Displays: Boosting perceptual fidelity
in IEEE Signal Processing Magazine, vol. 33, no. 5, IEEE, pp. 139-148, September 2016.
Adaptive Image-Space Sampling for Gaze-Contingent Real-time Rendering
in Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering EGSR), vol. 35, no. 4, pp. 129-139, July 2016.
EGSR'16 Best Paper Award
Related Projects
Digital Representations of the Real World
The book presents the state-of-the-art of how to create photo-realistic digital models of the real world. It is the result of work by experts from around the world, offering a comprehensive overview of the entire pipeline from acquisition, data processing, and modelling to content editing, photo-realistic rendering, and user interaction.
Eye-tracking Head-mounted Display
Immersion is the ultimate goal of head-mounted displays (HMD) for Virtual Reality (VR) in order to produce a convincing user experience. Two important aspects in this context are motion sickness, often due to imprecise calibration, and the integration of a reliable eye tracking. We propose an affordable hard- and software solution for drift-free eye-tracking and user-friendly lens calibration within an HMD. The use of dichroic mirrors leads to a lean design that provides the full field-of-view (FOV) while using commodity cameras for eye tracking.
Featuring more than 10 million pixels at 120 Hertz refresh rate, full-body motion capture, as well as real-time gaze tracking, our 5-meter ICG Dome enables us to research peripheral visual perception, to devise comprehensive foveal-peripheral rendering strategies, and to explore multi-user immersive visualization and interaction.
Scope of "Reality CG" is to pioneer a novel approach to modelling, editing and rendering in computer graphics. Instead of manually creating digital models of virtual worlds, Reality CG will explore new ways to achieve visual realism from the kind of approximate models that can be derived from conventional, real-world imagery as input.
The Virtual Video Camera research project is aimed to provide algorithms for rendering free-viewpoint video from asynchronous camcorder captures. We want to record our multi-video data without the need of specialized hardware or intrusive setup procedures (e.g., waving calibration patterns).