Parallax Panorama Video
Recent advances in consumer-grade panorama capturing setups enable personalized 360° experiences. Along with the improvement in head-mounted displays (HMDs), these allow to bring back memories at a previously unprecedented level of immersion. However, the lack of explicit depth and scene geometry prohibits any form of head movement, which is needed for a fully-immersive VR experience.
This research project explores methods for adding motion parallax to previously captured stereo panorama videos and enabling real-time playback of these enhanced videos in HMDs.
This work is being funded by the German Science Foundation (DFG) under the Reinhart Koselleck Project "Immersive Digital Reality" (DFG MA2555/15-1).
Multiview Panorama Alignment and Optical Flow Refinement
in Magnor M., Sorkine-Hornung A. (Eds.): Real VR – Immersive Digital Reality: How to Import the Real World into Head-Mounted Immersive Displays, Springer International Publishing, Cham, ISBN 978-3-030-41816-8, pp. 96-108, March 2020.
Immersion is the ultimate goal of head-mounted displays (HMD) for Virtual Reality (VR) in order to produce a convincing user experience. Two important aspects in this context are motion sickness, often due to imprecise calibration, and the integration of a reliable eye tracking. We propose an affordable hard- and software solution for drift-free eye-tracking and user-friendly lens calibration within an HMD. The use of dichroic mirrors leads to a lean design that provides the full field-of-view (FOV) while using commodity cameras for eye tracking.
With the advent of consumer-market Virtual Reality (VR) technology, the next revolution in visual entertainment is already on the horizon: real VR will enable us to experience live-action movies, sports broadcasts, concert videos, etc. in true visual (and aural) immersion. This book provides a comprehensive overview of the algorithms and methods that make it possible to immerse into real-world recordings. It brings together the expertise of internationally renowned experts from academia and industry who present the state of the art in this fascinating, interdisciplinary new research ﬁeld. Written by and for scientists, engineers, and practitioners, this book is the deﬁnitive reference for anyone interested in ﬁnding out about how to import the real world into head-mounted displays.