Real-Action VR
Abstract
Want to re-live your latest bungee jump? Share your incredible skateboard stunts with your friends in 360? Watch your last vacation adventures in full immersion and 3D? In this project we set out to pioneer the fully immersive experience of action camera recordings in VR headsets.
Real-Action VR: Immersive Experience of Action-Cam Video Recordings
a DFG project (no. 523421583)
Project Summary
Scientific goal of our project is the development of visual computing algorithms that enable the high-quality and fully immersive experience of real-world scenes as recorded by free-moving 360-degree action cameras. We set out to develop new methods for automatic drone swarm navigation and control, comprehensive dynamic scene reconstruction from video, 4D scene augmentation and resolution enhancement, real-time 6-DoF immersive video rendering, as well as cybersickness prevention. Our research lays the foundation for sharing and re-living personal adventures with unprecedented vivdness.
Researchers
Job Openings
We are always looking for talent. Want to join the project?
Publications
Measuring Velocity Perception Regarding Stimulus Eccentricity
in Proc. ACM Symposium on Applied Perception (SAP), no. 4, ACM, pp. 1-9, August 2024.
D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video
arXiv preprint, pp. 1-16, June 2024.
INPC: Implicit Neural Point Clouds for Radiance Field Rendering
arXiv preprint, March 2024.
Related Projects
Digital Representations of the Real World
The book presents the state-of-the-art of how to create photo-realistic digital models of the real world. It is the result of work by experts from around the world, offering a comprehensive overview of the entire pipeline from acquisition, data processing, and modelling to content editing, photo-realistic rendering, and user interaction.
Immersive Virtual Reality (VR) offers high flexibility to its users, i.e. viewers of 360° panoramic videos can freely chose their viewing direction while watching a movie that is playing all around them. But this raises the problem that viewers might miss important information because they are not looking into the right direction at the right time — which is an important problem for storytelling in VR.
This research project explores methods for unobtrusive visual attention guidance in 360° virtual environments.
Motivated by the advent of mass-market head-mounted immersive displays, we set out to pioneer the technology needed to experience recordings of the real world with the sense of full immersion as provided by VR goggles.
Recent advances in consumer-grade panorama capturing setups enable personalized 360° experiences. Along with the improvement in head-mounted displays (HMDs), these allow to bring back memories at a previously unprecedented level of immersion. However, the lack of explicit depth and scene geometry prohibits any form of head movement, which is needed for a fully-immersive VR experience.
This research project explores methods for adding motion parallax to previously captured stereo panorama videos and enabling real-time playback of these enhanced videos in HMDs.
Preventing Motion Sickness in VR
Motion sickness, also referred as Simulator Sickness, Virtual sickness, or Cyber sickness, is a problem common to all types of visual simulators consisting of motion sickness-like symptoms that may be experienced while and after being exposed to a dynamic, immersive visualization. It leads to ethical concerns and impaired validity of simulator-based research. Due to the popularity of virtual reality devices, the number of people exposed to this problem is increasing and, therefore, it is crucial to not only find reliable predictors of this condition before any symptoms appear, but also find alternatives to fully prevent its occurrence while experiencing VR content.
Scope of "Reality CG" is to pioneer a novel approach to modelling, editing and rendering in computer graphics. Instead of manually creating digital models of virtual worlds, Reality CG will explore new ways to achieve visual realism from the kind of approximate models that can be derived from conventional, real-world imagery as input.