Real VR - special issue of IEEE Computer Graphics and Applications

Since the times of the Lumière brothers, the way we watch movies has not fundamentally changed: whether in movie theaters, on mobile devices, or on TV at home, we still experience movies as outside observers, watching the action through a proscenium defined by the size of the screen. Unfortunately, the limited size of the screen causes our peripheral vision to remind us subconsciously that we are only outside observers and not “really there.” With the advent of consumer VR headsets, this outside-observer paradigm of visual entertainment has given way to a truly immersive experience. In immersive displays, the action completely encompasses the viewer, drawing us into the action and providing an acute sense of “being there.” Unsurprisingly, this paradigm shift has taken hold first in video gaming where content has always been completely computer-generated. Now, Real VR lays the foundations for the next step: How to import real-world video recordings into VR headsets? To attain this goal, two ambitious challenges must be tackled in unison. Suitable digital representations must be created from real-world footage that allow real-time rendering from arbitrary vantage points, including stereo vision and egomotion parallax. And the degree of perceptual realism of VR headsets must be enhanced to the point where depictions of real-world scenes appear truly genuine to our visual sense. Finding viable solutions to these challenges in Real VR will allow us to experience movies, concerts, or even live sports events with a sense of “being there,” which is unattainable with today’s existing technologies.