Real-time Editing of VR360 and VR180-Stereo Photos

Assigned Computer Science Master Thesis Games Research Lab Industry Cooperation

Motivation

With affordable high-quality standalone head-mounted displays (HMD), virtual reality is ready to move into the mass consumer market. With modern VR-cameras, images can be captured with a single camera shot covering the full sphere with 360x180 degrees around the camera (VR360). Other cameras have two wide-angle, forward-facing cameras and can capture stereoscopic content covering a half-sphere with 180x180 degrees. When these images are watched inside an HMD, the human perception is tricked into believing to really be at the scene. A common format to store such images in a 2D jpeg file is equirectangular. This leads to the issue that editing the image in a 2D paint program is much harder as the distortions have to be considered. Furthermore, when editing the stereoscopic images, the perceived depth has to be considered for any image changes for the left and right eye image.

Tasks

Investigate a real-time image editing approach for VR360 mono and VR180 stereo that can be used directly inside an HMD. What tools does the editing person need to be efficient? How can the depth consistency between the left and right eye image be maintained in stereoscopic images? Test the application on users and evaluate what works well and where problems remain for further research. The final software should be available as open source under a very open license, e.g., FreeBSD.

Prerequisites

• Excellent skills in C# and Unity
• Math knowledge for converting the equirectangular image data into a spherical projection
• Familiarity with Virtual Reality devices
• Familiarity with depth from stereo algorithms, e.g., being able to use these through OpenCV
• Interest in VR photography

Execution

The project will be executed together with a VR startup near Erlangen.

Contact Persons at the University Würzburg