VR Photo Processing - Stitching of two Fisheye Images into VR360

This project is already assigned.

Motivation

With affordable high-quality standalone head-mounted displays (HMD), virtual reality is ready to move into the mass consumer market. With modern VR-cameras, images can be captured with a single camera shot covering the full sphere with 360x180 degrees around the camera (VR360). Internally, these cameras consist of two fisheye lenses with each about 190° degrees field of view and two image sensors. After the shot, the two fisheye images are merged at the overlapping areas into a single image in an equirectangular format.

Figure 1: example of two 190° fisheye shots

Figure 2: example of a VR360 mono image in equirectangular format after stitching the two images from above

Figure 3: different stitching methods while merging at borders of the input images

Tasks

Conversion tools from VR camera manufacturers exist with mixed results in quality. Some work pretty well for most images, some have issues with certain situations. For this project, we start with an already existing, basic conversion tool in C++ that does the work and can be driven by various parameters which influence the outcome. The goal is to make many of these parameters automatic, for example, based on estimations of depth at the parts of the fisheye images that overlap for merging. Furthermore, pre-processing like changing the exposure levels or brightness might be required as the two input images might use different values. On top of that, in the final version of this project, we expect to have the task of stitching to VR360 running in Unity on a modern virtual reality device.

Prerequisites

• Excellent skills in C++, C# and Unity

• Basic math knowledge for converting the fisheye images into equirectangular images – we provide example code and references

• Familiarity with Virtual Reality devices

• Knowledge on using OpenCV is a plus, but can also be learned during the project

• Interest in VR photography is a plus

Execution

The project would be executed together with the VR startup “immerVR GmbH” near Erlangen. It could be in the form of an industry cooperation for a Bachelor’s thesis, Master’s thesis or an internship (“Pflichtpraktikum”) starting now.

Contact Persons at the University Würzburg

Prof. Dr. Sebastian von Mammen (Primary Contact Person)
Mensch-Computer-Interaktion, Universität Würzburg
sebastian.von.mammen@uni-wuerzburg.de

Daniel Pohl, CEO immerVR GmbH (Primary Contact Person)
Contact through website