Assessing the simulator-sickness level of a User in VR

Closed Human-Computer Interaction Computer Science Games Engineering eXtended Artificial Intelligence Master Project No Affiliation

xTai Lab VIA VR - Exposé

Motivation

Virtual Reality (VR) applications are commonly associated with the field of gaming, but they can also be a helpful tool in serious contexts, like therapy (for physiological and psychological issues) and rehabilitation. For example, the patient can be exposed to stressful situations or objects in VR as a form of virtual confrontation therapy. There are several advantages of virtual therapy: the situation in VR is completely predictable and controllable by a supervisor, and the therapy session can be conducted in any location.

To make sure the VR application has a positive effect on the patient, it is crucial that they do not feel any physical distress that is caused by VR itself. Nevertheless, it is not uncommon that people can experience cybersickness in virtual environments (Lawson, 2014). The symptoms of cybersickness are similar to those experienced when having motion sickness: headache, eye strain, stomach awareness, vomiting, nausea, fatigue, and disorientation. Experiencing these symptoms has a detrimental effect on the usefulness of virtual therapy sessions.

To detect cybersickness, the Simulator Sickness Questionnaire (SSQ) (Kennedy et al., 1993) can be used as a subjective measurement, but also the patient’s heart rate, galvanic skin response, head movement or EEG data can be monitored as objective physical measurements (Kim et al., 2005).

In this project, we want to develop a classifier that can detect if a person is experiencing cybersickness while they are using a VR application in real-time. This classifier can be a useful tool for the supervisor of the experiment, who is alerted by the system when the patient shows physiological symptoms of cybersickness, and who then can take according actions (like stopping the VR application or altering parameters in the virtual scenario).

Goal

The main question that led us to work on this project is: When does cybersickness occur? Answering this question made our way and our main goal which is; to develop a Neural Network for detecting cybersickness in real-time. To approach this goal, we split it into some short-time goals that have been elaborated in the following lines:

1. How does cybersickness affect people?

• Goal1: Reading paper about cybersickness and its effect.

1. How to measure cybersickness using sensors?

• Goal2: Working with sensors and finding out how they collect data.

1. How we can collect data?

• Goal3: Using customized measurement-engine and VR in parallel for collecting raw data.

1. How we should interpret data?

• Goal4: Clean collected data and prepare them for the training phase.

1. How we can develop a Neural Network which can detect cybersickness based on collected data?

• Goal5: Develop a Neural Network and train a model with the collected data.

1. How accurate is our model’s prediction?

• Goal6: Measuring validation parameters

Project Description

In order to get the data, which is used for training the model, an experiment in VR was conducted on different participants including ourselves. The experiment takes about two minutes riding a rollercoaster with a head-mounted display (HMD), while data from two sensors, Empatica E4 and Polar H10, is measured. The idea of measuring the heart-rate and additional data while riding a virtual rollercoaster is originally from a paper researching the effects of cybersickness and its correlation to body values from Blackmore and colleagues. (Blackmore et al. 2015)

Mapping the collected data with some ground truth values to detect if a person felt cybersick and training the model based on these mappings should result into a classifier, that can predict if a person feels cybersick right now, while doing an experiment and being connected to the sensors.

The two sensors we use are Empatica E4 and Polar H10, as mentioned already. They are worn on the wrist and around the chest. Additionally we collect the eye-tracking data from the Pico Neo 2 eye, which is the HMD we use to run the rollercoaster.

The ground truth is collected in two different ways. We use the SSQ (Kennedy et al., 1993) with three additional questions in a pre-post design to evaluate the participants well-being before and after the experiment. The additional questions are about previous experiment with VR and computer games and if the person usually feels sick in the car.
Furthermore, data is collected in Vivo. This technique was originally used to measure Presence in VR during the exposure from inside the experiment. For this, the participants have to press a controller button, whenever they feel cybersick, or any of the described symptoms occur. (Bouchard et al. 2008)

The experiment was done in the following steps:

1. Short introduction into the project and about the experiment

2. SSQ with additional questions

3. Setup of glasses and sensors, as well as the measurement engine, which is used to gather all the collected data

4. Instruction of the controller button, which should be pressed in case of cybersickness as well as information about stoping the experiment, when symptoms too strong

5. Start the rollercoaster and be quite to not break the presence of the participant in the VR

6. SSQ without additional questions

Based on answering the questions we made our plan.

Plan

Literature

R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal. Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 3(3):203–220, 1993.

B. Lawson. Motion Sickness Symptomatology and Origins, pp. 531–600. 09 2014. doi: 10.1201/b17360-29

Y. Youn Kim, H. Ju Kim, E. Nam Kim, H. Dong Ko, and H.-T. Kim. Characteristic changes in the physiological components of cybersickness. Psychophysiology, 42:616–25, 10 2005.

Blackmore et al. 2015 - Nalivaiko, Eugene & Davis, Simon & Blackmore, Karen & Vakulin, Andrew & Nesbitt, Keith. (2015). Cybersickness provoked by head-mounted display affects cutaneous vascular tone, heart rate and reaction time. Autonomic Neuroscience. 192. 63. 10.1016/j.autneu.2015.07.032.

Bouchard et al. 2008 - Stéphane Bouchard, Julie St-Jacques, Geneviève Robillard, Patrice Renaud; Anxiety Increases the Feeling of Presence in Virtual Reality. Presence: Teleoperators and Virtual Environments 2008; 17 (4): 376–391. doi: https://doi.org/10.1162/pres.17.4.376

Contact Persons at the University Würzburg