Using virtual reality and gamification within procedural generated environments to increase motivation during gait rehabilitation

Closed Computer Science Master Thesis No Affiliation

Organisation

The master thesis created in cooperation with the doctoral thesis written at the chair of psychology reviewed by Prof. Dr. Paul Pauli. The focus of this doctoral thesis is the evaluation in which extensive virtual reality supports the treadmill based rehabilitation process of patients with multiple sclerosis. The goal of the master thesis is the evaluation and integration of necessary hardware components and development of the virtual reality application. This project is realized by Florian Kern. The examiners are Prof. Dr. Marc Erich Latoschik and Dominik Gall from the chair of human computer interaction.

Introduction

Multiple Sclerosis

Multiple sclerosis (MS) is a complex neurodegenerative disease of the central nervous system. It manifests as a progressive disease through dissemination in time and space [1] and frequently leads to disability in young adults [2]. The worldwide total estimated number of people with MS has increased from 2.1 million in 2008 to 2.3 million in 2013. While MS found in every region of the world, the prevalence is very different. Being highest in North America with a population of 140 per 100.000 the value in Europe is much lower with 108 per 100.000. This amount indicates over 500.000 people with multiple sclerosis in Europe. [3, 4] Most commonly motor deficits affect the lower limb and leading to limitations in walking ability and daily activities [5, 6]. Compared to humans without gait restrictions people with multiple sclerosis frequently show reduced stride length, gait speed [7] and balance control [8]. More than 70% of patients with multiple sclerosis report mobility problems and gait disturbances as their main restriction [9–12]. Moreover, higher variability in lower limb kinematics and therefore lower balance control leads to a higher fall risk [8].

Virtual Rehabilitation

Virtual Reality (VR) for rehabilitation increasingly gain popularity in the last years [15]. VR is a computer-simulated reality, which replicates the real world and simulates physical presence [16, 17]. It also enables the user to interact with the objects in the virtual world. The computer could accurately record the user’s performance for later evaluation. [17]

Virtual rehabilitation enables the integration of multiple exercises, gaming techniques and direct feedback [2, 7, 15]. As already shown in past studies, virtual reality is a powerful tool for balance and gait training in clinical rehabilitation [15, 18]. Patients were able to improve the number of repetitions, gait speed and endurance by using a VR based treadmill training [7, 15, 19]. Besides this patients also experienced higher motivation, better focus on the task [15, 18, 20], a greater attitude [18] and some user’s also a reduced pain during the application usage [21].

The considered studies focused on motivating participants by inducing positive emotions using visual, auditive and tactile feedback, within tasks, consisting of independent challenges. Thereby these tasks are not coherent to each other, and representing separated parts of the game, without extended storytelling and social responsibilities for the patients within the game.

Hypothesis

Based on this results the target of the thesis is the development of a virtual reality application that involves the patients by assigning them a responsible and feasible role within the story. The patients should feel comfortable and became curious through an interactive virtual environment. The content of the application should be a familiar environment like a desert, grassland or forest. Furthermore, it should motivate the patients by including a reward system for training achievements. Therefore, we hypothesize that patients get more motivated by including an advanced storytelling and assigning them a responsible role within the game.

Concept

Virtual Reality Rehabilitation focuses on motivating people during their rehabilitation process. Within these training programs, the motivation of patients are fostered by inducing positive feedback and rewarding them for absolving particular tasks. Several studies integrated components of gamification within their application, and found out, that gamified challenges both increases physical performance, motivation and personal attitude.

Gamification

Self-Determination Theory describes the basic psychological needs of human being. An application, that should motivate a person for playing a game and reaching a goal, has to address this needs by foster competence, relatedness and autonomy [29]. Therefore, the application includes a gamified rewarding system to foster autonomous motivation of people during their training session [35, 36].

The center of the application is an extended storytelling combined with virtual reality, gamification and a procedural generated environment. Within the storytelling the patient gets a responsible role to increase his social relatedness. Next to this narrative, the patient gets a personal companion, named Max. Max is a small dog who accompanies the patient during the training session. Max is running in front of the patient, gives positive feedback by displaying rewarding and informational messages, barking and jumping. Therefore the primary purpose for Max is to motivate the patient to walk further, explore the environment and complete the tasks.

Rewarding

The foundation of the application is a continuous rewarding system consists of progression loops which reward the patient for achieving a particular walking distance. As visualized in figure 5.5 the overall primary goal is subdivided into secondary goals consisting of reachable tasks. Every secondary goal and task represents a rewarding which the patient gets by walking.

The point of rewarding is visualized by a yellow star together with the rewarding element. If the patient reaches a certain point, the companion Max receives the star and express his happiness by visual and auditive feedback in the form of text messages, barking, walking around and jumping.

A rewarding is a composition of visual and auditive components. The first component consists of visual elements like stones, grass, bushes or trees. The patient gets this reward every time he reaches a point of rewarding. The second component consists of a suitable background audio sound, that changes according to the current state of the environment like a windy desert or a lively forest.

Environment

Due to different walking abilities, the rewarding interval should be dynamically adapted accordingly to the configured walking distance. Therefore the application consists of a procedurally generated environment. A percentage value specifies the point of each rewarding. In that way, the rewarding depends on a relative percentage amount rather than a fixed meter-based number. The application uses a low-poly stylized environment to reduce the complexity of models and thereby to enable a real time shadowing and lighting without significant impacts on performance.

Safety

During the VR-based treadmill training, patients are not able to see their position on the treadmill. Therefore the application has to include an advanced solution for safety. Since the supervisor calibrates the system at the beginning, the application can visualize the borders of the treadmill walking space by an outline on the ground. In combination with the feet representation, the patient is able to determine his position on the treadmill. The third component mostly increases the safety of the system by displaying the HTC Vive RGB-Camera image. If the patient looks downwards, he can see the real world within a rectangle in the middle of his field-of-view. Figure 5.7 visualizes the three safety systems.

Homecoming: Training Session

The application, developed within the master thesis, is named “Homecoming”. At the beginning of the training session the supervisor configures and calibrates the system. Depending on the current walking ability of the patient, the supervisor configures the scheduled walking distance within the configuration. Afterwards, the training sessions starts with an introduction part.

The treadmill training starts with the introduction inside an empty and lifeless desert. Before the main walking task begins, the patient gets an emotional sequenced introduction by his companion Max. Within this part, Max introduces himself and tells the patient that a storm destroyed the home from him and his friends. Furthermore, he asks the patient to help him to create a new home for him and his friends. After this introduction, the patient gets informed about the safety systems and were asked to walk forward for beginning the training session.

Within the walking task, the patient walks inside the environment until he reaches the primary goal. The patient achieves several secondary goals divided into tasks by walking in the environment. Figure visualizes the application after finishing the primary goal, reached by walking a certain distance.

Hardware Environment

The application is developed by using Microsoft Windows 10 and the Unreal Engine 4.18 [41]. The computer consists of i7-6700K processor, 16 GB DDR4-Ram and the Nvidia GeForce GTX 1080. The audio playback doesn’t require any specific headphones. The application could run both in virtual reality and On-Screen. There are no special display requirements for the On-Screen visualization. The virtual reality hardware environment consists of the HTC Vive [42] head-mounted display.

As already mentioned the HTC Vive Tracker [43] used to track the current walking speed of the user. Therefore the Tracker are attached to the feet of the user.

The treadmill Cardiostrong TR30 [44] will be used during the pre-study. Within the primary study, the rehabilitation hospital uses their own treadmill.

References

1. Howard, J., Trevick, S. & Younger, D. S. Epidemiology of multiple sclerosis. Neurologic clinics 34, 919–939 (2016) (cit. on p. 2).
2. Peruzzi, A., Zarbo, I. R., Cereatti, A., Della Croce, U. & Mirelman, A. An innovative training program based on virtual reality and treadmill: effects on gait of persons with multiple sclerosis. Disability and rehabilitation 39, 1557– 1563 (2017) (cit. on pp. 2 sq., 6).
3. Browne, P. et al. Atlas of Multiple Sclerosis 2013: A growing global problem with widespread inequity. Neurology 83, 1022–1024 (2014) (cit. on p. 2).
4. Federation, M. S. I. Atlas of MS 2013 (2013) (cit. on p. 2).
5. Kempen, J. C., Doorenbosch, C. A., Knol, D. L., de Groot, V. & Beckerman, H. Newly identified gait patterns in patients with multiple sclerosis may be related to push-off quality. Physical therapy 96, 1744–1752 (2016) (cit. on p. 2).
6. Motl, R. W., Goldman, M. D. & Benedict, R. H. Walking impairment in patients with multiple sclerosis: exercise training as a treatment option. Neu- ropsychiatric disease and treatment 6, 767 (2010) (cit. on p. 2).
7. Peruzzi, A., Cereatti, A., Della Croce, U. & Mirelman, A. Effects of a virtual reality and treadmill training on gait of subjects with multiple sclerosis: a pilot study. Multiple sclerosis and related disorders 5, 91–96 (2016) (cit. on pp. 2 sq., 6).
8. Kalron, A., Fonkatz, I., Frid, L., Baransi, H. & Achiron, A. The effect of balance training on postural control in people with multiple sclerosis using the CAREN virtual reality system: a pilot randomized controlled trial. Journal of neuroengineering and rehabilitation 13, 13 (2016) (cit. on p. 2).
9. Crenshaw, S., Royer, T., Richards, J. & Hudson, D. Gait variability in people with multiple sclerosis. Multiple Sclerosis Journal 12, 613–619 (2006) (cit. on p. 2).
10. Bethoux, F. & Bennett, S. Evaluating walking in patients with multiple sclero- sis: which assessment tools are useful in clinical practice? International journal of MS care 13, 4–14 (2011) (cit. on p. 2).
11. Socie, M. J. & Sosnoff, J. J. Gait variability and multiple sclerosis. Multiple sclerosis international 2013 (2013) (cit. on p. 2).
12. Baram, Y. & Miller, A. Virtual reality cues for improvement of gait in patients with multiple sclerosis. Neurology 66, 178–181 (2006) (cit. on p. 2).
13. Kersten, S. et al. A pilot study of an exercise-based patient education program in people with multiple sclerosis. Multiple sclerosis international 2014 (2014) (cit. on p. 2).
14. Swinnen, E. et al. Treadmill training in multiple sclerosis: can body weight support or robot assistance provide added value? A systematic review. Multiple sclerosis international 2012 (2012) (cit. on p. 2).
15. Papegaaij, S., Morang, F. & Steenbrink, F. Virtual and augmented reality based balance and gait training (2017) (cit. on pp. 3, 5).
16. Plummer, P. Gait and balance training using virtual reality is more effective for improving gait and balance ability after stroke than conventional training without virtual reality [synopsis]. Journal of physiotherapy 63, 114 (2017) (cit. on p. 3).
17. Tong, Z. Virtual Reality in Neurorehabilitation. Int J Neurorehabilitation 3, 2376–0281 (2016) (cit. on p. 3).
18. Calabrò, R. S. et al. Robotic gait training in multiple sclerosis rehabilitation: Can virtual reality make the difference? Findings from a randomized controlled trial. Journal of the neurological sciences 377, 25–30 (2017) (cit. on pp. 3, 5 sq.).
19. Pilutti, L. A. et al. Effects of 12 weeks of supported treadmill training on functional ability and quality of life in progressive multiple sclerosis: a pilot study. Archives of physical medicine and rehabilitation 92, 31–36 (2011) (cit. on p. 3).
20. Massetti, T. et al. Virtual reality in multiple sclerosis–A systematic review. Multiple sclerosis and related disorders 8, 107–112 (2016) (cit. on pp. 3, 5).
21. Villiger, M. et al. Virtual Reality–Augmented Neurorehabilitation Improves Motor Function and Reduces Neuropathic Pain in Patients With Incomplete Spinal Cord Injury. Neurorehabilitation and neural repair 27, 675–683 (2013) (cit. on p. 3).
22. Howard, M. C. A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Computers in Human Behavior 70, 317–327 (2017) (cit. on p. 5).
23. Brütsch, K. et al. Virtual reality for enhancement of robot-assisted gait training in children with neurological gait disorders. Journal of Rehabilitation Medicine 43, 493–499 (2011) (cit. on p. 5).
24. Bryanton, C. et al. Feasibility, motivation, and selective motor control: virtual reality compared to conventional home exercise in children with cerebral palsy. Cyberpsychology & behavior 9, 123–128 (2006) (cit. on p. 5).
25. Calabrò, R. S. et al. The role of virtual reality in improving motor performance as revealed by EEG: a randomized clinical trial. Journal of neuroengineering and rehabilitation 14, 53 (2017) (cit. on pp. 5 sq.).
26. Peruzzi, A., Cereatti, A., Mirelman, A. & Della Croce, U. Feasibility and acceptance of a virtual reality system for gait training of individuals with mul- tiple sclerosis. European International Journal of Science and Technology 2, 171–181 (2013) (cit. on p. 6).
27. De Rooij, I., G. L. van de Port, I. & Meijer, J.-W. Feasibility and Effectiveness of Virtual Reality Training on Balance and Gait Recovery Early after Stroke: A Pilot Study. 5, 418 (July 2017) (cit. on p. 6).
28. Kılıc, M. M., Muratlı, O. C. & Catal, C. Virtual reality based rehabilitation system for Parkinson and multiple sclerosis patients in Computer Science and Engineering (UBMK), 2017 International Conference on (2017), 328–331 (cit. on pp. 6 sq.).
29. Ryan, R. M. & Deci, E. L. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American psychologist 55, 68 (2000) (cit. on pp. 7 sqq.).
30. Sailer, M., Hense, J. U., Mayr, S. K. & Mandl, H. How gamification moti- vates: An experimental study of the effects of specific game design elements on psychological need satisfaction. Computers in Human Behavior 69, 371–380 (2017) (cit. on pp. 7–10).
31. Ryan, R. M. & Deci, E. L. Intrinsic and extrinsic motivations: Classic defi- nitions and new directions. Contemporary educational psychology 25, 54–67 (2000) (cit. on pp. 7 sq.).
32. Gagné, M. & Deci, E. L. Self-determination theory and work motivation. Journal of Organizational behavior 26, 331–362 (2005) (cit. on pp. 7 sq.).
33. Mekler, E. D., Brühlmann, F., Tuch, A. N. & Opwis, K. Towards understand- ing the effects of individual gamification elements on intrinsic motivation and performance. Computers in Human Behavior 71, 525–534 (2017) (cit. on p. 8).
34. Deci, E. L., Koestner, R. & Ryan, R. M. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin 125, 627 (1999) (cit. on p. 8).
35. Werbach, K. & Hunter, D. For the win: How game thinking can revolutionize your business (Wharton Digital Press, 2012) (cit. on pp. 9 sq.).
36. Sailer, M., Hense, J., Mandl, H. & Klevers, M. Psychological perspectives on motivation through gamification. IxD&A 19, 28–37 (2013) (cit. on p. 9).
37. Deterding, S., Dixon, D., Khaled, R. & Nacke, L. From game design elements to gamefulness: defining gamification in Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (2011), 9–15 (cit. on p. 9).
38. Deterding, S., Khaled, R., Nacke, L. E. & Dixon, D. Gamification: Toward a definition in CHI 2011 gamification workshop proceedings 12 (2011) (cit. on p. 9).
39. Werbach, K. (Re) defining gamification: A process approach in International conference on persuasive technology (2014), 266–272 (cit. on p. 9).
40. Werbach, K. & Hunter, D. The Gamification Toolkit: Dynamics, Mechanics, and Components for the Win (Wharton Digital Press, 2015) (cit. on p. 10).
41. Epic Games. Make Something Unreal Accessed: 2018-03-13. https://www. unrealengine.com (2017) (cit. on p. 21).
42. HTC Corporation. Discover Virtual Reality Beyond Imagination Accessed: 2018-03-09. https://www.vive.com/ (2017) (cit. on p. 21).
43. HTC Corporation. Vive Tracker Accessed: 2018-03-15. https://www.vive. com/de/vive-tracker/ (2017) (cit. on p. 21).
44. Sport-Tiedje. cardiostrong Laufband TR30 Accessed: 2018-03-14. https: //www.sport-tiedje.de/cardiostrong-laufband-tr30-cst-tr30-3 (2018) (cit. on p. 21).

Contact Persons at the University Würzburg