Usability Evaluation of the VIA-VR 3D Environment Editor

Assigned Human-Computer Interaction Bachelor Thesis No Affiliation

1. Introduction

In the field of healthcare, virtual reality (VR) scenarios are often utilized to simu- late real medical situations, procedures, or therapeutic experiences. These scenarios are designed to provide healthcare professionals with a safe and controlled space for training, education, and patient interventions. For example, it allows psychother- apists to create threat scenarios as a form of exposure therapy (Rothbaum et al., 1997), or provide future doctors with opportunities to practice various medical pro- cedures, such as cardiac catheterization techniques. (Bettati et al., 2021).

Despite the potential benefits of VR in healthcare, creating and modifying VR sce- narios typically requires technical expertise. This can be a significant barrier for medical professionals who may not have the skills or resources to adapt or develop VR content on their own. The VIA-VR software platform offers a solution as a VR authoring tool for modeling VR experiences for medical training, prevention, ther- apy and rehabilitation. It includes features such as editing existing environments, drag-and-drop assets from various repositories, composition tools, contextual asset recommendations, and exporting the environment to a VR scene. This editor is de- signed to allow medical professionals, to create their own VR environments without prior technical knowledge.

In a former study Halbig et al., 2022 did research on the perceived usefulness and the ease of use of such a platform, as well as the attitude and ethical concerns the users might have. The feedback of that study was rather positive. However, the perceived usefulness was tied to the actual ease of use of the platform and sufficient support for learning and working with the platform.

These aspects of ease of use and the availability of sufficient support have been of great importance in the development of the VIA-VR Editor. Especially in the case of a software platform intended for people without a technical background, usability becomes a crucial factor. Therefore, it is essential to meet the requirements and challenges associated with creating an intuitive and user-friendly interface. Further- more, considering the diverse user population of healthcare professionals who can benefit from the VIA-VR Editor, including different backgrounds and knowledge levels, is essential.

By conducting an explorative user study, we aim to understand how medical profes- sionals perceive the usability of the current editor prototype, in particular the ease of use of this platform. Moreover, we want to get feedback on the tutorial, general difficulties and if they can imagine using such a program in the health sector. To sum up, the main research question is what is the perceived ease of use of the current prototype version of the VIA-VR Editor for medical professionals without special technical knowledge.

It is important to note that while this study focuses on usability evaluation, it is limited to specific aspects of the current version of the VIA-VR Editor. Our goal is to gain insight into the usability, difficulties and suggestions for improvement. By doing so, we hope to contribute to the optimization of the platform and hopefully improve its usability for healthcare professionals without special technical knowledge.

2. Methodology

The study will be conducted with participants from various healthcare sectors, in- cluding the therapy sector. This group will consist of individuals such as psychother- apists. Additionally, participants from the rehabilitation sector, such as physiother- apists and professionals working in rehabilitation centers, will be involved. The education sector will also be represented, with participants including first aid in- structors, paramedics, nurses, and doctors. People from the prevention sector, such as addiction counselors and similar professionals, will also be included. In addition to the medical professionals there will be a control group with Mensch-Computer- Systeme and Human-Computer-Interaction students. This comparison can show if a rather technical background makes a significant difference for the usability of the VIA-VR Editor.

Since all parts of the editor are in English, participants should have a basic knowl- edge of English. However, translations can be provided if needed. The study will be audio recorded for documentation purposes. Consent and information about the study will be given at the beginning, as well as a survey that includes demographic questions and questions about technical knowledge, such as how often participants use computers. The study will then start with two 1-click tests (Bailey & Wolfson, 2013). This means they are asked where they would intuitively click to achieve a certain behaviour. For example, where they would click or what they would do to increase an object. This is a way of measuring the effectiveness of a program and shows whether certain elements are expressive enough for people to notice the effect immediately. After these tests, the procedure will include a click-through tutorial and about two minutes of self-exploration of the program. Participants’ are asked to think aloud and their questions and problems should be documented.

After this first exploration the participants are given two tasks, where they are asked to recreate a scene which includes a medical room and where they have to position and scale a medicinal product. To measure information about the effi- ciency of the program, the amount of help needed, excluding language problems, is counted. This shows how quickly and easily the goal of the task can be achieved. As an additional measure of satisfaction, positive and negative comments can be noted. After both tasks, perceived difficulty can be measured with the one-item workload questionnaire, the SEA scale (Zijlstra, 1993). After all tasks have been completed, participants are asked to complete the QUESI (Naumann & Hurtienne, n.d.) and the System Usability Scale (SUS) (Brooke et al., 1996). The 14 items of the QUESI measure perceived effectiveness, mental efficiency and satisfaction. The SUS contains 10 items and is also an estimation measure of usability and customer satisfaction. It also asks about the likelihood of using such a system or whether support from a technical person would be needed. After this part, a short interview should provide further clarification on problems with the editor and suggestions for improvement. It will contain about three to five questions oriented on the aspects of the Technology Acceptance Model (TAM) like perceived ease of use and perceived usefulness.

This formative evaluation study should give more clarification whether a techni- cal background significantly influences the usability of the editor. In particular, it focuses on the perceived usability of the VIA-VR Editor and aims to gain insight into existing difficulties and general usability problems, as well as wishes and sug- gestions from healthcare professionals on how to improve the learning of this tool. By gathering this information, the study aims to provide valuable insights for the improvement of the Editor’s usability.

3. Work Schedule

Research needs to be done on the ideal study design and how best to test the editor, as well as getting to know the editor in order to create appropriate tasks for participants. Once the study has been designed, the first participants recruited and a pilot study conducted, the study itself can be conducted. Afterwards, the results need to be evaluated and interpreted to complete the bachelor’s thesis.

Bibliography

1. Bailey, B., & Wolfson, C. (2013). Firstclick usability testing. Web Usability (cit. on p. 4).
2. Bettati, P., Dormer, J. D., Young, J., Shahedi, M., & Fei, B. (2021). Virtual reality assisted cardiac catheterization. Medical Imaging 2021: Image-Guided Procedures, Robotic Interventions, and Modeling , 11598 , 621–628 (cit. on p. 3).
3. Brooke, J., et al. (1996). Sus-a quick and dirty usability scale. Usability evaluation in industry , 189 (194), 4–7 (cit. on p. 5).
4. Halbig, A., Babu, S. K., Gatter, S., Latoschik, M. E., Brukamp, K., & von Mammen, S. (2022). Opportunities and challenges of virtual reality in healthcare–a domain experts inquiry. Frontiers in Virtual Reality , 3 , 837616 (cit. on p. 3).
5. Naumann, A., & Hurtienne, J. (n.d.). Benchmarks for intuitive interaction with mobile devices. Proceedings of the 12th international conference on Human computer interaction with mobile devices and services , 401–402 (cit. on p. 5).
6. Rothbaum, B. O., Hodges, L., & Kooper, R. (1997). Virtual reality exposure therapy. Journal of Psychotherapy Practice & Research (cit. on p. 3).
7. Zijlstra, F. (1993). Efficiency in work behaviour: A design approach for modern tools(ph. d. thesis) (cit. on p. 5).

Contact Persons at the University Würzburg