Quality of Tracking Systems for Embodied Virtual Reality

Closed Human-Computer Interaction Master Thesis No Affiliation

1. Motivation

Embodiment, in cognitive science and philosophy of mind, describes the importance of sensorimotor skills for general intelligence and acquisition of knowledge, as well as the role of the body in shaping the mind and the subjective experience of ‘having’ and using a body (Blanke and Metzinger, 2009, Bickhard, 2008). Especially in fully immersive virtual reality the representation of a full-body avatar is important because, unlike as in cave-based virtual environments, the user is not able to see his own body while wearing a Head Mounted Display (HMD). The Illusion of Virtual Body Ownership (IVBO) describes the effect, that the users in a virtual environment feel a virtual body or its parts to be their own (Lugrin, Latt, and Latoschik, 2015).

While Virtual Reality is not only used in research areas and large companies, but also by a large number of people privately and in their leisure time, user motion tracking plays an important role here, because social and immersive virtual reality applications intend to offer the possibility for an embodied experience.

For the purpose of displaying the body movements in the virtual environment and thus create a sense of embodiment, it requires the use of body tracking (Spanlang et al., 2014). Tracking the body movements can be achieved by different systems, which can use e.g. a marker-based or markerless solution (Spanlang et al., 2014), which are the most common used systems in research papers in the last few years regarding to the survey of Caserman, Garcia-Agundez, and Goebel (2019). Since some marker-based solutions, like the one from Optitrack, is not intended to be purchased or used by the average consumer, there is a need for consumer-suitable or so called “low cost” systems. However, these systems should also be able to provide the best possible results to support, among other things, gaming experience and user comfort.

As each system offers its advantages and disadvantages a direct comparison of different systems could provide useful information for investigations on e.g. body ownership and presence. It can be assumed that a more precise and natural representation of the body movements are leading to a higher sense of embodiment and immersion, but how the user comfort and different tasks influence the embodiment has to be investigated additionally.

These findings lead to the motivation to compare two “low cost” systems, one marker-based and one markerless, with a “high-cost” marker-based system.

The main research questions are:

• Can “low cost” systems be suitable substitutes for full-body tracking in immersive virtual environments compared with a “high cost” system taking technical and psychological measurements into account?

• How strong do the objective measures (latency, accuracy, divergences) correlate with the subjective measures (system preference, usability, embodiment)?

• How well must a system perform to gain enough quality for embodiment?

2. Related Work

The experiment on the rubber hand illusion (RHI) showed that a seen artificial hand can be perceived as one’s own hand, if the artificial one and the not-seen corresponding real one are exposed to synchronous tactile stimulation (Botvinick et al., 1998). The Sense of Embodiment refers to all sensations which are developed by the sense of “being inside, having, and controlling a body” (Kilteni et al., 2012, p. 374f).

A lot of immersive virtual environments use some kind of motion tracking to be able to represent the movements of the users in virtual reality. In Roth et al. (2016) a motion capture system of OptiTrack was used for the body-tracking and a full body marker set was compared with a reduced rigid body marker set, which was supported by inverse kinematics. They make use of a marker-based motion capture system in order to have a good tracking quality with low latency and in addition at the inverse kinematics part a higher user-comfort.

As an alternative marker-based solution the Vive and additional trackers can be used for body tracking (Caserman, Garcia-Agundez, Konrad, et al., 2019) as well as an Inverse Kinematics approach (Roth et al., 2016), justified by a reasonable accuracy and low latency (Caserman, Garcia-Agundez, Konrad, et al., 2019) and a likewise cheaper option as the OptiTrack MoCap system.

Caserman, Garcia-Agundez, and Goebel (2019) mentioned that another common used solution for motion tracking in scientific papers of the last few years is the usage of markerless systems, which most frequently made use of a Kinect sensor. Although to a reported low framerate, researchers often use this sensor due to the benefit for developing, e.g., exergames without the requirement of attaching additional sensors on the user’s body. Capece, Erra, and Romaniello (2018) describe the use of a Microsoft Kinect v2 as a low-cost full body tracking system. They took this depth camera for developing a mesh painting system application in VR which uses body tracking to interact in the virtual environment. In the survey of Caserman, Garcia-Agundez, and Goebel (2019) just former versions of the kinect sensors (Kinect, Kinect v1 and Kinect v2) are analyzed, which leads to the motivation to use the most current version of the kinect, called Azure KinectDK.

3. Approach

The goal of this master thesis is to implement different tracking systems into an immersive virtual reality application and to gain insights of the impact of technical quality aspects on user experience and usability with a user study.

This master thesis is based on the IVBO Project of the HCI chair of the University of Würzburg (Roth, Lugrin, Latoschik, and Huber, 2017). The IVBO application was developed for the research of immersion and embodiment and includes full-body tracking, which receives motion capture data from the Optitrack Motive System and transfers it to an avatar (Roth, Lugrin, Latoschik, and Huber, 2017).

Like pointed out partly in the described related works, each system used in this thesis has advantages and disadvantages and the previous findings lead to the motivation to compare “low cost” with “high cost” tracking solutions in a user study. These “low-cost” solutions are the use of Inverse Kinematics, with the Vive Pro Eye and five Vive Trackers, and the capture by the Body Tracking SDK of the Azure Kinect Developer Kit (DK) and the “high cost” opponent is the Optitrack MoCap system with the software platform Motive.

In the master thesis different ground truths are used for the measurements, since the Optitrack system provides more suitable results for the calculation of the joint angles due to the full-body tracking and the Vive system can provide a reference point due to an additional tracker.

In the user study the participants will do simple movement tasks and should observe themselves in a virtual mirror.

3.1 Goals

The previous findings lead to the following goals that should be achieved in the master thesis:

• Implementation of additional full-body tracking systems in Unity
  • the system with additional Vive trackers accompanied with Inverse Kinematics (IK)
  • the movement data from the Azure Kinect rigged on an avatar
  • implementation of the additional systems in the IVBO Project
• Comparative analysis of Kinect, Vive and Optitrack System both objectively (performance, latency, accuracy measurements) and subjectively (user study)

3.2 Variables

Independent Variable:

• Coverage of System (Optitrack vs. Vive vs. Kinect)

Dependent Variables:

• Divergence of end-effectors
• Divergence of joint angles
• Embodiment
• System Preferences
• Usability

3.3 User Study

The study is a counterbalanced within-subjects design (Goodwin, 2010), therefore every participant will test every body-tracking system, whereas the order of the systems is randomized. Questionnaires will be given to the participants at the beginning, after each system and at the end of the user study.

As described in the goals the evaluation consists of an objective and a subjective part. Each part will be described below as well as the structure and procedure of the evaluation study itself.

The objective measurements of the study are described by the following methods, which will be examined in order to carry out the technical evaluation: videotape-based measuring of the latency (Friston and Steed, 2014), overlapping objects or intersection over union to measure the accuracy (Luckett, Key, Newsome, and Jones, 2019), system performance with each tracking solution and additional technical comparisons of the systems.

For the subjective analysis of the systems questionnaires will be taken to measure, e.g., the sense of embodiment with the Virtual Embodiment Questionnaire (Roth and Latoschik, 2019), the feeling of presence with the SPES (Hartmann et al., 2015) and the simulation sickness with the SSQ (Kennedy et al., 1993).

Additionally to these questionnaires the participants are asked about their system preferences and get some open questions.

Sequence:

1. Information about user study
2. Pre-Questionnaires
3. Time in VR (randomized order of systems)
   1. first system + questionnaires
   2. second system + questionnaires
   3. third system + questionnaires
4. Post-Questionnaires

4. References

Bickhard, M. H. (2008). 2—Is Embodiment Necessary? In P. Calvo & A. Gomila (Hrsg.), Handbook of Cognitive Science (S. 27–40). Elsevier. https://doi.org/10.1016/B978-0-08-046616-3.00002-5

Blanke, O. & Metzinger, T. (2009). Full-body illusions and minimal phenomenal selfhood. Trends in Cognitive Sciences, 13(1), 7–13.

Botvinick, M. & Cohen, J. (1998). Rubber hands ‘feel’ touch that eyes see. Nature, 391(6669), 756–756. Number: 6669 Publisher: Nature Publishing Group.

Capece, N., Erra, U., & Romaniello, G. (2018). A Low-Cost Full Body Tracking System in Virtual Reality Based on Microsoft Kinect. In L. De Paolis & P. Bourdot, Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018. (S. 623–635). Springer. https://doi.org/10.1007/978-3-319-95282-6_44

Caserman, P., Garcia-Agundez, A., & Goebel, S. (2019). A Survey of Full-Body Motion Reconstruction in Immersive Virtual Reality Applications. IEEE Transactions on Visualization and Computer Graphics, 1–1. https://doi.org/10.1109/TVCG.2019.2912607

Caserman, P., Garcia-Agundez, A., Konrad, R., Göbel, S., & Steinmetz, R. (2019). Real-time body tracking in virtual reality using a Vive tracker. Virtual Reality, 23(2), 155–168. https://doi.org/10.1007/s10055-018-0374-z

Friston, S., & Steed, A. (2014). Measuring Latency in Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, 20(4), 616–625. https://doi.org/10.1109/TVCG.2014.30

Goodwin, C. J. (2010). Research in psychology: Methods and design (6th ed). Wiley.

Hartmann, T., Wirth, W., Schramm, H., Klimmt, C., Vorderer, P., Gysbers, A., Böcking, S., Ravaja, N., Laarni, J., Saari, T., Gouveia, F., & Sacau, A. (2015). The Spatial Presence Experience Scale (SPES). Journal of Media Psychology: Theories, Methods, and Applications, 1, 1–15. https://doi.org/10.1027/1864-1105/a000137

Kennedy, R. S., Lane, N. E., Berbaum, K. S., & Lilienthal, M. G. (1993). Simulator sickness questionnaire: An enhanced method for quantifying simulator sickness. The international journal of aviation psychology, 3(3), 203-220.

Kilteni, K., Groten, R., & Slater, M. (2012). The Sense of Embodiment in Virtual Reality. Presence Teleoperators & Virtual Environments, 21. https://doi.org/10.1162/PRES_a_00124

Luckett, E., Key, T., Newsome, N., & Jones, J. (2019). Metrics for the Evaluation of Tracking Systems for Virtual Environments.

Lugrin, J.-L., Latt, J., & Latoschik, M. E. (2015). Avatar anthropomorphism and illusion of body ownership in VR. 2015 IEEE Virtual Reality (VR), 229–230. https://doi.org/10.1109/VR.2015.7223379

Roth, D., & Latoschik, M. E. (2019). Construction of a Validated Virtual Embodiment Questionnaire. arXiv:1911.10176 [cs]. http://arxiv.org/abs/1911.10176

Roth, D., Lugrin, J.-L., Büser, J., Bente, G., Fuhrmann, A., & Latoschik, M. E. (2016). A simplified inverse kinematic approach for embodied VR applications. 2016 IEEE Virtual Reality (VR), 275–276. https://doi.org/10.1109/VR.2016.7504760

Roth, D., Lugrin, J.-L., Latoschik, M. E., & Huber, S. (2017). Alpha IVBO - Construction of a Scale to Measure the Illusion of Virtual Body Ownership. Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA ’17, 2875–2883. https://doi.org/10.1145/3027063.3053272

Roth, D., Stauffert, J.-P., & Latoschik, M. E. (2019). Avatar Embodiment, Behavior Replication, and Kinematics in Virtual Reality. In VR Developer Gems. https://doi.org/10.1201/b21598-17

Spanlang, B., Normand, J.-M., Borland, D., Kilteni, K., Giannopoulos, E., Pomés, A., González-Franco, M., Perez-Marcos, D., Arroyo-Palacios, J., Muncunill, X. N., & Slater, M. (2014). How to Build an Embodiment Lab: Achieving Body Representation Illusions in Virtual Reality. Frontiers in Robotics and AI, 1. https://doi.org/10.3389/frobt.2014.00009

Contact Persons at the University Würzburg