Evaluate Full Body Pose Estimation Based on Sparse Tracking

Open Human-Computer Interaction Computer Science Games Engineering Bachelor Master Project Thesis No Affiliation

Motivation and Related Work

Applications for social interaction in virtual worlds are one of the promising application areas of virtual reality (VR). In social virtual reality applications, users are represented by avatars, which in most cases are only a torso with hands and a head due to limited tracking capabilities, although full-body avatars may be superior (Pan and Steed, 2019). There are different approaches to animate the full body of the avatar, either based on full body motion capture (Anvari et al., 2022) or based on the user’s controllers and headset movements using inverse kinematics (Aristidou et al., 2018; Anvari et al., 2022) or deep learning (Jiang et al., 2022; Winkler et al., 2022; Ponton et al. 2022; Anvari et al., 2022; Anvari et al., 2023).

Goal

The goal of this project is to integrate different approaches, e.g., inverse kinematics (Final IK) and deep learning (Meta Movement SDK), for composing the body posture based user’s controller and headset movements. The approaches will then be evaluated in a user study to compare the quality in terms of plausibility and humanity.

Tasks

The topic will focus on the following tasks:

• Literatur and state-of-the-art research
• Integration of approaches for composing the body posture
• Conduction of a user study
• Evaluation and presentation of results

Prerequisits

• Experience with Unity game engine

Literature

1. Pan, Y., & Steed, A. (2019, November). Avatar type affects performance of cognitive tasks in virtual reality. In Proceedings of the 25th ACM symposium on virtual reality software and technology (pp. 1-4).
2. Anvari, T., & Park, K. (2022, October). 3D Human Body Pose Estimation in Virtual Reality: A survey. In 2022 13th International Conference on Information and Communication Technology Convergence (ICTC) (pp. 624-628). IEEE.
3. Aristidou, A., Lasenby, J., Chrysanthou, Y., & Shamir, A. (2018, September). Inverse kinematics techniques in computer graphics: A survey. In Computer graphics forum (Vol. 37, No. 6, pp. 35-58).
4. Jiang, J., Streli, P., Qiu, H., Fender, A., Laich, L., Snape, P., & Holz, C. (2022). Avatarposer: Articulated full-body pose tracking from sparse motion sensing. In European Conference on Computer Vision (pp. 443-460). Springer, Cham.
5. Winkler, A., Won, J., & Ye, Y. (2022). QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars. arXiv preprint arXiv:2209.09391.
6. Ponton, J. L., Yun, H., Andujar, C., & Pelechano, N. (2022). Combining Motion Matching and Orientation Prediction to Animate Avatars for Consumer-Grade VR Devices. arXiv preprint arXiv:2209.11478.
7. Anvari, T., Park, K., & Kim, G. (2023). Upper Body Pose Estimation Using Deep Learning for a Virtual Reality Avatar. Applied Sciences, 13(4), 2460.

Contact Persons at the University Würzburg