Joint Range of Motion and Immersive 3D Sketching Accuracy and Quality

Open Human-Computer Interaction Bachelor Project Thesis No Affiliation

Background

Immersive design systems such as Tilt Brush, Open Brush or Gravity Sketch allow users to create 3D persistent sketches in the air around themselves using only hand gestures. Precisely sketching intended strokes in mid-air in all three dimensions is demanding (Keefe et al. (2007), Wiese et al. (2010), Monty et al. (2024)). Bio-mechanical constraints of the hand, wrist and arm contribute to fatigue and increased inaccuracy in unsupported VR sketching (Arora et al. (2017)). Sketch quality diminishes, users are less efficient, and spend less time actually sketching than with traditional sketching tools (Yang & Lee, (2020), Oti & Crilly, (2021), Alex et al. (2021)).

Accuracy in targeting tasks requires coordintated action involving multiple upper body joints (Aizawa et al. (2013), Gates et al, (2016), Mehler et al. (2017)). Range of motion increases as one moves upward along the joints of the arm from the fingers, wrist, elbow to the shoulder. With this increased range of motion, comes an increase in the capacity for targeting errors. We wish to quantify this error rate in user study that restricts sketch size, and restricts the available range of motion in sketching.

Goal

The aim of this project is to explore the role of joint range of motion on the quality and accuracy of 3D mid-air sketches. We will work with a real-time, immersive freehand sketching system that maps mid-air hand gestures to 3D mid-air ink strokes. We will explore how confining strokes to a specific sketching space limits the range of motion of the shoulder, elbow, and wrist. We will conduct a user study to determine the influence of sketch size on influences sketch quality, user efficiency, and system usability. We wish to address the following broad research question: What is the influence of stroke curvature correction on the accuracy and quality of sketches, as well as on user efficiency and system usability.

Tasks

The project will focus on the following tasks:

• Acquaintance with the topic and the theoretical background
• Design of a comparative user study
• Execution of a comparative user study
• Evaluation and presentation of results

Prerequisites

• Game engine experience (Unity)

• Familiarity with VR applications

Literature

• Junya Aizawa, Tadashi Masuda, Kashitaro Hyodo, Tetsuya Jinno, Kazuyoshi Yagishita, Koji Nakamaru, Takayuki Koyama, and Sadao Morita. 2013. “Ranges of Active Joint Motion for the Shoulder, Elbow, and Wrist in Healthy Adults.” Disability and Rehabilitation 35 (16): 1342–49. https://doi.org/10.3109/09638288.2012.731133.
• Marylyn Alex, Burkhard C. Wünsche, and Danielle Lottridge. 2021. Virtual Reality Art-Making for Stroke Rehabilitation: Field study and technology probe. International Journal of Human-Computer Studies, 145, 102481. https://doi.org/10.1016/j.ijhcs.2020.102481.
• Rahul Arora, Rubaiat Habib Kazi, Fraser Anderson, Tovi Grossman, Karan Singh, and George Fitzmaurice. 2017. Experimental Evaluation of Sketching on Surfaces in VR. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ‘17). Association for Computing Machinery, New York, NY, USA, 5643–5654. https://doi.org/10.1145/3025453.3025474
• Deanna H. Gates, Lisa Smurr Walters, Jeffrey Cowley, Jason M. Wilken, Linda Resnik; Range of Motion Requirements for Upper-Limb Activities of Daily Living. Am J Occup Ther January/February 2016, Vol. 70(1), 7001350010p1–7001350010p10. https://doi.org/10.5014/ajot.2016.015487.
• Daniel Keefe, Robert Zeleznik, and David Laidlaw. 2007. Drawing on Air: Input Techniques for Controlled 3D Line Illustration. IEEE Transactions on Visualization and Computer Graphics 13, 5 (2007), 1067–1081. https://doi.org/10.1109/tvcg.2007.1060.
• David Marc Anton Mehler, Alexandra Reichenbach, Julius Klein, and Jörn Diedrichsen. (2017) Minimizing endpoint variability through reinforcement learning during reaching movements involving shoulder, elbow and wrist. PLOS ONE 12(7): e0180803. https://doi.org/10.1371/journal.pone.0180803
• Samantha Monty, Florian Kern, Marc Erich Latoschik, Analysis of Immersive Mid-Air Sketching Behavior, Sketch Quality, and User Experience in Design Ideation Tasks, In 23rd IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE Computer Society, 2024.
• Alfred Oti and Nathan Crilly. 2021. Immersive 3D Sketching Tools: Implications for Visual Thinking and Communication. Computers & Graphics, 94, 111-123.
• Eva Wiese, Johann Habakuk Israel, Andrea Meyer, and Sara Bongartz. 2010. Investigating the Learnability of Immersive Free-Hand Sketching. In Proceedings of the Seventh Sketch-Based Interfaces and Modeling Symposium (SBIM ‘10). Eurographics Association, Goslar, DEU, 135–142.

• Eun Kyoung Yang, and Jee Hyun Lee. 2020. Cognitive Impact of Virtual Reality Sketching on Designers’ Concept Generation. Digital Creativity, 31(2):82–97. https://doi.org/10.1080/14626268.2020.1726964.

Contact Persons at the University Würzburg