Human-Computer Interaction

Enhancing Depth Cognition for Effective Mid-Air VR Sketching


This call for a thesis or project is open for the following modules:
If you are interested, please get in touch with the primary contact person listed below.

Background

Sketching is an intuitive, natural, and convenient way for people to explore ideas, capture experiences and convey information to others. Sketching is a cognitive tool that facilitates the deconstruction of problems, identification of critical details and evaluation of potential solutions. It has been shown to impact scientific analysis and innovation and has a positive effect on learning in all STEM disciplines. Room-scale virtual reality design systems empower users with tools to create sketches directly in mid-air, in all three dimensions using only simple hand gestures. Research has shown that people who create art with a VR design system experience a greater decrease in heart rate, negative affect and anxiety than those who create art with traditional tools. Yet despite the merits of immersive design systems, VR sketching systems are undervalued and underused.

Research has shown that while sketching in virtual reality, users spend more time changing the starting point of strokes, switching the observation position and correcting discrepancies of previously drawn strokes than on sketching new ideas. Precisely sketching intended strokes is a challenge in fully immersive virtual reality. Users must consider all three dimensions when positioning their hands in mid-air space to begin sketching. The addition of a third dimension contributes to a higher mental task load, and visual discrepancies in estimating the perceived depth lead to inaccurate and imprecise sketches. Furthermore, natural bio-mechanical constraints of the hand, wrist and arm cause fatigue in supported sketching, leading to additional inaccuracy and distortion of sketches. Consequently, sketches made with immersive design systems require more mental effort, are less accurate, and have inferior stroke aesthetic than those made with traditional two-dimensional tools. As a result, 3D mid-air sketches can be disorganized and difficult to understand, and lose their communicative effectiveness.

To exploit the benefits of sketching in virtual reality, we seek to increase the usability and effectiveness of fully immersive sketch systems. In this project we will explore the influence of sketch assistance methods on depth estimation, hand placement, spatial analysis, stroke accuracy and sketch aesthetic. We will develop a fully immersive sketching system that maps mid-air hand gestures to 3D mid-air ink strokes. We will develop methods to assist users with depth discrimination to improve sketch accuracy, as well as sketch beautification methods to automatically improve the aesthetic quality of sketches. In order to arrive at an effective tool, the software will continue to be developed following a user-centered engineering approach not only informed from requirements that become apparent when studying the scientific literature on sketch interfaces but also from requirements communicated by research experts in the field. This solution will then be evaluated in a user study to evaluate the effect of the sketch assistance method on stroke accuracy and aesthetic, system usability, cognitive load and other variables.

Tasks

The project will focus on the following tasks:

Prerequisites


Literature

  1. Armbrüster, C., Wolter, M., Kuhlen, T., Spijkers, W., & Fimm, B. (2008). Depth perception in virtual reality: distance estimations in peri-and extrapersonal space. Cyberpsychology & Behavior, 11(1), 9-15.
  2. Arora, R., Kazi, R. H., Anderson, F., Grossman, T., Singh, K., & Fitzmaurice, G. W. (2017, May). Experimental Evaluation of Sketching on Surfaces in VR. In CHI (Vol. 17, pp. 5643-5654).
  3. Arora, R., & Singh, K. (2021). Mid-air drawing of curves on 3D surfaces in virtual reality. ACM Transactions on Graphics (TOG), 40(3), 1-17.
  4. Barrera Machuca, M. D., Stuerzlinger, W., & Asente, P. (2019). The effect of spatial ability on immersive 3d drawing. In Proceedings of the 2019 on Creativity and Cognition (pp. 173-186).
  5. Barrera Machuca, M. D., Asente, P., Stuerzlinger, W., Lu, J., & Kim, B. (2018, October). Multiplanes: Assisted freehand vr sketching. In Proceedings of the 2018 ACM Symposium on Spatial User Interaction (pp. 36-47).
  6. Buxton, B. (2010). Sketching user experiences: getting the design right and the right design. Morgan Kaufmann.
  7. Keefe, D. F., & Laidlaw, D. H. Analysis of Performance in Precise 3D Curve Input Tasks in Virtual Reality.
  8. Lin, M., Chiang, I., Lee, L., & Lu, H. (2022). Examining performance of VR sketch modeling tool in personal sketches. DRS2022: Bilbao, 25.
  9. Oti, A., & Crilly, N. (2021). Immersive 3D sketching tools: Implications for visual thinking and communication. Computers & Graphics, 94, 111-123.
  10. Wiese, E., Israel, J. H., Meyer, A., & Bongartz, S. (2010, June). Investigating the learnability of immersive free-hand sketching. In Proceedings of the seventh sketch-based interfaces and modeling symposium (pp. 135-142).
  11. Yu, E., Arora, R., Stanko, T., Bærentzen, J. A., Singh, K., & Bousseau, A. (2021, May). Cassie: Curve and surface sketching in immersive environments. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1-14).


Contact Persons at the University Würzburg

Samantha Monty (Primary Contact Person)
Human-Computer Interaction, Universität Würzburg
samantha.monty@uni-wuerzburg.de

Prof. Dr. Marc Erich Latoschik
Human-Computer Interaction, Universität Würzburg
marc.latoschik@uni-wuerzburg.de

Legal Information