Investigating the Effects of Dynamic Control-Display Gain Manipulations on Virtual Hand Agency and Embodiment in Virtual Reality

Assigned Human-Computer Interaction. Bachelor Thesis No Affiliation

Thesis Title

• “Investigating the Effects of Dynamic Control-Display Gain Manipulations on Virtual Hand Agency and Embodiment in Virtual Reality”

Motivation/Goals

Virtual Reality (VR) enables users to experience a strong sense of embodiment, commonly defined through the components of body ownership, sense of agency, and self-location (Kilteni et al., 2012). Previous research has shown that these components can be influenced by factors such as avatar representation, visuomotor synchrony, and spatial transformations of the virtual body.

Recent work has demonstrated that manipulations of the control-display (C/D) ratio can be used to create pseudo-haptic effects such as perceived weight, resistance, or stiffness without requiring physical force feedback (Kim et al., 2022; Weiss et al., 2023). By altering the relationship between real and virtual movement, users may interpret these manipulations as properties of virtual objects while still maintaining a sense of control over their actions.

At the same time, research on hand redirection has shown that only a limited range of visuomotor distortions can remain unnoticed before users become aware of discrepancies between their physical and virtual movements (Zenner & Krüger, 2019). While previous work has investigated fixed gain manipulations and perceptual thresholds, little is known about how dynamically changing control-display mappings influences agency and embodiment.

The goal of this thesis is to investigate how fixed and dynamically changing C/D ratio manipulations affect agency, embodiment, and interaction performance in VR. In particular, the work aims to explore whether users adapt differently to changing visuomotor conditions than to stable distortions and how such manipulations influence the overall interaction experience.

Planned Methods and Concepts

System Design

The experiment will build upon the setup introduced by Fabian Unruh et al. (2024). Participants will be seated at a virtual table and interact using tracked virtual hands. To rule out effects caused by avatar appearance, no mirror or full-body avatar will be shown. Participants will only see natural-looking virtual hands throughout the experiment.

Participants will perform a hand-based interaction task that requires repeated movements within reachable space. At the beginning of each trial, the participant places their hand inside a predefined start area. A target then appears at a different location on the table. The participant must move their hand into the target area and remain there until a circular progress indicator is completed before returning to the start position. The next target appears after the previous trial has been completed. This task ensures continuous hand movement while allowing objective measurements of interaction performance.

In addition, a short exploratory interaction phase with simple virtual objects (e.g. balls, springs, or other manipulable objects) may be included to observe how the manipulations are experienced in more natural interactions.

The exact interaction task may be refined during implementation and pilot testing.

Control-Display Gain Manipulation

The experiment focuses on C/D ratio manipulation. Previous research has shown that control-display gain manipulation can be used to create pseudo-haptic effects such as perceived weight, resistance, and stiffness without requiring physical force feedback (Kim et al., 2022; Weiss et al., 2023). These studies show that users often interpret visuomotor distortions as properties of virtual objects rather than as tracking errors.

Research on hand redirection has further investigated how much gain manipulation users can tolerate before noticing discrepancies between their physical and virtual movements. Reported thresholds vary significantly across studies depending on the task and experimental setup. While some studies reported relatively large tolerated distortions, Zenner and Krüger (2019) employed a more conservative approach specifically aimed at estimating detection thresholds. Their results suggest that gain values between approximately 0.88 and 1.07 can often remain unnoticed, while larger deviations become increasingly detectable.

The gain values used in this experiment will intentionally exceed these thresholds in order to create noticeable visuomotor distortions. In contrast to previous work, which primarily investigated static gain values, this thesis focuses on dynamically changing gain manipulations and their influence on agency, embodiment, and interaction performance.

Experimental Conditions

The planned conditions are:

Baseline: Gain remains fixed at 1.0 throughout the interaction

Fixed Gain: A constant gain value below 1.0 (initially planned as 0.7)

Dynamic Gain: Gain changes continuously during interaction in three phases.

• Phase 1 - Gradual Adaptaion: Gain decreases gradually from 1.0 to 0.7

• Phase 2 - Strong Fluctuation: Gain fluctuates strongly between higher and lower values

  Example: 0.7 to 1.1 to 0.65 to 1.0 to 0.75

• Phase 3 - Stabilized Fluctuation: Gain remains centered around 0.7 and only fluctuates slightly

  Example: 0.7 to 0.65 to 0.7 to 0.75

Short questionnaire items will be presented between phases to record changes in user experience throughout the dynamic condition. Exact gain values will be determined through pilot testing

Hypotheses

• H1: Fixed gain manipulation will reduce perceived agency compared to the baseline condition.
• H2: Dynamic gain manipulation will reduce perceived agency more strongly than a fixed gain manipulation.
• H3: Dynamic gain manipulation will reduce embodiment ratings compared to baseline conditions.
• H4: Dynamic gain manipulation will negatively affect task performance, resulting in increased movement errors and/or longer completion times.
• H5: The strong fluctuation phase (Phase 2) will produce lower agency ratings than both Phase 1 and Phase 3.

Measurements

The primary measure will be the Virtual Embodiment Questionnaire (VEQ), which assesses embodiment using the subscales ownership, agency, and change (Roth & Latoschik, 2020). Secondary measures will consist of objective performance metrics recorded during the target-reaching task. These include target hit rate and movement completion time. These will provide an objective indication of how the different gain manipulations affect interaction performance.

Expected Contribution

Previous work has shown that C/D ratio manipulation can be used both for pseudo-haptic interaction and for hand redirection in VR. However, most existing studies focus on static gain manipulations and fixed gain values. This thesis extends previous research by investigating how users respond to dynamically changing control-display mappings. In particular, the work examines how different forms of dynamic gain manipulation influence agency, embodiment, and interaction performance over time.

Thesis Time Schedule

|WEEK | TASK | |——|——| | 1-2 | Literature review and refinement of methodology | | 3-5 | Implementation | | 6-8 | User study | | 9-10 | Evaluation of user study | | 11-12 | Writing the thesis |

References

Kilteni, K., Groten, R., & Slater, M. (2012). The Sense of Embodiment in Virtual Reality. Presence: Teleoperators and Virtual Environments, 21(4), 373–387. https://doi.org/10.1162/PRES_a_00124

Unruh, F., Lugrin, J.-L., & Latoschik, M. E. (2024). Out-Of-Virtual-Body Experiences: Virtual Disembodiment Effects on Time Perception in VR. Proceedings of the 30th ACM Symposium on Virtual Reality Software and Technology, VRST ’24, 1–11. https://doi.org/10.1145/3641825.3687717

Waltemate, T., Senna, I., Hülsmann, F., Rohde, M., Kopp, S., Ernst, M., & Botsch, M. (2016). The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality. Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology, VRST ’16, 27–35. https://doi.org/10.1145/2993369.2993381

Zenner, A., & Krüger, A. (2019). Estimating Detection Thresholds for Desktop-Scale Hand Redirection in Virtual Reality. 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 47–55. https://doi.org/10.1109/VR.2019.8798143

Kim, J., Kim, S., & Lee, J. (2022). The Effect of Multisensory Pseudo-Haptic Feedback on Perception of Virtual Weight. IEEE Access, 10, 5129–5140. https://doi.org/10.1109/ACCESS.2022.3140438

Weiss, Y., Villa, S., Schmidt, A., Mayer, S., & Müller, F. (2023). Using Pseudo-Stiffness to Enrich the Haptic Experience in Virtual Reality. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, CHI ’23, 1–15. https://doi.org/10.1145/3544548.3581223

Roth, D., & Latoschik, M. E. (2020). Construction of the Virtual Embodiment Questionnaire (VEQ). IEEE Transactions on Visualization and Computer Graphics, 26(12), 3546–3556. https://doi.org/10.1109/TVCG.2020.3023603

Contact Persons at the University Würzburg