Recent Publications

Abstract: This study explores how people interact in dyadic social eXtended Reality (XR), focusing on two main factors: the animation type of a conversation partner’s avatar and how immersed the user feels in the virtual environment. Specifically, we investigate how 1) idle behavior, 2) AI-generated gestures, and 3) motion-captured movements from a confederate (a controlled partner in the study) influence the quality of conversation and how that partner is perceived. We examined these effects in both symmetric interactions (where both participants use VR headsets and controllers) and asymmetric interactions (where one participant uses a desktop setup). We developed a social XR platform that supports asymmetric device configurations to provide varying levels of immersion. The platform also supports a modular avatar animation system providing idle behavior, real-time AI-generated co-speech gestures, and full-body motion capture. Using a 2×3 mixed design with 39 participants, we measured users’ sense of spatial presence, their perception of the confederate, and the overall conversation quality. Our results show that users who were more immersed felt a stronger sense of presence and viewed their partner as more human-like and believable. Surprisingly, however, the type of avatar animation did not significantly affect conversation quality or how the partner was perceived. Participants often reported focusing more on what was said rather than how the avatar moved.

Abstract: This systematic review investigates the current state of research on multimodal fusion methods, i.e., the joint analysis of multimodal inputs, for intentional, instruction-based human-computer interactions, focusing on the combination of speech and spatially expressive modalities such as gestures, touch, pen, and gaze. We examine 50 systems from a User-Centered Design perspective, categorizing them by modality combinations, fusion strategies, application domains and media, as well as reusability. Our findings highlight a predominance of descriptive late fusion methods, limited reusability, and a lack of standardized tool support, hampering rapid prototyping and broader applicability. We identify emerging trends in machine learning-based fusion and outline future research directions to advance reusable and user-centered multimodal systems.

Abstract:

Abstract:

Abstract: The use of VR for educational purposes provides the opportunity for integrating VR applications into assessments or graded examinations. Interacting with an VR environment requires specific human abilities, thus suggesting the existence of a VR competence. With regard to the emerging field of VR-based examinations, this VR competence might influence a candidate's final grade and hence should be taken into account. In this paper, we proposed and developed a VR competence assessment application. The application features eight individual challenges that are based on generic 3D interaction techniques. In a pilot study, we measured the performance of 18 users. By identifying significant correlations between VR competence score, previous VR experience and theoretically-grounded contributing human abilities and characteristics, we provide first evidence that our VR competence assessment is effective. In addition, we provide first data that a specific VR competence exists. Our analyses further revealed that mainly spatial ability but also immersive tendency correlated with VR competence scores. These insights not only allow educators and researchers to assess and potentially equalize the VR competence level of their subjects, but also help designers to provide effective tutorials for first-time VR users.

Abstract:

Abstract:

Abstract: