Recent Publications

Abstract:

Abstract: Immersive 3D mid-air sketching systems liberate users from the confines of traditional 2D sketching canvases. However, complications from perceptual challenges in Virtual Reality (VR), combined with the ergonomic and cognitive challenges of sketching in all three dimensions in mid-air lower the accuracy and aesthetic quality of 3D sketches. This paper explores how color-to-depth and opacity cues support users to create and perceive freehand, 3D strokes in room-scale sketching, unlocking a full 360° of freedom for creation. We implemented three graphic depth shader cues modifying the (1) alpha, (2) hue, and (3) value levels of a single color to dynamically adjust the color and transparency of meshes relative to their depth from the user. We investigated how these depth cues influence sketch efficiency, sketch quality, and total sketch experience with 24 participants in a comparative, counterbalanced, 4 x 1 within-subjects user study. First, with our graphic depth shader cues we could successfully transfer results of prior research in seated sketching tasks to room-scale scenarios. Our color-to-depth cues improved the similarity of sketches to target models. This highlights the usefulness of the color-to-depth approach even for the increased range of motion and depth in room-scale sketching. Second, our shaders assisted participants to complete tasks faster, spend a greater percentage of task time sketching, reduced the feeling of mental tiredness and improved the feeling of sketch efficiency in room-scale sketching. We discuss these findings and share our insights and conclusions to advance the research on improving spatial cognition in immersive sketching systems.

Abstract:

Abstract: Virtual reality (VR) exercise applications are promising tools, e.g., for at-home training and rehabilitation. However, existing applications vary significantly in key design choices such as environments, embodiment, and virtual coaching, making it difficult to derive clear design guidelines. A prominent design choice is the use of embodied virtual coaches, which guide user interaction and provide feedback. In a user study with 76 participants, we investigated how different levels of performance specificity in feedback from an embodied virtual coach affect intermediate factors, such as VR experience, motivation, and coach perception. Participants performed lower-body movement exercises, i.e., Leg Raises and Knee Extensions, commonly used in knee rehabilitation. We found that highly performance-specific feedback led to higher scores compared to medium specificity for perceived realism, as well as the anthropomorphism and sympathy of the virtual coach, but did not affect motivation. Based on our findings, we propose the design suggestion to include precise, performance-specific details when creating feedback for a virtual coach. We observed a descriptive pattern of higher scores in the low specificity condition compared to the medium condition on most measures, which raises the possibility that less specific feedback may, in some cases, be perceived more positively than moderately specific feedback. These findings provide valuable insights into how design choices impact relevant intermediate factors that are crucial for maximizing both workout effectiveness and the quality of the virtual coaching experience.

Abstract: This study explores how people interact in dyadic social eXtended Reality (XR), focusing on two main factors: the animation type of a conversation partner’s avatar and how immersed the user feels in the virtual environment. Specifically, we investigate how 1) idle behavior, 2) AI-generated gestures, and 3) motion-captured movements from a confederate (a controlled partner in the study) influence the quality of conversation and how that partner is perceived. We examined these effects in both symmetric interactions (where both participants use VR headsets and controllers) and asymmetric interactions (where one participant uses a desktop setup). We developed a social XR platform that supports asymmetric device configurations to provide varying levels of immersion. The platform also supports a modular avatar animation system providing idle behavior, real-time AI-generated co-speech gestures, and full-body motion capture. Using a 2×3 mixed design with 39 participants, we measured users’ sense of spatial presence, their perception of the confederate, and the overall conversation quality. Our results show that users who were more immersed felt a stronger sense of presence and viewed their partner as more human-like and believable. Surprisingly, however, the type of avatar animation did not significantly affect conversation quality or how the partner was perceived. Participants often reported focusing more on what was said rather than how the avatar moved.

Abstract: This systematic review investigates the current state of research on multimodal fusion methods, i.e., the joint analysis of multimodal inputs, for intentional, instruction-based human-computer interactions, focusing on the combination of speech and spatially expressive modalities such as gestures, touch, pen, and gaze. We examine 50 systems from a User-Centered Design perspective, categorizing them by modality combinations, fusion strategies, application domains and media, as well as reusability. Our findings highlight a predominance of descriptive late fusion methods, limited reusability, and a lack of standardized tool support, hampering rapid prototyping and broader applicability. We identify emerging trends in machine learning-based fusion and outline future research directions to advance reusable and user-centered multimodal systems.

Abstract:

Abstract: