Philipp Krop, M.Sc.

Abstract: With recent developments in spatial computing, work contexts might shift to augmented reality. Embodied AI - virtual conversa- tional agents backed by AI systems - have the potential to enhance these contexts and open up more communication channels than just text. To support knowledge transfer from virtual agent research to the general populace, we developed My CoWorker ChatGPT - an interactive demo where employees can try out various embodied AIs in a virtual office or their own using augmented reality. We use state-of-the-art speech synthesis and body-scanning technology to create believable and trustworthy AI assistants. The demo was shown at multiple events throughout Germany, where it was well received and sparked fruitful conversations about the possibilities of embodied AI in work contexts.

Abstract: The presence of a conspecific can mitigate autonomic responses to aversive stimuli, an effect known as social buffering. Nowadays, social interactions are often virtual, yet virtual social buffering effects remain poorly understood. This work presents five studies that systematically test the conditions required for virtual social buffering. We assessed participants’ emotion ratings and skin conductance responses when they were presented with neutral or fear-inducing sounds alone or in the presence of a virtual character with a varying extent of human-like features (virtual female or male person, wooden puppet, point cloud). The characters were presented using the same social framing, i.e., had the same social meaning. Our results show a significant reduction in SCR responses to fear-inducing sounds in the presence of a virtual character, but only if it is embodied as a woman or a wooden puppet. Clarifying the role of the social frame, a control study showed no social buffering effects if the wooden puppet was presented without the social frame. Our results show that the characteristics of a virtual character significantly moderate the social buffering of fear responses. Our findings shed light on the nature of virtual social buffering effects and are relevant for developing virtual applications for clinical and societal interventions.

Abstract: Even though people imagine different embodiments when asked which AI they would like to work with, most studies investigate trust in AI systems without specific physical appearances. This study aims to close this gap by combining influencing factors of trust to analyze their impact on the perceived trustworthiness, warmth, and competence of an embodied AI. We recruited 68 par- ticipants who observed three co-working scenes with an embodied AI, presented as expert/novice (expertise), human/AI (humanness), or congruent/slightly incongruent to the environment (congruence). Our results show that the expertise condition had the largest im- pact on trust, acceptance, and perceived warmth and competence. When controlled for perceived competence, the humanness of the AI and the congruence of its embodiment to the environment also influence acceptance. The results show that besides expertise and the perceived competence of the AI, other design variables are rele- vant for successful human-AI interaction, especially when the AI is embodied.

Abstract: Machine learning is an essential aspect of modern life that many educational institutions incorporate into their curricula. Often, students struggle to grasp how neural networks learn. Teaching these concepts could be assisted with pedagogical agents and serious games, which both have proven helpful for complex topics like engineering. We present "Traversing the Pass," a serious game that utilizes a mentor-like agent to explain the underlying machine learning concepts and provides feedback. We optimized the agent’s design in a pre-study before evaluating its effectiveness compared to a text-only user interface with experts and students. Participants performed better in a second assessment two weeks later if they played the game using the agent. Although criticized as repetitive, the game created an understanding of basic machine learning concepts and achieved high flow values. Our results indicate that agents could be used to enhance the beneficial effects of serious games with improved knowledge retention.

Abstract: The administrative burden for physicians in the hospital can affect the quality of patient care. The Service Center Medical Informatics (SMI) of the University Hospital Würzburg developed and implemented the smartphone-based mobile application (MA) ukw.mobile1 that uses speech recognition for the point-of-care ordering of radiological examinations. The aim of this study was to examine the usability of the MA workflow for the point-of-care ordering of radiological examinations. All physicians at the Department of Trauma and Plastic Surgery at the University Hospital Würzburg, Germany, were asked to participate in a survey including the short version of the User Experience Questionnaire (UEQ-S) and the Unified Theory of Acceptance and Use of Technology (UTAUT). For the analysis of the different domains of user experience (overall attractiveness, pragmatic quality and hedonic quality), we used a two-sided dependent sample t-test. For the determinants of the acceptance model, we employed regression analysis. Twenty-one of 30 physicians (mean age 34\,$\pm$\,8 years, 62\% male) completed the questionnaire. Compared to the conventional desktop application (DA) workflow, the new MA workflow showed superior overall attractiveness (mean difference 2.15\,$\pm$\,1.33), pragmatic quality (mean difference 1.90\,$\pm$\,1.16), and hedonic quality (mean difference 2.41\,$\pm$\,1.62; all p\,$<$\,.001). The user acceptance measured by the UTAUT (mean 4.49\,$\pm$\,0.41; min. 1, max. 5) was also high. Performance expectancy (beta\,=\,0.57, p\,=\,.02) and effort expectancy (beta\,=\,0.36, p\,=\,.04) were identified as predictors of acceptance, the full predictive model explained 65.4\% of the variance. Point-of-care mHealth solutions using innovative technology such as speech-recognition seem to address the users' needs and to offer higher usability in comparison to conventional technology. Implementation of user-centered mHealth innovations might therefore help to facilitate physicians' daily work.

Abstract: We present the gamified AR learning environment ARES. During the gameplay, learners guide the little dragon called ”Ares” through dungeons. Using AR, ARES threedimensionally integrates the dungeons in the real-world. Each dungeon represents a coordinate system with obstacles that ultimately create mathematical exercises, e.g. rotating a door to let the dragon fly through it. Overcoming such a challenge requires a learner to spatially analyze the exercise and apply the fundamental mathematical principles. ARES displays a debriefing screen at the end of a dungeon to further support the learning process. In a preliminary qualitative user study, preservice and in-service teachers saw great potential in ARES for providing further practical and motivating exercises to deepen the knowledge in the classroom and at home.

Abstract: Management of radiology requests in larger clinical contexts is characterized by a complex and distributed workflow. In our partner hospital, representing many similar clinics, these processes often still rely on exchanging physical papers and forms, making patient or case data challenging to access. This often leads to phone calls with long waiting queues, which are time-inefficient and result in frequent interrupts. We report on a user-centered design approach based on Rapid Contextual Design with an additional focus group to optimize and iteratively develop a new workflow. Participants found our prototypes fast and intuitive, the design clean and consistent, relevant information easy to access, and the request process fast and easy. Due to the COVID pandemic, we switched to remote prototype testing, which yielded equally good feedback and increased the participation rate. In the end, we propose best practices for remote prototype testing in hospitals with complex and distributed workflows.

Abstract: Multimodal Interfaces (MMIs) combining speech and spatial input have the potential to elicit minimal cognitive load. Low cognitive load increases effectiveness as well as user satisfaction and is regarded as an important aspect of intuitive use. While this potential has been extensively theorized in the research community, experiments that provide supporting observations based on functional interfaces are still scarce. In particular, there is a lack of studies comparing the commonly used Unimodal Interfaces (UMIs) with theoretically superior synergistic MMI alternatives. Yet, these studies are an essential prerequisite for generalizing results, developing practice-oriented guidelines, and ultimately exploiting the potential of MMIs in a broader range of applications. This work contributes a novel observation towards the resolution of this shortcoming in the context of the following combination of applied interaction techniques, tasks, application domain, and technology: We present a comprehensive evaluation of a synergistic speech & touch MMI and a touch-only menu-based UMI (interaction techniques) for selection and system control tasks in a digital tabletop game (application domain) on an interactive surface (technology). Cognitive load, user experience, and intuitive use are evaluated, with the former being assessed by means of the dual-task paradigm. Our experiment shows that the implemented MMI causes significantly less cognitive load and is perceived significantly more usable and intuitive than the UMI. Based on our results, we derive recommendations for the interface design of digital tabletop games on interactive surfaces. Further, we argue that our results and design recommendations are suitable to be generalized to other application domains on interactive surfaces for selection and system control tasks.

Abstract: Requests for radiological examinations in large medical facilities are a distributed and complex process with potential health-related risks for patients. A user-centered qualitative analysis with contextual interviews uncovered nine core problems, which hinder work efficiency and patient care: (1) Difficulties to access patient data & requests, (2) the large number of phone calls, (3) restricted & abused access rights, (4) request status difficult to track, (5) paper notes used for patient data, (6) lack of assistance for data entry, (7) frustration through documentation, (8) IT-systems not self-explanatory, and (9) conflict between physicians and radiologists.Contextual interviews were found to be a well fitting method to analyze and understand this complex process with multiple user roles. This analysis showed that there is room for improvement in the underlying IT systems, workflows and infrastructure. Our data gave useful insight into solutions to these problems and how we can use technology to improve all aspects of the request management. We are currently addressing those issues with a user-centered design process to design and implement a mobile application, which we will present in future work.

Abstract: The presence of a partner can attenuate physiological fear responses, a phenomenon known as social buffering. However, not all individuals are equally sociable. Here we investigated whether social buffering of fear is shaped by sensitivity to social anxiety (social concern) and whether these effects are different in females and males. We collected skin conductance responses (SCRs) and affect ratings of female and male participants when they experienced aversive and neutral sounds alone (alone treatment) or in the presence of an unknown person of the same gender (social treatment). Individual differences in social concern were assessed based on a well-established questionnaire. Our results showed that social concern had a stronger effect on social buffering in females than in males. The lower females scored on social concern, the stronger the SCRs reduction in the social compared to the alone treatment. The effect of social concern on social buffering of fear in females disappeared if participants were paired with a virtual agent instead of a real person. Together, these results showed that social buffering of human fear is shaped by gender and social concern. In females, the presence of virtual agents can buffer fear, irrespective of individual differences in social concern. These findings specify factors that shape the social modulation of human fear, and thus might be relevant for the treatment of anxiety disorders.