The Influence of Embodied Artificial Intelligence Presence on Phishing E-mail Recognition Performance in a Work Environment

Assigned Human-Computer Interaction Master Thesis No Affiliation

Motivation and Potential

Recent technological advances in artificial intelligence (AI) and augmented reality (AR) enable the development of embodied AI agents that can be integrated in real environments such as the workplace and promise intuitive, human-like interaction. The current study aims to find out whether agents using these current technologies may also introduce a side-effect that can be found in human social contexts called social facilitation. Social facilitation describes improved performance of simple or already learned tasks when an audience is present, whereas for hard or yet to learn tasks the opposite effect applies [1].

One possible application of AI-based systems is the workplace, where it can provide support in administrative, service-related and various other work environments. Research in the work environment can help to better adjust these systems to the needs of users in this environment and mitigate risks in later steps of the development [2]. A task prevalent in many workplaces is workers checking their email inboxes. From a German perspective, employees on average receive 26 emails per day [3]. In 2023, cyber-attacks were estimated to cause 148 billion € in damages, with 31% of those attacks being related to phishing. As such, the task of recognizing malicious emails is relevant for many businesses and email usage is widespread. Another argument for this task is a realistic possibility of actually co-operating with an AI agent during everyday processing of received emails.

Combining this motivation for embodied AI agents with effects from human social contexts and using email recognition as a representative work-related task proposes the following research question: How does the presence of an embodied AI agent displayed by an AR headset influence human detection of phishing emails in the work context?

Related Work

Agents

Socially Interactive Agents (SIAs) are a means to provide interfaces similar to human-human interaction that can be “more natural to human interactants by equipping the interface with a body that interacts multi-modally […]” [4]. Besides physical human-like agents, such as robots, SIAs can also be virtual by utilizing technologies such as AR, which enhances the real world with virtual objects [5]. In order for SIAs to offer such human-like ways of interaction, disciplines like natural language understanding (NLU) need to be integrated [6]. This is one example in which SIAs may facilitate AI, as recent developments such as neural networks based on the Transformer architecture [7] like OpenAI’s ChatGPT 4 [8] can offer NLU that enables interactions more akin to human speech.

Humans may apply stereotypes from human-human interaction to human-computer interaction, such as gender stereotypes when the computer uses respective male or female voices and treat them as “social actors” [9]. Even without human-like qualities or even humanoid appearances, users may apply certain social behaviours such as politeness [10] during their interaction with the system. Social facilitation and inhibition effects have also been found using embodied voice assistants. In Liu and Pu [11], participants’ response times during easy math tasks increased in the presence of a smart speaker, while they decreased for harder tasks. This influence on response times was similar to a condition where a human assistant was present.

Social Facilitation and Agents

Existing studies found social facilitation or social inhibition effects using virtual agents. Zanbaka et al. [12] had participants solve mathematical tasks in the presence of a virtual agent. Depending on the condition, the agent was either displayed using a head-mounted device (HMD) or projected onto a screen. It is unclear if and how the agent was framed. The agent’s behavior however was deliberately human, including audible noises like yawning and coughing that participants commented on post-experiment. At least one participant interpreted this coughing as a sign of impatience, which might point towards an unwanted source of perceived evaluation.

A study by Park and Catrambone [13] used anagram, maze and math-related tasks with a virtual agent being displayed on a computer monitor. The agent was framed as being AI-operated and “present to learn more about the tasks”. The study found significant differences between alone and virtual presence conditions for all tasks that suggest social facilitation and inhibition effects by virtual agents framed as AI.

Social Facilitation in AR

More recently, Miller et al. [14] measured task performance “in the presence of embodied agents” using a Microsoft HoloLens and an anagram task. The study features a 2x2 within-subjects design requiring participants to solve either easy or hard anagrams in a social or alone condition. Miller et al. [14] found large effect sizes [15] of d = 0.96 for facilitation of task performance in the social condition for easy anagrams and d = 0.83 for inhibition of task performance regarding hard anagrams respectively. The agent was introduced as a research aide and had a human-like appearance, including an idling animation and jaw movement synchronized to recorded audio.

In summary, related studies found effects of virtual agents facilitating or inhibiting task performance using different display media and framing of the agent. It is unclear however, if these findings could be transferred to more recent technologies and whether they could be replicated in a work environment. To close this gap, this study will consider these aspects. First, the agent will be framed as being AI-operated, preventing potential assumptions of human operation. Accordingly, a questionnaire regarding AI literacy will also be included. Second, instead of optical see-through (OST) AR, video see-through (VST) AR will be used. Finally, both sample and task will differ from typical academical background, by using a potentially more diverse sample of office workers with different educational backgrounds and from various age groups as well as using a work-related task as mentioned before.

Based on the findings of Miller et al. [14], which suggest that the presence of a virtual agent can facilitate or inhibit task performance, the following hypotheses are proposed:

The presence of an embodied AI agent will lead to …

• H1a: faster classification of easy emails
• H1b: more accurate classification of easy emails
• H2a: slower classification of hard emails
• H2a: less accurate classification of hard emails

… by the participants when compared to an alone condition.

In Miller et al. [14], the agent was matched to the participant’s sex. A study by Liu et al. [16], which did not use virtual agents but real humans, also investigated effects of mixed-sex social conditions. In both experiments conducted by the authors, social facilitation effects were further increased when at least one member of the audience was of the opposite sex, resulting in further reduction of time needed for visual search as well as math tasks. Based on these findings, the following hypothesis is proposed:

The presence of an embodied AI agent with a stereotypical appearance attributed to the opposite sex will lead to …

• H3a: faster classification of easy emails
• H3b: more accurate classification of easy emails

… by the participants when compared to task performance in the presence of an agent of stereotypical appearance attributed to the same sex as the participant.

Methodology

Pre-Study

As social facilitation and inhibition effects depend on task difficulty, emails must be categorized according to the difficulty participants may face when having to distinguish between phishing and non-malicious emails. To achieve this, a pre-study will be conducted.

Material

Phishing and non-malicious emails will be sourced by a variety of means. As the main study’s sample will consist of German employees, appropriate emails regarding language, context and culture are needed. These will be sourced from online resources, the researcher’s personal inbox and the cooperating organization’s IT department’s collection of malicious emails. Further enhancement by large language model (LLM) generated emails will also be considered. The amount of phishing and non-malicious emails will be balanced, prioritizing a task design more suitable to the area of research rather than being fully representative of the work context. The amount of easy and hard to recognize emails is also aimed to be balanced, based on an initial assessment by the researcher. Emails will be displayed in the survey using web technologies such as hypertext markup language (HTML). After the classification task, participants will be inquired about their demographic data, their previous experience regarding computer use and web security as well as whether they use personal or organizational email accounts. They will also be provided the Meta AI Literacy Scale (MAILS) [17].

Sample

The pre-study will be conducted online using SoSci Survey [18] and will be open for eligible German-speaking participants of the working population. Zanbaka et al. [12] base their easy and difficult math tasks on a study by Foos and Goolksian [19] which found significant differences between those conditions in reaction time and error rate using a sample size of 26 participants. A similar sample size of 30 participants will be targeted. Participants will be recruited using Prolific [20], receiving a monetary compensation of 10 pounds per hour proportionally.

Procedure

Participants will receive an introduction on how to recognize phishing emails based on public videos from the SECUSO research team [21]. They will then be provided six exemplary non-malicious and phishing emails. After this, participants will binary classify 60 emails while both the time required and accuracy of correct prediction will be measured.

Main study

Design

The study will use a 2x2x2 (agent x difficulty x audience) mixed design. The agent will be varied between-subject to either posses a stereotypical female or male appearance. The remaining factors are manipulated within-subject in order for participants to classify either easy or hard to recognize emails (difficulty) alone or in the audience of a virtual agent (audience).

Sample

Participants will be recruited from within a single organization located in Germany. The staff consists of female and male workers ranging from 18 to approximately 65 years of age and varying professions in fields such as finance, marketing, logistics, software development, graphic design and customer management. If targeted sample sizes should not be reached within the organization, the sample will be augmented by students through the university’s regular means of acquisition. Participants will be screened for epilepsy and are required fluency in German. Participants from within the organization can claim their time spent participating in the study as work time which will result in their usual monetary compensation. Furthermore, six vouchers of 50€ each will be raffled upon participants. Students will be able to earn study credits for participating.

A sample size of 50 participants will be targeted for the main study. This estimation is based on Zanbaka et al. [12], who reported a medium effect size of eta^2 = 0.11. Estimation, conversion and adaption to the design of the planned study was done using g*Power.

Setting

The study will be conducted in the organization’s headquarters. The room chosen includes typical office equipment like chairs, two tables and cabinets. It currently serves as a small conference room. The videoconferencing system will be removed for the days of data acquisition. The experimenter will leave the room once the first block has started. Participants will be told that the experimenter will only enter the room if they push a button of a handheld transceiver to cancel the experiment.

System

Phishing emails will be displayed using a virtual display set to a fixed position in the real room where a typical physical display would be placed. Visually this will be similar to Apple Vision Pro’s workspaces [22] or workrooms in Meta’s Quest devices [23]. The agent will sit beside the participant, outside of their personal space. Participants are not allowed to talk to the agent, nor will the agent respond to them. The agent will introduce themself as a new, AI-operated colleague, that is still in development. Its appearance will be based on the GenErika and GenErik assets currently being evaluated in a different study by Krop et al.. The sex of the agent will be randomly assigned, thus enabling comparison of same-sex and mixed-sex conditions. In the style of Park and Catrambone [13], participants will be told that the AI agent is present to learn the task and is currently being trained to detect phishing emails. As such, the agent is not engaged in another task and no direct competition takes place, thus being in line with two aspects mentioned in Guerrin [24]. As the agent has to pay attention to the task, no interaction or talking is possible during the task.

To further embrace that the agent is not there to evaluate the participant, voting an email to be phishing will be done using the left hand controller, whereas for non-malicious emails, the trigger on the right hand controller should be pressed. The agent can not see which trigger is being pressed and no visual feedback will be shown on the screen. Labeling of left and right hand controls will however be shown on the screen during the whole task to avoid confusion. Users will also receive haptic feedback upon button presses.

Technology

The application will be developed in Unity 2022.3.20f1 or newer and be displayed on a Quest 3 HMD. The software will run via Quest Link on a laptop facilitating a Nvidia Geforce RTX 3080 Ti Laptop GPU.

Procedure

Participants will be pre-questioned regarding epilepsy and are required to have sufficient reading comprehension in German. They will furthermore be inquired about existing visual impairments and their corrections. Participants will receive the same introductory material on how to recognize phishing emails [21] as provided in the pre-study. They will be asked to familiarize themselves with the material and fill out pre-questionnaires and screening questions beforehand off site. The Simulator Sickness Questionnaire (SSQ) [25] will be provided on site directly before the main procedure.

As part of the consent material, participants will be made aware that data collected is not available to the organization and that there are no consequences regarding their task performance and performance scores. All data will be provided anonymously and no direct comparisons between individuals’ scores will be made. Participants will however be told to try their best to correctly classify as many emails as possible in the given time frame. They will also be asked to provide their name to appear in the emails.

The study will begin with practice runs for three easy and three hard to classify emails. During these runs, the researcher will be present to answer any remaining questions. Before the first of four block starts, the researcher will leave the room. Participants will be given a fixed amount of 15 emails per block. All blocks as well as the practice runs will make use of AR technology. The four available blocks are based on the within-subject manipulation of difficulty (easy/hard) and audience (alone/social) which will be in random order. Between-subject, the agent will be randomly manipulated to either be GenErika (stereotypical female appearance) or GenErik.

Participants won’t receive direct feedback whether they classified an email correctly during the study. They can opt-in however, to receive a summary of their performance after all participants finished the study. Emails will either be phishing or non-malicious and participants will have to do a binary classification of these emails.

Measurements

Task performance will be calculated using F1-score. Furthermore, duration to classify each email will be measured. Post-experiment, participants will be inquired about demographic data such as age and gender, their prior knowledge regarding phishing e-mails and computers as well as their average usage time of computers and virtual reality. They also will be asked whether they have access to a personal email account provided by the organization. Participants will be provided with the MAILS [17] and the SSQ [25]. Perception of the agent will be measured using the Virtual Human Plausibility Scale [26] as well as the Uncanny Valley Index (UVI) [27]. The Networked Minds Measure of Social Presence [28] questionnaire will be used to measure both social presence as well as co-presence. All material will be provided in German.

Tasks & Timelines

Literature

• [1] B. Guerin and J. M. Innes. “Social facilitation and social monitoring: A new look at Zajonc’s mere presence hypothesis”. In: British Journal of Social Psychology 21 (1) (Feb. 1982), pp. 7–18.
• [2] Carolin Wienrich, Astrid Carolus, and Marc Erich Latoschik. “How to Enable Sovereign Human-AI Interactions at Work? Concepts of Graspable Testbeds Empowering People to Understand and Competently Use AI-Systems”. In: Proceedings of the Weizenbaum Conference 2022: Practicing Sovereignty - Interventions for Open Digital Futures. Ed. by Bianca Herlo and Daniel Irrgang. ISSN: 2510-7666. Berlin: Weizenbaum Institute for the Networked Society - The German Internet Institute, 2023, pp. 209–220.
• [3] Bitkom e.V. Durchschnittlich 26 berufliche E-Mails landen täglich im Postfach. Presse-information, Bitkom e. V. Mar. 22, 2021. URL: https://www.bitkom.org/Presse/Presseinformation/Durchschnittlich-26-berufliche-E-Mails (visited on 08/07/2024).
• [4] Birgit Lugrin. “Introduction to Socially Interactive Agents”. en. In: The Handbook on Socially Interactive Agents. Ed. by Birgit Lugrin, Catherine Pelachaud, and David Traum. 1st ed. New York, NY, USA: ACM, Sept. 2021, pp. 1–20.
• [5] Ronald T Azuma. “A Survey of Augmented Reality”. In: (1997).
• [6] Roberto Pieraccini. “Natural Language Understanding in Socially Interactive Agents”. In: The Handbook on Socially Interactive Agents. Ed. by Birgit Lugrin, Catherine Pelachaud, and David Traum. 1st ed. New York, NY, USA: ACM, Sept. 10, 2021, pp. 147–172.
• [7] Ashish Vaswani et al. “Attention is All you Need”. In: (2017).
• [8] OpenAI et al. GPT-4 Technical Report. Mar. 4, 2024. arXiv: 2303.08774[cs].
• [9] Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. “Computers are social actors”. en. In: Proceedings of the SIGCHI conference on Human factors in computing systems celebrating interdependence - CHI ’94. Boston, Massachusetts, United States: ACM Press, 1994, pp. 72–78.
• [10] Byron Reeves and Clifford Nass. “The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Pla”. In: Bibliovault OAI Repository, the University of Chicago Press (Jan. 1996).
• [11] Na Liu and Quanlin Pu. “Can Smart Voice Assistant Induce Social Facilitation Effect? A Preliminary Study”. In: Cross-Cultural Design. User Experience of Products, Services, and Intelligent Environments. Ed. by Pei-Luen Patrick Rau. Cham: Springer International Publishing, 2020, pp. 616–624.
• [12] Catherine Amine Zanbaka et al. “Social responses to virtual humans: implications for future interface design”. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI ’07. New York, NY, USA: Association for Computing Machinery, Apr. 29, 2007, pp. 1561–1570.
• [13] Sung Park and Richard Catrambone. “Social Facilitation Effects of Virtual Humans”. In: (2007).
• [14] Mark Roman Miller et al. “Social interaction in augmented reality”. en. In: PLOS ONE 14 (5) (May 2019). Publisher: Public Library of Science, e0216290.
• [15] Jacob Cohen. Statistical Power Analysis for the Behavioral Sciences. San Diego, UNITED STATES: Elsevier Science & Technology, 1977.
• [16] Na Liu et al. “Gender composition mediates social facilitation effect in co-action condition”. In: Scientific Reports 7 (1) (Nov. 8, 2017). Publisher: Nature Publishing Group, p. 15073.
• [17] Astrid Carolus et al. “MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies”. In: Computers in Human Behavior: Artificial Humans 1 (2) (Aug. 1, 2023), p. 100014.
• [18] Dominik Leiner. SoSci Survey professionelle Onlinebefragung made in Germany. 2024. URL: https://www.soscisurvey.de/ (visited on 08/20/2024).
• [19] Paul W. Foos and Paula Goolksian. “Presentation format effects in working memory: The role of attention”. In: Memory & Cognition 33 (3) (Apr. 1, 2005), pp. 499–513.
• [20] Prolific. Quickly find research participants you can trust. Prolific. 2024. URL: https://www.prolific.com (visited on 09/09/2024).
• [21] SECUSO. KIT - SECUSO Forschung - Ergebnisse - S&P Awarness/ Education/ Training - Sichere Kommunikation - NoPhish Videos. Feb. 27, 2024. URL: https://secuso.aifb.kit.edu/1047.php (visited on 09/02/2024).
• [22] Apple Inc. Create a workspace in Apple Vision Pro. Apple Support. 2024. URL: https://support.apple.com/en-us/118515 (visited on 08/24/2024).
• [23] Meta Platforms Inc. Horizon Workrooms virtual office and meetings. Meta for Work. URL: https://forwork.meta.com/horizon-workrooms/ (visited on 08/24/2024).
• [24] Bernard Guerin. Social Facilitation. European Monographs in Social Psychology. Cambridge: Cambridge University Press, 1993.
• [25] Robert S. Kennedy et al. “Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness”. In: The International Journal of Aviation Psychology 3 (3) (July 1993). Publisher: Taylor & Francis, pp. 203–220.
• [26] David Mal et al. “Virtual human coherence and plausibility–Towards a validated scale”. In: 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE. 2022, pp. 788–789.
• [27] Chin-Chang Ho and Karl F. MacDorman. “Measuring the Uncanny Valley Effect”. en. In: International Journal of Social Robotics 9 (1) (Jan. 2017), pp. 129–139.
• [28] Frank Biocca, Chad Harms, and Jennifer Gregg. “The Networked Minds Measure of Social Presence: Pilot Test of the Factor Structure and Concurrent Validity”. In: 4th annual International Workshop on Presence, Philadelphia (Jan. 2001).

Contact Persons at the University Würzburg