Human-Computer Interaction

Identification of user errors when using AI systems: The Role of AI Literacy

This project is already assigned.

Motivation/Goal

Artificial Intelligence (AI) has become an integral part of modern society, influencing various aspects of daily life, from virtual assistants on smartphones to AI-powered translation apps that facilitate communication. The AI Index Report 2024 [1] highlights this growing significance, noting that AI technologies are deeply embedded in everyday activities. As AI continues to evolve, developing comprehensive AI literacy — encompassing an understanding of AI’s mechanisms and effective interaction with AI—becomes essential [2, 3]. These competencies allow individuals to navigate a world increasingly shaped by AI and actively participate in its future developments [4].

Despite the widespread use of AI, a lack of AI literacy often leads to typical user errors, negatively affecting user experience and trust in AI technologies [2, 3, 4]. Common errors include overestimating AI capabilities, where users assume AI can perform beyond its actual functionality, leading to unrealistic expectations and potential disappointment [6]. For example, a user might rely on a virtual assistant to diagnose health symptoms, unaware that the AI is not equipped to replace professional medical advice. Conversely, underestimating AI’s potential may prevent users from fully leveraging available tools, resulting in missed benefits [8]. A user might, for instance, avoid using an AI-powered grammar checker under the impression that it can only detect basic errors, missing out on advanced style and tone suggestions.

Additionally, misinterpretation of AI outputs, often due to limited understanding, can lead users to accept AI-generated information without critical evaluation, risking the spread of misinformation or reliance on biased data [10]. Privacy concerns are also notable; users may inadvertently share sensitive personal information without understanding associated risks [11].

These errors underscore the importance of key AI literacy competencies, such as cognitive understanding of AI functionality (e.g., knowledge of machine learning and data processing), ethical sensitivity (awareness of privacy and discrimination issues), and practical application skills to interact purposefully with AI systems. A study by Wienrich and Koch (2024) emphasized the necessity of these competencies for assessing the possibilities and limitations of AI, promoting responsible and informed use of AI technologies.

Research indicates that AI literacy is still insufficiently developed in many users, leading to difficulties in responsibly navigating interactions with generative AI systems [2, 5, 6, 8]. For instance, users may rely too heavily on AI-generated suggestions without verifying their accuracy or might use AI tools for unintended purposes, such as seeking professional advice from chatbots despite limitations in the AI’s training data [10]. Ethical oversights, such as ignoring potential biases, may result in discriminatory practices or reinforcement of stereotypes [12]. These challenges highlight the impact of limited AI literacy on user interactions and the tendency toward typical user errors.

To address these issues, this study aims to capture and analyze frequently occurring errors that stem from users’ insufficient AI literacy. Specifically, it will examine the following research questions:

  1. What are potential sources of error that frequently occur?
  2. How do different levels of AI literacy impact error susceptibility in interactions with AI tools?

As discussed in the previous section, the role of AI literacy has become crucial as AI systems become increasingly integrated into everyday life. Various projects and studies have focused on identifying the challenges and opportunities in fostering AI literacy.

AI Literacy Initiatives

One such initiative is the MOTIV project at the University of Würzburg, which aims to enhance user competencies in interacting with voice-based AI [14]. The project developed the Digital Interaction Literacy (DIL) Model, encompassing three primary dimensions: Understanding Functional Principles, Mindful Usage, and User Group-Specific Competencies. The DIL Model is operationalized through an 18-module digital training platform, intended to improve users’ understanding of the potential and limitations of voice-based AI systems, thus fostering responsible and effective interaction with these technologies.

In addition to the MOTIV project, the ”Elements of AI” initiative, developed by the University of Helsinki and the company Reaktor, is another significant effort aimed at improving AI literacy [13]. Both projects share the goal of providing foundational knowledge about AI concepts, including machine learning, neural networks, and ethical implications. While MOTIV delivers this content in the context of interacting with voice-based AI systems and adopts a practice-oriented approach, ”Elements of AI” covers these concepts at a more general level without focusing on a specific application area.

Assessment of AI Literacy

The Meta AI Literacy Scale (MAILS), developed by [3], is a specialized questionnaire designed to systematically assess AI literacy. The scale covers various dimensions, including cognitive understanding, ethical sensitivity, and practical application skills. It serves as a practical tool for evaluating AI-related competencies in educational and professional contexts, helping to identify areas where skills may be lacking. However, it is not intended to function as a comprehensive approach to promoting AI literacy.

Challenges in AI Interaction

Despite initiatives like MOTIV and ”Elements of AI” aiming to enhance AI literacy, studies continue to reveal persistent challenges in this area. As noted by [6] and [8], users often place excessive trust in AI systems, accepting AI-generated outputs without critical scrutiny. This tendency to overrely on AI can result in misguided decisions, particularly in contexts where users assume the AI’s recommendations to be accurate and reliable. For instance, many users might follow health or financial advice generated by AI systems, unaware that these suggestions are based on probabilistic models rather than concrete facts.

A significant issue contributing to user errors is the misinterpretation of AI outputs. Users frequently perceive AI-generated content as objective and definitive, overlooking the fact that these systems operate on probabilistic algorithms and are limited by the data on which they were trained [8]. This misperception can lead to uncritical acceptance of AI-generated answers, especially in high-stakes areas like legal or medical advice, where accuracy is crucial.

Moreover, a lack of understanding about the potential biases inherent in AI systems exacerbates these challenges. According to chen2022, many users do not recognize that AI models can reflect biases present in their training data, which can result in discriminatory or skewed outputs. This lack of awareness may inadvertently reinforce stereotypes, especially when AI tools are used in sensitive areas like recruitment or financial lending.

Privacy concerns also emerge as a significant issue, as highlighted by pinski2024. Users often share personal data with AI applications without fully understanding how their information is collected, stored, or used, exposing them to potential privacy risks. This gap in knowledge underscores the need for users to be more informed about the implications of interacting with AI-driven platforms.

Finally, users frequently overestimate the capabilities of AI systems, assuming that these technologies possess human-like understanding and decision-making abilities. As [6] found, this misconception can lead users to rely on AI for tasks that require nuanced judgment, such as legal consultations or complex medical diagnoses, where AI is not yet equipped to provide reliable guidance.

These findings demonstrate that, despite efforts to improve AI literacy, there remain substantial gaps in users’ understanding of AI’s capabilities, limitations, and ethical implications.

Research Gap

Although projects like the MOTIV project [14] and the MAILS scale [3] have developed models for promoting and assessing AI literacy, studies that systematically examine how different levels of competence affect error susceptibility in interactions with AI systems are lacking. While the importance of cognitive, ethical, and practical competencies is emphasized [2, 3], the direct connection between deficiencies in these areas and typical user errors remains underexplored. For instance, the MAILS scale focuses on assessing competencies but does not thoroughly analyze how gaps in these competencies lead to specific errors [3]. Studies like those by Ng et al. [6] indicate that a lack of understanding often leads to overreliance on AI outputs, yet do not provide a comprehensive analysis of the underlying sources of these errors. Similarly, research by Su et al. [8] on AI literacy highlights that gaps in competence can lead to er- rors, but detailed data on the causes of these issues are still missing. Existing work predominantly relies on theoretical models and qualitative insights [5, 10], while comprehensive analyses linking competence levels to specific error sources remain scarce.

Hypotheses

To address these research gaps, the present study aims to investigate the relationship between AI literacy levels and error susceptibility in interactions with AI tools. Based on the literature and the identified gaps, the following hypotheses are proposed:

  1. Users with low levels of AI literacy exhibit a higher error susceptibility when interacting with AI tools compared to users with high levels of AI literacy.

This hypothesis is grounded in studies indicating that a lack of AI literacy constrains responsible AI use and leads to typical user errors [6, 8]. The empirical investigation aims to quantify how different levels of AI literacy affect error frequency.

  1. Certain error sources, such as misunderstandings of AI functionality or ethical concerns, occur more frequently among users with insufficient AI literacy.

Koch and Wienrich (2024) suggest that lacking cognitive and ethical competencies lead to specific errors in AI interactions [3]. This hypothesis seeks to identify and categorize these errors, providing insight into which competencies are most critical for effective AI interaction. By testing these hypotheses, the study seeks to contribute to a deeper understanding of how AI literacy impacts user interactions with AI systems and to identify common error sources attributable to insufficient AI literacy. This research will fill the existing gap by providing empirical evidence on the correlation between AI literacy levels and error susceptibility, ultimately informing the development of targeted educational interventions to enhance AI literacy.

Approach

To address the research questions and test the proposed hypotheses, a multiphase research design will be implemented. This methodology comprises four key stages: conducting a literature review, planning and executing a pre-study survey, developing a custom GPT model, and validating the model through a proof-of-concept study.

Literature Review

The initial phase involves a comprehensive literature review focusing on existing projects and frameworks related to AI literacy, such as the MOTIV project. This review aims to assess the applicability and effectiveness of these projects in enhancing AI literacy and reducing user errors. By critically analyzing their methodologies, outcomes, and limitations, the study seeks to identify best practices and potential gaps that the current research can address.

Pre-Study

Following the literature review, a pre-study will be designed and conducted using the SoSci Survey platform. The study will target a diverse sample of participants to assess their AI literacy using established scales such as the Meta AI Literacy Scale (MAILS). Additionally, the survey will collect data on common errors participants encounter when interacting with AI tools. This phase aims to identify frequent error sources and correlate them with varying levels of AI literacy.

Development of a Custom GPT Model

Based on the insights gained from the pre-study, a custom Generative Pre-trained Transformer (GPT) model will be developed. The model will be trained to recognize the frequently occurring errors identified earlier, which are attributable to insufficient AI literacy. The GPT model will be programmed to detect these errors during user interactions with AI tools and provide proactive feedback or guidance to users, aiming to mitigate errors before they occur.

Proof-of-Concept

To evaluate the effectiveness of the custom GPT model, a proof-of-concept validation study will be conducted. The participants, who will consist of fellow students, will be invited to interact with the GPT model in a controlled study environment. As an incentive, they will receive a chocolate bar for their par- ticipation. The purpose of this phase is to assess whether the model effectively helps reduce user errors and enhances the quality of interactions.

Timeline

Literature


Contact Persons at the University Würzburg

Philipp Krop (Primary Contact Person)
Human-Computer Interaction Group, University of Würzburg
philipp.krop@uni-wuerzburg.de

Follow our Social Media Channels
Contact
Lehrstuhl für
Mensch-Computer-Interaktion
Universität Würzburg
Emil-Fischer-Straße 50
97074 Würzburg
How to Find Us
University Main Page
About the University
University Contact
Legal Information