Exploring Player Experience of AI-Driven Dynamic Interpretative Gameplay Alteration in VR Social Games

Assigned Human-Computer Interaction. Master Project No Affiliation

Project Title

• “Exploring Player Experience of AI-Driven Dynamic Interpretative Gameplay Alteration in VR Social Games”

Introduction

Social Virtual Reality (Social VR) describes an immersive form of avatar-mediated, synchronous interaction in which multiple users share and interact within three-dimensional virtual environments (Dzardanova et al., 2018). Unlike conventional text- or image-based online communication, Social VR can approximate aspects of face-to-face interaction by combining spatial presence with verbal and nonverbal communication, including gaze behavior, gestures, body posture, and proximity (Dzardanova et al., 2018; Son & Rubo, 2025). Social VR games build on these characteristics by embedding shared, embodied interaction into gameplay, allowing players not only to communicate, but also to coordinate, compete, and act together within the same virtual game space (Lee et al., 2021; Liszio et al., 2017).

These environments offer considerable potential for multiplayer experiences, as they can foster social presence, shared activity, and meaningful interaction (Han & Bailenson, 2024). Research suggests that Social VR can reduce feelings of loneliness and social anxiety and may support social connectedness and well-being (Kenyon et al., 2023; Deighan et al., 2023). At the same time, Social VR games also introduce design challenges. Social entities in VR games can reduce perceived loneliness, but they do not necessarily enhance enjoyment and may even disrupt immersion if they are not designed carefully (Liszio et al., 2017). In addition, negative well-being effects may occur for vulnerable users when social VR involvement is high while social connectedness and self-esteem are low (Lee et al., 2021). Therefore, interactive systems in Social VR games need to enrich the shared experience without interrupting presence or game flow.

Artificial intelligence (AI) offers one possible direction for creating more adaptive and dynamic game experiences. Traditionally, AI in video games has often been associated with non-player characters, enemy behavior, pathfinding, and difficulty adjustment (Filipović, 2023). More recent developments extend this role toward procedural content generation, player experience modeling, adaptive gameplay, and generative content such as dialogue, quests, levels, or environmental changes (Filipović, 2023; Bonn, 2023).

In this way, AI can contribute to game experiences that are more variable, personalized, and difficult to realize through pre-written structures alone (del Bosque et al., 2025).

This HCI project builds on this potential by exploring Dynamic Interpretative Gameplay Alteration (DIGA) in a VR multiplayer first-person shooter. Instead of using generative AI only as a conversational agent or as a tool outside gameplay, the project investigates how player input can be interpreted by a Large Language Model and translated into changes to the game environment during play. The goal is to examine how such AI-driven gameplay alterations can be integrated into a fast-paced Social VR game while maintaining playability, immersion, and a positive user experience.

Recent Work

A first implementation of DIGA has been developed within the scope of the MAGE project. The prototype was realized as a browser-based multiplayer first-person shooter on the Captic/w3dge.io platform, where generative AI was integrated as an active and core gameplay element rather than only as a tool for content creation. During a match, players could collect a special AI item and enter a natural language prompt by text input, which was interpreted by an AI system and translated into changes of the game environment, such as adding or modifying movement-related elements including jump pads, speed pads, barriers, and elevators.

The implementation demonstrated that generative AI can be connected to multiplayer gameplay in a tangible way, but it also remained a proof of concept. Earlier ideas such as AI-generated weapons, 3D models, textures, or more complex social game scenarios were reduced due to technical limitations, generation time, and the need for short and easily understandable gameplay sessions. The final system therefore focused on lightweight map manipulation through an abstract map representation, allowing AI-driven changes to influence the match without interrupting the basic gameplay loop.

The evaluation showed that the general gameplay was perceived as accessible and enjoyable, particularly the movement-based mechanics. However, the AI feature was not always clearly noticed or understood by players, and the quality of the generated map changes was not evaluated systematically. These findings indicate that the previous prototype provides a useful foundation, but also leaves open questions regarding the visibility, playability, and systematic assessment of AI-generated gameplay alterations.

Planned Methodology

In the context of this project, a VR multiplayer first-person shooter prototype will be developed in the Unity Engine using a Meta Quest virtual reality headset. As a technical starting point, Unity’s VR Multiplayer Template and the “Set Up Your First Microgame” tutorial resource will be explored and combined where suitable. The VR Multiplayer Template provides a foundation for networked player interaction, while the FPS Microgame offers basic first-person shooter mechanics that can be adapted for a VR context.

Building on the previous web-based MAGE implementation, the goal is to recreate the core idea in a more game-oriented Unity and VR environment and to use it as a functional prototype for further exploration. The development will involve evaluating suitable technologies and workflows for VR interaction, multiplayer synchronization, FPS mechanics, and AI-based gameplay modification while gradually integrating these elements into a functional prototype.

In addition, suitable input methods for player prompts will be explored and integrated into the gameplay flow, aiming to allow AI interaction without interrupting the fast-paced multiplayer experience. The prototype will include core shooter mechanics, basic multiplayer functionality, VR interaction, and an initial version of the DIGA system. Particular focus will be placed on keeping AI-driven changes understandable, playable, and non-disruptive while maintaining a positive gameplay experience.

Next to the technical implementation, the project will explore how the quality of AI-generated gameplay alterations can be assessed. Possible criteria include whether generated changes are playable, whether important paths or items remain accessible, whether the changes create meaningful variation, and whether they support rather than interrupt the game flow.

To collect initial user feedback, the prototype will be presented at the HCI exposition in July 2026. The collected insights will then be incorporated into the prototype, which will be further explored and refined after the event. In addition, the prototype can serve as a testbed for a potential master thesis focusing on the evaluation and improvement of AI-generated or AI-altered gameplay.

Project Timetable

Mid and Final Prototype Milestones and Feature Scope

Mid prototype (17.07.2026) mandatory features / tests to pass

• Playable Unity VR First-Person-Shooter prototype
• Meta Quest-based VR interaction for FPS functionality
• Basic VR controller-based aiming, shooting, or interaction mechanic
• Simple 3D map that can be navigated by the player in VR
• Prompt-based DIGA interaction
• Usable prompt input method, at least as a basic prototype
• AI/LLM output is translated into a structured format Unity can interpret
• Runtime modification of the game environment, for example walls, barriers, jump pads, speed pads, or pickups
• Generated changes are visible to the player in VR
• Basic quality constraints so generated changes do not break the scene
• Important paths or objects remain accessible
• Prototype can be demonstrated to Expo visitors
• Initial user feedback can be collected

Final prototype (30.09.2026) mandatory features / tests to pass

• VR multiplayer FPS prototype in Unity
• Meta Quest-based VR interaction for FPS functionality
• Core FPS mechanics in VR, including movement, aiming, shooting, and feedback
• Basic multiplayer functionality with at least two players
• Players can see relevant actions or state changes of other players
• DIGA is integrated into the gameplay loop through an item, interaction point, or similar trigger
• Usable prompt input method for VR
• AI/LLM-generated changes are applied during runtime
• AI-driven map or gameplay changes are synchronized or consistently visible for all relevant players
• Basic quality assessment of generated changes
• Generated changes remain playable and do not block important paths or objects
• AI changes create meaningful variation without interrupting game flow
• Feedback from the HCI exposition is incorporated into the prototype
• Full adaption of MAGE prototype functionalities to Unity VR FPS prototype

Optional features, if time allows

• More advanced quality validation of AI-generated changes, such as path connectivity, object overlap, spawn accessibility, and pickup reachability.
• Automatic rejection or correction of AI-generated changes if they fail the defined quality criteria
• Experimental self-improvement loop in which the system can revise or regenerate an AI-altered map based on failed quality criteria
• Persistent logging of prompts, AI outputs, validation results, and applied changes for later analysis
• Comparison of different prompt structures or AI models to explore which approach produces more playable gameplay alterations
• Cosmetics like a color theme, designed avatars, animations etc.

References

• Dzardanova, E., Kasapakis, V., & Gavalas, D. (2018). Social virtual reality. In Encyclopedia of Computer Graphics and Games (pp. 1–3). https://doi.org/10.1007/978-3-319-08234-9_204-1
• Son, G., & Rubo, M. (2025). Social virtual reality elicits natural interaction behavior with self-similar and generic avatars. International Journal of Human-Computer Studies, 199, 103488. https://doi.org/10.1016/j.ijhcs.2025.103488
• Lee, H., Kim, S., & Uhm, J. (2021). Social virtual reality (VR) involvement affects depression when social connectedness and Self-Esteem are low: a moderated mediation on Well-Being. Frontiers in Psychology, 12. https://doi.org/10.3389/fpsyg.2021.753019
• Liszio, S., Emmerich, K., & Masuch, M. (2017). The influence of social entities in virtual reality games on player experience and immersion. Proceedings of the 12th International Conference on the Foundations of Digital Games, 1–10. https://doi.org/10.1145/3102071.3102086
• Han, E., & Bailenson, J. N. (2024). Social interaction in VR. Oxford Research Encyclopedia of Communication. https://doi.org/10.1093/acrefore/9780190228613.013.1489
• Kenyon, K., Kinakh, V., & Harrison, J. (2023). Social virtual reality helps to reduce feelings of loneliness and social anxiety during the Covid-19 pandemic. Scientific Reports, 13(1), 19282. https://doi.org/10.1038/s41598-023-46494-1
• Deighan, M. T., Ayobi, A., & O’Kane, A. A. (2023). Social Virtual Reality as a Mental Health Tool: How People Use VRChat to Support Social Connectedness and Wellbeing. Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23), 1–13. https://doi.org/10.1145/3544548.3581103
• Filipović, A. (2023). THE ROLE OF ARTIFICIAL INTELLIGENCE IN VIDEO GAME DEVELOPMENT. Kultura Polisa, 20(3), 50–67. https://doi.org/10.51738/kpolisa2023.20.3r.50f
• Bonn, N. F. (2023). The impact of integrating Artificial Intelligence into the video games industry: A case study [Master’s thesis, KTH Royal Institute of Technology].
• Del Bosque, A., Fernández-Arias, P., Lampropoulos, G., & Vergara, D. (2025). The role of artificial intelligence in gaming. Applied Sciences, 15(23), 12358. https://doi.org/10.3390/app152312358

Contact Persons at the University Würzburg