Virtual Audiences
Overview
This project investigates the implementation and performance of a Virtual Audience behaviour model for Virtual Reality. Our behaviour model is rule based and use two dimensions for the design of the virtual audiences. The valence is here defined as the opinion toward the speech and the arousal as the audience members’ level of alertness or engagement.
The system allows to easily build and experience a wide variety of attitudes with small or large groups of virtual agents for different domains of application: Training, Therapy and Entertainment. The plugin also gives a high level control tool of the virtual audience through a desktop graphical user interface or directly in virtual reality.
With regards to the trade-off between a fully autonomous simulation and a completely Wizard of Oz system where each agent would have to be individually controlled: for applications where replacing tutor expertise with an autonomous component is not desirable, our system can provide a component with high-level controls which may be easily used by the tutor during a simulation-based training session.
Virtual Audience System (VAS)
Research presentation training system
A first virtual training system prototype for bachelor students to practice their scientific presentation is currently being used in order to allow students to keep praticing despite the current pandemic. Hence, instead of training session alone or on a Zoom meeting they can face a virtual audience driven by our behaviour model. With the use of Decker the students are able to directly use their own slides in the virtual environment.
As for the instructors, we designed a web based supervision interface which provides a wide range of authoring tools for the audience behaviour and the virtual environment, e.g. change the audience attitued from interested to bored or restart the student’s timer. This interface also empower the intructors with the possibility to start pre-designed training scenarios which helps them focusing on the student’s presentation without losing the ability to manually adjust the virtual environment to the current presentation.
Thanks to our collaboration with the VIA-VR and the integration of their measurment engine we are now able to retreive physiological data from the students. The measurment engine is able to provide a stress classification using deep learning algorithm with empower the instructor with the possibility to design stress based scenarios ant to constantly monitor the student’s stress.
Virtual reality exposure therapy
On the initiative of the Virtual Elo project we are currently investigating how the VAS could be beneficial for therapists as a virtual exposure therapy system for public speaking anxiety. The VAS could provide an ecological environment which would be highly controlable and suitable for the design of therapeutic scenarios.
Technical overview
The behaviour model implementation is developed as a plugin for Unreal Engine 4 and is available. It works for windows virtual reality applications and also mobile applications, with the Oculus Quest for instance. The supervision interface uses a REST API that allows use to link other app like the measurment engine without being dependent to a specific technology. This also allows us to have the supervision interface through a phone or tablet.
Plugin’s Sources
Please contact Yann Glémarec to get access to the project’s source code. The plugin will be soon available on our GITLAB.
VAS’s Sources
A link to download the latest released version of the VAS will be soon available here.
French collaborators
|
The COMMEDIA team of the Lab-STICC CRNS UMR 6285 is also participating to the Virtual Audience project. Applications using the Virtual Audience plugin are currently under development in addition to the contribution of different researchers in virtual reality, system modelisation, artificial intelligence and cognitive science. Click on the logo for more information about the Lab-STICC and the CNRS ! |
First prototype’s video
Theses and projects
Closed
Funding and Collaboration
