A Web User Interface Framework for Real-time Interactive Systems

Closed Computer Science Bachelor Thesis No Affiliation

Motivation

Although today’s computing landscape shifts to web enabled applications supporting mobile devices, many applications fail to provide a web-based user interface. Projects like Cockpit (see Ref. 1), pyLoad (see Ref. 2) or Arch-AddOn for Kodi (see Ref. 3) show the advantages of having an accessible way to control and monitor applications remotely through a web browser. Furthermore they show that developing web interfaces is a time consuming task: Developers usually need to implement a REST interface for their application and then develop an individual web interface from scratch using frameworks like AngularJS (see Ref. 4) or Backbone.js (see Ref. 5).

Despite the complexity of these example frameworks, a simpler approach with more limitations on customization and features might still be a viable solution for many applications. Especially for research projects like SimulatorX, having a faster and simpler way to deploy web interfaces would allow developers to extend their applications with such interfaces.

This research targets such an open and easy-to implement solution, which allows any application to export a flexible web interface for monitoring and controlling, provided that both the application and the user’s device are connected in a common network.

Goal

This thesis describes the design and implementation of a framework to quickly implement web-based user interfaces for controlling, monitoring and generally interacting with remote applications, such as SimulatorX, in realtime.

The basic concept is a tree-like data structure containing all the data displayed to the user. It also contains hints, which are not displayed, but used in javascript to select the views, to format the data correctly and to specify the behaviour of containers and active components. This data structure is constantly kept in sync with a copy on the client side. Then it is then used to update the DOM accordingly. For a schematic overview of the different stages during the update process of the UI see Figure 1.

Figure 1: A diagram showing a schematic overview of how updates could be processed in such a framework.

Using this framework, the user is enabled to view and, if permitted, change the data in the user interface as well as send custom requests and call application procedures remotely. Additionally, a user management based on groups allows to filter data for different users to enable or disable parts of the interface based on the defined privileges of the group.

A concept overview is shown in Figure 2 with the primary components marked by a bold header. The main aspects are described in the following.

Figure 2: First prototype of the architectural design of the reference implementation. The diagram shows the different components involved and their relations.

Web Interface Library

Applications utilizing the framework are linked against this library. It provides an API to create data entries, to change their values, to set group privileges and to send or receive remote function call requests. The library handles all of the communication between the users’ devices, which display the web interface, and the remote application. All communication is done using a WebSocket server, which is part of this library. The WebSocket connections are used to communicate updates and remote function call requests in real time.

Web server

The web server provides the static files for the user’s web browser. Although the library may provide this component, a standalone web server is expected to provide better security and higher performance, especially when multiple clients are simultaneously requesting different assets.

Communication protocol stack

The communication protocol stack defines the data formats passed between the client and the server. These protocols will utilize the JSON data format (ECMA-404) and are built upon the WebSocket protocol (RFC 6455).

Client-side Javascript

The client side scripts provide the functionality to communicate with the server and to synchronize the data with the server. They also provide the logic to build user interfaces using hierarchical views which can be customized and extended by developers of applications. Additionally it implements the necessary functionality to send and receive remote function calls.

This solution follows a declarative approach: Application developers only need to declare the structure and properties of the components in the interface to the library and write code to regularly update the displayed values and to poll for the user’s RPC request. Despite that, the solution should be flexible enough to allow developers to add custom components, designs and behaviour to the interface to fit the application-specific requirements. In addition, these interfaces should run in realtime, which is a requirement for certain applications (e.g. previews of visualizations or quickly updating maps with markers for the characters’ positions in games).

To ensure the fast responsiveness of the library and the little impact on the applications performance, different approaches to improve the performance of the communication library will be investigated and the general performance will be evaluated in different benchmarks.

Tasks

The presented goals are planned to be realized by conducting the following main tasks:

1. Analysis of the requirements for a framework to achieve the described goals.
2. Analysis of the requirements for the data models and protocols used in such a framework.
3. Implementation of a reusable framework based on the results of the analysis of requirements and discussing this implementations features like data structures, data flow, algorithms and parallelization techniques.
4. Implementation and analysis of practical demonstrations using the framework (functional tests) and benchmarks of the framework (especially regarding different options to optimize performance)
5. Integration into SimulatorX and the proof-of-concept implementation of a web-based user interface for XRoads (see above)

Proof of Concept: User Interface for XRoads

As a proof of concept the framework shall be integrated into SimulatorX as a module. This enables researchers and developers using SimX to rapidly create a fully customizable web-based user interface for their application. This module will utilize the available functionality in SimulatorX to observe individual entities and to identify changes in their properties. The structure of the user interface and the collected data will be communicated to the library through a set of predefined widgets, which can easily be expanded for future projects using this framework.

In particular the framework will be showcased by developing a demo for a user interface for a mixed-reality board game called XRoads. It would allow users to view their character status and change their skill values when special events take place (e.g. the player’s character levels up). In addition, the interface should also allow the game master to have a different view on the user interface than the players and grant him different privileges than the player (e.g. the game master heals a player when he is close to death or kills some enemies when they are too strong for the group).

References

1. Cockpit, http://cockpit-project.org/ (accessed October 2016)
2. pyLoad, https://github.com/pyload/pyload (accessed October 2016)
3. Arch-AddOn for Kodi, https://github.com/abricot/webinterface.arch (accessed October 2016)
4. AngularJS, https://angularjs.org/ (accessed October 2016)
5. Backbone.js, http://backbonejs.org/ (accessed October 2016)

Contact Persons at the University Würzburg