Project Miona: Multi-Immersive Onco-Analysis

Closed Human-Computer Interaction Computer Science Games Engineering Bachelor Master Project Thesis Games Research Lab

Project Miona: Multi-Immersive Onko-Analysis

Introduction

Miona (Multi-Immersive Onko Analysis) aims to support researchers in life sciences - especially biology and medicine - who conduct in-vitro and in-silico experiments on multi-cellular tumor spheroids and organoids. Miona researches immersive, collaborative multi-user analysis tools for the exploration and qualitative and quantitative empirical data. It strives to seamlessly integrate simulated data and interactive simulations, encouraging the use of computational modeling. Miona aims for the joint, interactive, and immersive analysis and, anticipating future developments, the conduction, of in-vitro and in-silico experiments in cell biology. Individual applications should be developed in tight cooperation with the CCTB and ITAV in an agile process, elaborating on use cases and integrating domain experts tightly.

The project is a joint effort between the Würzburg University, represented by the Games Engineering Research Group of the Chair of Human-Computer-Interaction and the Center for Computational and Theoretical Biology, and the ITAV and the University Toulouse, France.

Background

Cell biology is a highly active research area that deepens the understanding of the multitude of mechanisms that drive cellular life. It is fundamental to the discovery and development of new drugs and treatment plans. 3D cell cultures are gaining in importance as experimental systems, because they resemble the spatial arrangement of cells and tissue structures more closely than traditional 2D cell cultures in Petri dishes. They come in different flavours: Multi-cellular tumor spheroids (MCTS) are 3D aggregates of tumor cells that are used (a) for drug screening or (b) to investigate fundamental processes in tumor biology. Organoids are formed from stem cells and mature to exhibit features of their tissue of origin. The quantitative analysis of 3D in-vitro cell cultures is often accompanied by agent-based in-silico modeling. In both model categories, the spatial arrangement of cells and tissue structures is a very relevant feature that has to be included in the data analysis. Biologists often retrieve large sets of volumetric data generated by a range of microscopy or tomography methods. The raw data is often analyzed in a layer-based fashion on typical flat screens. The data sets are further commonly segmented, where the intensity values of the image channels are used to extrapolate or infer the original features of the specimen, such as cell nuclei, boundaries, different types of tissue, etc. The exploration of this data is still usually tedious although various commercial and non-commercial applications exist. The main goal of our work is to support the analysis of empirical, segmented, and simulated data on MCTS and organoids by means of computational tools for the immersive, interactive visualisation and immersive analysis. Immersive visualization has been implemented successfully in many applications that require spatial understanding. Furthermore, immersive analysis can aid in understanding complex and abstract relations in large data sets. The specific analytical objectives include the exploration of the external and internal structure of the data, i.e. the exterior shape of the cell culture or the occurrence of cavities or structured features of the interior, and especially the classification and quantification of any 3D spatial patterns which arises during cell differentiation. In numerous conversations, biologists expressed their interest in visualizing and interacting with simulated data in the same way as with empirical data. We therefore aim to integrate interactive simulations with our applications whenever suit- able and possible. The effectiveness and the success of such tools highly depends on the support of the domain experts’ established workflows, to provide intuitive ways of using the tools to effectively harness the experts’ knowledge and the tools’ analytical functionality.

Research Topics and Applications

Within Project Miona, multiple applications and aspects are researched. In the following, we outline two of them.

GroundTruthVR

GroundTruthVR supports the segmentation process of raw imaging data by providing an immersive and intuitive tool for the collaborative definition of cells, nuclei, cell walls, and other relevant features. Such manually created segmentations forms the basis against which automated segmentation tools will be benchmarked and validated. Inherently, the process requires a good spatial understanding of the data, and manually segmented data needs to be proofed by multiple domain experts. Therefore, it is a well-suited target for an interactive, immersive, and collaborative application.

The tool that is currently most widely used for annotating fluorescence microscopy data is Fiji. Image stacks from 3D microscopy are loaded into Fiji and are annotated slice by slice. For annotation of structures that extend over several slices, the plugin Segmentation Editor in Fiji provides a more efficient method. In this editor, the structure does not have to be annotated in every slice, but the annotation of intermediate slices is obtained by interpolating the information from surrounding slices. Due to the restriction of the data to 2D image slices, annotating a nucleus centroid or identifying all neighbours of a given cell is particularly difficult.

Immersive Analysis Workbench

Current state-of-the-art workbenches for the structural analysis of spheroids are traditional desktop-based applications. The common measures can still not fully capture 3D spatial expression patterns. We would like to support ongoing efforts in the identification of novel quantitative measures for 3D spatial patterns. We plan to combine different kinds of visualisation modes with filtering of cells by different features. The selected information can be saved in CSV files for further analysis. Integration of the organoid visualisations with plots or other data formats will foster a thorough understanding. We expect the immersive application to extend, not replace the traditional desktop suite of applications. The latter is unlikely, as the high precision and resolution offered is still unmatched. We believe that such a cooperation can (a) benefit the domain experts through the addition of immersive scientific visualization and analysis tools of the data sets, and (b) provide a well-suited test- and benchmark-situation for such an immersive analytics workbench, due to the availability of existing tools and workflows to be used as reference.

Outline

After deciding on one the above applications, students would be asked to tackle the following fundamental tasks:

1. Research existing solutions for the definition of ground truth data for cell segmentation and select a benchmark (e.g. non-VR application) or existing algorithmic solution.
2. Elaborate user stories and use cases together with domain experts.
3. Iteratively develop a prototype that provides a sufficient amount of interactions and features to fulfill the desired use case.
4. Evaluate the prototype with an emphasis on one of the core aspects tackled in the project.

Within the project, students may identify and prioritize a specific core aspect:

1. Technological challenges
2. User interface design
3. Immersive Collaboration

Therefore, theses or projects can for example pursue either a more technical engineering focus or a human-centered design perspective, to be outlined in the expose. Also, consecutive work may be pursued (e.g. internship, project, thesis).

Contact Persons at the University Würzburg