Multi-shell diffusion MRI and Diffusion Spectrum Imaging are modern neuroimaging modalities that acquire diffusion weighted images at a high angular resolution, while also probing varying levels of diffusion weighting (b values). This yields large and intricate data for which very few interactive visualization techniques are currently available. We designed and implemented the first system that permits an interactive, iteratively refined classification of such data, which can serve as a foundation for isosurface visualizations and direct volume rendering. Our system leverages features learned by a Convolutional Neural Network. CNNs are state of the art for representation learning, but training them is too slow for interactive use. Therefore, we combine a computationally efficient random forest classifier with autoencoder based features that can be pre-computed by the CNN. Since features from existing CNN architectures are not suitable for this purpose, we design a specific dual-branch CNN architecture, and carefully evaluate our design decisions. We demonstrate that our approach produces more accurate classifications compared to learning with raw data, established domain-specific features, or PCA dimensionality reduction.

Frequency distributions (FD) are an important instrument when analyzing and investigating scientific data. In volumetric visualization, for example, frequency distributions visualized as histograms, often assist the user in the process of designing transfer function (TF) primitives. Yet a single point in the distribution can correspond to multiple features in the data, particularly in low-dimensional TFs that dominate time-critical domains such as health care. In this paper, we propose contributions to the area of medical volume data exploration, in particular Computed Tomography (CT) data, based on the decomposition of local frequency distributions (LFD). By considering the local neighborhood utilizing LFDs we can incorporate a measure for neighborhood similarity to differentiate features thereby enhancing the classification abilities of existing methods. This also allows us to link the attribute space of the histogram with the spatial properties of the data to improve the user experience and simplify the exploration step. We propose three approaches for data exploration which we illustrate with several visualization cases highlighting distinct features that are not identifiable when considering only the global frequency distribution. We demonstrate the power of the method on selected datasets.

This is a work in progress paper that describes a novel endoscope interface designed for use in an immersive virtual reality surgical simulator. We use an affordable off the shelf head mounted display to recreate the operating theatre environment. A hand held controller has been adapted so that it feels like the trainee is holding an endoscope controller with the same functionality. The simulator allows the endoscope shaft to be inserted into a virtual patient and pushed forward to a target position. The paper describes how we have built this surgical simulator with the intention of carrying out a full clinical study in the near future.

Personalized anatomical information of the heart is usually obtained from the visual analysis of patient-specific medical images with standard multiplanar reconstruction (MPR) of 2D orthogonal slices, volume rendering and surface mesh views. Commonly, medical data is visualized in 2D flat screens, thus hampering the understanding of 3D complex anatomical details, including incorrect depth/scaling perception, which is critical for some cardiac interventions such as medical device implantations. Virtual reality (VR) is becoming a valid complementary technology overcoming some of the limitations of conventional visualization techniques and allowing an enhanced and fully interactive exploration of human anatomy. In this work, we present VRIDAA, a VR-based platform for the visualization of patient-specific cardiac geometries and the virtual implantation of left atrial appendage occluder (LAAO) devices. It includes different visualization and interaction modes to jointly inspect 3D LA geometries and different LAAO devices, MPR 2D imaging slices, several landmarks and morphological parameters relevant to LAAO, among other functionalities. The platform was designed and tested by two interventional cardiologists and LAAO researchers, obtaining very positive user feedback about its potential, highlighting VRIDAA as a source of motivation for trainees and its usefulness to better understand the required surgical approach before the intervention.

The exploration of time-dependent measured or simulated blood flow is challenging due to the complex three-dimensional structure of vessels and blood flow patterns. Especially on a 2D screen, understanding their full shape and interacting with them is difficult. Critical regions do not always stand out in the visualization and may easily be missed without proper interaction and filtering techniques. The FlowLens [GNBP11] was introduced as a focus-and-context technique to explore one specific blood flow parameter in the context of other parameters for the purpose of treatment planning. With the recent availability of affordable VR glasses it is possible to adapt the concepts of the FlowLens into immersive VR and make them available to a broader group of users. Translating the concept of the Flow Lens to VR leads to a number of design decisions not only based around what functions to include, but also how they can be made available to the user. In this paper, we present a configurable focus-and-context visualization for the use with virtual reality headsets and controllers that allows users to freely explore blood flow data within a VR environment. The advantage of such a solution is the improved perception of the complex spatial structures that results from being surrounded by them instead of observing through a small screen.

We present a VR-based prototype for learning the hand anatomy. The prototype is designed to support embodied cognition, i.e., a learning process based on movements. The learner employs the prototype in VR by moving their own hand and fingers and observing how the virtual anatomical hand model mirrors this movement. The display of anatomical systems and their names can be adjusted. The prototype is deployed on the Oculus Quest and uses its native hand tracking capabilities to obtain the hand posture of the user. The potential of the prototype is shown with a small user study.

Specific phobias are among the most common mental diseases, affecting the lives of millions of people. Yet, many cases remain untreated and even undiagnosed, partly due to entry barriers such as waiting times and inconvenience of therapy. To improve the therapeutic options and convenience for the treatment of specific phobias, we implemented a virtual reality application for treating acrophobia (fear of heights) with in-virtuo exposure therapy. Our concept is based on principles from psychology and interaction design. This concept is then implemented using the game engine Unity and Oculus Rift headset as a target device for VR display. Our application has a wide range of customization options, which enables it to be personalized to individual patients. In addition, a number of motivational methods are integrated, which are intended to increase patient motivation, as motivation is essential for a successful therapy.

We introduce a Virtual Reality (VR) one-on-one tutoring system to support anatomy education. A student uses a fully immersive VR headset to explore the anatomy of the base of the human skull. A teacher guides the student by using the semi-immersive zSpace. Both systems are connected via network and each action is synchronized between both systems. The teacher is provided with various features to direct the student through the immersive learning experience. The teacher can influence the student’s navigation or provide annotations on the fly and hereby improve the student’s learning experience.

Discord only: Analysis pipelines for RNA sequencing data often produce long lists of differentially expressed genes. For functional analysis, these lists can be analyzed using Gene Ontology (GO) (Ashburner et al., 2000) enrichment analysis tools such as PANTHER (Mi et al., 2019) and DAVID (Sherman et al., 2007) resulting in lists of enriched GO terms with associated p-values. Since GO terms are organized roughly hierarchical, an enrichment of a child term propagates to enrichments of parent terms which can lead to redundancy in the list. Moreover, when GO enrichments are computed for multiple experimental conditions, it can be of interest to compare and summarize the results. With our tool we introduce a technique to reduce the redundancy in multiple lists of GO terms and provide an interactive visualization for list comparisons. We developed a web application that takes multiple lists of GO terms or genes as input. For lists of genes GO term enrichment is computed using the PANTHER API resulting in lists of GO terms. The GO terms are clustered hierarchically using a version of the REVIGO algorithm adapted to use multiple lists of GO terms simultaneously (Supek et al., 2011). REVIGO clusters GO terms based on their semantic similarity, p-values, and relatedness resulting in a hierarchical clustering where less dispensable terms are placed closer to the root. The clustering is visualized in a clustered heatmap showing the p-values for each term. With sliders users can filter dispensable GO terms and select a cutoff for the extraction of non-hierarchical clusters resulting in a set of GO terms with reduced redundancy and a meaningful grouping. In order to compare the clusters a treemap is visualized for each condition. Moreover, the treemaps can be compared by animation which facilitates viewing changes betweeen conditions and bar charts for detailed comparisons of single terms. In order to provide a global overview, the overall similarity of the lists is visualized in a PCA plot and a correlation heatmap. A detailed table shows further information about the GO terms. This tool will enable researchers to compare the functional analysis of multiple experimental conditions. Moreover, it can be extended to compare GO enrichment results of any other application where gene lists are produced, such as the analysis of clusters in gene co-expression networks or the comparison of differentially expressed genes in multiple species.

Discord only: Sequence diagrams (SD) are a common way to visualize the amino acid sequence of proteins. Usually, not only the sequence is represented, but also other relevant information like binding sites or the secondary structure, if available. Although SD are often used in conjunction with 3D visualizations of the protein structure, they are also useful by themselves for visually analyzing protein data. SD are often visualized as static, precomputed images. Therefore, we present an interactive SD visualization that shows not only the attributes of the protein that are stored in the RCSB Protein Data Bank, but can also visualize additional information provided by external analysis tools. Furthermore, we try to enhance the basic idea of SDs by adding per-atom information instead of showing only per-amino-acid attributes. Figure 1 (left) shows our SD, which was implemented as an interactive, web-based visualization using the JavaScript library D3. Different attributes per amino acid are represented in the rows: the type of amino acid, the chain ID of the amino acid chain, the secondary structure, the BFactor, the hydrophobicity, the predicted intrinsic disorder, and the binding sites. Most of these attributes are encoded using colored rectangles. BFactor, hydrophobicity, and disorder prediction are quantitative attributes for which individual color gradients are used. The secondary structure is graphically depicted via different types of helices and arrows. Binding sites are encoded as colored rings. To provide users with more detailed information, we extended the idea of the classical SD that only shows per-amino-acid attributes. For per-atom attributes like the BFactor, our system computes the overall minimum and maximum values, and the average value per residue. Figure 1 (right) shows three different possible visualizations offered by our extended SD that show either the summary statistics or the individual values per atom. Our SD allows users to interactively hide attribute rows, change the number of amino acid per row, and it offers a tooltip with additional information if the user hovers an amino acid.

Discord only: Pangenome graphs built from raw sets of alignments may have complex structures which can introduce difficulty in downstream analyses, visualization, mapping, and interpretation. Graph sorting aims to find the best node order for a 1D and 2D layout to simplify these complex regions. Pangenome graphs embed linear pangenomic sequences as paths in the graph, but to our knowledge, no algorithm takes into account this biological information in the sorting. Moreover, existing 2D layout methods struggle to deal with large graphs. We present a new layout algorithm to simplify a pangenome graph, by using path-guided stochastic gradient descent to move a single pair of nodes at a time. We exemplify how the 1D path-guided SGD implementation is a key step in general pangenome analyses such as pangenome graph linearization and simplification.

Discord only: Molecular representations are taking an important role in communicating ideas, in generating new hypotheses on biological mechanisms and in analysing molecular simulations. However, the current devices used to observe and manipulate these molecular systems are typically limited to the two dimensions of the computer screen combined with a keyboard and a mouse offering limited interaction capabilities. Nowadays, virtual reality headsets offer a more performant and accessible solution. However, adaptations are necessary to fully benefit from the advantages of using virtual reality for scientific visualisation. This poster presents a few examples implemented with the UnityMol software. In addition to immediate applications in teaching, the paradigm shift in interaction and the increased depth perception and shape comprehension of biological molecules are already easing the grasp of these complex systems and will certainly lead to the discovery of new scientific knowledge.

We present an approach for visual analysis of high-dimensional measurement data with varying sampling rates in the context of an experimental post-surgery study performed on a porcine surrogate model. The study aimed at identifying parameters suitable for diagnosing and prognosticating the volume state—a crucial and difficult task in intensive care medicine. In intensive care, most assessments not only depend on a single measurement but a plethora of mixed measurements over time. Even for trained experts, efficient and accurate analysis of such multivariate time-dependent data remains a challenging task. We present a linked-view post hoc visual analysis application that reduces data complexity by combining projection-based time curves for overview with small multiples for details on demand. Our approach supports not only the analysis of individual patients but also the analysis of ensembles by adapting existing techniques using normalization and non-parametric statistics. We evaluated the effectiveness and acceptance of our application through expert feedback with domain scientists from the surgical department using real-world data: the results show that our approach allows for detailed analysis of changes in patient state while also summarizing the temporal development of the overall condition. Furthermore, the medical experts believe that our method can be transferred from medical research to the clinical context, for example, to identify the early onset of a sepsis.

Deep learning is increasingly used in the field of glaucoma research. Although deep learning (DL) models can achieve high accuracy, issues with trust, interpretability, and practical utility form barriers to adoption in clinical practice. In this study, we explore whether and how visualizations of deep learning-based measurements can be used for glaucoma management in the clinic. Through iterative design sessions with ophthalmologists, vision researchers, and manufacturers of optical coherence tomography (OCT) instruments, we distilled four main tasks, and designed a visualization tool that incorporates a visual field (VF) prediction model to provide clinical decision support in managing glaucoma progression. The tasks are: (1) assess reliability of a prediction, (2) understand why the model made a prediction, (3) alert to features that are relevant, and (4) guide future scheduling of VFs. Our approach is novel in that it considers utility of the system in a clinical context where time is limited. With use cases and a pilot user study, we demonstrate that our approach can aid clinicians in clinical management decisions and obtain appropriate trust in the system. Taken together, our work shows how visual explanations of automated methods can augment clinicians’ knowledge and calibrate their trust in DL-based measurements during clinical decision making.

A brain lesion is an area of tissue that has been damaged through injury or disease. Its analysis is an essential task for medical researchers to understand diseases and find proper treatments. In this context, visualization approaches became an important tool to locate, quantify, and analyze brain lesions. Unfortunately, image uncertainty highly effects the accuracy of the visualization output. These effects are not covered well in existing approaches, leading to miss-interpretation or a lack of trust in the analysis result. In this work, we present an uncertainty-aware visualization pipeline especially designed for brain lesions. Our method is based on an uncertainty measure for image data that forms the input of an uncertainty-aware segmentation approach. Here, medical doctors can determine the lesion in the patient’s brain and the result can be visualized by an uncertainty-aware geometry rendering. We applied our approach to two patient datasets to review the lesions. Our results indicate increased knowledge discovery in brain lesion analysis that provides a quantification of trust in the generated results.

Clinical Decision Support Systems (CDSS) provide assistance to physicians in clinical decision-making. Based on patient-specific evidence items triggering the inferencing process, such as examination findings, and expert-modeled or machine-learned clinical knowledge, these systems provide recommendations in finding the right diagnosis or the optimal therapy. The acceptance of, and the trust in, a CDSS are highly dependent on the transparency of the recommendation’s generation. Physicians must know both the key influences leading to a specific recommendation and the contradictory facts. They must also be aware of the certainty of a recommendation and its potential alternatives. We present a glyph-based, interactive multiple views approach to explainable computerized clinical decision support. Four linked views (1) provide a visual summary of all evidence items and their relevance for the computation result, (2) present linked textual information, such as clinical guidelines or therapy details, (3) show the certainty of the computation result, which includes the recommendation and a set of clinical scores, stagings, etc., and (4) facilitate a guided investigation of the reasoning behind the recommendation generation as well as convey the effect of updated evidence items. We demonstrate our approach for a CDSS based on a causal Bayesian network representing the therapy of laryngeal cancer. The approach has been developed in close collaboration with physicians, and was assessed by six expert otolaryngologists as being tailored to physicians’ needs in understanding a CDSS.

Many biochemical and biomedical applications like protein engineering or drug design are concerned with finding functionally similar proteins, however, this remains to be a challenging task. We present a new imaged-based approach for identifying and visually comparing proteins with similar function that builds on the hierarchical clustering of Molecular Surface Maps. Such maps are two-dimensional representations of complex molecular surfaces and can be used to visualize the topology and different physico-chemical properties of proteins. Our method is based on the idea that visually similar maps also imply a similarity in the function of the mapped proteins. To determine map similarity we compute descriptive feature vectors using image moments, color moments, or a Convolutional Neural Network and use them for a hierarchical clustering of the maps. We show that image similarity as found by our clustering corresponds to functional similarity of mapped proteins by comparing our results to the BRENDA database, which provides a hierarchical function-based annotation of enzymes. We also compare our results to the TM-score, which is a similarity value for pairs of arbitrary proteins. Our visualization prototype supports the entire workflow from map generation, similarity computing to clustering and can be used to interactively explore and analyze the results.

Amyloid-beta fibrils are the result of the accumulation of misfolded amyloid precursor proteins along an axis. These fibrils play a crucial role in the development of Alzheimer’s disease, and yet its creation and structure are not fully understood. Visualization is often used to understand the structure of such fibrils. Unfortunately, existing algorithms require high memory consumption limiting their applications. In this paper, we introduce a ray marching algorithm that takes advantage of the inherent repetition in these atomic structures, requiring only a 2D density map to represent the fibril. During ray marching, the texture coordinates are transformed based on the position of the sample along the longitudinal axis, simulating the rotation of the fibrils. Our algorithm reduces memory consumption by a large margin and improves GPU cache hits, making it suitable for real-time visualizations. Moreover, we present several shading algorithms for this type of data, such as shadows or ambient occlusion, in order to improve perception. Lastly, we provide a simple yet effective algorithm to communicate the uncertainty introduced during reconstruction. During the evaluation process, we were able to show, that our approach not only outperforms the Standard Volume Rendering method by significantly lower memory consumption and high image quality for low resolution 2D density maps but also in performance.

When studying the function of proteins, biochemists utilize normal mode decomposition to enable the analysis of structural changes on time scales that are too long for molecular dynamics simulation. Such a decomposition yields a high-dimensional parameter space that is too large to be analyzed exhaustively. We present a novel approach to reducing and exploring this vast space through the means of interactive visualization. Our approach enables the inference of relevant protein function from single structure dynamics through protein tunnel analysis while considering normal mode combinations spanning the whole normal modes space. Our solution, based on multiple linked 2D and 3D views, enables the quick and flexible exploration of individual modes and their effect on the dynamics of tunnels with relevance for the protein function. Once an interesting motion is identified, the exploration of possible normal mode combinations is steered via a visualization-based recommendation system. This helps to quickly identify a narrow, yet relevant set of normal modes that can be investigated in detail. Our solution is the result of close cooperation between visualization and the domain. The versatility and efficiency of our approach are demonstrated in two case studies.

We present ANEULYSIS, a system to improve risk assessment and treatment planning of cerebral aneurysms. Aneurysm treatment must be carefully examined as there is a risk of fatal outcome during surgery. Aneurysm growth, rupture, and treatment success depend on the interplay of vascular morphology and hemodynamics. Blood flow simulations can obtain the patient-specific hemodynamics. However, analyzing the time-dependent, multi-attribute data is time-consuming and error-prone. ANEULYSIS supports the analysis and visual exploration of aneurysm data including morphological and hemodynamic attributes. Since this is an interdisciplinary process involving both physicians and fluid mechanics experts, we provide a redundancy-free management of aneurysm data sets according to a consistent structure. Major contributions are an improved analysis of morphological aspects, simultaneous evaluation of wall- and flow-related characteristics as well as multiple attributes on the vessel wall, the assessment of mechanical wall processes as well as an automatic classification of the internal flow behavior. It was designed and evaluated in collaboration with domain experts who confirmed its usefulness and clinical necessity.

Tubular flow analysis plays an important role in many fields, such as for blood flow analysis in medicine, e.g., for the diagnosisof cardiovascular diseases and treatment planning. Phase-contrast magnetic resonance imaging (PC-MRI) allows for non-invasive in vivo-measurements of such tubular flow, but may suffer from imaging artifacts. New acquisition techniques (orsequences) that are being developed to increase image quality and reduce measurement time have to be validated against thecurrent clinical standard. Computational Fluid Dynamics (CFD), on the other hand, allows for simulating noise-free tubularflow, but optimization of the underlying model depends on multiple parameters and can be a tedious procedure that may runinto local optima. Data assimilation is the process of optimally combining the data from both PC-MRI and CFD domains.We present an interactive visual analysis approach to support domain experts in the above-mentioned fields by addressingPC-MRI and CFD ensembles as well as their combination. We develop a multi-field similarity measure including both scalarand vector fields to explore common hemodynamic parameters, and visualize the evolution of the ensemble similarities in alow-dimensional embedding. Linked views to spatial visualizations of selected time steps support an in-detail analysis of thespatio-temporal distribution of differences. To evaluate our system, we reached out to experts from the PC-MRI and CFDdomains and summarize their feedback.

Computed Tomography Angiography (CTA) is one of the most commonly used modalities in the diagnosis of cerebrovasculardiseases like ischemic strokes. Usually, the anatomy of interest in ischemic stroke cases is the Circle of Willis and its periph-erals, the cerebral arteries, as these vessels are the most prominent candidates for occlusions. The diagnosis of occlusionsin these vessels remains challenging, not only because of the large amount of surrounding vessels but also due to the largenumber of anatomical variants. We propose a fully automated image processing and visualization pipeline, which provides afull segmentation and modelling of the cerebral arterial tree for CTA data. The model itself enables the interactive masking ofunimportant vessel structures e.g. veins like the Sinus Sagittalis, and the interactive planning of shortest paths meant to be usedto prepare further treatments like a mechanical thrombectomy. Additionally, the algorithm automatically labels the cerebralarteries (Middle Cerebral Artery left and right, Anterior Cerebral Artery short, Posterior Cerebral Artery left and right) detectsocclusions or interruptions in these vessels. The proposed pipeline does not require a prior non-contrast CT scan and achievesa comparable segmentation appearance as in a Digital Subtraction Angiography (DSA).