Research

The Virtual Reality and Visualization Group is part of many international research project consortia. Please find an overview of past and present projects below.

Objective: As part of the project Bauhaus Participation Lab, we are investigating the use of social mixed reality for citizen participation in municipal planning processes. Through an immersive walkthrough of digital designs and parametric models, we aim to provide a better understanding of planning concerns as well as spatial understanding of alternative designs that will enable citizens to form their own opinions and contribute informed proposals.

To this end, we are developing a pilot social mixed reality application that will be used in a total of three participatory workshops held on the "Bahnstadt Weimar" development concept. We will implement and evaluate novel approaches to address the special requirements that the application context places on interaction and navigation in different design variations.

The consortial project is funded by the Bundesministerium für Bildung und Forschung  as part of the initiative "Die digitale Kommune" and is carried out by the partners Bauhaus-Universität Weimar, Deutsches Institut für Urbanistik, Form Follows You GmbH, Consensive GmbH and Stadtentwicklungsamt Weimar.

Further information is available on the project website: bpl.vreval.de

 

Objective: Joint activities such as dancing, singing or movement games are based on the synchronisation of interpersonal interaction and are particularly well suited for evoking feelings of belonging and strengthening social bonds. GROOVE has set itself the goal of enabling social-virtual entrainment, i.e. experienced behavioural synchronisation, despite inevitable transmission latency in social virtual reality through intelligent interaction design. The aim is to specifically address the cognitive psychological factors that are the basis for the synchronisation of human behaviour. In addition, techniques will be developed to support behavioural adjustment, including rhythm generators within the virtual synchronised environment or the creation of synchronised movement illusions. On the technical side, virtual entrainment is to be supported effectively through a clock-synchronised environment with constant and low latency as well as through the algorithmic prediction and completion of periodic movement activity. Dancing, making music and playing games together can therefore also be done over long distances - albeit with slightly different rules. Interactive content creates incentives for activity and shared experiences that go far beyond the possibilities of phone calls and video conferencing. With the help of longitudinal studies, we want to document the emergence and development of social-virtual entrainment for our target groups (families, couples, close circles of friends). Empirical analyses of the psychological, physiological and behavioural factors should make it possible to operationalise the quality of virtual (co-)presence and to assess the conditions under which entrainment-optimised social virtuality is able to promote feelings of closeness and connectedness. 

The GROOVE project is funded by the Bundesministerium für Bildung und Forschung (BMBF) as part of the BMBF research programme ‘’Interactive Technologien für Gesundheit und Lebensqualität‘’ in the call ‘’Nähe über Distanz - Mit interaktiven Technologien zwischenmenschliche Verbundenheit ermöglichen‘’.

Further information can be found on the project website and the BMBF information page. 

Objective: The Teaching VR in VR project designs, implements and evaluates an immersive module for teaching, learning and testing of VR materials for the Master's course Virtual Reality of the Computer Science department at Bauhaus-Universität Weimar. Teachers demonstrate content to students in live interactive lectures in VR. Students can actively participate in these lectures by trying out VR techniques individually or together. Teaching and learning materials are made available in a persistent social virtual world as interactive VR recordings that can be accessed at any time and from any location. Teaching VR in VR is designed to enhance the experience of the content taught and to increase awareness and understanding of usability issues and limitations.

Teaching Virtual Reality in Virtual Reality is funded by the Thüringer Ministeriums für Wirtschaft, Wissenschaft und Digitale Gesellschaft (TMWDDG) and the Stifterverband.

Objective: In diesem Projekt kooperieren die Bauhaus-Universität Weimar (BUW) und die Europa-Universität Viadrina (EUV) mit dem Ziel, ein sozialwissenschaftliches KI-Lab für Forschendes Lernen (SKILL) aufzubauen. Studierende werden hier in den Prozess der Entwicklung und der Anwendung KI-basierter Technologien eingebunden. Diese Technologien werden in der Lage sein, die Struktur von Argumentationen in wissenschaftlichen Artikeln und politischen Arenen zu erkennen, zu analysieren und zu visualisieren und den Studierenden Erkenntnisse für die eigene universitäre Arbeit zur Verfügung zu stellen.

Das Projekt realisiert damit eine neue Qualität von Lern- und Lehrunterstützung für diskursive Analysen. Studierende lernen mit Unterstützung von SKILL Techniken des Argumentierens, sie lernen, schneller systematisches Wissen über den Stand einer Debatte zu erlangen, und sie gewinnen ein vertieftes Grundverständnis für die Funktionsweise, den Nutzen und die Grenzen von KI-Technologien.

SKILL wird vom Bundesministerium für Bildung und Forschung (BMBF) finanziert.

Objective: Im Rahmen des Verbundprojekts Goethe Live 3D werden immersive Lösungen zur Kultur- und Wissensvermittlung bei Museumserlebnissen entwickelt, umgesetzt und evaluiert. Das Goethe Nationalmuseum Weimar dient hierfür als Fokuspunkt für die Anforderungsanalyse, Anwendungsentwicklung und Evaluation. Unsere Projektergebnisse sollen in einer öffentlich zugänglichen Ausstellung vor Ort und ortsunabhängig in einer digitalen Museumskopie gemeinsam und zeitgleich  für lokale und nicht-lokale Besuchsgruppen erlebbar sein.

Das Verbundprojekt wird vom Bundesministerium für Bildung und Forschung (BMBF) finanziert und unter der Koordination der Weimarer Virtual Reality and Visualization Research Group mit den Partnern Universität Hamburg, Klassik Stiftung Weimar, ArcTron 3D GmbH, Curvature Games GmbH und Consensive GmbH umgesetzt. 

Weitere Information sind auf der Projektwebseite zu finden.

Objective: The APlausE-MR project investigates the plausibility of communication in virtual environments (VEs), focusing specifically on multi-party mixed reality scenarios involving groups of collocated and remote users. The project strives to identify factors that create and maintain plausibility, in order to apply this knowledge to the design of future VEs, and to test the hypothesis that such factors can compensate for each other in a multi-modal setting. To this end, we are developing a distributed VE that supports gestural communication by representing users as abstract and volumetric avatars, as well as supporting high-fidelity audio communication through binaural auralization of sound sources.

The APlausE-MR project is a collaboration with audio and communication experts at TU Ilmenau, and is part of the AUDICTIVE initiative.

Objective: Das Prioritätsvorhaben „Grundlagen der Digitalisierung in Denkmalpflege und Sanierung“ dient der Erschließung und Vermittlung innovativer Technologien für eine kontinuierliche digitale Bestands- und Zustandserfassung historischer Bauwerke sowie effektiver neuer Werkzeuge für die Bauwerkserhaltung und Sanierungsplanung. Darüber hinaus sollen Möglichkeiten und Synergieeffekte einer offenen Nutzung von 3D-Bauwerksmodellen als gemeinsame Plattform für die Planung und Umsetzung digitaler Innovationen in angrenzenden Bereichen der regionalen Bildung sowie des Kultur-und Tourismusmarketings untersucht werden. Extended Description

Links: https://www.vogtlandpioniere.de/ausschreibungen-projekte/projekte/

Objective: Mit MetaReal verfolgen wir die Vision, den Zugang zum Wissen durch den Einsatz von fortgeschrittenen Technologien der Virtual und Augmented Reality (VR, AR) sowie der Künstlichen Intelligenz (KI) auf die nächste Stufe zu heben. Wir werden Vorgehensweisen, Verfahren und Interfaces erforschen und entwickeln, um detaillierte digitale 3D-Modelle der realen Welt mit den im Internet vorhandenen komplexen Wissensbeständen anzureichern und für Personengruppen in VR- und AR-Umgebungen natürlichsprachlich im räumlichen Kontext erschließbar zu machen: An die Stelle der isolierten Interaktion mit Bild-schirmen tritt eine immersive und kollektive Erfahrung im virtuellen Raum in einer Art begehbarer Wikipedia, bei der Wissen durch körperliche Bewegungen und sprachliche Äußerungen abgerufen und angeeignet werden kann.

Das Projekt MetaReal wurde durch das Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft im Rahmen des Programms ProDigital (5575/10-5) gefördert.

Weitere Information sind auf der Projektwebseite zu finden. 

Beitrag in Zeit Campus

Objective: Today, touch interaction is very common and especially smartphones have established it in everyday life. Yet, visual analytics still almost exclusively relies on desktop setups controlled with mouse and keyboard. This project aims to make collaborative visual analytics processes more effective by developing innovative multitouch interaction methods. Such a system will allow multiple analysts to explore and gain insights into data. In particular the massive amounts of data that are collected during Earth observation pose a challenge and often rely on multiple linked views that visualize different aspects of the data at once. Consistent multitouch interactions for visualizations on large, high-resolution displays will eventually put the users at the center of visual analysis and the data at their finger tips.

This project is a cooperation with the Visual Analytics group at the Institute of Data Science of the German Aerospace Center (DLR). 

Projektziele: Ziel des KMU-innovativ Projekts Virtuelle Baugrube (ISBau) ist die Entwicklung neuer Kommunikations- und Interaktionstechnologien für die Fernüberwachung und die verteilte Steuerung von Baumaschinen. Am Beispiel von Telematikschnittstellen für Saugbagger werden die Anforderungen und Möglichkeiten zu deren intelligenten Steuerung in Zusammenarbeit mit ortsfernen Experten evaluiert. Dies umfasst den direkten Austausch über die Situation vor Ort durch Telepräsenztechnologien. Dazu wird eine Bausituation in Echtzeit dreidimensional aufgezeichnet, mit Plänen der Maschine und Sensordaten angereichert und als gemeinsame virtuelle Realität nutzbar gemacht. 4D-Aufzeichnungen dieser Situationen und Prozessen bieten auch realistische Trainingsmöglichkeiten für die Ausbildung. Extended Description 

Objective: Das Kooperationsprojekt AISTEC widmet sich innovativen digitalen Methoden zur Überwachung von Strukturbauwerken, insbesondere Brücken. Ziel ist die Entwicklung einer digitalen Prozesskette von der automatisierten Datenerfassung mit unbemannten Flugsystemen über die Modellbildung und semiautomatische Analyse bis zur Tragwerkssimulation und visuellen Prüfung in kollaborativer virtueller Realität. Durch die neuen Methoden kann der Zustand von Brücken und anderer Infrastruktur einfach, kostengünstiger und häufiger überwacht werden. 

Beitrag in der Fachzeitschrift Crisis Prevention
 

Objective: The Human Brain Project is an interdisciplinary European research project, in which scientists from neuroscience, medicine, physics and computer science work together to simulate the human brain in the computer. In the Neurorobotics subproject, the simulated brain models are connected to robots in order to investigate how the brains react to and deal with stimuli and to develop brain-inspired robot controls. Immersive virtual reality enables the realistic and plausible experience of simulated virtual robots and environments and offers important advantages for the observation, understanding and participation in virtual neurorobotics simulations.  

Objective: Data analysis techniques in the age of Big Data are becoming increasingly complex and combine a variety of intelligent technologies. Although the outcomes and results of data analyis are impressive, they are generally difficult to be reproduced and understood by users. Visualization is a first step towards presenting results in a comprehensible manner. However, the underlying motivation, the origin of the data, the transformation processes and the insights gained in the process remain hidden. The objective of the project is to make these processes more transparent to the users.

A research building recommended by the German Wissenschaftsrat, DFG and the state of Thuringia, funded by the federal government and the state of Thuringia.

Objective: The junior research group Big Data Analytics is focusing on scalable data analysis and mining techniques and their interplay with research fields like visualization, retrieval and algorithm engineering. Tools and algorithms to support analytic processes in big data will be developed in cooperation with experts from diverse application areas.

Objective: 3D-PITOTI is a European research project (FP7) that aims to develop novel technology for the acquisition and interactive visualization of ancient rock art like the Pitoti in northern Italy.

Objective: VR-HYPERSPACE (AAT-285681) was an ambitious project funded under the European Commission’s Seventh Framework Aeronautics and Air Transport (AAT) programme.  Begun in October 2011, this project consisted of 9 internationally leading universities, research institutes and industrial partners from 6 European countries. The project ended in September 2014. Adopting radical approaches using virtual reality and mixed reality technologies, and the latest studies in neuroscience and psychology, we are carrying out fundamental research and development leading to a paradigm shift in relation to passenger comfort.

Objective: Strategies for the robust design of structures.

Objective: Virtual environments hold the promise to provide a shared space in which such collaborative interactions of multiple users with virtual 3D models could occur as easily and smoothly as they would in the real world. Large projection-based stereoscopic displays are the most successful incarnation of such immersive virtual environments. These systems use a fixed view point or track the head position of a single user and compute a stereoscopic image pair for that point of view. Unfortunately, when non-tracked users see these images, the virtual environment may appear severely distorted and it changes shape when users are moving.The project IMVIS will carry out research to develop novel projection-based stereoscopic multi-view systems for co-located collaborative 3D displays for multiple users.

Our group develops software frameworks in different research and application areas which contribute essentially to our research projects. Please find an overview of our current developments here.

Avango/Guacamole is a Python/C++ framework for the development of complex virtual reality applications. It provides

• a modern scene graph,
• rapid application development in Python,
• support for various input devices,
• an efficient multi-GPU rendering back-end for different geometry types (including large point cloud rendering based on Lamure),
• 3D-reconstructions from RGBD-sensors,
• a programmable shading-pipeline for sophisticated rendering effects,
• and a network-distribution layer.

Please cite our work when using our software in your own research or publication.go to repository

Related publications:

• Schollmeyer, A., Schneegans, S., Beck, S., Steed, A., Froehlich, B.
  Efficient Hybrid Image Warping for High Frame-Rate Stereoscopic Rendering
  In IEEE Transactions on Visualization and Computer Graphics 23(4), pp. 1332 - 1341, 2017, and In Proceedings of IEEE VR 2017, Los Angeles, USA, March 2017. DOI: https://doi.org/10.1109/TVCG.2017.2657078
  [preprint][video]
• Kunert, A., Kulik, A., Beck, S., Froehlich B.
  Photoportals: Shared References in Space and Time
  In Proceedings of the 17th ACM conference on Computer supported cooperative work & social computing (CSCW '14). ACM, New York, NY, USA, pp. 1388-1399, February 2014
  DOI=10.1145/2531602.2531727   
  [abstract, paper and video]
• Schneegans, S., Lauer, F., Bernstein, A.-C., Schollmeyer, A., Froehlich, B.
  guacamole - An Extensible Scene Graph and Rendering Framework Based on Deferred Shading
  In Proceedings of the 7th Workshop on Software Engineering and Architectures for Realtime Interactive Systems (SEARIS '14) in conjunction with IEEE Virtual Reality 2014, Minneapolis, Minnesota, USA, 30th March, 2014
  [abstract, preprint]
• Beck, S., Kunert, A., Kulik, A., Froehlich B.
  Immersive Group-to-Group Telepresence (Best Paper Award)
  IEEE Transactions on Visualization and Computer Graphics, 19(4):616-625, March 2013 (Proceedings of IEEE Virtual Reality 2013, Orlando, Florida).
  [abstract][preprint][video]
• Kulik A., Kunert A., Beck S., Reichel R., Blach R., Zink A., Froehlich B.
  C1x6: A Stereoscopic Six-User Display for Co-located Collaboration in Shared Virtual Environments
  In Proceedings of the 2011 SIGGRAPH Asia Conference (SA '11). ACM, New York, NY, USA, Article 188, 12 pages.
  DOI=10.1145/2024156.2024222 doi.acm.org/10.1145/2024156.2024222 
  and
  ACM Transactions on Graphics 30, 6, Article 188 (December 2011), 12 pages.
  DOI=10.1145/2070781.2024222 doi.acm.org/10.1145/2070781.2024222
  [abstract, paper and video]

Related projects:

• Provenance Analytics, Big Data Analytics, 3D-Pitoti, VR-Hyperspace, IMVIS
  

Lamure is a complete standalone solution for preprocessing and real-time out-of-core rendering of very large 3D-point cloud models (including a basic support for large textured meshes). Multiple C/C++ example applications are included. Please cite our work when using our software in your own research or publication.go to repository

Related publications:

• Schollmeyer, A., Schneegans, S., Beck, S., Steed, A., Froehlich, B.
  Efficient Hybrid Image Warping for High Frame-Rate Stereoscopic Rendering
  In IEEE Transactions on Visualization and Computer Graphics 23(4), pp. 1332 - 1341, 2017, and In Proceedings of IEEE VR 2017, Los Angeles, USA, March 2017. DOI: https://doi.org/10.1109/TVCG.2017.2657078
  [preprint][video]

Related projects:

• Provenance Analytics, Big Data Analytics, 3D-Pitoti

Rgbd-calib is a C/C++ framework for the calibration of multiple RGD-sensors (Kinect version 1 and 2) into common coordinate system. Multiple C/C++ example applications are included. Please cite our work when using our software in your own research or publication.go to repository

Rgbd-recon is a C/C++ framework for the real-time 3D-reconstruction from multiple RGD-sensors (Kinect version 1 and 2) sensor streams on the GPU. Multiple C/C++ example applications are included. Please cite our work when using our software in your own research or publication.go to repository

Related publications:

• Beck, S., Froehlich, B.
  Sweeping-Based Volumetric Calibration and Registration of Multiple RGBD-Sensors for 3D Capturing Systems. In Proceedings of IEEE VR 2017, Los Angeles, USA, March 2017, pp. 167-176. [abstract, preprint and video]
• Beck, S., Froehlich, B.
  Volumetric Calibration and Registration of Multiple RGBD-Sensors into a
  Joint Coordinate System. In Proceedings of IEEE Symposium on 3D User Interfaces (3DUI), Arles, France, pp. 89-96, March 2015. DOI=10.1109/3DUI.2015.7131731
  [abstract, preprint and video]
• Liu, Y., Beck, S., Wang, R., Li, J., Xu, H., Yao, S., Tong, X., Froehlich, B.
  Hybrid Lossless-Lossy Compression for Real-Time Depth-Sensor Streams in 3D Telepresence Applications
  In: Ho YS., Sang J., Ro Y., Kim J., Wu F. (eds) Advances in Multimedia Information Processing -- PCM 2015. Lecture Notes in Computer Science, vol 9314. Springer, Cham, pp. 442-452.
  DOI=10.1007/978-3-319-24075-6_43
  [abstract and preprint]
• Beck, S., Kunert, A., Kulik, A., Froehlich B.
  Immersive Group-to-Group Telepresence (Best Paper Award)
  IEEE Transactions on Visualization and Computer Graphics, 19(4):616-625, March 2013 (Proceedings of IEEE Virtual Reality 2013, Orlando, Florida).
  [abstract][preprint][video]

Related projects:

• VR-Hyperspace
  

The schism C++ libraries, developed by Christopher Lux, offer object-oriented interfaces to the OpenGL C-language API with related interfaces for Nvidia CUDA and OpenCL compute language interoperability and various utilities to help with common real-time graphics related tasks. The libraries are written in standard C++ using OpenGL 3.3 to 4.2 core profile functionality and are useable on Windows as well as Linux operating systems. go to repository

Related publications:

• Lux, Christopher, Fröhlich, Bernd
  GPU-Based Ray Casting of Stacked Out-of-Core Height Fields
  In Proceedings of the 7th International Symposium on Visual Computing, LNCS vol. 6938, Springer-Verlag, Berlin, Heidelberg, pp 269-280, October 2011.
  [abstract][preprint]