Music Performances in Virtual Reality

Music Performances in Virtual Reality

Prof. Dr. Bernd Fröhlich
M.Sc. Sebastian Stickert

DegreeStudy ProgrammeExamination RegulationsECTS
B.Sc.Medieninformatikall15
M.Sc.Computer Science for 
Digital Media
PV18 and lower15
M.Sc.Computer Science for
Digital Media
PV2012
M.Sc.Computer Science and Mediaall15
M.Sc.Human-Computer InteractionPV17 and lower15
M.Sc.Human-Computer InteractionPV1912/18

Motivation

Music performances offer a vast design space that has historically evolved according to interactive multimedia systems. Artists and consumers alike appreciate a live music performance not only for auditory stimuli but for the sum of all aspects that can be found at a venue. Desktop systems, tablets and smartphones can only address this demand to a limited extent. That is why an increasing number of applications target VR technology to deliver compelling immersive experiences for both artists and audiences.

Our vision for a VR live concert experience includes an artistic representation for the performance, channels for communication of artist to crowd, crowd to artist and crowd to crowd as well as a suitable degree of agency for all participants. Tools and systems supporting such a concert experience require suitable integrations into the creative workflows of artists.

In this project, we will take first steps towards the aforementioned vision. Our investigations will begin by reviewing available applications and related work for VR performances and music-based creation of dynamic virtual environments. We will then evaluate the benefits and drawbacks of the various approaches. Finally, we will implement a set of promising techniques towards a system that enables music performances in VR.

Hardware

Due to the covid-19 pandemic, working from our lab may only be possible to a limited extent. Nevertheless, we want to supply you with state-of-the-art hardware and boost your journey into immersive virtual worlds.

For the duration of the project, each participant will be supplied with an Oculus Quest 2 HMD. We will also hand out desktop workstations to meet any additional hardware requirements, which your own machine cannot handle.

Depending on the scope of your contribution, we will consider acquiring additional hardware (such as a multi-channel audio interface) to integrate your own instruments or vocals.

Apart from deploying your applications using this hardware, we strongly encourage you to explore related VR applications in the wild!

Requirements

As well as willingness to work in a team, and enthusiasm for learning about multi-user VR application development, you should have the following competencies:

  • solid programming / scripting skills (C#, C++, Python or similar)
  • some practical experience in making music or audio technology

Additional advantages are experience in:

  • Unity3D development
  • digital signal processing
  • audio production & stage design
  • social VR development

If you are in doubt as to whether you fulfil the requirements, or if you have any further questions regarding the project, we are happy to have a discussion with you during the project fair on 11th of October. You can find us in our BigBlueButton room at the fair between 5pm and 6pm.

Assessment

The final assessment of your work will be conducted based on the project contributions of every team member, including:

  • Quality and innovation of your practical results
  • Active participation in the project during and in between weekly meetings
  • Intermediate and final project presentations
  • Documentation

Are you enthusiastic about music and feel inspired to apply your passion to virtual reality? Then, do not hesitate to visit us at the project fair.