I have tested how Unity can function as a live visual tool for concerts. For this I linked Ableton and Unity and used the midi information of a song from Ableton to control different particle systems in Unity.

The Song "AICS" stands for All I Can Say (...after a break-up of a long-term relationship) and is supposed to stand for a feeling of being uplifted after a break-up, which usually has a negative connotation but very often also brings a lot of positive sides and new ways, which are not seen first in our society, because for most of us single existence is bad and every person has to be dead sad after a break-up.

Bildschirmfoto 2022-03-31 um 13.54.13.png

_Visual: The particle systems show different lightweight elements like clouds, soap bubbles, flowers. These float out of my head, which I recreated with the help of the Blender plugin "Facebuilder".

Facebuilder: https://keentools.io/download/facebuilder-for-blender

Particle: https://quixel.com/bridge


Visual FX Package

_Audio: I wrote the song for a project of mine and it consists entirely of voice and voice samples that can be played keyboard-like with a midi keyboard. The particle systems are linked to the bass notes and change accordingly when the bass note changes. With the help of Bolt, after installing the necessary plug-ins, I was able to assign different actions to each midi note. For this example, I only used play particle system and stop playing.

Bildschirmfoto 2022-03-31 um 13.53.36.png

Plug Ins and tools for Ableton > Unity

MIDIMusical Instruments Digital Interface see also →OSC input plugin for Unity https://github.com/keijiro/MidiJack

MIDIMusical Instruments Digital Interface see also →OSC Input for New Input System -- A plugin that adds MIDIMusical Instruments Digital Interface see also →OSC input support to Unity's new Input System https://github.com/keijiro/Minis

virtualMIDI | Tobias Erichsen https://www.tobias-erichsen.de/software/virtualmidi.html

Setup MIDIMusical Instruments Digital Interface see also →OSC on macOS https://feelyoursound.com/setup-midi-os-x/

Screen Video:

Fake Live Trial:

Conclusion live context:

As can be seen in the screen video, my computer reaches its capacity limit with Ableton and Unity running, as Unity must always be in "Game" mode for the Bolt task to work and Ableton must also handle some live processing content such as effects or arpeggiator. In addition, the assignment of the midi notes is very confusing because it is only carried out in numbers. I made a table for this and assigned each number to the respective notes. This makes the preparation of a song very time-consuming, because you can only work with single Midi notes. Each midi note requires a new task. Which can be an advantage with more complex compositions, however. Despite these circumstances, I would like to continue working in this area, as I find the possibilities of Unity for visuals very exciting and would like to research the flexibility for the live context.

More tutorials:

visual fx graph unity object: https://www.youtube.com/results?search_query=visual+fx+graph+unity+object

Bolt: https://www.youtube.com/watch?v=aQceChK-kC4

Facebuilder: https://youtu.be/CfbZ_xP8yq8




(Teamwork by Lisa Zwinzscher, Christian Dähne and Alex Graeff)

> Since this is a group project, I will only briefly present the work.

Framed by the musicvideo-festival Bandstand of the European Centre of the Arts - HELLERAU, H. C. BEHRENDTSEN dig deeply into their sleeves of tricks. This endeavor results in a pocket-sized blockbuster - virtually a crime thriller at marshmallow-landia. Allusions to Ibiza-gate involving H. C. Strache and its whole pandora of embarrassing scandals within Austria’s right-wing conservative establishment are perpetually spilling forth. Amongst the “Red Bull Brother from Austria”, Kanzler Kurz’s spirit animals, nocturnal data cleansings and peculiarly unsettling chat protocols, H. C. BEHRENDTSEN’s digital lowpoly-duplicates are floundering and gazing across the scenes, having to concern: WTF!?!

H. C. BEHRENDTSEN create sounds between experimental music and math rock. Eruptive spontaneity, intricate grooves and mashupy textures between the uplifting and the bizarre are consequences of both: infatuation for contemporary composing as well as for the heroes of their adolescence.

Christian Dähne: idea and concept, creation & animation H. C. BEHRENDTSEN low poly characters, particle systems, camera, skyboxes

Lisa Zwinzscher: design and composition island set up and props, animation, particle systems, creation of textures, camera

Alex Graeff: director

Realisation of group work:

Conception and collection of ideas via Miroboard https://miro.com

Bildschirmfoto 2022-03-31 um 17.27.04.png

Working on a Unity project via the "Collaborate" function


Since it is not possible to work on a project at the same time, a good schedule must be set up.