GMU:Bits, Beats & Pieces/Lalalolo/lalalolo/Projekt: Difference between revisions

Revision as of 21:37, 21 March 2019

<automata>

Context:

'Life is a simulation' or is it? The quest for finding a set of rules to simulate life in some shape or form has been around for ever. Whether it is in search for the free will or just to find out what makes things living. One mathematical approach got introduced to world in 1970 by the British mathematician John Horton Conway, the Game of Life. The so called Game of Life (or GoL in short) is a 'cellular automaton', a type of simulation that works on a regular grid of cells and in discrete time-steps. For every time-step a set of rules is enforced for every cell, determining it's state. In most cellular automaton there are only two states 'living' and 'dead'. Depending on the rules and other constrains of the simulation, complex patterns might occur and this project will try to take these patterns and represent them as music.

@@ Line 1: / Line 1: @@
-==Never ending match cuts==
+==<automata>==
 ===Context:===
-Match cuts, being the art of transitioning from one scene to another while continuing the context in some form, are a common trope in filmmaking.
+'Life is a simulation' or is it?
-Keeping the context using a match cut may be archived by keeping a similar frame-composition, motion-path or audio backdrop.
+The quest for finding a set of rules to simulate life in some shape or form has been around for ever.
-Match cuts may be spatial, temporal or metaphorical as long as continuity may be perceived.
+Whether it is in search for the free will or just to find out what makes things living.
+One mathematical approach got introduced to world in 1970 by the British mathematician John Horton Conway, the Game of Life.
+The so called Game of Life (or GoL in short) is a 'cellular automaton', a type of simulation that works on a regular grid of cells and in discrete time-steps. For every time-step a set of rules is enforced for every cell, determining it's state. In most cellular automaton there are only two states 'living' and 'dead'. Depending on the rules and other constrains of the simulation, complex patterns might occur and this project will try to take these patterns and represent them as music.
-From cinematic classics like Stanley Kubrick's 2001 A Space Odyssey to  modern music videos like Radiohead’s Daydreaming match cuts create transitions without stopping the general flow of an audio-visual impression.
 ===Concept:===
-Match cuts are usually crafted by hand with care and attention to detail. In contrast to that I will try to create match cuts programmatically and in the spirit of “Me and the Machine” use an algorithm to find fitting cuts in found footage videos to create an endless stream of matching videos.
-The idea is to use the endless stream of audio visual content of the modern day world to create something new and unique that looks and feels continuous and might resemble something handcrafted with context.
-Due to  technical constraints the techniques used might vary, but if possible, I would like to use all forms of match cuts from simple audio matches to complex contextual or even metaphorical matches.
-To further give the work meaning, I might limit the pool of videos to choose from or possibly even include interaction by the observer, while keeping the idea of an endless audio-visual stream.
 ===Approach:===
-To achieve this endless stream continuous image and audio analysis is needed. Both of theses tasks are of high computational cost and no easily done in a live-context.
-Visual match cuts might be programmatically solved by comparing the last frame of a chosen sequence with frames of another video using different approaches.
-The first approach might be as easy as comparing histograms, which is fast, but is only really feasible for colored imagery and might fail to guarantee continuity.
-Feature matching might be possible for finding similar image compositions and or things  of similar shape in two frames. This method might be able to recreate metaphorical and spatial match cuts.
-Deep Ranking might give best results in finding similar images, creating the feeling of continuous imagery, but feasible solutions are scarce and training a neural network takes time and a lot of prepared data. This approach might still be possible using an existing framework or image-comparison API.
-Using audio feature extraction, classification and segmentation one might be able to find similar audio clips in videos thus creating audio match cut.