Medien Wiki - User contributions [en]

GMU:BioArt Forum/Lichens workshop

2022-11-08T08:48:45Z

Js: added links

BioArt Forum: Lichens workshop with Dr. Jan Vondrak

Meeting: Marienstrasse 7b, room 204<br>
Meeting point: Marienstrasse 7b, room 204<br>
Max participants: 20

* Saturday, 12.11.2022. 10.00 -18.00
* Sunday, 13.11.2022. 10.00 - 15.00

''Dr. Jan Vondrak is an academic assistant at the [https://botanika.prf.jcu.cz/english/ University of South Bohemia, Faculty of Biological Sciences, Department of Botany in Budweis Jeske Budovice (CZ)]. Jan Vondrak is one of the few lichen specialists in the world and an experienced mycologist.''

==EN==
(scroll down for the version in German)

For our first workshop of the Environmental Nature Labs we have the very experienced mushroom and lichen researcher Dr. Jan Vondrak from the [https://botanika.prf.jcu.cz/english/ botanical institute of the University of Budweis] was invited to Weimar. There are only a few such proven connoisseurs of lichens as him, most of them come from the Czech Republic, the reasons for this will be explained later. Jan is willing to go on two excursions with the participants in the city/outskirts of Weimar and present lichens and their ecology living on natural and building sites. Bringing a strong magnifying glass and umbrellas is recommended. However, we also provide a set of magnifying glasses on loan, and there is also the opportunity to learn about the study and identification and preservation of these creatures in the Environmenatal Biolaboratory (microscopy, color tests, etc.).

Lichens are inconspicuous, often annoying, but so widespread that they make an enormous contribution to the regulation of CO2 and nitrogen oxides in the atmosphere. It is also about getting a different perspective and looking at the colonization of lichens in gardens, house walls and paths in a positive way and forgoing their chemical or physical removal.

In addition, lichens are partners from two organisms that have blessedly come together: a fungus and a green or blue-green algae (cyanobacteria). The fungus and its partner live together on a substrate (rock, humus, bark). The fungus receives energy-rich carbohydrates from the partner that carries out photosynthesis and in return the fungus supplies water and mineral salts from the subsoil and forms pigments that protect the algae/bacteria from excessive UV radiation. Lichens grow on bark, leaves, rocks, dry stone walls or on the ground. Lichens grow very slowly compared to mosses and flowering plants. Some only a few millimeters to a centimeter per year. They can live for hundreds to thousands of years. This makes them one of the longest-lived creatures on earth. Like mosses, they can survive longer periods of drought "in their sleep" and start photosynthesis and continue to grow when humidity increases. Like mosses, they are regarded as pioneer organisms. That means they are found in extreme locations where little to no other plants survive. For example, barren nutrient-poor rocks or on walls and walls. Lichens are not parasites and do not harm the tree. They can even protect it from bacteria and fungi. They do not penetrate the living part of the tree (cambium), but sit on the outermost layer, the bark.

==DE==
(English above)

Wir haben für unseren ersten Workshop der Environmental Nature Labs den sehr erfahrenen Pilz- und Flechtenforscher Dr. Jan Vondrak vom [https://botanika.prf.jcu.cz/english/ botanischen Institut der Universität Budweis] nach Weimar eingeladen. Es gibt nur wenige so ausgewiesene Kenner von Flechten wie Ihn, die meisten stammen aus Tschechien, die Gründe dafür werden noch erläutert. Jan wird mit den Teilnehmern zwei Exkursionen im Stadt/Randgebiet von Weimar unternehmen und dort auf Natur- und Bauflächen lebende Flechten und Ihre Ökologie vorzustellen. Die Mitnahme einer starken Lupe und von Regenschirmen ist empfohlen. Wir stellen jedoch auch ein Set von Lupen leihweise zur Verfügung. Es besteht auch die Möglichkeit etwas über die Untersuchung und Identifizierung und Bewahrung dieser Lebewesen im Environmenatal Biolabor zu erfahren (Mikroskopie, Farbtests etc.).
Flechten sind unauffällig, oft lästig, aber derart verbreitet, daß sie einen enorm hohen Beitrag zur Regulation von CO2 und Stickoxiden in der Atmosphäre leisten. Es geht auch darum einen anderen Blick zu bekommen und die Ansiedlung von Flechten in Gärten, Hausmauern und Wegen positiv zu betrachten und auf Ihre chemische oder physikalische Beseitigung zu verzichten.

Flechten sind Partnerwesen aus zwei Organismen, die sich segensreich zusammen geschlossen haben: Einem Pilz und einer Grün- oder Blaualge (Cyanobakterien). Der Pilz und sein Partner leben zusammen auf einem Substrat (Gestein, Humus, Rinde). Der Pilz erhält von dem Partner, der Fotosynthese betreibt, energiereiche Kohlenhydrate und im Gegenzug liefert der Pilz Wasser und Mineralsalze aus dem Untergrund und bildet Pigmente, die die Algen/Bakterien vor zu starker UV-Strahlung schützen. Flechten wachsen an Rinde, Blättern, Felsen, Trockenmauern oder auf dem Boden. Flechten wachsen im Vergleich zu Moosen und Blütenpflanzen sehr langsam. Manche nur wenige Millimeter bis einen Zentimeter pro Jahr. Sie können mehrere hundert bis tausend Jahre alt werden. Damit gehören sie zu den langlebigsten Lebewesen der Erde. Ebenso wie Moose können sie längere Trockenperioden “im Schlaf“ überleben und bei steigender Feuchtigkeit mit der Fotosynthese beginnen und weiter wachsen. Sie gelten wie Moose als Pionierorganismen. Das heißt, sie kommen an extremen Standorten vor, wo wenige bis keine anderen Pflanzen überleben. Zum Beispiel karge nährstoffarme Felsen oder an Mauern und Wänden. Flechten sind keine Parasiten und schaden dem Baum nicht. Sie können ihn sogar vor Bakterien und Pilzen schützen.Sie dringen nicht in den lebendigen Teil des Baumes (Kambium) vor, sondern sitzen auf der äußersten Schicht, der Rinde auf.

==Photos==

<gallery>
File:k_DSC02317.jpeg
File:k_DSC02322.jpeg
File:k_DSC02327.jpeg
File:k_DSC02350.jpeg
File:k_DSC02390.jpeg
File:k_DSC02398.jpeg
File:k_DSC02400.jpeg
File:k_DSC02481.jpeg
File:k_DSC02631.jpeg
File:k_IMG_7190.jpeg
File:k_IMG_7376.jpeg
File:k_IMG_7385.jpeg
File:k_IMG_7401.jpeg
File:k_IMG_7460.jpeg
File:k_IMG_7656.jpeg
File:k_IMG_8308.jpeg
</gallery>

GMU:BioArt Forum/Lichens workshop

2022-11-08T08:41:48Z

Js: polished DE version

BioArt Forum: Lichens workshop with Dr. Jan Vondrak

Meeting: Marienstrasse 7b, room 204<br>
Meeting point: Marienstrasse 7b, room 204<br>
Max participants: 20

* Saturday, 12.11.2022. 10.00 -18.00
* Sunday, 13.11.2022. 10.00 - 15.00

''Dr. Jan Vondrak is an academic assistant at the University of South Bohemia, Faculty of Biological Sciences, Department of Botany in Budweis Jeske Budovice (CZ). Jan Vondrak is one of the few lichen specialists in the world and an experienced mycologist.''

==EN==
(scroll down for the version in German)

For our first workshop of the Environmental Nature Labs we have the very experienced mushroom and lichen researcher Dr. Jan Vondrak from the botanical institute of the University of Budweis was invited to Weimar. There are only a few such proven connoisseurs of lichens as him, most of them come from the Czech Republic, the reasons for this will be explained later. Jan is willing to go on two excursions with the participants in the city/outskirts of Weimar and present lichens and their ecology living on natural and building sites. Bringing a strong magnifying glass and umbrellas is recommended. However, we also provide a set of magnifying glasses on loan, and there is also the opportunity to learn about the study and identification and preservation of these creatures in the Environmenatal Biolaboratory (microscopy, color tests, etc.).

Lichens are inconspicuous, often annoying, but so widespread that they make an enormous contribution to the regulation of CO2 and nitrogen oxides in the atmosphere. It is also about getting a different perspective and looking at the colonization of lichens in gardens, house walls and paths in a positive way and forgoing their chemical or physical removal.

In addition, lichens are partners from two organisms that have blessedly come together: a fungus and a green or blue-green algae (cyanobacteria). The fungus and its partner live together on a substrate (rock, humus, bark). The fungus receives energy-rich carbohydrates from the partner that carries out photosynthesis and in return the fungus supplies water and mineral salts from the subsoil and forms pigments that protect the algae/bacteria from excessive UV radiation. Lichens grow on bark, leaves, rocks, dry stone walls or on the ground. Lichens grow very slowly compared to mosses and flowering plants. Some only a few millimeters to a centimeter per year. They can live for hundreds to thousands of years. This makes them one of the longest-lived creatures on earth. Like mosses, they can survive longer periods of drought "in their sleep" and start photosynthesis and continue to grow when humidity increases. Like mosses, they are regarded as pioneer organisms. That means they are found in extreme locations where little to no other plants survive. For example, barren nutrient-poor rocks or on walls and walls. Lichens are not parasites and do not harm the tree. They can even protect it from bacteria and fungi. They do not penetrate the living part of the tree (cambium), but sit on the outermost layer, the bark.

==DE==
(English above)

Wir haben für unseren ersten Workshop der Environmental Nature Labs den sehr erfahrenen Pilz- und Flechtenforscher Dr. Jan Vondrak vom botanischen Institut der Universität Budweis nach Weimar eingeladen. Es gibt nur wenige so ausgewiesene Kenner von Flechten wie Ihn, die meisten stammen aus Tschechien, die Gründe dafür werden noch erläutert. Jan wird mit den Teilnehmern zwei Exkursionen im Stadt/Randgebiet von Weimar unternehmen und dort auf Natur- und Bauflächen lebende Flechten und Ihre Ökologie vorzustellen. Die Mitnahme einer starken Lupe und von Regenschirmen ist empfohlen. Wir stellen jedoch auch ein Set von Lupen leihweise zur Verfügung. Es besteht auch die Möglichkeit etwas über die Untersuchung und Identifizierung und Bewahrung dieser Lebewesen im Environmenatal Biolabor zu erfahren (Mikroskopie, Farbtests etc.).
Flechten sind unauffällig, oft lästig, aber derart verbreitet, daß sie einen enorm hohen Beitrag zur Regulation von CO2 und Stickoxiden in der Atmosphäre leisten. Es geht auch darum einen anderen Blick zu bekommen und die Ansiedlung von Flechten in Gärten, Hausmauern und Wegen positiv zu betrachten und auf Ihre chemische oder physikalische Beseitigung zu verzichten.

Flechten sind Partnerwesen aus zwei Organismen, die sich segensreich zusammen geschlossen haben: Einem Pilz und einer Grün- oder Blaualge (Cyanobakterien). Der Pilz und sein Partner leben zusammen auf einem Substrat (Gestein, Humus, Rinde). Der Pilz erhält von dem Partner, der Fotosynthese betreibt, energiereiche Kohlenhydrate und im Gegenzug liefert der Pilz Wasser und Mineralsalze aus dem Untergrund und bildet Pigmente, die die Algen/Bakterien vor zu starker UV-Strahlung schützen. Flechten wachsen an Rinde, Blättern, Felsen, Trockenmauern oder auf dem Boden. Flechten wachsen im Vergleich zu Moosen und Blütenpflanzen sehr langsam. Manche nur wenige Millimeter bis einen Zentimeter pro Jahr. Sie können mehrere hundert bis tausend Jahre alt werden. Damit gehören sie zu den langlebigsten Lebewesen der Erde. Ebenso wie Moose können sie längere Trockenperioden “im Schlaf“ überleben und bei steigender Feuchtigkeit mit der Fotosynthese beginnen und weiter wachsen. Sie gelten wie Moose als Pionierorganismen. Das heißt, sie kommen an extremen Standorten vor, wo wenige bis keine anderen Pflanzen überleben. Zum Beispiel karge nährstoffarme Felsen oder an Mauern und Wänden. Flechten sind keine Parasiten und schaden dem Baum nicht. Sie können ihn sogar vor Bakterien und Pilzen schützen.Sie dringen nicht in den lebendigen Teil des Baumes (Kambium) vor, sondern sitzen auf der äußersten Schicht, der Rinde auf.

==Photos==

<gallery>
File:k_DSC02317.jpeg
File:k_DSC02322.jpeg
File:k_DSC02327.jpeg
File:k_DSC02350.jpeg
File:k_DSC02390.jpeg
File:k_DSC02398.jpeg
File:k_DSC02400.jpeg
File:k_DSC02481.jpeg
File:k_DSC02631.jpeg
File:k_IMG_7190.jpeg
File:k_IMG_7376.jpeg
File:k_IMG_7385.jpeg
File:k_IMG_7401.jpeg
File:k_IMG_7460.jpeg
File:k_IMG_7656.jpeg
File:k_IMG_8308.jpeg
</gallery>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T13:11:00Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

= implementation notes =
The underlying Max patch for tracking the ISS and playing audio looks like this:
[[File:ISS_overhead_patch.png|400px]]

[[File:ISS_overhead_2021-06-14.maxpat|Max patch]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

= references =
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T13:10:24Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

= implementation notes =
The underlying Max patch for tracking the ISS and playing audio looks like this:
[[File:ISS_overhead_patch.png|400px]]

[[File:ISS_overhead_2021-06-14.maxpat Max patch]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

= references =
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T13:10:07Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

= implementation notes =
The underlying Max patch for tracking the ISS and playing audio looks like this:
[[File:ISS_overhead_patch.png|400px]]

[[File:ISS_overhead_2021-06-14.maxpat]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

= references =
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

File:ISS overhead 2021-06-14.maxpat

2021-06-17T13:09:46Z

Js: File uploaded with MsUpload

File uploaded with MsUpload

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T13:07:04Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

= implementation notes =
The underlying Max patch for tracking the ISS and playing audio looks like this:
[[File:ISS_overhead_patch.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

= references =
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

File:ISS overhead patch.png

2021-06-17T13:06:06Z

Js: File uploaded with MsUpload

File uploaded with MsUpload

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:22:24Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

= implementation notes =
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

= references =
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:21:56Z

Js: /* project: distance communication */

Max and I, Max and Me class notebook

2021-06-17T12:21:53Z

Js: /* project: i see you and you see me */

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:21:22Z

Js: /* project: i see you and you see me */

Max and I, Max and Me class notebook

2021-06-17T12:21:18Z

Js: /* project: video theremin */

Max and I, Max and Me class notebook

2021-06-17T12:17:52Z

Js: /* project: color based movement tracker */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:16:21Z

Js: /* project: video theremin */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:16:06Z

Js: /* project: color based movement tracker */

Max and I, Max and Me class notebook

2021-06-17T12:16:02Z

Js: /* process: retrieve images from a camera */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:15:34Z

Js: /* process: record and save audio and video data */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

Max and I, Max and Me class notebook

2021-06-17T12:15:31Z

Js: /* project: theremin */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:15:17Z

Js: /* project: screamie */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:13:12Z

Js: /* process: retrieve images from a camera */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

Max and I, Max and Me class notebook

2021-06-17T12:13:10Z

Js: /* project: theremin */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:12:49Z

Js: /* project: theremin */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

Max and I, Max and Me class notebook

2021-06-17T12:12:46Z

Js: /* process: other peoples work to investigate */

Max and I, Max and Me class notebook

2021-06-17T12:11:32Z

Js: Created page with "= process: other peoples work to investigate = * [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXP..."

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:11:17Z

Js: /* process: other peoples work to investigate */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T12:10:52Z

Js: /* references */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]
* [[Max and I, Max and Me class notebook]]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:56:41Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup. The data about the ISS crew is scraped from the internet using custom written software.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:52:11Z

Js: /* references */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]
* [https://www.howmanypeopleareinspacerightnow.com/ how many people are in space right now?]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:50:02Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:49:33Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition. A handbell is rung and the names of the people currently in outer space are read out loud.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

File:People in space.mp3

2021-06-17T11:47:05Z

Js: Js uploaded a new version of File:People in space.mp3

File uploaded with MsUpload

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:41:46Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

The Max patch was [https://docs.cycling74.com/max8/vignettes/standalone_building exported to a standalone application]. It is now run on a small computer board to reduce the complexity of the setup.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:37:32Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:37:19Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

{{Listen
|title="ISS announcement"
|filename=people_in_space.mp3}}

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:34:48Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

[[:File:people_in_space.mp3]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

File:People in space.mp3

2021-06-17T11:34:28Z

Js: File uploaded with MsUpload

File uploaded with MsUpload

File:People in space graham.wav

2021-06-17T11:34:08Z

Js: File uploaded with MsUpload

File uploaded with MsUpload

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:28:55Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

[[File:ISS_overhead.png|400px]]

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

File:ISS overhead.png

2021-06-17T11:28:41Z

Js: File uploaded with MsUpload

File uploaded with MsUpload

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:14:47Z

Js: /* implementation notes */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:08:57Z

Js: /* final project: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation notes ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-17T11:08:29Z

Js: /* proof of concept: 2021-05-19 */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== implementation note ==s
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:46:10Z

Js:

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== proof of concept: 2021-05-19 ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:45:23Z

Js:

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== proof of concept: 2021-05-19 ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

__FORCETOC__

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:41:09Z

Js:

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= final project 3: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== proof of concept: 2021-05-19 ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:40:52Z

Js: /* final project 3: ISS overhead */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:25:48Z

Js: /* project 1: i see you and you see me */

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:25:30Z

Js: /* final project 2: distance communication */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project 1: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= project: distance communication =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

= final project 3: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== proof of concept: 2021-05-19 ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]

GMU:Max and I, Max and Me/Johannes Schneemann

2021-06-10T12:25:08Z

Js: /* final project 1: i see you and you see me */

This page documents my work and experiments done during the "Max and I, Max and Me" course. Feel free to copy, but give attribution where appropriate.

= process: other peoples work to investigate =
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/F.Z._Ayg%C3%BCler F.Z._Aygüler AN EYE TRACKING EXPERIMENT]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/WeiRu_Tai Does social distancing change the image of connections between friends, family and strangers? (includes patch walkthrough)]
* [https://www.uni-weimar.de/kunst-und-gestaltung/wiki/GMU:Max_and_the_World/Paulina_Magdalena_Chwala Paulina Magdalena Chwala (VR, OSC, Unity)]

= project: theremin =
Assignment: Please do your own first patch and upload onto the wiki under your name

I did build a simple [https://en.wikipedia.org/wiki/Theremin Theremin] to get to know the workflow with Max. Move your mouse cursor around to control the pitch and volume.

[[File:simple_mouse_theremin_screenshot.png|400px]]

[[:File:simple_mouse_theremin.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/loadbang loadbang]
* [https://docs.cycling74.com/max8/refpages/mousestate mousestate]
* [https://docs.cycling74.com/max8/refpages/cycle~ cycle~]
* [https://docs.cycling74.com/max8/refpages/slider slider]
* [https://docs.cycling74.com/max8/refpages/ezdac~ ezdac~]

= process: retrieve images from a camera =
This simple patch allows you to grab (still) images from the camera.

[[File:retrieve_images_from_camera.png|400px]]
[[:File:2021-04-23_retrieve_images_from_camera_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab]

= project: screamie =
If you scream loud enough ... you will get a selfie.

[[File:screamie_photo.png|400px]]

This little art is a first shot. Also: snarky comment on social media and selfie-culture.
[[File:screamie.png|400px]]

[[:File:2021-04-25_screamie_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/levelmeter~ levelmeter~] - to display the audio signal level
* [https://docs.cycling74.com/max8/refpages/peakamp~ peakamp~] - find peak in signal amplitude (of the last second)
* [https://docs.cycling74.com/max8/refpages/past past] - send a bang when a number is larger then a given value
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get image from camera
* [https://docs.cycling74.com/max8/refpages/jit.matrix jit.matrix] - to buffer the image and write to disk

= process: record and save audio and video data =
This patch demonstrates how to get audio/video data and how to write it to disk.
[[File:audio_video_recorder.png|400px]]

[[:File:2021-04-23_audio_video_recorder_JS.maxpat]]

Objects used:
* [https://docs.cycling74.com/max8/refpages/ezadc~ ezadc~] - to get audio input
* [https://docs.cycling74.com/max8/refpages/meter~ meter~] - to display (audio) signal levels
* [https://docs.cycling74.com/max8/refpages/metro metro] - to get a steady pulse of bangs
* [https://docs.cycling74.com/max8/refpages/counter counter] - to count bangs/toggles
* [https://docs.cycling74.com/max8/refpages/select select] - to detect certain values in number stream
* [https://docs.cycling74.com/max8/refpages/delay delay] - to delay a bang (to get DSP processing a head start)
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - to get images from camera
* [https://docs.cycling74.com/max8/refpages/jit.vcr jit.vcr] - to write audio and video data to disk

= project: color based movement tracker =
This movement tracker uses color as a marker for an object to follow. Hold the object in front of the camera an click on it to select an RGB value to track. The object color needs to be distinctly
different from the background in order for this simple approach to work. This patch yields both a bounding box and its center point to process further.

The main trick is to overlay the video display area (jit.pwindow) with a color picker (suckah) to get the color. A drop-down menu is added as a convenience function to select the input camera (e.g. an external one vs. the build-in device).
The rest is math.

[[File:color_based_movement_tracker.png|400px]]

[[:File:2021-04-23_color_based_movement_tracker_JS.maxpat]]

The following objects are used
* [https://docs.cycling74.com/max8/refpages/metro metro] - steady pulse to grab images from camera
* [https://docs.cycling74.com/max8/refpages/jit.grab jit.grab] - actually grab an image from the camera upon a bang
* [https://docs.cycling74.com/max8/refpages/prepend prepend] - put a stored message in front of any input message
* [https://docs.cycling74.com/max8/refpages/umenu umenu] - a drop-down menu with selection
* [https://docs.cycling74.com/max8/refpages/jit.pwindow jit.pwindow] - a canvas to display the image from the camera
* [https://docs.cycling74.com/max8/refpages/suckah suckah] - pick a colour from the underlying part of the screen (yields RGBA as 4-tuple of floats (0.0 .. 1.0))
* [https://docs.cycling74.com/max8/refpages/swatch swatch] - display a colour (picked)
* [https://docs.cycling74.com/max8/refpages/vexpr vexpr] - for (C-style) math operations with elements in lists
* [https://docs.cycling74.com/max8/refpages/jit.findbounds findbounds] - search values in a given range in the video matrix
* [https://docs.cycling74.com/max8/refpages/jit.lcd jit.lcd] - a canvas to draw on
* [https://docs.cycling74.com/max8/refpages/send send] & [https://docs.cycling74.com/max8/refpages/receive receive] - avoid cable mess

= project: video theremin =
Assignment: Combine your first patch with sound/video conversion/analysis tools

This patch combines the previously developed color-based movement tracker with the first theremin.

[[File:video_theremin.png|400px]]
[[:File:2021-04-23_video_theremin_JS.maxpat]]

= project 1: i see you and you see me =

''The project did not grew legs upon more detailed examination. It therefore was not persued further.''

* in collaboration with [[Luise Krumbein]]

Video conference calls replaced in-person meetings on a large scale in a short amount of time. Most software tries to mimic a conversation bus actually either forces people into the spotlight and into monologues or off to the side into silence. One to one communication allows a user to concentrate on the (one) other, but group meetings blur the personalities and pigeonhole expression into a cookie-cutter shape of what is technically efficient and socially barely tolerable. The final project in this course seeks to explore different representations in synchronous online group communication.

To shift the perception data will be gathered from multiple (ambient) sources. This means tapping into public data sets for broad context. It also means gathering hyper-local/personal data via an arduino or similar microcontroller (probably ESP8266) for sensing the individual. These data streams will be combined to produce a visual output using jitter. This result aims to represent the other humans in a manner to emphasize personality and not the emulation of the technicality in face to face exchanges.

= <s>final project 2: distance communication </s> =

''Different implementations of the algorithm were tried in Max. None yielded a convergence of the phases of the oscillators. Therefore the project was cancelled.''

The installation "distance communication" explores and reflects on (romantic) relationships at a time where partners are apart. Their lives don't take place in the same environment anymore and are open to different influences. It takes effort to stay in sync even though a distance may be bridged by modern communication means. Two blinking LEDs represent the partners, the blinking rhythm their respective lives.

The technical setup consists of two LEDs driven by a microcontroller and sensed via a light-dependent resistor. A small computer (laptop or small form factor PC) runs both Max and a video conference software to display "the other". The Max patch and the "other" are displayed on a screen or video projector. The Max patch tries to synchronize the respective blinking pattern by leveraging the well studied behaviour of fireflies. A microphone on the floor picks up ambient sound. If this audio level exceeds a given a threshold the blinking of one LEDs is triggers / reset to introduce disturbance in the process. The installation may be presented with both parts in the same room or one part off-site somewhere else.

== implementation notes ==
The firefly model turned out to be insufficient for the current setup due to the small number of agents. The interaction has been remodeled from first principles based on two harmonic oscillators and the Kuramoto–Daido model for phase locking.

[[File:lover_equations_2021-05-24.jpg|400px]]

Observing the equation system over time shows the expected behavior (when implemented in Python).

[[File:lover_oscillators_2021-05-24.png|400px]]

Max implementation of the Kuramoto-Daido model

[[File:portrait_of_two_lovers_at_a_distance_2021-05-31.png|400px]]

== references ==
* [https://www.springer.com/gp/book/9783540071747 Kuramoto, Yoshiki (1975). H. Araki (ed.). Lecture Notes in Physics, International Symposium on Mathematical Problems in Theoretical Physics. 39. Springer-Verlag, New York. p. 420.)]
* <s>[https://www.jstor.org/stable/2808377 Buck, John. (1988). Synchronous Rhythmic Flashing of Fireflies. The Quarterly Review of Biology, September 1988, 265 - 286.]</s>
* <s>[https://www.journals.uchicago.edu/doi/10.1086/414564 Carlson, A.D. & Copeland, J. (1985). Flash Communication in Fireflies. The Quarterly Review of Biology, December 1985, 415 - 433.]</s>
* <s>[https://deepblue.lib.umich.edu/handle/2027.42/56374 Lloyd, J. E. Studies on the flash communication system in Photinus fireflies 1966] </s>
* <s>[https://academic.oup.com/icb/article/44/3/225/600910 Nobuyoshi Ohba: Flash Communication Systems of Japanese Fireflies]</s>

= final project 3: ISS overhead =

The International Space Station is currently the only extraterrestrial habitat for humans. It circles the earth approximately every 90 minutes and thus can look down on a vast planet. Humans on the other hand rarely look up and grasp what we accomplished and what we can reach for. This small installation tracks the International Space Station and emits a notification if it is above the place of exhibition.

The technical setup consists of a computer running Max and some means of notification like a bell or a speaker. The current position of the International Space Station is calculated from its orbital data and set into relation to the place of exhibition (e.g. Weimar). If the ISS is at a place which can be considered overhead the notification is triggered. If a speaker is used the sound synthesis can encode different parameters.

== proof of concept: 2021-05-19 ==
The basic Max patch for tracking the ISS and sending a bang is done.
[[File:ISS_notifier_2021-05-19.png|400px]]

Processing TLE files and orbital data was sidestepped by relying on [http://open-notify.org/Open-Notify-API/ISS-Location-Now/ open-notify.org] and processing [https://json.org/ JSON] data. The distance (on earth) to the city center of Weimar (at 50.979444, 11.329722) calculated using the euclidian distance as a metric. To simplify the math it is assumed that the world is flat and the [https://en.wikipedia.org/wiki/World_Geodetic_System#WGS84 WGS 84] reference system is ignored. This is ok for small distances where the curvature of the earth does not result in too much of an error.

Lessons learned:
* sometimes you can find an API for exactly the data you need
* occasionally ignoring the material reality is feasible on very narrow bounds
* school math turned out to be useful

Todo:
* extend to use different satellites
* leverage their orbital data
* attach a loudspeaker for announcements

== references ==
* [https://en.wikipedia.org/wiki/International_Space_Station wikipedia: International Space Station]
* [https://en.wikipedia.org/wiki/Two-line_element_set wikipedia: two-line element set (TLE)]
* [https://nssdc.gsfc.nasa.gov/nmc/SpacecraftQuery.jsp NASA Space Science Data Coordinated Archive Master Catalog Search]