IFD:MSOS SoSe22

Making Sense of Sounds

Fachmodul
Instructor: Clemens Wegener
Credits: 6 ECTS, 2 SWS
Capacity: max. 10 students
Language: English
Location: BigBlueButton Online Streaming
Meeting Time: Mondays 9:15 am

Description

We are constantly surrounded by electronic interfaces: elevators, automatic doors, ticket machines, remote controls, voice recognition systems, etc. With a high degree of networking between our devices, interfaces must seamlessly integrate into ecosystems of information acquisition and processing. Thus, ergonomic sensor placement, which integrates discreetly into our accustomed environment, is gaining ground.

The course explores the possibilities of this ergonomic, embedded interaction using the example of acoustic interfaces. We will develop electronic sensor systems for the detection of vibrations (structure-borne sound) in integrated surfaces and experiment with machine learning methods to differentiate between vibration gestures (e.g. knocking noises, friction noises, etc.).

The Course will be taught in video sessions and regular assignments every week.

If you are interested, please send a letter of motivation and portfolio to clemens.wegener ( at ) uni-weimar (dot) de.

Course Introduction Video

Admission requirements

Things you need

• computer or laptop
• Teensy 4.0 + audio shield
• a pair of headphones or speakers
• a soldering iron would be a plus

We will use either the Teensy 4.0 hardware platform or the Raspberry Pi to classify sounds in the environment. Therefore, a budget of (30-40€) should be planned for the Teensy and other components.

Educational Requirements

Introductory level of programming is a requirement!

Evaluation

Successful completion of 50% of the assignments. Realization and documentation of a completed semester prototype and a final video presentation. Please refer to the Evaluation Rubric for more details.

Eligible participants

Qualified MFA Medienkunst/-gestaltung, MFA Media Art and Design, MSc MediaArchitecture candidates

Syllabus

• Week 1: Meet and Greet
• Week 2: C++ Fundamentals
• Week 3: Soldering a Piezo Disc Preamplifier
• Week 4: Programming a Clusterer in C++ 1/2
• Week 5: Programming a Clusterer in C++ 2/2
• Week 6: Implementing the Clusterer on the Teensy 4.0
• Week 7: Inspiration Session on Sound Recognition
• Week 8: Presentation of Project Ideas
• Until End: Consultations and Project Work
• July, 26th: Final presentations
• August, 16th: Hand in documentation

Course Material

After each session you will find the new course material here:

• course_material/Session 1. Meet and Greet and Inpiration
• course_material/Session 2: Data Types, Classes and Piezo Amp
• course_material/Session 3: File Structuring, Pointers and Addresses
• course_material/Session 4: Programming the Classifier Part1
• course_material/Session 5: Soldering
• course_material/Session 6: Getting Started with Teensyduino
• course_material/Session 7: Coding_the_envelope_follower

Further Reading

Zamborlin, Bruno. 2015. Studies on customisation-driven digital music instruments. Doctoral thesis, Goldsmiths, University of London.

Murray-Smith, Roderick & Williamson, John & Hughes, Stephen & Quaade, Torben. (2008). Stane: Synthesized surfaces for tactile input. Conference on Human Factors in Computing Systems - Proceedings. 1299-1302. 10.1145/1357054.1357257.

Rasamimanana, N.H., Bevilacqua, F., Schnell, N., Guédy, F., Fléty, E., Maestracci, C., Zamborlin, B., Frechin, J., & Petrevski, U. (2010). Modular musical objects towards embodied control of digital music. Tangible and Embedded Interaction.

Links

• Course Introduction Video
• Blendie
• Gary Hill, "Wall Piece"
• Hyper Surfaces
• Mogees Play
• ATV AFrame
• CHAIR.AUDIO Tickle
• The Table Recorder
• PebbleBox
• Modular Musical Objects
• Tangible Acoustic Interfaces for Computer-Human Interaction: Sound Table
• Sven König - sCrAmBlEd?HaCkZ!