File:A Pd framework for the Xth Sense-enabling computers to sense human kinetic behaviour.pdf

A Pd framework for the Xth Sense: enabling computers to sense human kinetic behaviour Marco DONNARUMMA Sound Design, ACE, The University of Edinburgh

The Xth Sense is an interactive system for the biophysical generation and control of music. It makes use of muscle sounds1 produced by a performer as both raw sonic material and control data. Presently the Xth Sense (XS) technology consists of low-cost, wearable biosensors and a Pure Data2 based application for capture, analysis, real time processing and playback of human muscle sounds. The technical implementation of the XS biosensors has been recently illustrated in [2]. This paper describes the design of the XS software; it is a program that enables a computer to “listen” to the MMG signals transduced by the XS biosensors, to understand the performance main features, and therefore to interact with the performer. After a brief introduction on the nature of the interaction fostered by the XS technology, I focus on the framework main features such as: the XS library, a tabbed dynamic interface (TDI), a MMG features extraction unit, and a graph-on-parent3 (GOP) routing system for dynamic mapping of gesture to sound. Keywords Biophysical music, muscle sounds, modular DSP framework.