Plug your cam – Extending Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]] the modular way

Author: IOhannes zmölnig

Download full paper: Media:Plug your cam-Extending Gem the modular way.pdf

In this paper we present the new plugin infrastructure in the upcoming release of Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]], that aims to cleanse the core library of superfluous dependencies.

Traditionally, Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]] is built as a monolithic library of Pd[[Pure Data]] a dataflow programming environment objects, with linked in support for various system features like image acquisition or output methods.

This has been redesigned, in order to allow to extend Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]]'s interfacing capabilities in a plugin based way, allowing developers to easily extend the system with new backends. On the other hand, end users need not install the full feature set, but can choose what they need and eventually build a minimal system.

Plugins do not extend Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]]'s set of objectclasses, but rather provide “backends” to one (or several) given objectclasses. The motivation for externalizing these parts of Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]] into a plugin system, was mainly driven by two aspects: lowering dependencies and providing a uniform interface across platforms.

Lowering dependencies aims at easing installation of binary packages (and thus the maintenance of such packages). For instance, on linux Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]] (0.92) can be compiled with support for five different movie reading libraries, some of them being (partially) patent encumbered, outdated or otherwise hard to obtain. In order to support the widest range of film footage, one is tempted to link against all possible libraries. However, this also means that the end user has to install these libraries first in order to make use of Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]], because the Pd[[Pure Data]] a dataflow programming environment (or rather: the operating system) will refuse to load Gem[[Tracking Motion Detection#Gem|Graphics Environment for Multimedia]] an OpenGL extension for →[[Pure Data|Pd]] if one of those libraries is missing. Even if they are not interested in video playback at all! (The alternative to making the end user install a number of libraries, is to provide them in the release, which bloats the package, eventually introducing legal problems)

From the patching side of things, the new plugin infrastructure provides a uniform interface to device/backend dependent settings. Different backends provide different possibilities to interact with e.g. an image acquisition device. What's worse, different devices can have different features, the user might want to control. (E.g. a webcam could provide a means to control pan/tilt/zoom, whereas a video capture card might allow to switch between different inputs). In the past this divergence has led to incompatible implementations of e.g. the [pix_video] object, leading to patches that are not portable across operating systems.

The plugin infrastructures provides a way to query and set virtually all available controls for a given device/backend in a uniform way (though the available controls will obviously vary), and to keep controls persistens when switching between backends.

So far, [pix_video] (for live video acquisition), [pix_film] (for film footage acquisition) and [pix_record] (for video output), have been switched. Still image acquisition ([pix_image], but also [pix_buffer] and the like) will probably be converted to a plugin system by time of the Pd[[Pure Data]] a dataflow programming environment-convention as well.


Kreativfonds Bauhaus-Univeristät WeimarElectronic Arts Blog für digitale SpielkulturThe Mozilla FoundationAllied Vision TechnologiesReality Jockey Ltd.Freistaat ThüringenBauhaus-Universität WeimarHochschule für Musik Franz Liszt WeimarFraunhofer Institute for Digital Media Technology IDMTStadt WeimarKlassik Stiftung WeimarNKFaculty of MediaStudio for electro-acoustic MusicKulturTragWerk e.V.Elektronisches Studio der TU BerlinMaschinenraum Hackerspace WeimarParlomar5 e.V.Lab for Electronic Arts and PerformanceRadio Lotte WeimarSponsors and partners of the 4th internationals Pure Data Convention in Weimar 2011

4th international Pure Data Convention 2011 Weimar ~ Berlin