“Death and the Powers” is an interactive opera by Tod Machover.

On their website they explain more detailed how the technology behind functions.