GMU:Digital Puppetry Lab/Group A Midsummer Night's Dream

From Medien Wiki

A Midsummer Night's Dream

Group work by Liu Yun & Li Qianqian & Fan Xiangzhen & Wu Junyuan & Tan Yuxin

Introduction

Have you ever looked at the moon with the breeze at night? Have you ever heard the harmony of the birds and frogs? Do you remember the wonderful summer nights? We just want to give you a midsummer night's dream. This project is based on processing with the face detection and data collection, and then send the data message into the MaxMsp. We divide four regions in MaxMsp. The various sounds you hear will change according to your different position. In addition, in the unity we tried to establish a vivid picture of summer. Imagine that grass is moving with the wind in a beautiful moonlight accompanied by bursts of sounds of insects. Let`s enjoy a summer dream together.

Diagram

FLowDiagram.jpg

Demonstration

Demonstration Viedo

Demo1.jpg

Demo2.jpg

Appendix

FaceDetectionOutput.jpg

Data Input:
· Getting data (the position X and Y of the center of face and the size of face) by face detection
· Sending data from processing to Max/Msp by OSC
· Another processing file sending the same data to Unity by using different port

ProcessingToMaxMsp.jpg

Data Transfer:
· Receiving data from processing by OSC
· Only keeping number from the message by using 'zl slice'
· Dividing number into three usable data by using 'unpack'
· Sending data separately from Yun's laptop to Xiangzhen's laptop in Max/Msp by OSC

SoundEffect&BG.jpg

Audio Output:
· Receiving data from Yun's Max/Msp by OSC
· Control volumes of 4 corner's sound effects by building different functions with data received

ProcessingToUnity.jpg

Data Using in Unity:
· Receiving data from processing by OSC
· Dealing with function call between js and C# file
· Using 'x' and 'y' data sent by Yun's laptop to control the quaternion of the camera
· Connect TUIO app with Laptop

TuioPad.jpg

ScreenTouch.jpg

Data Getting by phone:
· Using the app TuioPad to get screen touch data by phone

UnityTerrain.jpg

Terrain Creation and Particular System in Unity:
· Build a terrain -- tree (5 styles) --build grass --adjust parameters
· Create skybox, light environment, windzone
· Create glow warm and falling leaves particle system

Scripts 1.jpg

Scripts_1:'
· Refer to unity asset "firstcharactorcontroller"
· Variable definitions
· Initializing the transform of camera
· Getting data from osc

Scripts 3.jpg

Scripts_2:'
· Set the range of data
· Using function of Quaternion to change rotation of the camera