GMU:Tutorials/Performance Platform/Using the tracking system: Difference between revisions

From Medien Wiki
No edit summary
No edit summary
 
(2 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Introduction ==
Controlling Objects in Unity with The Captury<br>


== 1.open unity and create a new Unity project.==
- Cover the OSC syntax and how to get nodes as matrix / vector, absolute and relative<br>
- How to create objects in Unity and arrange them so they represent the nodes<br>
- How to animate these Objects <br>


== Rotation matrix, Quaternion ==
At first we should understand some concepts.There are three ways to represent rotation that is rotation matrix, quaternion and Euler, theCaputry uses rotation matrix, Unity uses  Quaternion and  Euler. This tutorial focus on  Rotation matrix, Quaternion.<br>


[[Image:unity start.png|thumb|left|800px]]
'''4×4 Rotation matrix'''<br>
<br style="clear:both">
TheCaputry uses it to represent rotation. Unity had only provided a 4 × 4 matrix class Matrix4x4, it contains transform T, rotation R and scaling information. You can refer to [https://en.wikipedia.org/wiki/Rotation_matrix].<br>
Advantage:<br>
Any vector can be the rotation axis;<br>
Disadvantage:<br>
1.  In fact, to rotate a object only need 4 values ,a vector(x,y,z) and a angle, but the matrix has 16 elements;
2.  It will increase the amount of calculation when doing multiplication operation and resultin some waste of space and time;


== 2. create a simple Unity scene.==
'''Quaternion'''<br>
Quaternions are used to represent rotations. Mostly you can use Quaternion AngleAxis(float angle, Vector3 axis) to create a quaternion.Unity internally uses Quaternions to represent all rotations. you can refer to:[https://docs.unity3d.com/ScriptReference/Quaternion.html]<br>


Advantage:<br>
1.  They are compact, don't suffer from gimbal lock and can easily be interpolated;<br>
2.  It only need a 4-dimensional quaternion through the origin then can execute arbitrary rotation. In some cases it has higher efficiency than rotation matrix;<br>
3.  It can provide a smooth interpolation;<br>


For the purposes of this tutorial, it is recommended to creating a simple plane and shape objects.
Disadvantage:<br>
1.Compare to Euler it has one more dimension, so it will be difficult to understand.


here I create a cube and a sphere.
== Conversion between Rotation matrix and Quaternion ==
We might need to convert rotation matrix to quaternion depend on your project needs after we get data from theCaptury in matrix format.There i present two ways to convert there two.


[[Image:add new object.png|thumb|left|800px]]
'''Convert quaternion to 4×4 Rotation matrix.'''<br>
<br style="clear:both">
<source lang="csharp">
[[Image:new object in scene.png|thumb|left|800px]]
Quaternion q = Quaternion.LookRotation(new Vector3(0,0.5,1)); 
<br style="clear:both">
Matrix4x4 rot = new Matrix4x4(); 
rot.SetTRS(new Vector3(0,0,0),q,new Vector3(1,1,1)); 
</source>


==3. add component to camera==
'''Convert  4×4 Rotation matrix to quaternion.'''<br>
<source lang="csharp">
Matrix4x4 rot = new Matrix4x4(); 
rot.SetTRS(new Vector3(0,0,0),q,new Vector3(1,1,1));  //It’s the same “q” that used above
Vector4 vy = rot.GetColumn(1); 
Vector4 vz = rot.GetColumn(2); 
Quaternion newQ = Quaternion.LookRotation(new Vector3(vz.x,vz.y,vz.z),new Vector3(vy.x,vy.y,vy.z)); 
</source>




[[Image:select camera.png|thumb|left|800px]]
'''Other conversion:'''<br>
<br style="clear:both">
[http://www.euclideanspace.com/maths/geometry/rotations/conversions/eulerToQuaternion/index.htm Euler to Quaternion]<br>
select camera in hierarchy,single click it.
[http://www.euclideanspace.com/maths/geometry/rotations/conversions/quaternionToEuler/index.htm Quaternion To Euler]<br>
[http://www.euclideanspace.com/maths/geometry/rotations/conversions/angleToQuaternion/index.htm AngleAxis to Quaternion]<br>
[http://www.euclideanspace.com/maths/geometry/rotations/conversions/quaternionToAngle/index.htm Quaternion to AngleAxis]<br>


[[Image:add component.png|thumb|left|800px]]
== Cover the OSC syntax and how to get nodes as matrix / vector, absolute and relative ==
<br style="clear:both">
Cover the OSC syntax and get nodes as matrix / vector<br>
click the right of table to add component to the camera.
With OSC you can subscribe to the following:
1. subscribe to a bone position,rotation as vector in world coordinates for positions:<br>
/subscribe/<name of person1>/blender/@/vector<br>
[[Image:vector.png|left|Vector]]<br>
<br style=“clear:both">


[[Image:new script.png|thumb|left|300px]]
<br style="clear:both">
select a new script


[[Image:script name.png|thumb|left|300px]]
2. subscribe to a bone position,rotation as vector in relative coordinates for positions:<br>
<br style="clear:both">
/subscribe/<name of person1>/blender/@/upper/vector<br>
enter a name for this script
[[Image:vector_upper.png|left|Relative vector]]<br>
<br style=“clear:both">


[[Image:write code.png|thumb|left|800px]]
<br style="clear:both">
double click the script in Assets,then you will get a script widow, you can write code in it.


3. subscribe to a bone position,rotation as matrix in world coordinates:<br>
/subscribe/<name of person1>/blender/@/matrix<br>
[[Image:matrix.png|left|matrix]]<br>
<br style=“clear:both”>


==4.write the code.==


this code is for camera, then create a gameobject, this gameobject is the cube, the capture system will send the position of the root bone to the cube, then the script will make the camera follow the cube.
4. subscribe to a bone position,rotation as matrix in relative coordinates:<br>
/subscribe/<name of person1>/blender/@/upper/matrix<br>
[[Image:matrix_upper.png|left|Relative matrix]]<br>
<br style=“clear:both”>


[[Image:code.png|thumb|left|800px]]
You can use transform.localPosition to get position of the transform relative to the parent transform.
<br style="clear:both">


== How to create objects in Unity and arrange them so they represent the nodes ==
1.  Create a gameobject or you can import your own model into Unity<br>
[[Image:create_cube.png|Relative vector]]


==5.set the data related to the capture system.==
2. Arrange the cube in a certain hierarchy<br>
[[Image:hierarchy.png|hierarchy]]


[[Image:right table.png|thumb|left|400px]]
 
<br style="clear:both">
or you can import your model then automap it.<br>
Then in the right table, you can set the remote IP and the port of the main computer.
[[Image:avatar_1.png|avatar]]
[[Image:avatar_2.png|avatar]]
 
== How to animate these Objects ==
Asking for data;<br>
<source lang="javascript">
msg = handler.StringToOscMessage("/subscribe/fiona/blender/Root/vector 50.0 0.0 20.0");
handler.Send(msg);
</source>
 
2.  Getting data;<br>
<source lang="javascript">
switch (msgAddress){
            case "/Fiona_fit/blender/Root/vector":
            hip[0] = values[0];
            hip[1] = values[1];
            hip[2] = values[2];
            break;}
</source>
 
3.  Giving the data to the object.<br>
<source lang="javascript">
var transHip = GameObject.Find("Cube_hip1");
transHip.transform.position = Vector3(hip[0], -hip[2], hip[1]);
</source>
 
4.  [https://www.youtube.com/watch?v=VVhXqrPgMJ0&feature=youtu.be '''Video'''] In this video i use cube to replace every bone.

Latest revision as of 15:42, 31 July 2016

Introduction

Controlling Objects in Unity with The Captury

- Cover the OSC syntax and how to get nodes as matrix / vector, absolute and relative
- How to create objects in Unity and arrange them so they represent the nodes
- How to animate these Objects

Rotation matrix, Quaternion

At first we should understand some concepts.There are three ways to represent rotation that is rotation matrix, quaternion and Euler, theCaputry uses rotation matrix, Unity uses Quaternion and Euler. This tutorial focus on Rotation matrix, Quaternion.

4×4 Rotation matrix
TheCaputry uses it to represent rotation. Unity had only provided a 4 × 4 matrix class Matrix4x4, it contains transform T, rotation R and scaling information. You can refer to [1].

Advantage:
Any vector can be the rotation axis;
Disadvantage:
1. In fact, to rotate a object only need 4 values ,a vector(x,y,z) and a angle, but the matrix has 16 elements; 2. It will increase the amount of calculation when doing multiplication operation and resultin some waste of space and time;

Quaternion
Quaternions are used to represent rotations. Mostly you can use Quaternion AngleAxis(float angle, Vector3 axis) to create a quaternion.Unity internally uses Quaternions to represent all rotations. you can refer to:[2]

Advantage:
1. They are compact, don't suffer from gimbal lock and can easily be interpolated;
2. It only need a 4-dimensional quaternion through the origin then can execute arbitrary rotation. In some cases it has higher efficiency than rotation matrix;
3. It can provide a smooth interpolation;

Disadvantage:
1.Compare to Euler it has one more dimension, so it will be difficult to understand.

Conversion between Rotation matrix and Quaternion

We might need to convert rotation matrix to quaternion depend on your project needs after we get data from theCaptury in matrix format.There i present two ways to convert there two.

Convert quaternion to 4×4 Rotation matrix.

Quaternion q = Quaternion.LookRotation(new Vector3(0,0.5,1));  
Matrix4x4 rot = new Matrix4x4();  
rot.SetTRS(new Vector3(0,0,0),q,new Vector3(1,1,1));

Convert 4×4 Rotation matrix to quaternion.

Matrix4x4 rot = new Matrix4x4();  
rot.SetTRS(new Vector3(0,0,0),q,new Vector3(1,1,1));  //It’s the same “q” that used above
Vector4 vy = rot.GetColumn(1);  
Vector4 vz = rot.GetColumn(2);  
Quaternion newQ = Quaternion.LookRotation(new Vector3(vz.x,vz.y,vz.z),new Vector3(vy.x,vy.y,vy.z));


Other conversion:
Euler to Quaternion
Quaternion To Euler
AngleAxis to Quaternion
Quaternion to AngleAxis

Cover the OSC syntax and how to get nodes as matrix / vector, absolute and relative

Cover the OSC syntax and get nodes as matrix / vector
With OSC you can subscribe to the following: 1. subscribe to a bone position,rotation as vector in world coordinates for positions:
/subscribe/<name of person1>/blender/@/vector

Vector




2. subscribe to a bone position,rotation as vector in relative coordinates for positions:
/subscribe/<name of person1>/blender/@/upper/vector

Relative vector




3. subscribe to a bone position,rotation as matrix in world coordinates:
/subscribe/<name of person1>/blender/@/matrix

matrix




4. subscribe to a bone position,rotation as matrix in relative coordinates:
/subscribe/<name of person1>/blender/@/upper/matrix

Relative matrix



You can use transform.localPosition to get position of the transform relative to the parent transform.

How to create objects in Unity and arrange them so they represent the nodes

1. Create a gameobject or you can import your own model into Unity
Relative vector

2. Arrange the cube in a certain hierarchy
hierarchy


or you can import your model then automap it.
avatar avatar

How to animate these Objects

Asking for data;

	msg = handler.StringToOscMessage("/subscribe/fiona/blender/Root/vector 50.0 0.0 20.0"); 
	handler.Send(msg);

2. Getting data;

switch (msgAddress){
            case "/Fiona_fit/blender/Root/vector":
            hip[0] = values[0];
            hip[1] = values[1];
            hip[2] = values[2];
            break;}

3. Giving the data to the object.

	var transHip = GameObject.Find("Cube_hip1");
	transHip.transform.position = Vector3(hip[0], -hip[2], hip[1]);

4. Video In this video i use cube to replace every bone.