GMU:Tutorials/Networking/Controlling Skeletons in Unity with The Captury

From Medien Wiki

Introduction

This tutorial will show you how to set up a rigged Model in Blender, how to export it into Unity and how to control it with The Cpatury's OSC Messages.

Shortcuts (Blender)

TAB = Toggling between Edit- and Object-Mode
Shift + Right-Click = Multi-Select
Spacebar = Open Finder -> type in commands and select them
X = delete
S = scale
Shift + D = Duplicate Object
Ctrl + J = Combine Objects to one Mesh
G = "Grab"


Creating the Rig in Blender


Step1
Open a new .blend File. Delete the Cube which is set as default with "x". Go to Add -> Armature and select Human (MetaRig).
If there is no Rig then go to: File -> User Prefrences and select the Addons Tab and type in "Rigify". Make sure the Addon is checked.
If not simply click on the square and Save User Settings at the bottom.

Create the Rig
Go to User Prefrences
Check if Rigify is activated
This is how the Rig looks






Step2
Select your Rig and switch to the Outliner on the right. Double click on metarig and type in the name you wish: e.g "Cubeboy".

Rename your Rig






Step3
Now switch back to the 3d-Rig and select it with right-click and press "TAB" to switch into "Edit Mode".
Because we don't need any fingers and unnacessary bones we are going to delete the Fingers: Hold "Shift" and right-click to multi-select the bones.
After you selected all fingers go and press "x" to delete the bones. Repeat the Process on the other side. We are also going to delete the "Heel bones".

Multiselect "Shift + Right Click"
Delete bones with "X"
Deleting the Heels






Our Rig should now look like this (I expanded the bone structure in the "outliner" by clicking on the plus sypmbol).

Bone Structure






Quick Helper
Sometimes you encounter that this little red and white rounded thing was moved accidently from it's original position which is actually the "Cursor"
centerd in the X,Y,Z Coordinate = 0,0,0. If you press Spacebar you can simply type in "Snap Cursor to Center" and click on the slected Field to reposition it.
This is Importanrt because when we create a new Object we want it to be fully centerd. (This will make life easier).

Set cursor to 0,0,0






Step4
Switch back to "Object Mode". Go to the Properties Window on the right under the Outliner and click the "Armature" tab. Activate X-ray so you can watch trough the Mesh afterwards.

Activate X-Ray






Step5
We are going to add now the Mesh for our Cubeboy so it will be visible in "Unity" afterwards because we won't see the bones.
So go again to "Add" in your 3D-View and add a Cube. Be sure to be in "Object Mode".
Scale it down till it fits as a had with "S". Move it up to the "Head-Bone" with left clicking the blue "Z-Axis" Arrow.

Add Cube in Object Mode
Position and Scaling of Cubes






Our next step would be to duplicate the main Cube and put it on some parts of the Rig. Press "Shift + D" to duplicate the cube.
The new Cube will be outlined with white. Now press "Z", "Y" or "X" to move it along one Axis and confirm it with "Left-Click".
Repeat these steps till you are happe with your Cubeboy.

Duplicate Cubes






Step6
Mr. Cubeboy should now look like this. What we need now is to combine all the cubes because we only need one Mesh.
So select all the Cubes and press "Ctrl + J" to combine them. Go to the Outliner and Rename the "Cube.015" to e.g "Cubeboy_Mesh".

How Cubeboy should look like
Combine all the Cubes into one Mesh






Step7
In order to move our Rig together wit the Mesh we have to combine both. First select the Mesh and at last the Rig and combine them with "Ctrl + P" and select "With Automatic Weights".
Blender will set the weights of each bone so you don't have to do it manually.

To be sure everything went well switch to "Pose Mode". When we now select a bone it will be rendered blue.
With "G" you can grab and move around the bones with your Mouse. Release them with Right-Click so they get back in Position.

Switch to Pose Mode
bone Selection Pose Mode
Check if Bones are animation ready






Exporting the Rig


After everything is checked go back into Object-Mode and select the Rig and the Mesh. Go to File -> Export -> .fbx
In Export Settings we only need the Armature and Mesh. Notice also make sure that "Selected Objects" is checked. Name your .fbx file properly and export it.

How to Export
Export Settings






Importing the Rig in Unity


Open Unity and create a new project. Call it as you like.

Creat New Project






Right click in your Assets folder -> Create -> Folder and name it. "Model".
To import your Model simply drag it from your destination Folder into Model or right click "Import new Asset" and select your fbx file.
Drag and Drop the Model into the "Hierarchy" on the left side. Move the character towards the Camera so we can see it better.

Prepare Folder Steructures
Import_Model






Once we extend the Cubeboy's Rig in the Hierarchy we can see our Bones. Ignone the once with a "_end" Tag. You also notice why we called our Mesh "Cubeboy_Mesh" so we can't get confused.

Hierarchy of Bones






Setting up the Scripts


Our next step will be to prepare an empty Gameobject where we will put in our OSC Scripts.
Go to Hierarchy right click -> "Create Empty" and call it "OSC_Script".

Create an Empty Game Object






Download the OSChandler Files on Github | OSC for Unity and Copypaste the Plugin Folder from Assets into your Asset Folder.
Download this modified OSC_Reciever Script and paste it to a new Folder called "Scripts". Select your Empty called "OSC_Script" and click on "Add Component" and search for all three Scripts and add them. It should look like this:

Adding Scripts to Object
Script Setup






Bone Overview The Captury


Switch to The Captury and go into Playback Mode. Select your Recording.
On the right Panel you can see the "Skeleton Types" tab. Click on it and a whole list will appear which every single bone.
The bone names we need are the ones with the white background. These are the bones which we need to set up in our OSC_Reciever script.

Playback Mode
Skeleton Types






Modifying the OSC Script in Unity


Since these steps are going to be repetitive I will show you how to set up two bones for the OSC Messages between Unity and The Captury.
Open the OSC_Reciever script in Unity by default with "Viusal Studio" or "Monodevelop". Go to line "60" and "63" and change the "yourname" according to your recording.

    msg = handler.StringToOscMessage("/subscribe/yourname/blender/Root/vector 50.0 0.0 20.0"); 
    handler.Send(msg);

    msg = handler.StringToOscMessage("/subscribe/yourname/blender/Head/vector 50.0 0.0 20.0");
    handler.Send(msg);

Also change it in line "101" adn "107".

    switch (msgAddress){

     
        case "/yourname/blender/Root/vector":
            hips_x = values[0];
            hips_y = values[1];
            hips_z = values[2];
            break;

        case "/yourname/blender/Head/vector":
            head_x = values[0];
            head_y = values[1];
            head_z = values[2];
            break;

        default:
            //
            break;
    }


Notice: You can also change the Bone types coming from The Captury. E.g you don't want to animate the "Head" bone, change it to which ever bone you like. Save your Script.

Now go back to Unity. In the Editor on the right. Once the OSC_Script is selected, type in the bones from your Model not the ones from The Captury. You can see them in the Hierarchy.

Modifying the Script






Run Everything


Be sure everything is set up correctly (Message string / Bonenames / Modelbones / Ports / Remote IP) before you run The Captury and Unity.
Run The Captury in "Playback Mode" looping. Run Unity. Your Character shall move now.


Thank you!