[[Tutorial]]




#contents

This tutorial introduces a method to control a human avatar using KinectV2 sensor

* Control of an agent by Kinect [#s03fafab]

※This tutorial is valid for SIGVerse 2.1.0 or later.
※This tutorial requires OpenNI in your client computer. Sample program NiUserTracker is also required to be executable.
※This tutorial requires Windows 8.1 and Kinect for Windows SDK 2.0. 

This tutorial introduces a method to control a human avatar using Kinect sensor.
※This tutorial is valid for SIGVerse 2.2.0 or later.

** Version of OpenNI [#qd9610ff]
- OpenNI Stable Build for Windows x86 (32-bit) v1.5.2.23 Development Edition
-- [[You can download from here>http://75.98.78.94/Downloads/OpenNIModules.aspx]]
- PrimeSene NITE Stable Build for Windows x86 (32-bit) v1.5.2.21 Development
-- Download to select OpenNI Compliant Middleware Binaries on the same page.
- PrimeSense Sensor Module for OpenNI Version 5.1.0.25　
-- [[You can download from here>https://github.com/avin2/SensorKinect]]
※This tutorial requires that OpenCV be installed on your PC.

** Set up at client side [#hc45c4cb]
※ To build SIGService for KinectV2 from the source code, you will require OpenNI to be setup in your PC.

A plug-in module for the SIGViewer receives motion data from Kinect, calculates quaternion of each joint, and send the data to avatar controller on the server.
------
If you have any questions after reading this page: Please feel free to email at:
sigverse-users@googlegroups.com
-------

*** Download [#e4ddcefe]

Please download SIGNiUserTracker_<version>.zip from [[KINECT Service]]
** Kinect for Windows SDK 2.0 [#qd9610ff]

-[[You can download SDK from here>http://www.microsoft.com/en-us/download/details.aspx?id=44561]]

*** Installation [#u458405b]
** Set up at client side on Windows 8.1 [#hc45c4cb]

Extract the zip file and find SIGNiUserTracker.sig file. Put the SIGNiUserTracker.sig in a folder which has the OpenNI sample program. The defold folder name is:
 C:\Program Files (x86)\OpenNI\Samples\Bin\Release
(※In the case of Windows7 64bit）
A plug-in module for the SIGViewer receives motion data from Kinect, calculates Quaternion of each joint, and send the data to avatar controller on the server.

Connect your Kinect to the client PC.
*** Building the KinectV2 SIGService from source [#nefbf579]

For building the SIGService, Visual C++ 2013 Express Edition has been used.
However, you can use  Visual C++ 2010 Express Edition as well.

The Source is uploaded on GitHub which you can clone on your disk.

    $ git clone https://github.com/SIGVerse/Client.git
    $ cd Plugin/KinectV2
    
The instructions to build are given on the Github page 
https://github.com/SIGVerse/Client/tree/master/Plugin/KinectV2

** Set up at server side [#vbb0fd2f]

A controller receives joint angle data; controls the avatar's joints.

*** Controller [#vc7899c9]

Create a controller file kinectController.cpp as follows:

#highlight(cpp){{
#include <string>
#include "Controller.h"
#include "Logger.h"
#include "ControllerEvent.h"
#gist(raghavendrajain/bbe0100ecaa008a3951b)

#define PI 3.141592
#define DEG2RAD(DEG) ( (PI) * (DEG) / 180.0 )

using namespace std;

class AgentController : public Controller
{
public:
  double onAction(ActionEvent &evt);
  void onRecvMsg(RecvMsgEvent &evt);
  void onInit(InitEvent &evt);

private:
  double m_posx, m_posy, m_posz;
  double m_yrot;
  double m_range;
  int m_maxsize;     // maximum number of joints

  BaseService *m_kinect;
};

void AgentController::onInit(InitEvent &evt)
{
  m_kinect = NULL;
  SimObj *my = getObj(myname());

  // Get initial position
  Vector3d pos;
  my->getPosition(pos);
  m_posx = pos.x();
  m_posy = pos.y();
  m_posz = pos.z();

  // Get initial rotation
  Rotation rot;
  my->getRotation(rot);
  double qw = rot.qw();
  double qy = rot.qy();

  m_yrot = acos(fabs(qw))*2;
  if(qw*qy > 0)
    m_yrot = -1*m_yrot;

  m_range = 0.1;
  m_maxsize = 15;
}

double AgentController::onAction(ActionEvent &evt)
{
  // Frequent check whether service is available or not
  bool available = checkService("SIGKINECT");

  // If service is available
  if(available && m_kinect == NULL){
    // Connect to the service
    m_kinect = connectToService("SIGKINECT");
  }
  // If service is not working
  else if (!available && m_kinect != NULL){
    m_kinect = NULL;
  }
  return 1.0;
}

void AgentController::onRecvMsg(RecvMsgEvent &evt)
{
  std::string sender = evt.getSender();

  SimObj *my = getObj(myname());

  // Receiving message
  char *all_msg = (char*)evt.getMsg();

  char *msg = strtok(all_msg," ");
  if(strcmp(msg,"KINECT_DATA") == 0)
    {
      int i = 0;
      while(true)
        {
          i++;
          if(i == m_maxsize+1) break;
          char *type = strtok(NULL,":");

          // Position of the avatar
          if(strcmp(type,"POSITION") == 0)
            {
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              // Coordinate transform from Kinect coordinate to SIGVerse coordinate
              double gx = cos(m_yrot)*x - sin(m_yrot)*z;
              double gz = sin(m_yrot)*x + cos(m_yrot)*z;
              my->setPosition(m_posx+gx,m_posy+y,m_posz+gz);
              continue;
            }

          // Orientation of the whole body
          else if(strcmp(type,"WAIST") == 0)
            {
              double w = atof(strtok(NULL,","));
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              my->setJointQuaternion("ROOT_JOINT0",w,x,y,z);
              continue;
            }

          else if(strcmp(type,"END") == 0)  break;

          // Rotation of joints
          else
            {
              double w = atof(strtok(NULL,","));
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              double angle = acos(w)*2;
              double tmp = sin(angle/2);
              double vx = x/tmp;
              double vy = y/tmp;
              double vz = z/tmp;
              double len = sqrt(vx*vx+vy*vy+vz*vz);
              if(len < (1 - m_range) || (1 + m_range) < len) continue;
              my->setJointQuaternion(type,w,x,y,z);
              continue;
            }
        }
    }
}

extern "C"  Controller * createController ()
{
  return new AgentController;
}
}}

This controller receives messages from Kinect; rotates each joint of an avatar.

Compile with the following command.

 $ ./sigmake.sh kinectController.cpp

*** World file [#kcdef075]

Create world file kinectWorld.xml as follows:

#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
 <world name="myworld2">
#gist(raghavendrajain/b5fb732e835db76eabcc)

 <!--Gravity-->
   <gravity x="0.0" y="-980.7" z="0.0"/>
Just an avatar will be appeared in this world. This sample controls only one human avatar by the Kinect.

 <!--Initialization of Avatar, Man-nii-->
   <instanciate class="Man-nii.xml">

 <!--Name of the avatar-->
   <set-attr-value name="name" value="kinect_man"/>

 <!--Set C++ as controller language-->
   <set-attr-value name="language" value="c++"/>

 <!--Set kinectController.so as a controller-->
   <set-attr-value name="implementation" value="./kinectController.so"/>

 <!--Do not use dynamics simulation-->
   <set-attr-value name="dynamics" value="false"/>

 <!--Position of the avatar(x,y,z)-->
   <set-attr-value name="x" value="0.0"/>
   <set-attr-value name="y" value="60.0"/>
   <set-attr-value name="z" value="0.0"/>

  </instanciate>
</world>
}}

Just an avator will be appeared in this world. This sample controls only one human avatar by the Kinect.

** Execution [#s58961b9]

*** SIGVerse server [#fe0a59dc]

Start the SIGVerse server
 $ sigserver.sh -w ./kinectWorld.xml

Next, run the SIGViewer and connect to the sigserver. You can see an avatar is standing.

*** Registration of user interface plug-in [#i3bdc209]

Click the [Service]-->[Add] menu on the top-left of the SIGViewer window bar.

#ref(SIGNi_1.PNG)

You then can see the following window.
Click the Add button to registrate SIGNiUserTracker.sig

#ref(./SIGNi_2.PNG,80%)

After the registration of SIGNiUserTracker.sig to the Service list, click the OK button to close the window.

Next, click the [Service]-->[Start] menu; then select the SIGNiUserTracker.sig.
Next, click the [Service]-->[Start] menu; then select the kinectV2_2013.sig

If the Kinect's depth map window is appeared and you see the following message on the server, it means the application is successfully started.
 [SYS]  Service: "SIGKINECT" is available

Click the "START" button to start the simulation.

You can control the human avatar with the Kinect.

#ref(./SIGNi_3.PNG,40%)
#ref(./SIGNi_3.JPG,25%)

** Compilation of source code of user interface plug-in [#n9886846]

In this tutorial Visual C++ 2008 is used to compile the user interface plug-in.

*** Download of the source code of user interface plug-in [#g1622ce0]

Please download the source code from [[KINECT Service]]
//#ref(SIGNiUserTracker_1-0-0.zip)

Move the SIGNiUserTracker folder, which is contained in the ZIP file, to a folder which has sample codes of OpenNI.


C:\Program Files (x86)\OpenNI\Samples

*** Compile [#od39f64a]
Next, double-click the solution file in the SIGNiUserTracker folder with administrative right.

※You have to install boost library in the case of the version number is 1.0.0.

//VC++が起動したらソリューションエクスプローラの中のソリューション名を右クリックしてビルドを選択します。ビルドが成功すると以下に実行ファイルSIGNiUserTracker.exeが作成されます。
Right-click the solution name in the solution explorer of VC++ to build the source code. If the build succeeds, SIGNiUserTracker.exe will be created at 
C:\Program Files (x86)\OpenNI\Samples\Bin\Release

You can registrate the plug-in module to SIGViewer after changing the suffix from .exe to .sig.

#highlight(end)
Front page   New List of pages Search Recent changes   Help   RSS of recent changes