[[Tutorial]]

#contents

* Control of an agent by Kinect [#s03fafab]

※This tutorial is valid for SIGVerse 2.1.0 or later.
※This tutorial requires OpenNI in your client computer. Sample program NiUserTracker is also required to be executable.

This tutorial introduces a method to control a human avatar using Kinect sensor.

** Version of OpenNI [#qd9610ff]
- OpenNI Stable Build for Windows x86 (32-bit) v1.5.2.23 Development Edition
-- [[You can download from here>http://75.98.78.94/Downloads/OpenNIModules.aspx]]
- PrimeSene NITE Stable Build for Windows x86 (32-bit) v1.5.2.21 Development
-- Download to select OpenNI Compliant Middleware Binaries on the same page.
- PrimeSense Sensor Module for OpenNI Version 5.1.0.25　
-- [[You can download from here>https://github.com/avin2/SensorKinect]]

** Set up at client side [#hc45c4cb]

A plug-in module for the SIGViewer receives motion data from Kinect, calculates quaternion of each joint, and send the data to avatar controller on the server.

*** Download [#e4ddcefe]

Please download SIGNiUserTracker_<version>.zip from [[KINECT Service]]


*** Installation [#u458405b]

Extract the zip file and find SIGNiUserTracker.sig file. Put the SIGNiUserTracker.sig in a folder which has the OpenNI sample program. The defold folder name is:
 C:\Program Files (x86)\OpenNI\Samples\Bin\Release
(※In the case of Windows7 64bit）

Connect your Kinect to the client PC.


** Set up at server side [#vbb0fd2f]

A controller receives joint angle data; controls the avatar's joints.

*** Controller [#vc7899c9]

Create a controller file kinectController.cpp as follows:

#highlight(cpp){{
#include <string>
#include "Controller.h"
#include "Logger.h"
#include "ControllerEvent.h"

#define PI 3.141592
#define DEG2RAD(DEG) ( (PI) * (DEG) / 180.0 )

using namespace std;

class AgentController : public Controller
{
public:
  double onAction(ActionEvent &evt);
  void onRecvMsg(RecvMsgEvent &evt);
  void onInit(InitEvent &evt);

private:
  double m_posx, m_posy, m_posz;
  double m_yrot;
  double m_range;
  int m_maxsize;     // maximum number of joints

  BaseService *m_kinect;
};

void AgentController::onInit(InitEvent &evt)
{
  m_kinect = NULL;
  SimObj *my = getObj(myname());

  // Get initial position
  Vector3d pos;
  my->getPosition(pos);
  m_posx = pos.x();
  m_posy = pos.y();
  m_posz = pos.z();

  // Get initial rotation
  Rotation rot;
  my->getRotation(rot);
  double qw = rot.qw();
  double qy = rot.qy();

  m_yrot = acos(fabs(qw))*2;
  if(qw*qy > 0)
    m_yrot = -1*m_yrot;

  m_range = 0.1;
  m_maxsize = 15;
}

double AgentController::onAction(ActionEvent &evt)
{
  // Frequent check whether service is available or not
  bool available = checkService("SIGKINECT");

  // If service is available
  if(available && m_kinect == NULL){
    // Connect to the service
    m_kinect = connectToService("SIGKINECT");
  }
  // If service is not working
  else if (!available && m_kinect != NULL){
    m_kinect = NULL;
  }
  return 1.0;
}

void AgentController::onRecvMsg(RecvMsgEvent &evt)
{
  std::string sender = evt.getSender();

  SimObj *my = getObj(myname());

  // Receiving message
  char *all_msg = (char*)evt.getMsg();

  char *msg = strtok(all_msg," ");
  if(strcmp(msg,"KINECT_DATA") == 0)
    {
      int i = 0;
      while(true)
        {
          i++;
          if(i == m_maxsize+1) break;
          char *type = strtok(NULL,":");

          // Position of the avatar
          if(strcmp(type,"POSITION") == 0)
            {
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              // Coordinate transform from Kinect coordinate to SIGVerse coordinate
              double gx = cos(m_yrot)*x - sin(m_yrot)*z;
              double gz = sin(m_yrot)*x + cos(m_yrot)*z;
              my->setPosition(m_posx+gx,m_posy+y,m_posz+gz);
              continue;
            }

          // Orientation of the whole body
          else if(strcmp(type,"WAIST") == 0)
            {
              double w = atof(strtok(NULL,","));
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              my->setJointQuaternion("ROOT_JOINT0",w,x,y,z);
              continue;
            }

          else if(strcmp(type,"END") == 0)  break;

          // Rotation of joints
          else
            {
              double w = atof(strtok(NULL,","));
              double x = atof(strtok(NULL,","));
              double y = atof(strtok(NULL,","));
              double z = atof(strtok(NULL," "));
              double angle = acos(w)*2;
              double tmp = sin(angle/2);
              double vx = x/tmp;
              double vy = y/tmp;
              double vz = z/tmp;
              double len = sqrt(vx*vx+vy*vy+vz*vz);
              if(len < (1 - m_range) || (1 + m_range) < len) continue;
              my->setJointQuaternion(type,w,x,y,z);
              continue;
            }
        }
    }
}

extern "C"  Controller * createController ()
{
  return new AgentController;
}
}}

This controller receives messages from Kinect; rotates each joint of an avatar.

Compile with the following command.

 $ ./sigmake.sh kinectController.cpp

*** World file [#kcdef075]

Create world file kinectWorld.xml as follows:

#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
 <world name="myworld2">

 <!--Gravity-->
   <gravity x="0.0" y="-980.7" z="0.0"/>

 <!--Initialization of Avatar, Man-nii-->
   <instanciate class="Man-nii.xml">

 <!--Name of the avatar-->
   <set-attr-value name="name" value="kinect_man"/>

 <!--Set C++ as controller language-->
   <set-attr-value name="language" value="c++"/>

 <!--Set kinectController.so as a controller-->
   <set-attr-value name="implementation" value="./kinectController.so"/>

 <!--Do not use dynamics simulation-->
   <set-attr-value name="dynamics" value="false"/>

 <!--Position of the avatar(x,y,z)-->
   <set-attr-value name="x" value="0.0"/>
   <set-attr-value name="y" value="60.0"/>
   <set-attr-value name="z" value="0.0"/>

  </instanciate>
</world>
}}

Just an avator will be appeared in this world. This sample controls only one human avatar by the Kinect.

** Execution [#s58961b9]

*** SIGVerse server [#fe0a59dc]

Start the SIGVerse server
 $ sigserver.sh -w ./kinectWorld.xml

Next, run the SIGViewer and connect to the sigserver. You can see an avatar is standing.

*** Registration of user interface plug-in [#i3bdc209]

Click the [Service]-->[Add] menu on the top-left of the SIGViewer window bar.

#ref(SIGNi_1.PNG)

You then can see the following window.
Click the Add button to registrate SIGNiUserTracker.sig

#ref(./SIGNi_2.PNG,80%)

After the registration of SIGNiUserTracker.sig to the Service list, click the OK button to close the window.

Next, click the [Service]-->[Start] menu; then select the SIGNiUserTracker.sig.

If the Kinect's depth map window is appeared and you see the following message on the server, it means the application is successfully started.
 [SYS]  Service: "SIGKINECT" is available

Click the "START" button to start the simulation.

You can control the human avatar with the Kinect.

#ref(./SIGNi_3.PNG,40%)

** Compilation of source code of user interface plug-in [#n9886846]

In this tutorial Visual C++ 2008 is used to compile the user interface plug-in.

*** Download of the source code of user interface plug-in [#g1622ce0]

Please download the source code from [[KINECT Service]]
//#ref(SIGNiUserTracker_1-0-0.zip)

Move the SIGNiUserTracker folder, which is contained in the ZIP file, to a folder which has sample codes of OpenNI.


C:\Program Files (x86)\OpenNI\Samples

*** Compile [#od39f64a]
Next, double-click the solution file in the SIGNiUserTracker folder with administrative right.

※You have to install boost library in the case of the version number is 1.0.0.

//VC++が起動したらソリューションエクスプローラの中のソリューション名を右クリックしてビルドを選択します。ビルドが成功すると以下に実行ファイルSIGNiUserTracker.exeが作成されます。
Right-click the solution name in the solution explorer of VC++ to build the source code. If the build succeeds, SIGNiUserTracker.exe will be created at 
C:\Program Files (x86)\OpenNI\Samples\Bin\Release

You can registrate the plug-in module to SIGViewer after changing the suffix from .exe to .sig.

#highlight(end)
Front page   New List of pages Search Recent changes   Help   RSS of recent changes