Controlling agent with Kinect
[
Front page
] [
New
|
List of pages
|
Search
|
Recent changes
|
Help
]
Start:
Up:[[Tutorial]] Previous:[[Eye movement]] Nex...
----
#contents
* Control of an agent by Kinect [#s03fafab]
※This tutorial is valid for SIGVerse 2.1.0 or later.
※This tutorial requires OpenNI in your client computer. S...
This tutorial introduces a method to control a human avat...
** Version of OpenNI [#qd9610ff]
- OpenNI Stable Build for Windows x86 (32-bit) v1.5.2.23 ...
-- [[You can download from here>http://75.98.78.94/Downlo...
- PrimeSene NITE Stable Build for Windows x86 (32-bit) v1...
-- Download to select OpenNI Compliant Middleware Binarie...
- PrimeSense Sensor Module for OpenNI Version 5.1.0.25
-- [[You can download from here>https://github.com/avin2/...
** Set up at client side [#hc45c4cb]
A plug-in module for the SIGViewer receives motion data f...
*** Download [#e4ddcefe]
Please download SIGNiUserTracker_<version>.zip from [[KIN...
*** Installation [#u458405b]
Extract the zip file and find SIGNiUserTracker.sig file. ...
C:\Program Files (x86)\OpenNI\Samples\Bin\Release
(※In the case of Windows7 64bit)
Connect your Kinect to the client PC.
** Set up at server side [#vbb0fd2f]
A controller receives joint angle data; controls the avat...
*** Controller [#vc7899c9]
Create a controller file kinectController.cpp as follows:
#highlight(cpp){{
#include <string>
#include "Controller.h"
#include "Logger.h"
#include "ControllerEvent.h"
#define PI 3.141592
#define DEG2RAD(DEG) ( (PI) * (DEG) / 180.0 )
using namespace std;
class AgentController : public Controller
{
public:
double onAction(ActionEvent &evt);
void onRecvMsg(RecvMsgEvent &evt);
void onInit(InitEvent &evt);
private:
double m_posx, m_posy, m_posz;
double m_yrot;
double m_range;
int m_maxsize; // maximum number of joints
BaseService *m_kinect;
};
void AgentController::onInit(InitEvent &evt)
{
m_kinect = NULL;
SimObj *my = getObj(myname());
// Get initial position
Vector3d pos;
my->getPosition(pos);
m_posx = pos.x();
m_posy = pos.y();
m_posz = pos.z();
// Get initial rotation
Rotation rot;
my->getRotation(rot);
double qw = rot.qw();
double qy = rot.qy();
m_yrot = acos(fabs(qw))*2;
if(qw*qy > 0)
m_yrot = -1*m_yrot;
m_range = 0.1;
m_maxsize = 15;
}
double AgentController::onAction(ActionEvent &evt)
{
// Frequent check whether service is available or not
bool available = checkService("SIGKINECT");
// If service is available
if(available && m_kinect == NULL){
// Connect to the service
m_kinect = connectToService("SIGKINECT");
}
// If service is not working
else if (!available && m_kinect != NULL){
m_kinect = NULL;
}
return 1.0;
}
void AgentController::onRecvMsg(RecvMsgEvent &evt)
{
std::string sender = evt.getSender();
SimObj *my = getObj(myname());
// Receiving message
char *all_msg = (char*)evt.getMsg();
char *msg = strtok(all_msg," ");
if(strcmp(msg,"KINECT_DATA") == 0)
{
int i = 0;
while(true)
{
i++;
if(i == m_maxsize+1) break;
char *type = strtok(NULL,":");
// Position of the avatar
if(strcmp(type,"POSITION") == 0)
{
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
// Coordinate transform from Kinect coordin...
double gx = cos(m_yrot)*x - sin(m_yrot)*z;
double gz = sin(m_yrot)*x + cos(m_yrot)*z;
my->setPosition(m_posx+gx,m_posy+y,m_posz+g...
continue;
}
// Orientation of the whole body
else if(strcmp(type,"WAIST") == 0)
{
double w = atof(strtok(NULL,","));
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
my->setJointQuaternion("ROOT_JOINT0",w,x,y,...
continue;
}
else if(strcmp(type,"END") == 0) break;
// Rotation of joints
else
{
double w = atof(strtok(NULL,","));
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
double angle = acos(w)*2;
double tmp = sin(angle/2);
double vx = x/tmp;
double vy = y/tmp;
double vz = z/tmp;
double len = sqrt(vx*vx+vy*vy+vz*vz);
if(len < (1 - m_range) || (1 + m_range) < l...
my->setJointQuaternion(type,w,x,y,z);
continue;
}
}
}
}
extern "C" Controller * createController ()
{
return new AgentController;
}
}}
This controller receives messages from Kinect; rotates ea...
Compile with the following command.
$ ./sigmake.sh kinectController.cpp
*** World file [#kcdef075]
Create world file kinectWorld.xml as follows:
#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
<world name="myworld2">
<!--Gravity-->
<gravity x="0.0" y="-980.7" z="0.0"/>
<!--Initialization of Avatar, Man-nii-->
<instanciate class="Man-nii.xml">
<!--Name of the avatar-->
<set-attr-value name="name" value="kinect_man"/>
<!--Set C++ as controller language-->
<set-attr-value name="language" value="c++"/>
<!--Set kinectController.so as a controller-->
<set-attr-value name="implementation" value="./kinectC...
<!--Do not use dynamics simulation-->
<set-attr-value name="dynamics" value="false"/>
<!--Position of the avatar(x,y,z)-->
<set-attr-value name="x" value="0.0"/>
<set-attr-value name="y" value="60.0"/>
<set-attr-value name="z" value="0.0"/>
</instanciate>
</world>
}}
Just an avator will be appeared in this world. This sampl...
** Execution [#s58961b9]
*** SIGVerse server [#fe0a59dc]
Start the SIGVerse server
$ sigserver.sh -w ./kinectWorld.xml
Next, run the SIGViewer and connect to the sigserver. You...
*** Registration of user interface plug-in [#i3bdc209]
Click the [Service]-->[Add] menu on the top-left of the S...
#ref(SIGNi_1.PNG)
You then can see the following window.
Click the Add button to registrate SIGNiUserTracker.sig
#ref(./SIGNi_2.PNG,80%)
After the registration of SIGNiUserTracker.sig to the Ser...
Next, click the [Service]-->[Start] menu; then select the...
If the Kinect's depth map window is appeared and you see ...
[SYS] Service: "SIGKINECT" is available
Click the "START" button to start the simulation.
You can control the human avatar with the Kinect.
#ref(./SIGNi_3.PNG,40%)
** Compilation of source code of user interface plug-in [...
In this tutorial Visual C++ 2008 is used to compile the u...
*** Download of the source code of user interface plug-in...
Please download the source code from [[KINECT Service]]
//#ref(SIGNiUserTracker_1-0-0.zip)
Move the SIGNiUserTracker folder, which is contained in t...
C:\Program Files (x86)\OpenNI\Samples
*** Compile [#od39f64a]
Next, double-click the solution file in the SIGNiUserTrac...
※You have to install boost library in the case of the ver...
//VC++が起動したらソリューションエクスプローラの中のソリ...
Right-click the solution name in the solution explorer of...
C:\Program Files (x86)\OpenNI\Samples\Bin\Release
You can registrate the plug-in module to SIGViewer after ...
#highlight(end)
Up:[[Tutorial]] Previous:[[Eye movement]] Nex...
#counter
End:
Up:[[Tutorial]] Previous:[[Eye movement]] Nex...
----
#contents
* Control of an agent by Kinect [#s03fafab]
※This tutorial is valid for SIGVerse 2.1.0 or later.
※This tutorial requires OpenNI in your client computer. S...
This tutorial introduces a method to control a human avat...
** Version of OpenNI [#qd9610ff]
- OpenNI Stable Build for Windows x86 (32-bit) v1.5.2.23 ...
-- [[You can download from here>http://75.98.78.94/Downlo...
- PrimeSene NITE Stable Build for Windows x86 (32-bit) v1...
-- Download to select OpenNI Compliant Middleware Binarie...
- PrimeSense Sensor Module for OpenNI Version 5.1.0.25
-- [[You can download from here>https://github.com/avin2/...
** Set up at client side [#hc45c4cb]
A plug-in module for the SIGViewer receives motion data f...
*** Download [#e4ddcefe]
Please download SIGNiUserTracker_<version>.zip from [[KIN...
*** Installation [#u458405b]
Extract the zip file and find SIGNiUserTracker.sig file. ...
C:\Program Files (x86)\OpenNI\Samples\Bin\Release
(※In the case of Windows7 64bit)
Connect your Kinect to the client PC.
** Set up at server side [#vbb0fd2f]
A controller receives joint angle data; controls the avat...
*** Controller [#vc7899c9]
Create a controller file kinectController.cpp as follows:
#highlight(cpp){{
#include <string>
#include "Controller.h"
#include "Logger.h"
#include "ControllerEvent.h"
#define PI 3.141592
#define DEG2RAD(DEG) ( (PI) * (DEG) / 180.0 )
using namespace std;
class AgentController : public Controller
{
public:
double onAction(ActionEvent &evt);
void onRecvMsg(RecvMsgEvent &evt);
void onInit(InitEvent &evt);
private:
double m_posx, m_posy, m_posz;
double m_yrot;
double m_range;
int m_maxsize; // maximum number of joints
BaseService *m_kinect;
};
void AgentController::onInit(InitEvent &evt)
{
m_kinect = NULL;
SimObj *my = getObj(myname());
// Get initial position
Vector3d pos;
my->getPosition(pos);
m_posx = pos.x();
m_posy = pos.y();
m_posz = pos.z();
// Get initial rotation
Rotation rot;
my->getRotation(rot);
double qw = rot.qw();
double qy = rot.qy();
m_yrot = acos(fabs(qw))*2;
if(qw*qy > 0)
m_yrot = -1*m_yrot;
m_range = 0.1;
m_maxsize = 15;
}
double AgentController::onAction(ActionEvent &evt)
{
// Frequent check whether service is available or not
bool available = checkService("SIGKINECT");
// If service is available
if(available && m_kinect == NULL){
// Connect to the service
m_kinect = connectToService("SIGKINECT");
}
// If service is not working
else if (!available && m_kinect != NULL){
m_kinect = NULL;
}
return 1.0;
}
void AgentController::onRecvMsg(RecvMsgEvent &evt)
{
std::string sender = evt.getSender();
SimObj *my = getObj(myname());
// Receiving message
char *all_msg = (char*)evt.getMsg();
char *msg = strtok(all_msg," ");
if(strcmp(msg,"KINECT_DATA") == 0)
{
int i = 0;
while(true)
{
i++;
if(i == m_maxsize+1) break;
char *type = strtok(NULL,":");
// Position of the avatar
if(strcmp(type,"POSITION") == 0)
{
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
// Coordinate transform from Kinect coordin...
double gx = cos(m_yrot)*x - sin(m_yrot)*z;
double gz = sin(m_yrot)*x + cos(m_yrot)*z;
my->setPosition(m_posx+gx,m_posy+y,m_posz+g...
continue;
}
// Orientation of the whole body
else if(strcmp(type,"WAIST") == 0)
{
double w = atof(strtok(NULL,","));
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
my->setJointQuaternion("ROOT_JOINT0",w,x,y,...
continue;
}
else if(strcmp(type,"END") == 0) break;
// Rotation of joints
else
{
double w = atof(strtok(NULL,","));
double x = atof(strtok(NULL,","));
double y = atof(strtok(NULL,","));
double z = atof(strtok(NULL," "));
double angle = acos(w)*2;
double tmp = sin(angle/2);
double vx = x/tmp;
double vy = y/tmp;
double vz = z/tmp;
double len = sqrt(vx*vx+vy*vy+vz*vz);
if(len < (1 - m_range) || (1 + m_range) < l...
my->setJointQuaternion(type,w,x,y,z);
continue;
}
}
}
}
extern "C" Controller * createController ()
{
return new AgentController;
}
}}
This controller receives messages from Kinect; rotates ea...
Compile with the following command.
$ ./sigmake.sh kinectController.cpp
*** World file [#kcdef075]
Create world file kinectWorld.xml as follows:
#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
<world name="myworld2">
<!--Gravity-->
<gravity x="0.0" y="-980.7" z="0.0"/>
<!--Initialization of Avatar, Man-nii-->
<instanciate class="Man-nii.xml">
<!--Name of the avatar-->
<set-attr-value name="name" value="kinect_man"/>
<!--Set C++ as controller language-->
<set-attr-value name="language" value="c++"/>
<!--Set kinectController.so as a controller-->
<set-attr-value name="implementation" value="./kinectC...
<!--Do not use dynamics simulation-->
<set-attr-value name="dynamics" value="false"/>
<!--Position of the avatar(x,y,z)-->
<set-attr-value name="x" value="0.0"/>
<set-attr-value name="y" value="60.0"/>
<set-attr-value name="z" value="0.0"/>
</instanciate>
</world>
}}
Just an avator will be appeared in this world. This sampl...
** Execution [#s58961b9]
*** SIGVerse server [#fe0a59dc]
Start the SIGVerse server
$ sigserver.sh -w ./kinectWorld.xml
Next, run the SIGViewer and connect to the sigserver. You...
*** Registration of user interface plug-in [#i3bdc209]
Click the [Service]-->[Add] menu on the top-left of the S...
#ref(SIGNi_1.PNG)
You then can see the following window.
Click the Add button to registrate SIGNiUserTracker.sig
#ref(./SIGNi_2.PNG,80%)
After the registration of SIGNiUserTracker.sig to the Ser...
Next, click the [Service]-->[Start] menu; then select the...
If the Kinect's depth map window is appeared and you see ...
[SYS] Service: "SIGKINECT" is available
Click the "START" button to start the simulation.
You can control the human avatar with the Kinect.
#ref(./SIGNi_3.PNG,40%)
** Compilation of source code of user interface plug-in [...
In this tutorial Visual C++ 2008 is used to compile the u...
*** Download of the source code of user interface plug-in...
Please download the source code from [[KINECT Service]]
//#ref(SIGNiUserTracker_1-0-0.zip)
Move the SIGNiUserTracker folder, which is contained in t...
C:\Program Files (x86)\OpenNI\Samples
*** Compile [#od39f64a]
Next, double-click the solution file in the SIGNiUserTrac...
※You have to install boost library in the case of the ver...
//VC++が起動したらソリューションエクスプローラの中のソリ...
Right-click the solution name in the solution explorer of...
C:\Program Files (x86)\OpenNI\Samples\Bin\Release
You can registrate the plug-in module to SIGViewer after ...
#highlight(end)
Up:[[Tutorial]] Previous:[[Eye movement]] Nex...
#counter
Page: