Speech recognition using Julius
[
Front page
] [
New
|
List of pages
|
Search
|
Recent changes
|
Help
]
Start:
//
// This page was written by tafifet & guezout
//
Up:[[Tutorial]] Previous:[[Joystick service]]
----
This tutorial shows how to use Julius engine with SIGVers...
#contents
*Overview [#ed1b8135]
The Speech recognition Service consists in two parallel p...
- The Julius Speech recognition process Julius.exe which...
- The SIGService Process Takeit.sig which reads recognize...
&ref(JuliusOverview.PNG,,80%);
*Speach recognition plug in [#ed1b9196]
** Configuration [#ed1b8156]
You need to run services as an administrator, go to Speec...
To start the service, add the takeIt.sig file to SIGViewe...
Ps: Don't forget to configure the microphone before.
*Speech recognition grammar [#ed1b9136]
** Building grammar [#ed1b8176]
Use Language Model file which contains a large list of wo...
Each word in grammar file has an associated list of phone...
Acoustic model is associated to grammar, it contain a sta...
Recognition grammar is separated into two files:
• the ".grammar" file which defines a set of rules govern...
• the ".voca" file which defines the actual "Word Candida...
-User has to use grammar generator to create new grammars
-Please refer to this tutorial for more information about...
You can use the file VoxForgeDict that contains the dicti...
PATH-TO-SERVICE\SIGVerseGrammar\VoxForgeDict
** Compiling grammar [#ed1b8156]
The .grammar and .voca files need to be compiled into "....
-Compile your files (sample.grammar and sample.voca) as f...
-Command to execute:
PATH-TO-SERVICE/mkdfa.pl .\grammar\simple
This will generate the expected grammar
&ref(GrammarGen.PNG,,80%);
-It generates sample.dfa and sample.term files which cont...
Now user can perform recognition on the new defined gramm...
*Controller [#ed1b8158]
The controller receives messages from the SIGVerse servic...
voiceRecognition.cpp:
#include "Controller.h"
#include "Logger.h"
#include <unistd.h>
#include "ControllerEvent.h"
#include <sstream>
class voiceRecognition : public Controller
{
public:
void onInit(InitEvent &evt);
double onAction(ActionEvent &evt);
void onRecvMsg(RecvMsgEvent &evt);
public:
RobotObj *my;
double velocity;
int i;
double direction1;
double direction2;
int j;
double Weel_one;
double Weel_two;
};
void voiceRecognition::onInit(InitEvent &evt)
{
Weel_one = 0.0;
Weel_two = 0.0;
my = this->getRobotObj(this->myname());
my->setWheel(10.0, 10.0);
}
double voiceRecognition::onAction(ActionEvent &evt)
{
my->setWheelVelocity(Weel_one,Weel_two);
return 0.01;
}
void voiceRecognition::onRecvMsg(RecvMsgEvent &evt)
{
std::string sender = evt.getSender();
std::string msg = evt.getMsg();
LOG_MSG(("message : %s", msg.c_str()));
if(strcmp("moveforward",msg.c_str())==0)
{
Weel_one = 3.0;
Weel_two = 3.0;
}
else if(strcmp("back",msg.c_str())==0)
{
Weel_one = -3.0;
Weel_two = -3.0;
}
else if(strcmp("turnleft",msg.c_str())==0)
{
Weel_one = 0.78;
Weel_two = -0.78;
}
else if(strcmp("turnright",msg.c_str())==0)
{
Weel_one = -0.78;
Weel_two = 0.78;
}
}
extern "C" Controller * createController ()
{
return new voiceRecognition;
}
If you want to Enable/disable the speech recognition serv...
Enable service:
sendMsg(“VoiceReco_Service”,”Start_Recognition”)
Disable service:
sendMsg(“VoiceReco_Service”,”Stop_Recognition”)
*World file [#ed1b8156]
VoiceRecognition.xml:
<?xml version="1.0" encoding="utf-8"?>
<world name="myworld1">
<gravity x="0.0" y="-980.7" z="0.0"/>
<instanciate class="WheelRobot-nii-v1.xml" type="Robot">
<set-attr-value name="name" value="robot_000"/>
<set-attr-value name="language" value="c++"/>
<set-attr-value name="implementation"
value="./voiceRecognition.so"/>
<set-attr-value name="dynamics" value="false"/>
<set-attr-value name="x" value="-100.0"/>
<set-attr-value name="y" value="30.0"/>
<set-attr-value name="z" value="-130.0"/>
<set-attr-value name="collision" value="true"/>
<!--stereo camera right-->
<camera id="1"
link="REYE_LINK"
direction="0.0 -1.0 1.0"
position="0.0 0.0 3.0"/>
<!--stereo camera left-->
<camera id="2"
link="LEYE_LINK"
direction="0.0 -1.0 1.0"
position="0.0 0.0 3.0"/>
<!--distance sensor-->
<camera id="3"
link="WAIST_LINK0"
direction="0.0 0.0 1.0"
position="0.0 -5.0 20.0"/>
<!--monitoring camera-->
<camera id="4"
link="WAIST_LINK2"
direction="0 0 1"
quaternion="0.0 0.0 -0.966 0.259"
position="0.0 40.0 120.0"/>
</instanciate>
</world>
*Running the service [#ed1b6236]
Here are steps to follow to run the project:
- First, you need to run the controller, go to the contro...
$ sigserver.sh -w ./VoiceRecognition.xml
- After that, you have to run the SIGViewer.
- Then, run the takeIt.exe service for speech recognition...
To make the robot moving, the allowed voice commands are :
move forward, move backward, turn left and turn right.
*Speech Recognition Service Source code [#ed1b8135]
** Julius process [#ed1b8136]
The Julius project can be downloaded from the [[official ...
It is located in : path_to_julius_service\julius\output_s...
*** Source code explanation [#ed1b8138]
Below is the main lines for retriving the recognized text...
output_stdout.cpp file:
#highlight(cpp){{
hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE, // use paging f...
NULL, // default secu...
PAGE_READWRITE, // read/write a...
0, // maximum obje...
BUF_SIZE, // maximum obje...
szName); // name of mapp...
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not create file mapping object...
GetLastError());
return 1;
}
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to m...
FILE_MAP_ALL_ACCESS, // read/writ...
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return 1;
}
}}
Create a new file mapping object, and initialize a buffer.
#highlight(cpp){{
int i;
if(firstLoop == 1){
ZeroMemory(
(PVOID)pBuf,
(_tcslen(pBuf) * sizeof(pBuf) )
);
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
printf("close the file mapping \n");
}
firstLoop=1;
strcpy (resultReco," ");
if (seq != NULL) {
for (i=1;i<n-1;i++) {
strcat(resultReco,winfo->woutput[seq[i]]);
}
}
myprintf("%s",resultReco);
}}
This part of code allows to retrieve the recognized text ...
** SIGService process [#ed1b8137]
This is the SIGVerse service for speech recognition, its ...
*** Source code explanation [#ed1b8338]
recognition.cpp:
#highlight(cpp){{
TCHAR szName[]=TEXT("Global\\MyFileMappingObject");
}}
Initialize a TCHAR variable with the name of the mapping ...
#highlight(cpp){{
system("start .\\julius.exe -input mic -C .\\SIGVerseGra...
}}
Start the julius service.
#highlight(cpp){{
hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE, // use paging f...
NULL, // default secu...
PAGE_READWRITE, // read/write a...
0, // maximum obje...
BUF_SIZE, // maximum obje...
szName); // name of mapp...
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not create file mapping object...
GetLastError());
return 1;
}
}}
Create and initialize the file mapping object.
#highlight(cpp){{
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to m...
FILE_MAP_ALL_ACCESS, // read/writ...
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return 0.1;
}
}}
Create a buffer.
std::string send_msg =(std::string) pBuf;
this->sendMsg("robot_000",send_msg.c_str());
Send the recognized text to the controller.
#highlight(cpp){{
if (strcmp(s.c_str(),"Stop_Reco")==0)
{
Enable = false;
}
else if(strcmp(s.c_str(),"Start_Reco")==0)
{
Enable = true;
}
}}
Enable/Disable the recognition when needed.
**Compiling the project [#ed1b8438]
You first need to download the whole project from the GIT...
Go to "SIGVerseJulius\msvc" and open the VS 2008 project....
In the solution explorer window, 4 projects are listed, y...
1) libsent
2) libjulius
3) julius
4) takeIt
*Downloading the project [#ed1b8736]
To download the project from GIT repository, use the foll...
https://github.com/SIGVerse/samples/tree/master/SpeechRec...
#highlight(end)
Up:[[Tutorial]] Previous:[[Joystick service]]
#counter
End:
//
// This page was written by tafifet & guezout
//
Up:[[Tutorial]] Previous:[[Joystick service]]
----
This tutorial shows how to use Julius engine with SIGVers...
#contents
*Overview [#ed1b8135]
The Speech recognition Service consists in two parallel p...
- The Julius Speech recognition process Julius.exe which...
- The SIGService Process Takeit.sig which reads recognize...
&ref(JuliusOverview.PNG,,80%);
*Speach recognition plug in [#ed1b9196]
** Configuration [#ed1b8156]
You need to run services as an administrator, go to Speec...
To start the service, add the takeIt.sig file to SIGViewe...
Ps: Don't forget to configure the microphone before.
*Speech recognition grammar [#ed1b9136]
** Building grammar [#ed1b8176]
Use Language Model file which contains a large list of wo...
Each word in grammar file has an associated list of phone...
Acoustic model is associated to grammar, it contain a sta...
Recognition grammar is separated into two files:
• the ".grammar" file which defines a set of rules govern...
• the ".voca" file which defines the actual "Word Candida...
-User has to use grammar generator to create new grammars
-Please refer to this tutorial for more information about...
You can use the file VoxForgeDict that contains the dicti...
PATH-TO-SERVICE\SIGVerseGrammar\VoxForgeDict
** Compiling grammar [#ed1b8156]
The .grammar and .voca files need to be compiled into "....
-Compile your files (sample.grammar and sample.voca) as f...
-Command to execute:
PATH-TO-SERVICE/mkdfa.pl .\grammar\simple
This will generate the expected grammar
&ref(GrammarGen.PNG,,80%);
-It generates sample.dfa and sample.term files which cont...
Now user can perform recognition on the new defined gramm...
*Controller [#ed1b8158]
The controller receives messages from the SIGVerse servic...
voiceRecognition.cpp:
#include "Controller.h"
#include "Logger.h"
#include <unistd.h>
#include "ControllerEvent.h"
#include <sstream>
class voiceRecognition : public Controller
{
public:
void onInit(InitEvent &evt);
double onAction(ActionEvent &evt);
void onRecvMsg(RecvMsgEvent &evt);
public:
RobotObj *my;
double velocity;
int i;
double direction1;
double direction2;
int j;
double Weel_one;
double Weel_two;
};
void voiceRecognition::onInit(InitEvent &evt)
{
Weel_one = 0.0;
Weel_two = 0.0;
my = this->getRobotObj(this->myname());
my->setWheel(10.0, 10.0);
}
double voiceRecognition::onAction(ActionEvent &evt)
{
my->setWheelVelocity(Weel_one,Weel_two);
return 0.01;
}
void voiceRecognition::onRecvMsg(RecvMsgEvent &evt)
{
std::string sender = evt.getSender();
std::string msg = evt.getMsg();
LOG_MSG(("message : %s", msg.c_str()));
if(strcmp("moveforward",msg.c_str())==0)
{
Weel_one = 3.0;
Weel_two = 3.0;
}
else if(strcmp("back",msg.c_str())==0)
{
Weel_one = -3.0;
Weel_two = -3.0;
}
else if(strcmp("turnleft",msg.c_str())==0)
{
Weel_one = 0.78;
Weel_two = -0.78;
}
else if(strcmp("turnright",msg.c_str())==0)
{
Weel_one = -0.78;
Weel_two = 0.78;
}
}
extern "C" Controller * createController ()
{
return new voiceRecognition;
}
If you want to Enable/disable the speech recognition serv...
Enable service:
sendMsg(“VoiceReco_Service”,”Start_Recognition”)
Disable service:
sendMsg(“VoiceReco_Service”,”Stop_Recognition”)
*World file [#ed1b8156]
VoiceRecognition.xml:
<?xml version="1.0" encoding="utf-8"?>
<world name="myworld1">
<gravity x="0.0" y="-980.7" z="0.0"/>
<instanciate class="WheelRobot-nii-v1.xml" type="Robot">
<set-attr-value name="name" value="robot_000"/>
<set-attr-value name="language" value="c++"/>
<set-attr-value name="implementation"
value="./voiceRecognition.so"/>
<set-attr-value name="dynamics" value="false"/>
<set-attr-value name="x" value="-100.0"/>
<set-attr-value name="y" value="30.0"/>
<set-attr-value name="z" value="-130.0"/>
<set-attr-value name="collision" value="true"/>
<!--stereo camera right-->
<camera id="1"
link="REYE_LINK"
direction="0.0 -1.0 1.0"
position="0.0 0.0 3.0"/>
<!--stereo camera left-->
<camera id="2"
link="LEYE_LINK"
direction="0.0 -1.0 1.0"
position="0.0 0.0 3.0"/>
<!--distance sensor-->
<camera id="3"
link="WAIST_LINK0"
direction="0.0 0.0 1.0"
position="0.0 -5.0 20.0"/>
<!--monitoring camera-->
<camera id="4"
link="WAIST_LINK2"
direction="0 0 1"
quaternion="0.0 0.0 -0.966 0.259"
position="0.0 40.0 120.0"/>
</instanciate>
</world>
*Running the service [#ed1b6236]
Here are steps to follow to run the project:
- First, you need to run the controller, go to the contro...
$ sigserver.sh -w ./VoiceRecognition.xml
- After that, you have to run the SIGViewer.
- Then, run the takeIt.exe service for speech recognition...
To make the robot moving, the allowed voice commands are :
move forward, move backward, turn left and turn right.
*Speech Recognition Service Source code [#ed1b8135]
** Julius process [#ed1b8136]
The Julius project can be downloaded from the [[official ...
It is located in : path_to_julius_service\julius\output_s...
*** Source code explanation [#ed1b8138]
Below is the main lines for retriving the recognized text...
output_stdout.cpp file:
#highlight(cpp){{
hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE, // use paging f...
NULL, // default secu...
PAGE_READWRITE, // read/write a...
0, // maximum obje...
BUF_SIZE, // maximum obje...
szName); // name of mapp...
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not create file mapping object...
GetLastError());
return 1;
}
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to m...
FILE_MAP_ALL_ACCESS, // read/writ...
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return 1;
}
}}
Create a new file mapping object, and initialize a buffer.
#highlight(cpp){{
int i;
if(firstLoop == 1){
ZeroMemory(
(PVOID)pBuf,
(_tcslen(pBuf) * sizeof(pBuf) )
);
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
printf("close the file mapping \n");
}
firstLoop=1;
strcpy (resultReco," ");
if (seq != NULL) {
for (i=1;i<n-1;i++) {
strcat(resultReco,winfo->woutput[seq[i]]);
}
}
myprintf("%s",resultReco);
}}
This part of code allows to retrieve the recognized text ...
** SIGService process [#ed1b8137]
This is the SIGVerse service for speech recognition, its ...
*** Source code explanation [#ed1b8338]
recognition.cpp:
#highlight(cpp){{
TCHAR szName[]=TEXT("Global\\MyFileMappingObject");
}}
Initialize a TCHAR variable with the name of the mapping ...
#highlight(cpp){{
system("start .\\julius.exe -input mic -C .\\SIGVerseGra...
}}
Start the julius service.
#highlight(cpp){{
hMapFile = CreateFileMapping(
INVALID_HANDLE_VALUE, // use paging f...
NULL, // default secu...
PAGE_READWRITE, // read/write a...
0, // maximum obje...
BUF_SIZE, // maximum obje...
szName); // name of mapp...
if (hMapFile == NULL)
{
_tprintf(TEXT("Could not create file mapping object...
GetLastError());
return 1;
}
}}
Create and initialize the file mapping object.
#highlight(cpp){{
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to m...
FILE_MAP_ALL_ACCESS, // read/writ...
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
_tprintf(TEXT("Could not map view of file (%d).\n"),
GetLastError());
CloseHandle(hMapFile);
return 0.1;
}
}}
Create a buffer.
std::string send_msg =(std::string) pBuf;
this->sendMsg("robot_000",send_msg.c_str());
Send the recognized text to the controller.
#highlight(cpp){{
if (strcmp(s.c_str(),"Stop_Reco")==0)
{
Enable = false;
}
else if(strcmp(s.c_str(),"Start_Reco")==0)
{
Enable = true;
}
}}
Enable/Disable the recognition when needed.
**Compiling the project [#ed1b8438]
You first need to download the whole project from the GIT...
Go to "SIGVerseJulius\msvc" and open the VS 2008 project....
In the solution explorer window, 4 projects are listed, y...
1) libsent
2) libjulius
3) julius
4) takeIt
*Downloading the project [#ed1b8736]
To download the project from GIT repository, use the foll...
https://github.com/SIGVerse/samples/tree/master/SpeechRec...
#highlight(end)
Up:[[Tutorial]] Previous:[[Joystick service]]
#counter
Page: