old/Speech recognition
[
Front page
] [
New
|
List of pages
|
Search
|
Recent changes
|
Help
]
Start:
//
// Last-modified: 2012-03-28 (Wed) by Aurelian
// Too old, for SIGViewer 1.0
//
Up:[[Tutorial]] Previous:[[Samples/Communication...
#contents
* Speech recognition [#ed1b8136]
This section explains how to perform speech recognition i...
※ NII server will work properly with version 101104 or la...
** Speech recognition of voice data (wave files) [#i34d15...
Here you will learn how to recognize the voice of audio d...
*** Create controller [#z30b38e5]
Let's create a controller of the agent that receives the ...
$ cd ~/sigverse-<version>/bin/NewWorld
$ emacs soundRecog.cpp
soundRecog.cpp
#highlight(cpp){{
#include <Controller.h>
#include <ControllerEvent.h>
#include <RawSound.h>
#include <Logger.h>
#include <Text.h>
class AgentController : public Controller
{
public:
// Declare the use of onRecvSound function to be call...
void onRecvSound(RecvSoundEvent &evt);
};
void AgentController::onRecvSound(RecvSoundEvent &evt)
{
// Get the audio data.
RawSound * sound = evt.getRawSound();
// Get the audio header.
RawSoundHeader &h = sound->getHeader();
LOG_MSG(("channel = %d", h.getChannelNum()));
LOG_MSG(("sampling rate = %u", h.getSamplingRate()));
// Get the endian.
RawSoundEndian endian = h.getEndian();
LOG_MSG(("endian : %s", endian == RAW_SOUND_LITTLE_END...
// Get the raw WAV data.
int datalen = sound->getDataLen();
char * data = sound->getData();
// Gets the text to speech recognition results.
Text * text = getText(*sound);
if (text) {
// Display the results of speech recognition.
const char * str = text->getString();
LOG_MSG(("text : %s", str));
// Then delete the text data when you no longer need.
delete text;
}
}
extern "C" Controller * createController ()
{
return new AgentController;
}
}}
Here is an example of how to display the results of speec...
*** Compile [#w1365d1b]
Modify the Makefile.
$ emacs Makefile
Specify the soundRecog.so as the object file.
# Specify the object file
OBJS = soundRecog.so
#ref(makefile_sound)
And then compile it.
$ make
*** World file creation [#xf737d28]
$ cd ..
$ emacs xml/soundRecog.xml
soundRecog.xml
#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
<world name="myworld4">
<gravity x="0.0" y="-9.8" z="0.0"/>
<!-- Configure the Agent Robot-nii to receive voice data...
<instanciate class="Robot-nii.xml">
<set-attr-value name="name" value="robot_000"/>
<set-attr-value name="language" value="c++"/>
<!-- Specify the recipient's voice data controller -->
<set-attr-value name="implementation"
value="./NewWorld/soundRecog.so"/>
<!-- Dynamics calculation is set to false -->
<set-attr-value name="dynamics" value="false"/>
<!-- The location of the agent (x, y, z) -->
<set-attr-value name="x" value="0.0"/>
<set-attr-value name="y" value="60.0"/>
<set-attr-value name="z" value="-60.0"/>
</instanciate>
</world>
}}
You're ready to roll.
*** Start of SIGVerse and Julius [#dec687ff]
Start SIGVerse first.
$ ./sigserver.sh -p 9001 -w soundRecog.xml &
Start Julius next.
$ ./sigjsp.sh -p 9001
This starts the shell script named Julius sigjsp.sh. It t...
Julius has been successfully started If you see a message...
Stat: adin_tcpip: waiting connection...
===================================
julius adinnet port connect ok!
Stat: server-client: connect from 127.0.0.1
Stat: adin_tcpip: connected
===================================
***************************************
*** julius successfully started ***
***************************************
*******************************************************
connect to SimServer (localhost:9111) ok
*******************************************************
clerkName (socio.iir.nii.ac.jp)
send COMM_REQUEST_PROVIDE_SERVICE ok
waiting for connection ... (accept sock: 7)
[SYS] socio.iir.nii.ac.jp:7500 : service provider
[SYS] 127.0.0.1 connected
*** Transmission of voice data from SIGViewer [#i5255de9]
When Julius has been started in SIGVerse, start the SIGVi...
To start the simulation, in the SIM_STRL_CMD, choose STAR...
#ref(sound_1.jpg)
A dialog box is displayed to select the audio data to be ...
C:\Program Files\SIGViewer_<version>\SIGViewer\release
Select the sample.wav in, and sends the audio data.
The following message appears then,
I think that speech recognition results to be displayed "...
connection accepted. (sock=8)
waiting for connection ... (accept sock: 7)
connection accepted. (sock=8)
waiting for connection ... (accept sock: 7)
[MSG] robot_000(127.0.0.1) channel = 0
[MSG] robot_000(127.0.0.1) sampling rate = 0
[MSG] robot_000(127.0.0.1) endian : little
[MSG] robot_000(127.0.0.1) text : 貧乏参考ください。
So that said "Please three individual apple" and I have h...
※ Please note that there is only Tsuzu Julius will start ...
$ ps aux | grep <username>
** Julius setup [#j7d0f42e]
SIGVerse Communication is performed using the module mode...
Julius If the port number for communication simulation Ju...
*** Julius server settings [#z00e82f3]
Copy the home directory of your own dictation kit. (Or yo...
$ cp -r /usr/local/share/dictation-kit-v4.0 ~/
Already has a configuration file that is configured for S...
Please replace if you download the configuration file here.
#ref(fast.jconf)
$ emacs ~/dictation-kit-v4.0/fast.jconf
Set the port number the server module mode, adinnet here....
- The port number of the server module mode
########################################################...
#### Server module mode
########################################################...
-Module module starting with the # server mode
↓
########################################################...
#### Server module mode
########################################################...
-Module module boot server mode # 10511
-adinnet port number
########################################################...
#### Audio input source
########################################################...
## Please select one (default: mfcfile)
(Optional)
-input adinnet
↓
########################################################...
#### Audio input source
########################################################...
## Please select one (default: mfcfile)
(Optional)
-input adinnet
-adport 10522
10511 Here, we have the module number and port number 105...
*** Setting up the simulation server [#ba0cc5d7]
Sets the simulation server then.
Simulation server reads the configuration file on startup...
$ emacs /home/<usrname>/sigverse-<version>/share/sigvers...
The default is as follows.
# ******************************************************...
# sigjsp configuration file
# ******************************************************...
# ======================================================
# SimServer hostname and port number
# ======================================================
SERVER=localhost
PORT=7000
# ======================================================
# Port number that sigjsp waiting for connection
# ======================================================
SERVICE_PORT=7500
# ======================================================
# julius port number
# ======================================================
JULIUS_MODULE_PORT=10500
JULIUS_ADINNET_PORT=5530
# ======================================================
# julius path (not directory, path to executable)
# ======================================================
JULIUS_PATH=/usr/local/bin/julius
# ======================================================
# julius config file
#
# [remark]
# when relative path is specified, actual path is
# calculated from current directory.
# (ex)
# JULIUS_CONFIG_FILE=fast.jconf
# $ pwd
# /home/somebody
# $ sigjsp
# ---> sigjsp searches for /home/somebody/fast.jconf
# ======================================================
JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/f...
# ======================================================
# some wait time to finish starting julius (in msec)
# ======================================================
# DELAY_TIME_FOR_START_JULIUS=3000
Here you can set the following.
- The port number of the simulation server (default 7000)
-Julius service provider's port number (default 7500)
-Julius port number of the module (default 10500)
-Julius adinnet port number (default 5530)
-Julius location of the executable file of
-Julius location of the configuration file
These port numbers are set to the value of the default po...
JULIUS_MODULE_PORT=10500
JULIUS_ADINNET_PORT=5530
↓
JULIUS_MODULE_PORT=10511
JULIUS_ADINNET_PORT=10522
Change the number Julius difficult to duplicate the port ...
SERVICE_PORT=7500
↓
SERVICE_PORT=10533
Change the location of the configuration file then Julius.
JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/f...
↓
JULIUS_CONFIG_FILE=/home/<username>/dictation-kit-v4.0/f...
In addition, these port numbers and configuration files c...
% sigjsp.sh [-p <number>] [-sp <number>] [-mp <n...
OPTION :
-p <number> : port number of the simulation server
-sp <number> : Julius port number of service providers
-mp <number> : Julius module mode port number
-ap <number> : adinnet port number
-e <path> : the location of the Julius (default: / us...
-C <config.jconf;> : Julius location of the configuratio...
Setup is now complete. Please make sure that Julius start...
*** If the result of speech recognition would have been d...
Please try and run the file add the following line to you...
-charconv EUC-JP UTF-8
** Speech recognition of voice data (input from the micro...
This section describes how to speech recognition in SIGVe...
※ It is assumed that the microphone is connected to the c...
*** Edit controller [#c2f898cf]
Edit the controller soundRecog.cpp you just created.
$ cd ~/sigverse-<version>/bin/NewWorld
$ emacs soundRecog.cpp
#highlight(cpp:firstline[38]){{
// Display the results of speech recognition.
const char * str = text->getString();
LOG_MSG(("text : %s", str));
}}
And add the following code under the. (Speech recognition...
#highlight(cpp:firstline[41]){{
if(strstr(str, "こんにちは"))
{
// Get yourself
SimObj *my = getObj(myname());
// You raise your hands.
my->setJointAngle("RARM_JOINT2", DEG2RAD(90));
}
}}
Raise one's hand is that if the recognition result of "He...
Let's compile and run So.
$ make
*** Recognition of the recorded audio data [#jfa78904]
Start the simulation server.
$ ./sigserver.sh -p 9001 -w soundRecog.xml &
Start Julius.
$ ./sigjsp.sh -p 9001
Connect to the server to start the SIGViewer, start the s...
#ref(sound_2.jpg)
Please try talking with "Hello" into the microphone. When...
#ref(sound_3.jpg)
Select the Capture.wav was recorded just click the "Send ...
I think if the robot raises his hand and speech recogniti...
#highlight(end)
// ** Send voice data to other agents
Up:[[Tutorial]] Previous:[[Samples/Communication...
End:
//
// Last-modified: 2012-03-28 (Wed) by Aurelian
// Too old, for SIGViewer 1.0
//
Up:[[Tutorial]] Previous:[[Samples/Communication...
#contents
* Speech recognition [#ed1b8136]
This section explains how to perform speech recognition i...
※ NII server will work properly with version 101104 or la...
** Speech recognition of voice data (wave files) [#i34d15...
Here you will learn how to recognize the voice of audio d...
*** Create controller [#z30b38e5]
Let's create a controller of the agent that receives the ...
$ cd ~/sigverse-<version>/bin/NewWorld
$ emacs soundRecog.cpp
soundRecog.cpp
#highlight(cpp){{
#include <Controller.h>
#include <ControllerEvent.h>
#include <RawSound.h>
#include <Logger.h>
#include <Text.h>
class AgentController : public Controller
{
public:
// Declare the use of onRecvSound function to be call...
void onRecvSound(RecvSoundEvent &evt);
};
void AgentController::onRecvSound(RecvSoundEvent &evt)
{
// Get the audio data.
RawSound * sound = evt.getRawSound();
// Get the audio header.
RawSoundHeader &h = sound->getHeader();
LOG_MSG(("channel = %d", h.getChannelNum()));
LOG_MSG(("sampling rate = %u", h.getSamplingRate()));
// Get the endian.
RawSoundEndian endian = h.getEndian();
LOG_MSG(("endian : %s", endian == RAW_SOUND_LITTLE_END...
// Get the raw WAV data.
int datalen = sound->getDataLen();
char * data = sound->getData();
// Gets the text to speech recognition results.
Text * text = getText(*sound);
if (text) {
// Display the results of speech recognition.
const char * str = text->getString();
LOG_MSG(("text : %s", str));
// Then delete the text data when you no longer need.
delete text;
}
}
extern "C" Controller * createController ()
{
return new AgentController;
}
}}
Here is an example of how to display the results of speec...
*** Compile [#w1365d1b]
Modify the Makefile.
$ emacs Makefile
Specify the soundRecog.so as the object file.
# Specify the object file
OBJS = soundRecog.so
#ref(makefile_sound)
And then compile it.
$ make
*** World file creation [#xf737d28]
$ cd ..
$ emacs xml/soundRecog.xml
soundRecog.xml
#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
<world name="myworld4">
<gravity x="0.0" y="-9.8" z="0.0"/>
<!-- Configure the Agent Robot-nii to receive voice data...
<instanciate class="Robot-nii.xml">
<set-attr-value name="name" value="robot_000"/>
<set-attr-value name="language" value="c++"/>
<!-- Specify the recipient's voice data controller -->
<set-attr-value name="implementation"
value="./NewWorld/soundRecog.so"/>
<!-- Dynamics calculation is set to false -->
<set-attr-value name="dynamics" value="false"/>
<!-- The location of the agent (x, y, z) -->
<set-attr-value name="x" value="0.0"/>
<set-attr-value name="y" value="60.0"/>
<set-attr-value name="z" value="-60.0"/>
</instanciate>
</world>
}}
You're ready to roll.
*** Start of SIGVerse and Julius [#dec687ff]
Start SIGVerse first.
$ ./sigserver.sh -p 9001 -w soundRecog.xml &
Start Julius next.
$ ./sigjsp.sh -p 9001
This starts the shell script named Julius sigjsp.sh. It t...
Julius has been successfully started If you see a message...
Stat: adin_tcpip: waiting connection...
===================================
julius adinnet port connect ok!
Stat: server-client: connect from 127.0.0.1
Stat: adin_tcpip: connected
===================================
***************************************
*** julius successfully started ***
***************************************
*******************************************************
connect to SimServer (localhost:9111) ok
*******************************************************
clerkName (socio.iir.nii.ac.jp)
send COMM_REQUEST_PROVIDE_SERVICE ok
waiting for connection ... (accept sock: 7)
[SYS] socio.iir.nii.ac.jp:7500 : service provider
[SYS] 127.0.0.1 connected
*** Transmission of voice data from SIGViewer [#i5255de9]
When Julius has been started in SIGVerse, start the SIGVi...
To start the simulation, in the SIM_STRL_CMD, choose STAR...
#ref(sound_1.jpg)
A dialog box is displayed to select the audio data to be ...
C:\Program Files\SIGViewer_<version>\SIGViewer\release
Select the sample.wav in, and sends the audio data.
The following message appears then,
I think that speech recognition results to be displayed "...
connection accepted. (sock=8)
waiting for connection ... (accept sock: 7)
connection accepted. (sock=8)
waiting for connection ... (accept sock: 7)
[MSG] robot_000(127.0.0.1) channel = 0
[MSG] robot_000(127.0.0.1) sampling rate = 0
[MSG] robot_000(127.0.0.1) endian : little
[MSG] robot_000(127.0.0.1) text : 貧乏参考ください。
So that said "Please three individual apple" and I have h...
※ Please note that there is only Tsuzu Julius will start ...
$ ps aux | grep <username>
** Julius setup [#j7d0f42e]
SIGVerse Communication is performed using the module mode...
Julius If the port number for communication simulation Ju...
*** Julius server settings [#z00e82f3]
Copy the home directory of your own dictation kit. (Or yo...
$ cp -r /usr/local/share/dictation-kit-v4.0 ~/
Already has a configuration file that is configured for S...
Please replace if you download the configuration file here.
#ref(fast.jconf)
$ emacs ~/dictation-kit-v4.0/fast.jconf
Set the port number the server module mode, adinnet here....
- The port number of the server module mode
########################################################...
#### Server module mode
########################################################...
-Module module starting with the # server mode
↓
########################################################...
#### Server module mode
########################################################...
-Module module boot server mode # 10511
-adinnet port number
########################################################...
#### Audio input source
########################################################...
## Please select one (default: mfcfile)
(Optional)
-input adinnet
↓
########################################################...
#### Audio input source
########################################################...
## Please select one (default: mfcfile)
(Optional)
-input adinnet
-adport 10522
10511 Here, we have the module number and port number 105...
*** Setting up the simulation server [#ba0cc5d7]
Sets the simulation server then.
Simulation server reads the configuration file on startup...
$ emacs /home/<usrname>/sigverse-<version>/share/sigvers...
The default is as follows.
# ******************************************************...
# sigjsp configuration file
# ******************************************************...
# ======================================================
# SimServer hostname and port number
# ======================================================
SERVER=localhost
PORT=7000
# ======================================================
# Port number that sigjsp waiting for connection
# ======================================================
SERVICE_PORT=7500
# ======================================================
# julius port number
# ======================================================
JULIUS_MODULE_PORT=10500
JULIUS_ADINNET_PORT=5530
# ======================================================
# julius path (not directory, path to executable)
# ======================================================
JULIUS_PATH=/usr/local/bin/julius
# ======================================================
# julius config file
#
# [remark]
# when relative path is specified, actual path is
# calculated from current directory.
# (ex)
# JULIUS_CONFIG_FILE=fast.jconf
# $ pwd
# /home/somebody
# $ sigjsp
# ---> sigjsp searches for /home/somebody/fast.jconf
# ======================================================
JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/f...
# ======================================================
# some wait time to finish starting julius (in msec)
# ======================================================
# DELAY_TIME_FOR_START_JULIUS=3000
Here you can set the following.
- The port number of the simulation server (default 7000)
-Julius service provider's port number (default 7500)
-Julius port number of the module (default 10500)
-Julius adinnet port number (default 5530)
-Julius location of the executable file of
-Julius location of the configuration file
These port numbers are set to the value of the default po...
JULIUS_MODULE_PORT=10500
JULIUS_ADINNET_PORT=5530
↓
JULIUS_MODULE_PORT=10511
JULIUS_ADINNET_PORT=10522
Change the number Julius difficult to duplicate the port ...
SERVICE_PORT=7500
↓
SERVICE_PORT=10533
Change the location of the configuration file then Julius.
JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/f...
↓
JULIUS_CONFIG_FILE=/home/<username>/dictation-kit-v4.0/f...
In addition, these port numbers and configuration files c...
% sigjsp.sh [-p <number>] [-sp <number>] [-mp <n...
OPTION :
-p <number> : port number of the simulation server
-sp <number> : Julius port number of service providers
-mp <number> : Julius module mode port number
-ap <number> : adinnet port number
-e <path> : the location of the Julius (default: / us...
-C <config.jconf;> : Julius location of the configuratio...
Setup is now complete. Please make sure that Julius start...
*** If the result of speech recognition would have been d...
Please try and run the file add the following line to you...
-charconv EUC-JP UTF-8
** Speech recognition of voice data (input from the micro...
This section describes how to speech recognition in SIGVe...
※ It is assumed that the microphone is connected to the c...
*** Edit controller [#c2f898cf]
Edit the controller soundRecog.cpp you just created.
$ cd ~/sigverse-<version>/bin/NewWorld
$ emacs soundRecog.cpp
#highlight(cpp:firstline[38]){{
// Display the results of speech recognition.
const char * str = text->getString();
LOG_MSG(("text : %s", str));
}}
And add the following code under the. (Speech recognition...
#highlight(cpp:firstline[41]){{
if(strstr(str, "こんにちは"))
{
// Get yourself
SimObj *my = getObj(myname());
// You raise your hands.
my->setJointAngle("RARM_JOINT2", DEG2RAD(90));
}
}}
Raise one's hand is that if the recognition result of "He...
Let's compile and run So.
$ make
*** Recognition of the recorded audio data [#jfa78904]
Start the simulation server.
$ ./sigserver.sh -p 9001 -w soundRecog.xml &
Start Julius.
$ ./sigjsp.sh -p 9001
Connect to the server to start the SIGViewer, start the s...
#ref(sound_2.jpg)
Please try talking with "Hello" into the microphone. When...
#ref(sound_3.jpg)
Select the Capture.wav was recorded just click the "Send ...
I think if the robot raises his hand and speech recogniti...
#highlight(end)
// ** Send voice data to other agents
Up:[[Tutorial]] Previous:[[Samples/Communication...
Page: