Up:[[Tutorial]]　　 　　Previous：[[Samples/Communication between agents]]　　　　　Next：[[Samples/Operation on visual]]

#contents

* Speech recognition [#ed1b8136]
This section explains how to perform speech recognition in SIGVerse.

※ NII server will work properly with version 101104 or later of SIGVerse. Also, in order to perform speech recognition, you must specify the location of the dictation-kit and (make a speech recognition software) Julius during the installation of the SIGVerse. How to do it [[here>Tutorial/Installation of SIGVerse server on socio]]
** Speech recognition of voice data (wave files) [#i34d150b]
Here you will learn how to recognize the voice (wave file) of audio data samples that are prepared in advance.

*** Create controller [#z30b38e5]
Let's create a controller of the agent that receives the audio data to perform speech recognition.
 $ cd ~/sigverse-<version>/bin/NewWorld
 $ emacs soundRecog.cpp

soundRecog.cpp

#highlight(cpp){{
 #include <Controller.h>
 #include <ControllerEvent.h>
 #include <RawSound.h>
 #include <Logger.h>
 #include <Text.h>
 
 
 class AgentController : public Controller
 {
 public:
    // Declare the use of onRecvSound function to be called when it receives a voice data.
   void  onRecvSound(RecvSoundEvent &evt);
 };
 
 void AgentController::onRecvSound(RecvSoundEvent &evt)
 {
   // Get the audio data.
   RawSound * sound = evt.getRawSound(); 
  
  // Get the audio header.
   RawSoundHeader &h = sound->getHeader();
   LOG_MSG(("channel = %d", h.getChannelNum()));
   LOG_MSG(("sampling rate = %u", h.getSamplingRate()));
 
  // Get the endian.
   RawSoundEndian endian = h.getEndian();
   LOG_MSG(("endian : %s", endian == RAW_SOUND_LITTLE_ENDIAN? "little": "big"));
 
   // Get the raw WAV data.
   int datalen = sound->getDataLen();
   char * data = sound->getData();
 
   // Gets the text to speech recognition results.
   Text * text = getText(*sound);
   if (text) { 
 
     // Display the results of speech recognition.
     const char * str = text->getString();
     LOG_MSG(("text : %s", str));
 
     // Then delete the text data when you no longer need.
     delete text;
   }
 
 }
 
 extern "C"  Controller * createController ()
 {
   return new AgentController;
 }
}}
Here is an example of how to display the results of speech recognition.

*** Compile [#w1365d1b]
Modify the Makefile.
 $ emacs Makefile

Specify the soundRecog.so as the object file.

 # Specify the object file
 OBJS     = soundRecog.so

#ref(makefile_sound)


And then compile it.
 $ make

*** World file creation [#xf737d28]
 $ cd ..
 $ emacs xml/soundRecog.xml

soundRecog.xml
#highlight(xml){{
<?xml version="1.0" encoding="utf8"?>
 <world name="myworld4">
 
   <gravity x="0.0" y="-9.8" z="0.0"/>
 
 <! - Configure the Agent Robot-nii to receive voice data ->
   <instanciate class="Robot-nii.xml"> 
 
         <set-attr-value name="name" value="robot_000"/>
         <set-attr-value name="language" value="c++"/> 
 
 <! - Specify the recipient's voice data controller ->
         <set-attr-value name="implementation"
 value="./NewWorld/soundRecog.so"/>
 
 <! - Dynamics calculation is set to false ->
         <set-attr-value name="dynamics" value="false"/>
 
 <! - The location of the agent (x, y, z) ->
        <set-attr-value name="x" value="0.0"/>
        <set-attr-value name="y" value="60.0"/>
        <set-attr-value name="z" value="-60.0"/>
 
 </instanciate>
}}
You're ready to roll.

*** Start of SIGVerse and Julius [#dec687ff]
Start SIGVerse first.
 $ ./sigserver.sh -p 9001 -w soundRecog.xml &
Start Julius next. To execute the command press the Enter key.
 $ ./sigjsp.sh -p 9001
Start the shell script named Julius sigjsp.sh. It tells the simulation server that Julius has started with the -p option to specify the port number the server is running the simulation at this time.

Julius has been successfully started If you see a message like the following. If you do not start properly, please refer to the setting of Julius [[]].

 Stat: adin_tcpip: waiting connection...
 ===================================
   julius adinnet port connect ok!
 Stat: server-client: connect from 127.0.0.1
 Stat: adin_tcpip: connected
 ===================================
 ***************************************
 ***   julius successfully started   ***
 ***************************************
 *******************************************************
   connect to SimServer (localhost:9111) ok
 *******************************************************
 clerkName (socio.iir.nii.ac.jp)
 send COMM_REQUEST_PROVIDE_SERVICE ok
 waiting for connection ... (accept sock: 7)
 [SYS]  socio.iir.nii.ac.jp:7500  : service provider
 [SYS]  127.0.0.1 connected

*** Transmission of voice data from SIGViewer [#i5255de9]

When Julius has been started in SIGVerse, start the SIGViewer on the client side, and connect to the server. When the connection is complete, you will see a standing robotic agent.

To start the simulation, in the SIM_STRL_CMD, choose START in the list, then press the "send" button. Next, click the Command tab in the upper right corner. The top entry field lets you select a voice and general. Here you select the voice. Click on the robotic agent Click the button "Choose to be transmitted" then. Click the "Send Voice" button.

#ref(sound_1.jpg)

A dialog box is displayed to select the audio data to be sent, where you have installed the SIGViewer, by default

C:\Program Files\SIGViewer_<version>\SIGViewer\release

Select the sample.wav in, and sends the audio data.
The following message appears then,
I think that speech recognition results to be displayed "Please reference poverty."

 connection accepted. (sock=8)
 waiting for connection ... (accept sock: 7)
 connection accepted. (sock=8)
 
 waiting for connection ... (accept sock: 7)
 [MSG]  robot_000(127.0.0.1) channel = 0
 [MSG]  robot_000(127.0.0.1) sampling rate = 0
 [MSG]  robot_000(127.0.0.1) endian : little
 [MSG]  robot_000(127.0.0.1) text : 貧乏参考ください。

So that said "Please three individual apple" and I have heard the view (sample.wav) audio data, it has been recognized a little wrong you know.

※ Please note that there is only Tsuzu Julius will start even if you exit the SIGVerse. You can see the process running the following command.

 $ ps aux | grep <username>

** Julius setup [#j7d0f42e]
SIGVerse Communication is performed using the module mode of Julius. Also, use the audio input port over the network to provide voice and data Julius that adinnet.
Julius If the port number for communication simulation Julius and server has been duplicated with other users may not boot properly. This section describes how to configure these port numbers.

*** Julius server settings [#z00e82f3]
Copy the home directory of your own dictation kit. (Or you have downloaded from the site.)
 $ cp -r /usr/local/share/dictation-kit-v4.0　~/
Already has a configuration file that is configured for SIGVerse Julius in the directory and then copy. Modify the configuration file.

Please replace if you download the configuration file here.
#ref(fast.jconf)
 $ emacs ~/dictation-kit-v4.0/fast.jconf

Set the port number the server module mode, adinnet here. Modify the following two points.

- The port number of the server module mode

 ######################################################################
 #### Server module mode
 ######################################################################
 -Module module starting with the # server mode

　　　↓
 ######################################################################
 #### Server module mode
 ######################################################################
 -Module module boot server mode # 10511


-adinnet port number

 ######################################################################
 #### Audio input source
 ######################################################################
 ## Please select one (default: mfcfile)
 
 (Optional)
  -input adinnet

　　　↓

 ######################################################################
 #### Audio input source
 ######################################################################
 ## Please select one (default: mfcfile)
 
 (Optional)
 -input adinnet 
 -adport 10522

10511 Here, we have the module number and port number 10522 adinnet port. It is recommended that you change the number difficult to duplicate with other users.

*** Setting up the simulation server [#ba0cc5d7]
Sets the simulation server then.
Simulation server reads the configuration file on startup following Julius. Let's see.

 $ Emacs / home / <usrname> / sigverse-<version> / share / sigverse / etc / sigjsp.conf

The default is as follows.

 # ************************************************* ***********
 # Sigjsp configuration file
 # ************************************************* ***********
 
 # ================================================= =====
 # SimServer hostname and port number
 # ================================================= =====
 SERVER = localhost
 PORT = 7000
 
 
 # ================================================= =====
 # Port number that sigjsp waiting for connection
 # ================================================= =====
 SERVICE_PORT = 7500
  
 
 # ================================================= =====
 # Julius port number
 # ================================================= =====
 JULIUS_MODULE_PORT = 10500
 JULIUS_ADINNET_PORT = 5530
  
 
 # ================================================= =====
 # Julius path (not directory, path to executable)
 # ================================================= =====
 JULIUS_PATH = / usr / local / bin / julius
 
 # ================================================= =====
 # Julius config file
 #
 # [Remark]
 # When relative path is specified, actual path is
 # Calculated from current directory.
 # (Ex)
 # JULIUS_CONFIG_FILE = fast.jconf
 # $ Pwd
 # / Home / somebody
 # $ Sigjsp
 # ---> Sigjsp searches for / home / somebody / fast.jconf
 # ================================================= =====
 JULIUS_CONFIG_FILE = / usr/local/share/dictation-kit-v4.0/fast.jconf
 
 
 # ================================================= =====
 # Some wait time to finish starting julius (in msec)
 # ================================================= =====
 # DELAY_TIME_FOR_START_JULIUS = 3000

Here you can set the following.
- The port number of the simulation server (default 7000)
-Julius service provider's port number (default 7500)
-Julius port number of the module (default 10500)
-Julius adinnet port number (default 5530)

-Julius location of the executable file of
-Julius location of the configuration file

These port numbers are set to the value of the default port number is set by Julius. Change the port number that has been previously set these values.
 
 JULIUS_MODULE_PORT = 10500
 JULIUS_ADINNET_PORT = 5530

↓
 JULIUS_MODULE_PORT = 10511
 JULIUS_ADINNET_PORT = 10522

Change the number Julius difficult to duplicate the port number of the next service provider.

 SERVICE_PORT = 7500
↓
 SERVICE_PORT = 10533

Change the location of the configuration file then Julius.

 JULIUS_CONFIG_FILE = / usr/local/share/dictation-kit-v4.0/fast.jconf
↓
 JULIUS_CONFIG_FILE = / home / <username> / dictation-kit-v4.0/fast.jconf

In addition, these port numbers and configuration files can also be specified when the execution of the sigjsp.sh. Will be the value specified in the configuration file above if you do not specify anything.

         % Sigjsp.sh [-p <number>] [-sp <number>] [-mp <number>] [-ap <number>] [-e <path>] [-C <config.jconf>]

 OPTION:
-P <number>: port number of the simulation server
-Sp <number>: Julius port number of service providers
-Mp <number>: Julius module mode port number
-Ap <number>: adinnet port number
-E <path>: the location of the Julius (default: / usr / local / bin / julius)
-C <config.jconf;>: Julius location of the configuration file

Setup is now complete. Please make sure that Julius starts successfully.

If the result of speech recognition *** would have been displayed as (null) [# yc262c9c]
Please try and run the file add the following line to your configuration fast.jconf of Julius.
 -Charconv EUC-JP UTF-8

Speech recognition of voice data ** (input from the microphone) [# w8e9ffc0]
This section describes how to speech recognition in SIGVerse audio data recorded on the microphone next.

※ that the microphone is connected to the client pc is assumed.

Edit controller *** [# a43d5b9a]
Edit the controller soundRecog.cpp you just created.

 $ Cd ~ / sigverse-<version> / bin / NewWorld
 $ Emacs soundRecog.cpp

# Highlight (cpp: firstline [38]) {{
    / / Display the results of speech recognition.
    const char * str = text-> getString ();
    LOG_MSG (("text:% s", str));
}}
And add the following code under the. (Speech recognition can also be done without it)
# Highlight (cpp: firstline [41]) {{
    if (strstr "hello" (str,))
      {
        / / Get yourself
        SimObj * my = getObj (myname ());
 
        / / You raise your hands.
         my-> setJointAngle ("RARM_JOINT2", DEG2RAD (90));
      }
}}
Raise one's hand is that if the recognition result of "Hello" This was obtained.

Let's compile and run So.
 $ Make

Recognition of the recorded audio data *** [# ycb258d6]
Start the simulation server.
 $. / Sigserver.sh-p 9001-w soundRecog.xml &

Start the Julius.
 $. / Sigjsp.sh-p 9001

Connect to the server to start the SIGViewer, start the simulation, as shown above, click Select, the voice the comm tab in the upper right corner, specify the subject sent next. To start recording, click the "Rec Start" button and.
# Ref (sound_2.jpg)

Please try talking with "Hello" into the microphone. When you finish recording, click the "Rec End" button. Audio data is normally recorded at the lower left of SIGViewer "appears and captured voice save to (Capture.wav) at this time.

# Ref (sound_3.jpg)

Select the Capture.wav was recorded just click the "Send Voice" button. Has been created in the same location as the sample file is Capture.wav sample.wav of earlier.

I think if the robot raises his hand and speech recognition, "Hello". Please try to give a hand robot challenge.

# Highlight (end)
/ / ** Send voice data to other agents [# b4d2b5db]

[Operation on visual []]: Next [] exchange of messages between the [agent]: Up: [[Tutorial]] Previous
Front page   New List of pages Search Recent changes   Help   RSS of recent changes