![[PukiWiki]](image/Concept_logo.jpg "[PukiWiki]"){kind=logo}
// // Last-modified: 2012-03-28 (Wed) by Aurelian // Too old, for SIGViewer 1.0 // Up:[[Tutorial]] Previous：[[Samples/Communication between agents]] Next：[[Samples/Operation on visual]] # Contents #contents * Speech recognition [# t40318dd] [#ed1b8136] * Speech recognition [#ed1b8136] This section explains how to perform speech recognition in SIGVerse. ※ In NII server will work properly in version 101104 of SIGVerse. In addition, in order to perform speech recognition, you must specify the location of the dictation-kit and (make a speech recognition software) Julius during the installation of the SIGVerse. How to specify the [installation of> SIGVerse here [(NII server)]] Speech recognition of voice data ** (wave files) [# pc99087f] Here I would like to recognize the voice (wave file) of audio data samples that are prepared in advance. ※ NII server will work properly with version 101104 or later of SIGVerse. Also, in order to perform speech recognition, you must specify the location of the dictation-kit and of the speech recognition software Julius during the installation of the SIGVerse. How to do it [[here>Tutorial/Installation of SIGVerse server on socio]] ** Speech recognition of voice data (wave files) [#i34d150b] Here you will learn how to recognize the voice of audio data samples (wave files) that are prepared in advance. Create controller *** [# m6feace7] Create a controller of the agent receives the audio data, speech recognition is performed. $ Cd ~ / sigverse-<version> / bin / NewWorld $ Emacs soundRecog.cpp *** Create controller [#z30b38e5] Let's create a controller of the agent that receives the audio data to perform speech recognition. $ cd ~/sigverse-<version>/bin/NewWorld $ emacs soundRecog.cpp soundRecog.cpp # Highlight (cpp) {{ # Include <Controller.h> # Include <ControllerEvent.h> # Include <RawSound.h> # Include <Logger.h> # Include <Text.h> #highlight(cpp){{ #include <Controller.h> #include <ControllerEvent.h> #include <RawSound.h> #include <Logger.h> #include <Text.h> class AgentController: public Controller class AgentController : public Controller { public: // Declare the use of onRecvSound function to be called when it receives a voice data. void onRecvSound(RecvSoundEvent &evt); }; / / Declare the use of onRecvSound function to be called when it receives a voice data. void onRecvSound (RecvSoundEvent & evt); ;} void AgentController :: onRecvSound (RecvSoundEvent & evt) void AgentController::onRecvSound(RecvSoundEvent &evt) { / / Get the audio data. RawSound * sound = evt.getRawSound (); // Get the audio data. RawSound * sound = evt.getRawSound(); / / Get the audio header. RawSoundHeader & h = sound-> getHeader (); LOG_MSG (("channel =% d", h.getChannelNum ())); LOG_MSG (("sampling rate =% u", h.getSamplingRate ())); // Get the audio header. RawSoundHeader &h = sound->getHeader(); LOG_MSG(("channel = %d", h.getChannelNum())); LOG_MSG(("sampling rate = %u", h.getSamplingRate())); / / Get the endian. RawSoundEndian endian = h.getEndian (); LOG_MSG (? ("Endian:% s", endian == RAW_SOUND_LITTLE_ENDIAN "little": "big")); // Get the endian. RawSoundEndian endian = h.getEndian(); LOG_MSG(("endian : %s", endian == RAW_SOUND_LITTLE_ENDIAN? "little": "big")); / / Get the raw WAV data. int datalen = sound-> getDataLen (); char * data = sound-> getData (); // Get the raw WAV data. int datalen = sound->getDataLen(); char * data = sound->getData(); / / Gets the text to speech recognition results. Text * text = getText (* sound); if (text) { // Gets the text to speech recognition results. Text * text = getText(*sound); if (text) { / / Display the results of speech recognition. const char * str = text-> getString (); LOG_MSG (("text:% s", str)); // Display the results of speech recognition. const char * str = text->getString(); LOG_MSG(("text : %s", str)); / / Then delete the text data when you no longer need. // Then delete the text data when you no longer need. delete text; } } extern "C" Controller * createController () extern "C" Controller * createController () { return new AgentController; } }} Here is an example of how to display the results of speech recognition. Compile *** [# gdda3ca6] *** Compile [#w1365d1b] Modify the Makefile. $ Emacs Makefile $ emacs Makefile Specifies the soundRecog.so where specified in the object file. Specify the soundRecog.so as the object file. # Specify the object file OBJS = soundRecog.so OBJS = soundRecog.so # Ref (makefile_sound) #ref(makefile_sound) And then compile it. $ Make $ make File creation world *** [# l5f5ce8b] $ Cd .. $ Emacs xml / soundRecog.xml *** World file creation [#xf737d28] $ cd .. $ emacs xml/soundRecog.xml soundRecog.xml # Highlight (xml) {{ <? Xml version = "1.0" encoding = "utf8"?> #highlight(xml){{ <?xml version="1.0" encoding="utf8"?> <world name="myworld4"> <gravity x="0.0" y="-9.8" z="0.0"/> <! - Configuring the Agent Robot-nii receive voice data -> <instanciate class="Robot-nii.xml"> <!-- Configure the Agent Robot-nii to receive voice data --> <instanciate class="Robot-nii.xml"> <set-attr-value name="name" value="robot_000"/> <set-attr-value name="language" value="c++"/> <set-attr-value name="language" value="c++"/> <! - Specify the recipient's voice data controller -> <Set-attr-value name = "implementation" value = ". / NewWorld / soundRecog.so" /> <!-- Specify the recipient's voice data controller --> <set-attr-value name="implementation" value="./NewWorld/soundRecog.so"/> <! - Dynamics calculation is set to false -> <!-- Dynamics calculation is set to false --> <set-attr-value name="dynamics" value="false"/> <! - The location of the agent (x, y, z) -> <!-- The location of the agent (x, y, z) --> <set-attr-value name="x" value="0.0"/> <set-attr-value name="y" value="60.0"/> <set-attr-value name="z" value="-60.0"/> </ Instanciate> </instanciate> </world> }} You're ready to roll. Start *** SIGVerse, of Julius [# q211f39e] Start the first SIGVerse. $. / Sigserver.sh-p 9001-w soundRecog.xml & Start the next Julius. To continue with the command by pressing the Enter key. $. / Sigjsp.sh-p 9001 Start the shell script named Julius sigjsp.sh. It tells the simulation server that Julius has started with the-p option to specify the port number the server is running the simulation at this time. *** Start of SIGVerse and Julius [#dec687ff] Start SIGVerse first. $ ./sigserver.sh -p 9001 -w soundRecog.xml & Start Julius next. $ ./sigjsp.sh -p 9001 This starts the shell script named Julius sigjsp.sh. It tells the simulation server to start Julius, with the -p option specifying the port number of the server running the simulation. Julius has been successfully started If you see a message like the following. If you do not start properly, please refer to the [> # w9e5b872] setting of [Julius]. Julius has been successfully started If you see a message like the following. If you do not start properly, please refer to the [[Julius setup>#j7d0f42e]]. Stat: adin_tcpip: waiting connection ... Stat: adin_tcpip: waiting connection... =================================== julius adinnet port connect ok! Stat: server-client: connect from 127.0.0.1 Stat: adin_tcpip: connected =================================== *************************************** *** Julius successfully started *** *** julius successfully started *** *************************************** ************************************************** ***** connect to SimServer (localhost: 9111) ok ************************************************** ***** ******************************************************* connect to SimServer (localhost:9111) ok ******************************************************* clerkName (socio.iir.nii.ac.jp) send COMM_REQUEST_PROVIDE_SERVICE ok waiting for connection ... (accept sock: 7) [SYS] socio.iir.nii.ac.jp: 7500: service provider [SYS] 127.0.0.1 connected [SYS] socio.iir.nii.ac.jp:7500 : service provider [SYS] 127.0.0.1 connected Transmission of voice data from SIGViewer *** [# jac958b8] *** Transmission of voice data from SIGViewer [#i5255de9] In SIGVerse, Julius has been started, start the SIGViewer on the client side, and connect to the server. You'll see they are standing When the connection is complete robotic agent. When Julius has been started in SIGVerse, start the SIGViewer on the client side, and connect to the server. When the connection is complete, you will see a standing robotic agent. Start the simulation by pressing the, "send" button on the START condition has become SIM_STRL_CMD is, and then click the Command tab in the upper right corner. The top entry field has to be able to select a voice and general. Here you select the voice. Click on the robotic agent Click the button "Choose to be transmitted" then. Click the "Send Voice" button. To start the simulation, in the SIM_STRL_CMD, choose START in the list, then press the "send" button. Next, click the Command tab in the upper right corner. The top entry field lets you select a voice and general. Here you select the voice. Click on the robotic agent Click the button "Choose to be transmitted" then. Click the "Send Voice" button. # Ref (sound_1.jpg) #ref(sound_1.jpg) Dialog box is displayed to select the audio data to be sent, where you have installed the SIGViewer, by default A dialog box is displayed to select the audio data to be sent, where you have installed the SIGViewer, by default C: \ Program Files \ SIGViewer_ <version> \ SIGViewer \ release C:\Program Files\SIGViewer_<version>\SIGViewer\release Select the sample.wav in, and sends the audio data. The following message appears then, I think that speech recognition results to be displayed "Please reference poverty." connection accepted. (sock = 8) connection accepted. (sock=8) waiting for connection ... (accept sock: 7) connection accepted. (sock = 8) connection accepted. (sock=8) waiting for connection ... (accept sock: 7) [MSG] robot_000 (127.0.0.1) channel = 0 [MSG] robot_000 (127.0.0.1) sampling rate = 0 [MSG] robot_000 (127.0.0.1) endian: little [MSG] robot_000 (127.0.0.1) text: Please reference poverty. [MSG] robot_000(127.0.0.1) channel = 0 [MSG] robot_000(127.0.0.1) sampling rate = 0 [MSG] robot_000(127.0.0.1) endian : little [MSG] robot_000(127.0.0.1) text : 貧乏参考ください。 So that said "Please three individual apple" and I have heard the view (sample.wav) audio data, it has been recognized a little wrong you know. ※ Please note that there is only Tsuzu Julius will start even if you exit the SIGVerse. You can see the process running the following command. $ Ps aux | grep <username> $ ps aux | grep <username> Setting ** Julius [# w9e5b872] SIGVerse Communication is performed using the module mode of julius. Also, use the audio input port over the network to provide voice and data julius that adinnet. Julius If the port number for communication simulation julius and server has been duplicated with other users may not boot properly. This section describes how to configure these port numbers. ** Julius setup [#j7d0f42e] SIGVerse Communication is performed using the module mode of Julius. Also, use the audio input port over the network to provide voice and data Julius that adinnet. Julius If the port number for communication simulation Julius and server has been duplicated with other users may not boot properly. This section describes how to configure these port numbers. Julius server settings *** [# ee41288b] *** Julius server settings [#z00e82f3] Copy the home directory of your own dictation kit. (Or you have downloaded from the site.) $ Cp-r / usr/local/share/dictation-kit-v4.0 ~ / $ cp -r /usr/local/share/dictation-kit-v4.0 ~/ Already has a configuration file that is configured for SIGVerse Julius in the directory and then copy. Modify the configuration file. Please replace if you download the configuration file here. # Ref (fast.jconf) $ Emacs ~ / dictation-kit-v4.0/fast.jconf #ref(fast.jconf) $ emacs ~/dictation-kit-v4.0/fast.jconf Set the port number the server module mode, adinnet here. Modify the following two points. - The port number of the server module mode # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Server module mode # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ###################################################################### #### Server module mode ###################################################################### -Module module starting with the # server mode ↓ # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Server module mode # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # ↓ ###################################################################### #### Server module mode ###################################################################### -Module module boot server mode # 10511 -Adinnet port number -adinnet port number # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Audio input source # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Please select one (default: mfcfile) ###################################################################### #### Audio input source ###################################################################### ## Please select one (default: mfcfile) (Optional) -Input adinnet -input adinnet ↓ ↓ # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Audio input source # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # # Please select one (default: mfcfile) ###################################################################### #### Audio input source ###################################################################### ## Please select one (default: mfcfile) (Optional) -Input adinnet -Adport 10522 -input adinnet -adport 10522 10511 Here, we have the module number and port number 10522 adinnet port. It is recommended that you change the number difficult to duplicate with other users. Setting up the simulation server *** [# t8869d80] *** Setting up the simulation server [#ba0cc5d7] Sets the simulation server then. Simulation server reads the configuration file on startup following Julius. Let's see. $ Emacs / home / <usrname> / sigverse-<version> / share / sigverse / etc / sigjsp.conf $ emacs /home/<usrname>/sigverse-<version>/share/sigverse/etc/sigjsp.conf The default is as follows. # ************************************************* *********** # Sigjsp configuration file # ************************************************* *********** # ************************************************************ # sigjsp configuration file # ************************************************************ # ================================================= ===== # SimServer hostname and port number # ================================================= ===== SERVER = localhost PORT = 7000 # ====================================================== # SimServer hostname and port number # ====================================================== SERVER=localhost PORT=7000 # ================================================= ===== # Port number that sigjsp waiting for connection # ================================================= ===== SERVICE_PORT = 7500 # ====================================================== # Port number that sigjsp waiting for connection # ====================================================== SERVICE_PORT=7500 # ================================================= ===== # Julius port number # ================================================= ===== JULIUS_MODULE_PORT = 10500 JULIUS_ADINNET_PORT = 5530 # ====================================================== # julius port number # ====================================================== JULIUS_MODULE_PORT=10500 JULIUS_ADINNET_PORT=5530 # ================================================= ===== # Julius path (not directory, path to executable) # ================================================= ===== JULIUS_PATH = / usr / local / bin / julius # ====================================================== # julius path (not directory, path to executable) # ====================================================== JULIUS_PATH=/usr/local/bin/julius # ================================================= ===== # Julius config file # ====================================================== # julius config file # # [Remark] # When relative path is specified, actual path is # Calculated from current directory. # (Ex) # JULIUS_CONFIG_FILE = fast.jconf # $ Pwd # / Home / somebody # $ Sigjsp # ---> Sigjsp searches for / home / somebody / fast.jconf # ================================================= ===== JULIUS_CONFIG_FILE = / usr/local/share/dictation-kit-v4.0/fast.jconf # [remark] # when relative path is specified, actual path is # calculated from current directory. # (ex) # JULIUS_CONFIG_FILE=fast.jconf # $ pwd # /home/somebody # $ sigjsp # ---> sigjsp searches for /home/somebody/fast.jconf # ====================================================== JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/fast.jconf # ================================================= ===== # Some wait time to finish starting julius (in msec) # ================================================= ===== # DELAY_TIME_FOR_START_JULIUS = 3000 # ====================================================== # some wait time to finish starting julius (in msec) # ====================================================== # DELAY_TIME_FOR_START_JULIUS=3000 Here you can set the following. - The port number of the simulation server (default 7000) -Julius service provider's port number (default 7500) -Julius port number of the module (default 10500) -Julius adinnet port number (default 5530) -Julius location of the executable file of -Julius location of the configuration file These port numbers are set to the value of the default port number is set by Julius. Change the port number that has been previously set these values. JULIUS_MODULE_PORT = 10500 JULIUS_ADINNET_PORT = 5530 JULIUS_MODULE_PORT=10500 JULIUS_ADINNET_PORT=5530 ↓ JULIUS_MODULE_PORT = 10511 JULIUS_ADINNET_PORT = 10522 ↓ JULIUS_MODULE_PORT=10511 JULIUS_ADINNET_PORT=10522 Change the number Julius difficult to duplicate the port number of the next service provider. SERVICE_PORT = 7500 ↓ SERVICE_PORT = 10533 SERVICE_PORT=7500 ↓ SERVICE_PORT=10533 Change the location of the configuration file then Julius. JULIUS_CONFIG_FILE = / usr/local/share/dictation-kit-v4.0/fast.jconf ↓ JULIUS_CONFIG_FILE = / home / <username> / dictation-kit-v4.0/fast.jconf JULIUS_CONFIG_FILE=/usr/local/share/dictation-kit-v4.0/fast.jconf ↓ JULIUS_CONFIG_FILE=/home/<username>/dictation-kit-v4.0/fast.jconf In addition, these port numbers and configuration files can also be specified when the execution of the sigjsp.sh. Will be the value specified in the configuration file above if you do not specify anything. % Sigjsp.sh [-p <number>] [-sp <number>] [-mp <number>] [-ap <number>] [-e <path>] [-C <config.jconf>] % sigjsp.sh [-p <number>] [-sp <number>] [-mp <number>] [-ap <number>] [-e <path>] [-C <config.jconf>] OPTION: -P <number>: port number of the simulation server -Sp <number>: Julius port number of service providers -Mp <number>: Julius module mode port number -Ap <number>: adinnet port number -E <path>: the location of the Julius (default: / usr / local / bin / julius) -C <config.jconf;>: Julius location of the configuration file OPTION : -p <number> : port number of the simulation server -sp <number> : Julius port number of service providers -mp <number> : Julius module mode port number -ap <number> : adinnet port number -e <path> : the location of the Julius (default: / usr / local / bin / julius) -C <config.jconf;> : Julius location of the configuration file Setup is now complete. Please make sure that Julius starts successfully. If the result of speech recognition *** would have been displayed as (null) [# yc262c9c] *** If the result of speech recognition would have been displayed as (null) [#h508ced3] Please try and run the file add the following line to your configuration fast.jconf of Julius. -Charconv EUC-JP UTF-8 -charconv EUC-JP UTF-8 Speech recognition of voice data ** (input from the microphone) [# w8e9ffc0] ** Speech recognition of voice data (input from the microphone) [#s23c7bf4] This section describes how to speech recognition in SIGVerse audio data recorded on the microphone next. ※ that the microphone is connected to the client pc is assumed. ※ It is assumed that the microphone is connected to the client pc. Edit controller *** [# a43d5b9a] *** Edit controller [#c2f898cf] Edit the controller soundRecog.cpp you just created. $ Cd ~ / sigverse-<version> / bin / NewWorld $ Emacs soundRecog.cpp $ cd ~/sigverse-<version>/bin/NewWorld $ emacs soundRecog.cpp # Highlight (cpp: firstline [38]) {{ / / Display the results of speech recognition. const char * str = text-> getString (); LOG_MSG (("text:% s", str)); #highlight(cpp:firstline[38]){{ // Display the results of speech recognition. const char * str = text->getString(); LOG_MSG(("text : %s", str)); }} And add the following code under the. (Speech recognition can also be done without it) # Highlight (cpp: firstline [41]) {{ if (strstr "hello" (str,)) #highlight(cpp:firstline[41]){{ if(strstr(str, "こんにちは")) { / / Get yourself SimObj * my = getObj (myname ()); // Get yourself SimObj *my = getObj(myname()); / / You raise your hands. my-> setJointAngle ("RARM_JOINT2", DEG2RAD (90)); // You raise your hands. my->setJointAngle("RARM_JOINT2", DEG2RAD(90)); } }} Raise one's hand is that if the recognition result of "Hello" This was obtained. Let's compile and run So. $ Make $ make Recognition of the recorded audio data *** [# ycb258d6] *** Recognition of the recorded audio data [#jfa78904] Start the simulation server. $. / Sigserver.sh-p 9001-w soundRecog.xml & $ ./sigserver.sh -p 9001 -w soundRecog.xml & Start the Julius. $. / Sigjsp.sh-p 9001 Start Julius. $ ./sigjsp.sh -p 9001 Connect to the server to start the SIGViewer, start the simulation, as shown above, click Select, the voice the comm tab in the upper right corner, specify the subject sent next. To start recording, click the "Rec Start" button and. # Ref (sound_2.jpg) #ref(sound_2.jpg) Please try talking with "Hello" into the microphone. When you finish recording, click the "Rec End" button. Audio data is normally recorded at the lower left of SIGViewer "appears and captured voice save to (Capture.wav) at this time. # Ref (sound_3.jpg) #ref(sound_3.jpg) Select the Capture.wav was recorded just click the "Send Voice" button. Has been created in the same location as the sample file is Capture.wav sample.wav of earlier. I think if the robot raises his hand and speech recognition, "Hello". Please try to give a hand robot challenge. # Highlight (end) / / ** Send voice data to other agents [# b4d2b5db] #highlight(end) // ** Send voice data to other agents [Operation on visual []]: Next [] exchange of messages between the [agent]: Up: [[Tutorial]] Previous Up:[[Tutorial]] Previous：[[Samples/Communication between agents]] Next：[[Samples/Operation on visual]]
Site admin: inamura
PukiWiki 1.5.0 Copyright © 2001-2006 PukiWiki Developers Team. License is GPL.