![[PukiWiki]](image/Concept_logo.jpg "[PukiWiki]"){kind=logo}
[[Tutorial using ROS for ver.3]] #contents * Overview [#me21a0b1] A human and a robot log in to the same virtual space. The human can give a cleanup instruction to the robot by pointing. The human logs in to the virtual space using Windows + VR device (Oculus Rift).~ The robot logs in to the virtual space using Windows + Ubuntu (ROS: Robot Operating System). Humans and robots (ROS) are connected via the Internet and can communicate in the same virtual space. #ref(CleanupUsingCloudAndVR_Overview.png) * Prerequisites [#r10a7b3e] ** Common prerequisites [#ofa841b3] Using networks is a common prerequisite for humans and robots. [[Photon Realtime>https://www.photonengine.com/en-US/Realtime]] is used for the cloud, and the port numbers listed on the following page are used when connecting to the Internet. ~ Check whether the port is available beforehand. ~ https://doc.photonengine.com/en-us/realtime/current/connection-and-authentication/tcp-and-udp-port-numbers If the port numbers are not available, the cloud cannot be used, but it is possible to replace Photon Server on the local network. ~ Photon Server is only compatible with Windows. ~ If the port is not available, the cloud cannot be used, but you can use the local network Photon Server instead. ~ Photon Server can only be installed on Windows. ~ https://doc.photonengine.com/en-us/server/current/operations/requirements The environment constructions when using the cloud and when using the local server are explained in [[Environment construction>#Environments]]. ** Prerequisites for Human (VR) [#s38cc158] -Windows environment ~ In addition to [[basic requirements for using SIGVerse assets>Tutorial for ver.3#SystemRequirements]], [[requirements for using Oculus Rift>https://support.oculus.com/248749509016567/]] is also required. -Oculus Rift ~ Please use Oculus Rift CV1 or Oculus Rift S. ~ Please install [[Oculus Software>https://www.oculus.com/setup/]] on Windows and prepare a basic environment to use Oculus Rift. ** Prerequisites for Robot (ROS) [#y682edd3] -Windows environment and Ubuntu environment ~ See [[here>Tutorial using ROS for ver.3#SystemRequirements]] for details. * Environment Construction [#Environments] The procedure on the human (VR) side and the procedure on the robot (ROS) side are the same. When using the cloud, follow [[4a.Build Cloud environment>#PhotonCloud]] procedure.~ When using a local server, follow [[4b. Build Local server environment>#PhotonServer]] procedure. ** 1. Create a Photon account [#CreatePhotonAccount] ** 1. Create Photon account [#CreatePhotonAccount] First you need to create a Photon account. ~ Create it according to the following procedure. + Go to [[Photon official site>https://www.photonengine.com/]]. + Create a Photon account by signing in at the top right of the page. ** 2. Import Oculus Utilities [#k0ec0826] You need to import Oculus Utilities into your Unity project. + Download Oculus Utilities for Unity ver.1.28.0 from the link below. ~ https://developer.oculus.com/downloads/package/oculus-utilities-for-unity-5/1.28.0/ + Unzip the downloaded zip file. + Open the SIGVerse project in Unity. + Click [Assets]-[Import Package]-[Custom Package ...] in Unity menu. + Select "OculusUtilities.unitypackage" in the extracted folder. + Click [Import] button. + Click [Yes] button on the "Update Oculus Utilities Plugin" window. + Click [Restart] button on the "Restart Unity" window. ** 3. Using Oculus and PUN libraries [#y0decaa9] + Click [SIGVerse]-[SIGVerse Settings] to open the SIGVerse settings. + Check [Use Oculus] and [Use PUN] in [Define symbols Settings]. #ref(SIGVerseSettingsOculusPUN.png) ** 4a. Build Cloud environment [#PhotonCloud] The free plan of Photon Cloud can connect up to 20 people at the same time. See the next page for details. ~ https://www.photonengine.com/en-US/Realtime/pricing To connect to the cloud, use [[PUN (Photon Unity Networking)>https://www.photonengine.com/en-US/PUN]]. ~ The PUN environment construction procedure is as follows: + Sign in to [[Photon official site>https://www.photonengine.com/]]. + Get application ID. ++ Click [CREATE A NEW APP] button on the dashboard after signing in. ++ Set [Photon Type] to "Photon PUN" and set the application name. (For example, the application name is "CleanupVR") ++ Click [CREATE] button. ++ Check that the PUN app is added to the dashboard app list, and write down the "App ID". ~ (The App ID is 36 digits. Click to display all digits.) #ref(PhotonAppId.png) + Import PUN2 assets into Unity project. ++ Open the SIGVerse project in Unity. ++ Access [[PUN2 page>https://assetstore.unity.com/packages/tools/network/pun-2-free-119922]] of Unity Asset Store. ++ Sign in to your Unity account from the top right of the page. ++ When you click the [Open in Unity] button, PUN2 is displayed on the [Asset Store] tab of Unity. ++ Uncheck some libraries.~ Please uncheck ''"Photon/PhotonLibs/WebSocket/websocket-sharp.dll".'' Since websocket-sharp.dll is already included in the asset, it will conflict. #ref(PUN2ExcludeWebsocket-sharp.png) ++ Click [Import] button.~ ++ [PUN Wizzard] will be opened. Enter the above "AppID" in the "AppId" field and press the [Setup Project] button. #ref(PunWizardCloud.png) The following [PhotonServerSettings] window opens. #ref(PhotonServerSettingsForCloud.png) (This window can also be opened from [Window]-[Photon Unity Networking]-[PUN Wizzard]-[Locate PhotonServerSettings]. AppId is set to "App Id Realtime".) + Click [Assets]-[Reimport All] in the Unity menu. + Click [Reimport] button to rebuild. ** 4b. Build Local server environment [#PhotonServer] You can also use Photon Server on a local network instead of the cloud. + Sign in to [[Photon official site>https://www.photonengine.com/]]. + Download a license file. ++ Click [TRY FREE LICENSE] in [[Photon Server page>https://www.photonengine.com/en-US/Server]]. ++ Click [DOWNLOAD FREE] to get a license file. ~ The license file is named "E-mail address+Photon-vX.free.100-ccu.license". + Install Photon Server. ++ Access [[Photon Server page>https://www.photonengine.com/en-US/sdks#server-sdkserverserver]]. ++ Click the version number to the right of "Download SDK" to download the executable file. ++ Run the downloaded file and extract it to the directory you want to install. ++ Open "Photon-OnPremise-Server-SDK_vxxxxxxx\deploy\bin_Win64" in the expanded folder in Explorer. ++ Place the license file. For example, the following figure. #ref(PhotonServerFolder.png) + Start Photon Server. (''After restarting Windows, Photon Server must also be started.'') &Aname(LaunchPhotonServer); ++ Double-click PhotonControl.exe in the bin_Win64 folder to start the application. ~ (App icon appears on the Windows taskbar) ++ Click the PhotonControl icon on the Windows taskbar, click [Game Server IP Config], and then select the appropriate server IP address. #ref(PhotonServerIP.png) ++ Click [LoadBalancing (MyCloud)]-[Start as application] to start Photon Server. ++ Click [LoadBalancing (MyCloud)]-[Run TestClient (LoadBalancing] to test the server. + Import PUN2 assets into Unity project. ++ Open the SIGVerse project in Unity. ++ Access [[PUN2 page>https://assetstore.unity.com/packages/tools/network/pun-2-free-119922]] of Unity Asset Store. ++ Sign in to your Unity account from the top right of the page. ++ When you click the [Open in Unity] button, PUN2 is displayed on the [Asset Store] tab of Unity. ++ Uncheck some libraries.~ Please uncheck ''"Photon/PhotonLibs/WebSocket/websocket-sharp.dll".'' Since websocket-sharp.dll is already included in the asset, it will conflict. #ref(PUN2ExcludeWebsocket-sharp.png) ++ Click [Import] button.~ ++ [PUN Wizzard] will be opened, so press the [Skip] button. #ref(PunWizardLocalServer.png) ++ [PhotonServerSettings] will be opened. Make the following settings. --- Uncheck [Use Name Server] --- Enter the IP address of Photon Server in [Server] --- Enter "5055" in [Port]. #ref(PhotonServerSettingsForLocalServer.png) (This window can also be opened from [Window]-[Photon Unity Networking]-[PUN Wizzard]-[Locate PhotonServerSettings].) + Click [Assets]-[Reimport All] in the Unity menu. + Click [Reimport] button to rebuild. * Execution Procedure[#s9ea839c] If you want to run as a human (VR), please follow the [[this procedure>#human_execution]]. If you want to run as a human (VR), please follow [[here>#human_execution]]. If you want to run as a robot (ROS), please follow the [[this procedure>#robot_execution]]. If you want to run as a robot (ROS), please follow [[here>#robot_execution]]. It doesn't matter which one you run first, but if you want to use a local server, please [[Launch Photon Server>#LaunchPhotonServer]] first. * Human (VR) Execution procedure [#human_execution] ** Startup Procedure [#vafd19d3] '' * If you want to use a local server, please [[Launch Photon Server>#LaunchPhotonServer]] first. '' *** Windows startup procedure [#t197903c] + Make sure that Oculus Rift is connected and ready for use. + Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupVR(.unity)] scene with reference to [[here>Tutorial using ROS for ver.3#open_scene]]. + Click the [Human] button to log in as a human. + If the robot is not logged in, wait for the robot to log in. Windows (Login Screen) #ref(HSRCleanupVRLogin.png) ** Run [#v04386d7] Operate the human avatar and give cleanup instructions to the robot. The basic flow is as follows. + Send the message "Pick it up!" to the robot while pointing at the target object. + Send the message "Clean up!" to the robot while pointing at the destination. The operation method of Oculus Touch is as follows. -Pulling the middle finger trigger puts the avatar's hand in the pointing posture. -When the A or X button is pressed, the message "Pick it up!" is sent to the robot. -When the B or Y button is pressed, the message "Clean up!" is sent to the robot. -When you tilt the left stick, the avatar moves in the tilted direction. -When the right stick is pressed, the message "Good!" is sent to the robot. -When the left stick is pressed, the message "Bad!" is sent to the robot. Windows #ref(HSRCleanupVRHuman.png) * Robot (ROS) Execution procedure [#robot_execution] ** Startup Procedure [#vafd19d3] ~'' * If you want to use a local server, please [[Launch Photon Server>#LaunchPhotonServer]] first. '' Start the Ubuntu side and then the Windows side. *** Ubuntu side startup procedure [#qd09b4ab] Open a new terminal and run the following command: $ roslaunch sigverse_hsr_teleop_key teleop_key.launch To start rviz, execute the following command. $ roslaunch sigverse_hsr_teleop_key teleop_key_with_rviz.launch *** Windows startup procedure [#t197903c] + Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupVR(.unity)] scene with reference to [[here>Tutorial using ROS for ver.3#open_scene]]. + Click the [Robot] button to log in as a robot. + If a human is not logged in, wait for the human to log in. Windows (Login Screen) #ref(HSRCleanupVRLogin.png) ** Run [#v04386d7] You can operate HSR by operating the keyboard on the terminal of teleop_key.launch on Ubuntu side. You can control HSR by keyboard operation on the teleop_key.launch terminal. Operate the robot and grasp one of the objects on the desk, then bring it to the place where the human points. ~* Check the terminal for details of the operation. If you want to finish, stop the Unity side and then the ROS side. Ubuntu side #ref(HSRCleanupVRRobotUbuntu.png) Windows side (Logged in) #ref(HSRCleanupVRRobotWindows.png) Windows side (Pointing) #ref(HSRCleanupVRRobotWindowsPointing.png)
Site admin: inamura
PukiWiki 1.5.0 Copyright © 2001-2006 PukiWiki Developers Team. License is GPL.