IROS2019/Using Cloud-based VR
[
Front page
] [
New
|
List of pages
|
Search
|
Recent changes
|
Help
]
Start:
#ref(Tutorial at IROS2019/IROS2019_logo.png,70%)
* Overview [#me21a0b1]
A human and a robot log in to the same virtual space. The...
The human logs in to the virtual space using Windows + VR...
The robot logs in to the virtual space using Windows + Ub...
Humans and robots (ROS) are connected via the Internet an...
In this example, [[Photon Realtime>https://www.photonengi...
#ref((HSR)Cleanup Task using Cloud and VR/CleanupUsingClo...
//* Prerequisites [#r10a7b3e]
//** Common prerequisites [#ofa841b3]
//Using networks is a common prerequisite for humans and ...
//[[Photon Realtime>https://www.photonengine.com/en-US/Re...
//~* Make sure that anti-virus software does not block co...
//The environment constructions when using the cloud and ...
//** Prerequisites for Human (VR) [#s38cc158]
//-Windows environment ~
//In addition to [[basic requirements for using SIGVerse ...
//-Oculus Rift ~
//Please use Oculus Rift CV1 or Oculus Rift S. ~
//Please install [[Oculus Software>https://www.oculus.com...
//** Prerequisites for Robot (ROS) [#y682edd3]
//-Windows environment and Ubuntu environment ~
//See [[here>Tutorial using ROS for ver.3#SystemRequireme...
* Environment Construction [#Environments]
//The procedure on the human (VR) side and the procedure ...
** Import Oculus Utilities [#k0ec0826]
You need to import Oculus Utilities into your Unity proje...
+ Open the SIGVerse project on Unity Editor.
+ Click [Assets]-[Import Package]-[Custom Package ...] in...
+ Select "OculusUtilities.unitypackage" in the "ovr_unity...
+ Click [Open] button.
+ Click [Import] button.
+ Click [Yes] button on the "Update Oculus Utilities Plug...
+ Click [Restart] button on the "Restart Unity" window.
** Modify SIGVerse Settings [#y0decaa9]
+ Click [SIGVerse]-[SIGVerse Settings] to open the SIGVer...
+ Check [Use Oculus] and [Use PUN] in [Define symbols Set...
#ref((HSR)Cleanup Task using Cloud and VR/SIGVerseSetting...
** Setup Photon Local server [#PhotonServer]
~* Make sure that anti-virus software does not block comm...
//A Photon Server should be set in a local network.
+ Open [Photon-OnPremise-Server-SDK_v4-0-29-11263\deploy\...
+ Execute "PhotonControl.exe". (App icon appears on the W...
+ Click Photon server application in the taskbar of Windo...
+ Click [Game Server IP Config]-[Set local IP: xxx.xxx.xx...
#ref((HSR)Cleanup Task using Cloud and VR/PhotonServerIP....
+ Click [LoadBalancing (MyCloud)]-[Start as application] ...
+ Click [LoadBalancing (MyCloud)]-[Run TestClient (LoadBa...
+ Close the console window of the test client.
** Import PUN2 assets [#qb0ad89e]
+ Open the SIGVerse project on Unity Editor.
+ Access [[PUN2 page>https://assetstore.unity.com/package...
+ Sign in to your Unity account from the top right of the...
+ Click the [Open in Unity] button. "PUN2 - Free" is disp...
+ Uncheck ''"Photon/PhotonLibs/WebSocket/websocket-sharp....
#ref((HSR)Cleanup Task using Cloud and VR/PUN2ExcludeWebs...
+ Click [Import] button.~
+ Click [Skip] button on the [PUN Wizzard] window.
#ref((HSR)Cleanup Task using Cloud and VR/PunWizardLocalS...
+ Click [Close] button.
+ [PhotonServerSettings] will be opened. Modify the setti...
(You can open the PhotonServerSettings from [Window]-[Pho...
--- Uncheck [Use Name Server]
--- Enter the IP address of Photon Server in [Server]
--- Enter "5055" in [Port].
#ref((HSR)Cleanup Task using Cloud and VR/PhotonServerSet...
+ Click [Assets]-[Reimport All] in the Unity menu.
+ Click [Reimport] button to rebuild.
* Execution Procedure[#s9ea839c]
In the case of using the local server, Photon server shou...
//You should [[launch Photon Server>#LaunchPhotonServer]]...
//To execute the virtual avatar, please follow thw [[foll...
//To execute the robot software, please follow the [[foll...
** Procedure to Login as a Robot (ROS) [#robot_execution]
~* ROS nodes should be launched on the Ubuntu side before...
*** Ubuntu side startup procedure [#qd09b4ab]
+ Open a new terminal and execute the following command:
$ roslaunch sigverse_hsr_teleop_key teleop_key.launch
//To start rviz, execute the following command.
// $ roslaunch sigverse_hsr_teleop_key teleop_key_with_rv...
*** Windows startup procedure [#t197903c]
+ Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupV...
+ Click the [Robot] button to log in as a robot.
//+ If a human is not logged in, wait for the human to lo...
Windows (Login Screen)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRLog...
*** Robot Operation [#v04386d7]
You can control HSR by keyboard operation on the teleop_k...
Keyboard commands are shown on a terminal.
Operate the robot and grasp one of the objects on the des...
~* When you stop the program, please stop the Unity side ...
Ubuntu side
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
Windows side (Logged in)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
Windows side (Pointing)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
** Procedure to Login as a Human Avatar (VR) [#human_exec...
~* This step requires a VR ready PC and a VR headset (Ocu...
Please see [[requirements for using Oculus Rift>https://s...
[[Oculus Software>https://www.oculus.com/setup/]] should ...
*** Windows startup procedure [#t197903c]
+ Make sure that Oculus Rift is connected and ready for u...
+ Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupV...
+ Click the [Human] button to log in as a human.
//+ If the robot is not logged in, wait for the robot to ...
Windows (Login Screen)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRLog...
*** Instruction to the Robot [#v04386d7]
Operate the human avatar and give cleanup instructions to...
The basic flow is as follows.
+ Send the message "Pick it up!" to the robot while point...
+ Send the message "Clean up!" to the robot while pointin...
The operation method of Oculus Touch is as follows.
-Pulling the middle finger trigger puts the avatar's hand...
-When the A or X button is pressed, the message "Pick it ...
-When the B or Y button is pressed, the message "Clean up...
-When you tilt the left stick, the avatar moves in the ti...
-When the right stick is pressed, the message "Good!" is ...
-When the left stick is pressed, the message "Bad!" is se...
Windows
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRHum...
----
#counter
Top: [[Tutorial at IROS2019]]
End:
#ref(Tutorial at IROS2019/IROS2019_logo.png,70%)
* Overview [#me21a0b1]
A human and a robot log in to the same virtual space. The...
The human logs in to the virtual space using Windows + VR...
The robot logs in to the virtual space using Windows + Ub...
Humans and robots (ROS) are connected via the Internet an...
In this example, [[Photon Realtime>https://www.photonengi...
#ref((HSR)Cleanup Task using Cloud and VR/CleanupUsingClo...
//* Prerequisites [#r10a7b3e]
//** Common prerequisites [#ofa841b3]
//Using networks is a common prerequisite for humans and ...
//[[Photon Realtime>https://www.photonengine.com/en-US/Re...
//~* Make sure that anti-virus software does not block co...
//The environment constructions when using the cloud and ...
//** Prerequisites for Human (VR) [#s38cc158]
//-Windows environment ~
//In addition to [[basic requirements for using SIGVerse ...
//-Oculus Rift ~
//Please use Oculus Rift CV1 or Oculus Rift S. ~
//Please install [[Oculus Software>https://www.oculus.com...
//** Prerequisites for Robot (ROS) [#y682edd3]
//-Windows environment and Ubuntu environment ~
//See [[here>Tutorial using ROS for ver.3#SystemRequireme...
* Environment Construction [#Environments]
//The procedure on the human (VR) side and the procedure ...
** Import Oculus Utilities [#k0ec0826]
You need to import Oculus Utilities into your Unity proje...
+ Open the SIGVerse project on Unity Editor.
+ Click [Assets]-[Import Package]-[Custom Package ...] in...
+ Select "OculusUtilities.unitypackage" in the "ovr_unity...
+ Click [Open] button.
+ Click [Import] button.
+ Click [Yes] button on the "Update Oculus Utilities Plug...
+ Click [Restart] button on the "Restart Unity" window.
** Modify SIGVerse Settings [#y0decaa9]
+ Click [SIGVerse]-[SIGVerse Settings] to open the SIGVer...
+ Check [Use Oculus] and [Use PUN] in [Define symbols Set...
#ref((HSR)Cleanup Task using Cloud and VR/SIGVerseSetting...
** Setup Photon Local server [#PhotonServer]
~* Make sure that anti-virus software does not block comm...
//A Photon Server should be set in a local network.
+ Open [Photon-OnPremise-Server-SDK_v4-0-29-11263\deploy\...
+ Execute "PhotonControl.exe". (App icon appears on the W...
+ Click Photon server application in the taskbar of Windo...
+ Click [Game Server IP Config]-[Set local IP: xxx.xxx.xx...
#ref((HSR)Cleanup Task using Cloud and VR/PhotonServerIP....
+ Click [LoadBalancing (MyCloud)]-[Start as application] ...
+ Click [LoadBalancing (MyCloud)]-[Run TestClient (LoadBa...
+ Close the console window of the test client.
** Import PUN2 assets [#qb0ad89e]
+ Open the SIGVerse project on Unity Editor.
+ Access [[PUN2 page>https://assetstore.unity.com/package...
+ Sign in to your Unity account from the top right of the...
+ Click the [Open in Unity] button. "PUN2 - Free" is disp...
+ Uncheck ''"Photon/PhotonLibs/WebSocket/websocket-sharp....
#ref((HSR)Cleanup Task using Cloud and VR/PUN2ExcludeWebs...
+ Click [Import] button.~
+ Click [Skip] button on the [PUN Wizzard] window.
#ref((HSR)Cleanup Task using Cloud and VR/PunWizardLocalS...
+ Click [Close] button.
+ [PhotonServerSettings] will be opened. Modify the setti...
(You can open the PhotonServerSettings from [Window]-[Pho...
--- Uncheck [Use Name Server]
--- Enter the IP address of Photon Server in [Server]
--- Enter "5055" in [Port].
#ref((HSR)Cleanup Task using Cloud and VR/PhotonServerSet...
+ Click [Assets]-[Reimport All] in the Unity menu.
+ Click [Reimport] button to rebuild.
* Execution Procedure[#s9ea839c]
In the case of using the local server, Photon server shou...
//You should [[launch Photon Server>#LaunchPhotonServer]]...
//To execute the virtual avatar, please follow thw [[foll...
//To execute the robot software, please follow the [[foll...
** Procedure to Login as a Robot (ROS) [#robot_execution]
~* ROS nodes should be launched on the Ubuntu side before...
*** Ubuntu side startup procedure [#qd09b4ab]
+ Open a new terminal and execute the following command:
$ roslaunch sigverse_hsr_teleop_key teleop_key.launch
//To start rviz, execute the following command.
// $ roslaunch sigverse_hsr_teleop_key teleop_key_with_rv...
*** Windows startup procedure [#t197903c]
+ Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupV...
+ Click the [Robot] button to log in as a robot.
//+ If a human is not logged in, wait for the human to lo...
Windows (Login Screen)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRLog...
*** Robot Operation [#v04386d7]
You can control HSR by keyboard operation on the teleop_k...
Keyboard commands are shown on a terminal.
Operate the robot and grasp one of the objects on the des...
~* When you stop the program, please stop the Unity side ...
Ubuntu side
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
Windows side (Logged in)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
Windows side (Pointing)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRRob...
** Procedure to Login as a Human Avatar (VR) [#human_exec...
~* This step requires a VR ready PC and a VR headset (Ocu...
Please see [[requirements for using Oculus Rift>https://s...
[[Oculus Software>https://www.oculus.com/setup/]] should ...
*** Windows startup procedure [#t197903c]
+ Make sure that Oculus Rift is connected and ready for u...
+ Start the [Assets/SIGVerse/SampleScenes/HSR/HsrCleanupV...
+ Click the [Human] button to log in as a human.
//+ If the robot is not logged in, wait for the robot to ...
Windows (Login Screen)
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRLog...
*** Instruction to the Robot [#v04386d7]
Operate the human avatar and give cleanup instructions to...
The basic flow is as follows.
+ Send the message "Pick it up!" to the robot while point...
+ Send the message "Clean up!" to the robot while pointin...
The operation method of Oculus Touch is as follows.
-Pulling the middle finger trigger puts the avatar's hand...
-When the A or X button is pressed, the message "Pick it ...
-When the B or Y button is pressed, the message "Clean up...
-When you tilt the left stick, the avatar moves in the ti...
-When the right stick is pressed, the message "Good!" is ...
-When the left stick is pressed, the message "Bad!" is se...
Windows
#ref((HSR)Cleanup Task using Cloud and VR/HSRCleanupVRHum...
----
#counter
Top: [[Tutorial at IROS2019]]
Page: