This example is a simple example that automatically grasps an object.
TurtleBot3 has OpenManipulator.
Object recognition by darknet_ros (YOLO) and Point Cloud data are used.
So CUDA must be installed in the Ubuntu side in order to recognize objects using YOLO.
Although it is possible to check this example scene using CPU without installing CUDA, the object recognition speed is very slow.
The outline of the operation is as follows.
When the above operation is performed, TurtleBot3 tries to grasp the object as follows:
Please refer to here for the specifications of Open Manipulator.
Please refer to here for camera specifications.
Because it is convenient to shorten the minimum distance of the depth sensor, RealSense SR300 is used instead of RealSense R200.
In this example, darknet_ros (YOLO) needs to be installed in Ubuntu environment.
It is recommended to check the operation of normal darknet without using ros.
We confirmed that it works with darknet_ros version 1.1.4.
The procedure is as follows:
$ cd ~/catkin_ws/src $ git clone --recursive https://github.com/leggedrobotics/darknet_ros.git $ git checkout ac666ab8e8e3dd23a8a95d891fb90874e63c8cb5 $ cd ~/catkin_ws/src/darknet_ros/darknet_ros/yolo_network_config/weights/ $ wget http://pjreddie.com/media/files/yolo.weights
$ cd ~/catkin_ws $ catkin_make -DCMAKE_BUILD_TYPE=Release
Start the Ubuntu side and then the Windows side.
Open a new terminal and run the following command:
$ roslaunch sigverse_turtlebot3_open_manipulator grasping_auto.launch
Start the [Assets/SIGVerse/SampleScenes/Turtlebot3/OpenManipulatorSR300(.unity)] scene with reference to here.
You can operate TurtleBot3 by operating the keyboard on the terminal named grasping_auto on Ubuntu side.
* Check the terminal for details of the operation.
If you want to finish, stop the Unity side and then the ROS side.
Ubuntu side
Windows side
Windows side (grasped "clock")