This sample is a simple sample that automatically grips an object.
TurtleBot3 has OpenManipulator.
Please refer to here for the specifications of Open Manipulator.
Please refer to here for camera specifications.
Object recognition by darknet_ros (YOLO) and Point Cloud data are used.
In addition, CUDA must be installed in the Ubuntu environment in order to perform YOLO object recognition.
Although it is possible to operate using the CPU without installing CUDA, the object recognition speed is very slow.
The outline of the operation is as follows.
When the above operation is performed, an attempt is made to grip the object in the following flow.
Please refer to here for the specifications of Open Manipulator.
Please refer to here for camera specifications.
Also, since it was more convenient for the camera to have the shortest distance from the depth sensor, use RealSense SR300 instead of RealSense R200.
In this sample, darknet_ros (YOLO) needs to be installed in Ubuntu environment.
It is recommended to check the operation of [[normal darknet> https://github.com/pjreddie/darknet]] without using ros.
The following darknet_ros has been confirmed to work with version 1.1.4.
The procedure is as follows
$ cd ~/catkin_ws/src $ git clone --recursive https://github.com/leggedrobotics/darknet_ros.git $ git checkout ac666ab8e8e3dd23a8a95d891fb90874e63c8cb5 $ cd ~/catkin_ws/src/darknet_ros/darknet_ros/yolo_network_config/weights/ $ wget http://pjreddie.com/media/files/yolo.weights
$ cd ~/catkin_ws $ catkin_make -DCMAKE_BUILD_TYPE=Release
First, start Ubuntu. Then start Windows.
Open a new terminal and run the following command: $ roslaunch sigverse_turtlebot3_open_manipulator grasping_auto.launch
Start the [Assets/SIGVerse/SampleScenes/Turtlebot3/OpenManipulatorSR300(.unity)] scene with reference to here.
You can operate TurtleBot3 by key operation on the terminal named grasping_auto on Ubuntu side.
* Check the terminal for details of the operation.
If you want to finish, stop the Unity side and then the ROS side.
Ubuntu side
Windows side
Windows side (grasped "clock")