[[Tutorial using ROS for ver.3]] &color(red){&size(40){Noetic Edition Under Construction};}; * Grasping with TurtleBot3 automatically [#e93a5390] This example is a simple example that automatically grasps an object. TurtleBot3 has OpenManipulator. Object recognition by darknet_ros (YOLO) and Point Cloud data are used. ~ So CUDA must be installed in the Ubuntu in order to recognize objects using YOLO. ~ Although it is possible to check this example scene using CPU without installing CUDA, the object recognition speed is very slow. The outline of the operation is as follows. + Move the TurtleBot3 to the front of the target object by keyboard operation. + Move the TurtleBot3 so that the target object comes to the center of the color image. + Enter a number to specify the object to be grasped. When the above operation is performed, TurtleBot3 tries to grasp the object as follows: + The darknet_ros (YOLO) performs object recognition using a color image. + Estimate the 3D position of the target object using color images and Point Cloud information. + Grasp the calculated 3D position. Please refer to [[here>http://emanual.robotis.com/docs/en/platform/turtlebot3/manipulation/#manipulation]] for the specifications of Open Manipulator. ~ Please refer to [[here>http://emanual.robotis.com/docs/en/platform/turtlebot3/appendix_realsense/]] for camera specifications.~ Because it is convenient to shorten the minimum distance of the depth sensor, RealSense SR300 is used instead of RealSense R200. ** Build Ubuntu environment [#c709f1fe] In this example, darknet_ros (YOLO) needs to be installed in Ubuntu environment. ~ It is recommended to check the operation of [[normal darknet>https://github.com/pjreddie/darknet]] without using ros. The procedure is as follows: + If you want to use CUDA, install the NVIDIA driver. (It is possible to run using the CPU without installing CUDA.) + If you want to use CUDA, download and install it from [[here>https://developer.nvidia.com/cuda-downloads]]. ~ (For details, please adjust according to your PC and NVIDIA Driver environment.) + Git clone darknet_ros. ~ Download the commit that have been verified to work. $ cd ~/catkin_ws/src $ git clone --recursive https://github.com/leggedrobotics/darknet_ros.git $ cd darknet_ros $ git checkout 1027a280 + Install darknet_ros. $ cd ~/catkin_ws $ catkin_make -DCMAKE_BUILD_TYPE=Release ~* Depending on the combination of GPU and CUDA, you may get an error like "Unsupported gpu architecture 'compute_30'" during catkin_make. ~ In that case, you can comment out the line related to compute_30 in CMakeLists.txt of darknet_ros to avoid the error. ~* For example, in the case of NVIDIA Tesla T4, delete the line "compute_30" and add "compute_75". ** Startup Procedure [#bc27b1e9] Start the Ubuntu side and then the Windows side. *** Ubuntu side startup procedure [#l39593fd] Open a new terminal and run the following command: $ roslaunch sigverse_turtlebot3_open_manipulator grasping_auto.launch *** Windows side startup procedure [#df821e14] Start the [Assets/SIGVerse/ExampleScenes/Turtlebot3/OpenManipulatorSR300(.unity)] scene with reference to [[here>Tutorial using ROS for ver.3#open_scene]]. ** Run [#sd88b953] You can control TurtleBot3 by keyboard operation on the grasping_auto terminal. ~* Check the terminal for details of the operation. ~* Because of the high image processing load, the robot's camera image transmission interval is set to 1000 [ms]. ~ (If you want to shorten the sending interval, please reduce the Sending Interval of the TurtleBot3PubSR300RGBController attached to turtlebot3_with_open_manipulator_SR300/RosBridgeScripts) If you want to finish, stop the Unity side and then the ROS side. Ubuntu side (The details may differ from the latest version) #ref(TurtleBot3GraspAutoUbuntu.png) Windows side (The details may differ from the latest version) #ref(TurtleBot3GraspAutoWindows.png) Windows side (grasped "clock") (The details may differ from the latest version) #ref(TurtleBot3GraspAutoWindowsGrasping.png) ---- ''Reference Video'' (Image recognition is slow because no GPU is used.) #youtube(eZgu1bO7KMM) ---- #counter