<slide_break></slide_break> ### Introduction * This small but tricky assignment pulls together many of the strands of robotics engineering that we've been covering * You will work with basic image processing and computer vision, PID control, and more complex behaviors * This will not require any machine learning but feel free to use it yourself * You will Write a program to follow a distinct line on the ground using the robots camera * You will work with and process camera RGB data * You will learn how to use line detection, image filtering, and contour detection to identify already known features <slide_break></slide_break> ### Expected preparation * (Re-) read [Chapter 12 of PRR](http://cosi119r.s3-website-us-west-2.amazonaws.com/content/topics/ros_book/120_prr_follow_bot/) * Learn how to use openCV in ROS * Learn how to subscribe and use the camera image topics * Understand the RGB representation of an image <slide_break></slide_break> ### Additional references and resources * [OpenCV](https://docs.opencv.org/master/d6/d00/tutorial_py_root.html) * [OpenCV to ROS compatible image bridge](http://wiki.ros.org/cv_bridge/Tutorials/ConvertingBetweenROSImagesAndOpenCVImagesPython) * [Contouring](https://docs.opencv.org/3.4/d4/d73/tutorial_py_contours_begin.html) * [Line recognition](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_houghlines/py_houghlines.html) <slide_break></slide_break> ### Requirements for the PA * You will solve this both in Sim and with an actual robot * The robot will begin at a spot just before the line. * It will Follow a distinct line on the ground/floor * It is your task to keep the robot as close to the line as possible for as much of the length of the path as possible * Much of this comes out of the book, but make sure you understand it And here are some Additional Challenges * Robot starts somewhere where the line is not immediately visible * Robot is able to double back along the line allowing it to follow the line infinitely * Robot uses Lidar (/scan) to detect an obstacle in the way. Place an object in gazebo of your choice to move around. The more "challenging" choice of obstacle(s) will mean more extra points awarded <slide_break></slide_break> ### Steps * Subscribe to the compressed image topic * Convert the callback message compressed image to a opencv compatible image * Apply the image manipulation techniques to identify features of the line * Use the feature information to control the robots movement to keep following the line * Also it will be very helpful to find a way to debug your algorithms * Think: What information can you use about the location of your detected line to help control the robot? * Think: How can you debug your computer vision algorithms in Rviz? <slide_break></slide_break> ### Tips for the simulation <pre><code class="language-bash"> # You will be provided with a basic gazebo world without any walls, # and a single line. First make sure you have it. cw cd src/cosi119_src git pull # And now launch it with this command. NOTE that it says model:=waffle roslaunch line_follower line_follower_sim.launch model:=waffle </code></pre> <slide_break></slide_break> ### What to submit * A correctly structured ROS package including * Nicely commented python source files * Appropriate launch file * Readme explaining how the code works * Video of your program running * One in gazebo and one IRL * Best to put the video on youtube and give the link <slide_break></slide_break> ### Example Videos * [Sample Solution - Sim](https://www.youtube.com/watch?v=tygWRKdvnPo&list=PLWp7_Yk4l1aPcMGxCCvqKCSwnkTBBInI3&index=75) * [Sample Solution - Real](https://www.youtube.com/watch?v=8AC597dVKHg&list=PLWp7_Yk4l1aPcMGxCCvqKCSwnkTBBInI3&index=80) ## Notes 1. Make the callback to the image topic subscriber as leightweight as possible. In this case, it's not good to follow PRR's line following code, which puts all of its logic in the image callback function. This approach works in simulation, but when you work with a real robot, you'll quickly run into performance issues. 1. Even though the starter code includes a my_odom.py, you are not required to use it, and you are also welcome to modify it. <slide_break></slide_break> ### What skills this assignment covers <table class="table table-striped table-sm table-responsive table-hover"><thead><tr><th>Skills</th></tr></thead><tr><td>ROS App: Basic structure of a ros app, ROSCore and Nodes</td></tr><tr><td>launch and run: Launch files, bringing up robots, running with simulation</td></tr><tr><td>Motion: Control motion with cmd_vel</td></tr><tr><td>Pubsub: publish and subscribe</td></tr><tr><td>pid: PID as key control algorithm</td></tr><tr><td>calc: Algorithms, calculations and state management</td></tr><tr><td>concurrent: highly concurrent programming</td></tr><tr><td>cv: images, cameras and opencv</td></tr><tr><td>lines: Basic opencv for line detection</td></tr></table>