Cosi119a

* prev: <a href="/content/topics/robotics/ros_book/115_prr_move_base/">10 part3: Move Base</a>

## Introduction

* How can the robot follow lines on the ground?
    * Acquire images
    * Filter the images
    * Steer the robot

## Acquiring Images

* Message: `sensor_msgs/Image`
* Run Gazebo to simulate a robot with a camera
* Since the TB3 Waffle has a camera, we will use those models, and then run gazebo

<pre><code class="language-bash">
# Set model to WAFFLE!!!
$ export TURTLEBOT3_MODEL="waffle"
$ export TB3_MODEL="waffle"
$ roslaunch turtlebot3_gazebo turtlebot3_world.launch
$ rostopic list | grep /camera
</code></pre>

* Note all the camera related topics being published
* The image itself can be found under `/camera/rgb/image_raw`
* `../compressed` is good for sending the images over wifi
* `../theora` applies even more compression

<slide_break></slide_break>

### Look at image

* Lets visualize the image
* Choose from each of the topics relating to the camera
* Move the robot in Gazebo and watch the picture change

<pre><code class="language-bash">
$ rqt_image_view
</code></pre>

<slide_break></slide_break>

### Run the simplest program

<pre><code class="language-bash">
$ rosrun samples follower.py
$ rosnode samples follower
--------------------------------------------------------------------------------
Node [/follower]
Publications: 
 * /rosout [rosgraph_msgs/Log]

Subscriptions: 
 * /camera/rgb/image_raw [sensor_msgs/Image]
 * /clock [rosgraph_msgs/Clock]

Services: 
 * /follower/get_loggers
 * /follower/set_logger_level

contacting node http://10.0.0.95:39103/ ...
Pid: 17179
Connections:
 * topic: /rosout
    * to: /rosout
    * direction: outbound
    * transport: TCPROS
 * topic: /clock
    * to: /gazebo (http://10.0.0.95:42857/)
    * direction: inbound
    * transport: TCPROS
 * topic: /camera/rgb/image_raw
    * to: /gazebo (http://10.0.0.95:42857/)
    * direction: inbound
    * transport: TCPROS

</code></pre>

* First section is information determined from roscore about what /follower is doing
* Second section is information directly from the /follower node about it's connections
    * We see that it is publishing to /rosout
    * And is subscribed to `/clock` (from gazebo)
    * And `/camera/rgb/image_raw` (from gazebo too)
* Lets find out the frame rate (simulated by gazebo)

<pre><code class="language-bash">
rostopic hz /camera/rgb/image_raw
subscribed to [/camera/rgb/image_raw]
WARNING: may be using simulated time
average rate: 10.204
	min: 0.019s max: 0.131s std dev: 0.03037s window: 10
</code></pre>
* So, we are receiving about 10 images per second
* Which is why my computer is so slow :)

## Race Course

**NB: We diverge from the book a little. Note different roslaunch**

* Kill everything from before
* Load a new world into gazebo

<pre><code class="language-bash">
$ roslaunch turtlebot3_gazebo turtlebot3_autorace.launch
</code></pre>

* You will see in gazebo, a TB3 Waffle on a little racetrack
* Here's the track, and the view from the camera:

## Introducing OpenCV

* Our goal is to detect the line using OpenCV, so we can then actually follow it
* Technique will be simplistic: filter a block of rows by image color and drive the robot towards the center of the pixels that pass the filte
* Note that this will fail in many normal cases!
* First, just demonstrate that OpenCV and ROS Talk:
```
$ rosrun samples follower_opencv.py
```

* Simple program captures each image from camera
* Gives it to OpenCV to display in a window

<slide_break></slide_break>

### Detecting the yellow line

* Image and Pattern Recognition is arbitrarily complex
* Here we will take a simplistic approach
* An interesting relevant link: [Tracking Colored Objects](http://aishack.in/tutorials/tracking-colored-objects-opencv/)
* Note that we customized the values to recognize yellow as `HSV [40,0,0]:[120,255,255]`
* These values are very rough and could be improved

```
$ rosrun samples follower_color_filter.py
```

* Will display three windows: the original image, the mask and the original as masked
* We will really only need the masked image

<slide_break></slide_break>

### Now, use yellow line to pick a direction

* First we take a 20 pixel slice near the bottom of the filtered image, which contains ony the yellow line
* We process the image further to locate the ["centroid"](https://www.learnopencv.com/find-center-of-blob-centroid-using-opencv-cpp-python/)
* And we paint a red circle there to show where that is
* Notice that while the image is still, its actually being regenerated 10 times per second, once for each  frame. Move the robot manually in Gazebo to notice this.

```
$ rosrun samples follower_line_finder.py
```
<slide_break></slide_break>

### Actually following the line
* We will use the location of the centroid
* We offset it because the yellow line is not in the middle of the road
* We move the robot forward and give it a little turn
* Based on how far from the middle the centroid is
* Notice this is a PID controller!

```
$ rosrun samples follower_p.py
```