* **Reading: <a href="/content/topics/homeworks/hw_readings_119/190_hw_119_prr10_1/">Read PRR Chapter 10</a>** *Reminder: Readings are your responsibility. You will be expected to come to class prepared, having read the material, and ready to participate in the discussion* <slide_break></slide_break> ### Logistics * How is Lab time going? * Tour of Labnotebook2! <slide_break></slide_break> ## Localization * What exactly is the problem with knowing where you are? * Remember all of this needs an agreed upon coordinate system! * How would you do it with your eyes closed? * Recall **odometry** is like **dead reckoning** * Lots of noise in the signal; accuracy varies; errors build up <slide_break></slide_break> ### What kinds of landmarks can be used? * Lidar detected fixed obstacles * Vision detected fixed obstacles * How can one obstacle be distinguished from another? * Color, Size, Shape * Note that in addition to recognizing, we need to know the distance <slide_break></slide_break> ### Lidar detection * Robot has a 'map' of fixed obstacles * Robot compares that map with the apparent, transient, map from Lidar * Calculates a probability distribution of where it might be on that map * Process is called AMCL - Adaptive Monte Carlo Localization <slide_break></slide_break> ### Visual Detection * Robot has a collection of scenes it can recognize. * Needs to meet certain characteristics for it to work * With each scene is the coordinates that correspond to the recognized scene * CV is constantly analyzing what is seen by the camera * As soon as it identifies an image, it can use that to figure out where it is * Note! It has to also figure out where it is relative to the image <slide_break></slide_break> ### Requirements Visual localization * CV (computer vision) algorithms need to be able to identify and differentiate it from other images * A coordinate in 3D space is required * It needs to stay put and not move * Examples: facade of a building, a particular tree, a wall, etc. * Bad examples: a parked car; a person <slide_break></slide_break> ### Fiducial "SLAM" * Special CV Algorithm identifies a known pattern -- a fiducial * Each fiducial has a unique ID and a known 3D pose in the room * Algorithm determines the TF relationship between camera and a fiducial * As fiducial has a known 3D pose relative to the room, this gives a 6D pose estimate for the camera * This is then used to localize the robot relative to the map * Need to be able to see the fiducial and identify it * Distance? Size of Fiducial? Is it in frame? Is it occluded? * Need to see more than one fiducial to get a position fix <slide_break></slide_break> ## AMCL - Important algorithm for localization * Adaptive Monte Carlo Localization * Very sophisticated (and standard) mathematical technique * Can be used with different kinds of sensors. <slide_break></slide_break> ## SLAM and AMCL * <a href="/content/topics/robotics/ros_book/90_prr_building_maps/">9: Building Maps</a> * <a href="/content/topics/robotics/ros_book/95_prr_building_maps_additional/">9a: Building Maps</a> <slide_break></slide_break> ### How AMCL works with Lidar 1. Requires a map of Lidar visible obstacles, with a coordinate system, and anchored in the real world 1. Given a stationary robot, and a lidar scan, what does it see? 1. Look for a match on the map 1. Form a probability distribution of where the robot MIGHT be 1. Move the robot a little. 1. Compute what the lidar would see given what is known about the motion 1. Update the probabilities <slide_break></slide_break> ### SLAM * What if there is no map yet? * Simultaneous localization and mapping * Create a theoretical map based on view of the LIDAR * Move the robot a little and update the map * Travel through the relevant region (using e.g. Teleop) * Use the map to localize, or where there is no map yet, extend the map. <slide_break></slide_break> ### GOAT - Go To Any Thing <iframe width="560" height="315" src="https://www.youtube.com/embed/Ozb8YrVbPUM?si=m7TP1n3iciIHQc8t" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe> * https://theophilegervet.github.io/projects/goat/ * https://github.com/facebookresearch/home-robot <slide_break></slide_break> <h2 class="shadow p-3 bg-white rounded">Thank you. Questions?<img class="img-fluid w-100" src="https://picsum.photos/800/100.jpg" /> <small> (random Image from picsum.photos)</small></h2>