Vision-Based Motion Planning for an Autonomous Motorcycle on Ill Structured RoadsDARPA Grand Challenge 2005Ill-Structured RoadsBerkeley Blue TeamTraditional Vision-based NavigationVision-based Navigation for MotorcycleSlide 7Related Workstructured roadsStructured roadsUnstructured EnvironmentsSlide 12Related Work: Sensing MethodsMonocular vision and ill structure roadsVideo Camera: Pin-hole ModelCamera configurationProblem definitionStep 1V2-SpaceAlgorithm Step1: Construction of V2-SpaceColor CorrectionSurface Verification – Gaussian MethodSurface Verification: One ExampleDirection ExtractionDirection Extraction: One ExampleV2-Space ConstructionStep 2Slide 28Candidate trajectoriesRoad Following Quality (RFQ)Select optimal trajectoryGenerate trajectory velocity profileConsider vehicle geometric constraintMotion Planning (cont’d)Experimental ResultsSlide 36ConclusionsSlide 38Vision-Based Motion Planning for an Autonomous Motorcycle on Ill Structured Roads Dezhen SongHyunnam LeeJingang YiTexas A&M UniversityAnthony LevandowskiEnscon, Inc.2DARPA Grand Challenge 20053Ill-Structured Roads4Berkeley Blue TeamMore than 20 undergraduate & graduate students from UC Berkeley and TAMU campusesFinalist in 2004 Semi-finalists in 2005 (43 out of 200)http://www.ghostriderrobot.com(a) 2004 (a) 2004 platformplatform (b) 2005 (b) 2005 platformplatform5Traditional Vision-based NavigationRoad detection Obstacle detection Path generation Trajectory following Low level control Road representationsensesenseplanplanactact6Vision-based Navigation for MotorcycleRoad detection Obstacle detection Path generation Trajectory following Low level control sensesenseplanplanactactRoad representationVehicle geometric, kinematics, dynamic limits7Vision-based Navigation for MotorcycleRoad detection Obstacle detection Path generation Trajectory following Low level control sensesenseplanplanactactVision vector space (V2-Space)Vehicle geometric, kinematics, dynamic limits8Related WorkSurveys –Broggi and Berte 1995,–Desouza and Kak 2002, –Bertozzi et al. 2002,–Sun et al. 2006 Classifications of vision-based navigation–Road conditions–Sensing methods–Vision algorithms9structured roadsNo Hands Across America (1995)–Navlab5 employed RALPH algorithm–from Pittsburgh, PA to San Diego, CA –2797/2849 miles (98.2%) autonomously.The ARGO project(1998) –ARGO vehicle employed GOLD algorithm–2000km tour throughout ItalyEkinci et al. 2000, He et al. 2004 …Navlab5ARGO10Structured roadsRoad edge-based navigation+Road region-based navigationOriginal imageprojection image 3 winning curvaturesArea used for computing road color Road extraction Final ResultHe et al. 200411Unstructured Environments• Manduchi et al. 2005• Volpe et al. 2000• Lorigo et al. 1997• Stephen et al. 1989• Matthies et al. 1998• Ibanez-Guzman et al. 200412Ill-Structured Roads• Shadow and illumination changes• Non-uniform road surface•No lane markings or clear boundary•No/little prior knowledge•Drastic change of road conditions13Related Work: Sensing MethodsMonocular vision–Happold et al. 2006, Dahlkamp et al. 2006–Lieb et al. 2005, Michels et al. 2005–Rasmussen 2004Stereo vision–Kolesnik et al. 1998Vision+LADAR–Rasmussen 2006–Manduchi 200514Monocular vision and ill structure roadsInternal directions, dominating directions–Broggi and Berte 1998–Zhang and Russell 2005–Rasmussen 2004Dominant OrientationsThe Vanishing point15Video Camera: Pin-hole ModelPoint in world frame W related to point in image frame I by Perspective projection16Camera configurationBrake Brake actuatoractuatorBatteryBatteryThrottle Throttle actuatoractuatorSteering Steering actuatoractuatorSpeedometSpeedometererSteering Steering sensorsensorCameraCameraGPS/IMUGPS/IMUComputer Computer & & electronicselectronics17Problem definitionRoad detection Obstacle detection Path generation Trajectory following Low level control sensesenseplanplanactactVision vector space (V2-Space)Vehicle geometric, kinematics, dynamic limitsInput: images + GPSOutput: Trjaectory18Step 1V2-Space ConstructionPath generation Trajectory following Low level control sensesenseplanplanactact(V2-Space)Vehicle geometric, kinematics, dynamic limitsInput: images + GPSOutput: Trjaectory19V2-Space Image size in pixels:Color image frame in matrix form:Vision vector space is a collision-free direction vectors in image frame where .20Algorithm Step1: Construction of V2-Space21Color Correctionc1c2c3 model: shadow-invariant(a) Original (a) Original imageimage (b) After color (b) After color correctioncorrection ( (cc33))22Surface Verification – Gaussian MethodReference Reference area.area.23Surface Verification: One Example(a) Original (a) Original imageimage (b) Surface (b) Surface outputoutput24Direction ExtractionFor each pixel, check pixel similarity in surrounding directions5-10 pixels are checked along each direction for noise reductionStatistical criterion is used to determine the similarity(b) Similarity check (b) Similarity check around (around (u,vu,v)) (a) Pixel (a) Pixel ((u,vu,v)) (c) Extracted direction (c) Extracted direction at (at (u,vu,v))25Direction Extraction: One Example(a) Original (a) Original imageimage (b) Direction extraction (b) Direction extraction outputoutput26V2-Space ConstructionRemarks–Open framework–Allow integration of sophisticated methods•Polymonial Mahalanobis Distance: Grudic and Mulligan 2006•Texture matching: Schaffalitzky and Zisserman 2001–Variable resolution–More important: facilitates motion planning.27Step 2V2-Space ConstructionPath generation Trajectory following Low level control sensesenseplanplanactact(V2-Space)Vehicle geometric, kinematics, dynamic limitsInput: images + GPSOutput: Trjaectory28Motion planningStep 2V2-Space ConstructionLow level control sensesenseplanplanactact(V2-Space)Input: images + GPSOutput: Trjaectory29Generic trajectoryCandidate trajectories where R -- path radius, d -- a binary direction variable. 06543 21Candidate circular arcs –Controllability of Nonholonomic vehicle: Ma, et al 1999Seven candidate arcs–Resolution of the trajectory following30: boundary.Road Following Quality (RFQ)Project a given finite circular trajectories in to the image spaceCalculate the