Autonomous Navigation of a Quadrotor Helicopter Using GPS and Vision Control Project GoalsSoftware and CompassAccomplishedCompassGetting the compass to workImporting Serial data to MATLABWhat the code is doingFuture PlansGPS Integration and HardwareProgressCurrent Set-upAccuracy TestsQuadrotor TestThe Next Step:Control SystemDeliverablesControl System DesignModel AssumptionsPretty PicturesPretty PicturesPretty PicturesImage Processing GoalsThe CMUCam2+Obstacles in Using the CMUCamRelevant CMUCam FunctionsWhat's NextArduino Microcontroller and CommunicationAccomplishedBenefits of On Board Data CollectionDisadvantages of on Board Data CollectionWork LeftPossible On Board ControlAutonomous Navigation of a Quadrotor Helicopter Using GPS and Vision ControlGroup 1Photo of the Ascending Technologies Hummingbird Autopilot Quadrocopter removed due to copyright restrictions.Project Goals• Fly helicopter to a predetermined location using GPS feedback• Take pictures at this location• Fly a planned path along GPS coordinates• Take pictures along the reference path• Use GPS and camera feedback to visually servo to and land on a marked targetAutonomous landing ontargetObtaincompass data ObtainGPS data Communicatebetween XBees Pick a controlalgorithmWrite MATLABflight control codeTest code ona simulationCollect datawirelessly Put data intoMATLABSend commandsand receive datato and from quadrotorMount hardwareon quadrotorHover in place,fly to position,and fly to multiplepositions autonomouslyImage processingFlight ControlObtain imagedata from cameraSend commands to camera Communicatebetween XBeeswith cameraTake pictureswirelesslyPut data intoMATLAB Recognize target with cameraFigure out vertical distance from targetFind error inX, Y in imageTurn signal into visible imageFind error in Quadrotor’s positionSend data tocontrollerLand autonomously on marked targetSoftware and CompassStudent AAccomplished-> Serial connection with XBees-> Wireless connection between the compass and my laptop->Importing the readings to a MATLAB variableCompassWireless serial connectionSends data to the laptop via XBee. SIGNAL FROM THE COMPASS MUST BE INVERTED BEFORE IT IS FED TO THE XBEE!Mounted on the vehicleInverting Op-AmpImages from the OpenClipArt Library and mangonha on Flickr.Getting the compass to work• In order to collect data, the output signal from the compass must be inverted.Receiving serial data from the compass via XBee. This data can be imported to MATLAB.Importing Serial data to MATLAB• Luckily, we already have a VC++ code that can read the heading of the compass. •“MexFunction” and some other lines are added to the C Code in order to send the heading to matlab…………….#define YP_OUT plhs[0]void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray* prhs[] ) { //start of the mexfunction.double *yp; float tempheading;opencompass(); //opens compasstempheading = readcompass();WaitForSingleObject(headingMutex, INFINITE);heading = tempheading;ReleaseMutex(headingMutex);.................What the code is doing• The mex function “compmat” reads the heading of the compass at that instant. (returns one float type number)• Example: code that reads the heading for 10 seconds (10 readings per sec)for i=1:100heading(i) = compmat;pause(0.1);endresultsFuture Plans• Solder the whole thing up so it can be attached to the vehicle.• The XBee and the compass needs 3.3V. Figure out how to get this voltage from the quadrotor’s battery.• Try out the control code by flying the quadrotor with the compass and the GPS attached.• After we start getting decent results, we can take off all the XBees and replace it with an Arduino; which will be much more elegant.GPS Integration and HardwareStudents C and EProgress Read GPS signal directly connected to computer Successful XBee communication Read GPS signal through XBee communication Transmit GPS signal from Quadrotor Send data to MATLABCurrent Set-upImages from the OpenClipArt Library and mangonha on Flickr.Accuracy TestsGPS Only ±20mWAAS Enabled GPS ±5mQuadrotor TestThe Next Step:• Get MATLAB to read data from the XBee• Build a more permanent housing for the GPS Fly to preset GPS coordinate  Maintain position Fly a planned path based on GPSMission Goals:Read GPSPut data into MATLABSend to controller Position commandControl SystemStudent BDeliverablesDemonstrate closed loop control on a LTI model of the quadrotorDemonstrate closed loop control of the quadrotorControl System Design• Yaw is set initially and is static• Depends on ability of quadrotor to pivot• Controlled variables– Yaw rate: points at the target– Roll: keeps on line to target– Pitch: determines speed forward– Thrust: offsets gravity and brings rotor to correct height• Measured variables– Yaw: compass– X, Y positions: GPS, camera– Height: Pressure sensor, GPSStrategy: Point and goModel Assumptions• Linear Time Invariant• Small angle pitch and roll• Max, Min thrust = 1.25*m*g and 0.75*m*g• Delta t=0.001 seconds• Added in a lot of random noise to x_dot, y_dot, z_dot and yawPretty Pictures-20 0 20 40 60 80 100 120-12-10-8-6-4-202Position in X, YX (meters)Y (meters)Pretty Pictures0 100 200 300 400 500 600 700 800 900 1000020406080100120Distance from target in X-Y planeTime (seconds)Distance (meters)Pretty Pictures0 100 200 300 400 500 600 700 800 900 1000-20246810121416HeightTime (seconds)Height (meters)Image Processing Goals• Take pictures of a predetermined location and also along a reference flightpath• Track a landing target at a known location• Visually servo to the target using feedback from the imageThe CMUCam2+z Specificationsz Low res: 87x142z Hi res: 174x254z Featuresz Tracks motionz Tracks colorsz Makes real-time histogramsz Face recognitionPhoto of the CMUCam2+ removed due to copyright restrictions.Low Resolution Image High Resolution ImageObstacles in Using the CMUCamz Poor implementationz GUI can directly execute functions, but the data goes nowherez GUI can't take frames and data simultaneouslyz Serial communicationsz CMU's recommendations don't workz Crashes MATLABz Camera cannot directly save an imageRelevant CMUCam Functionsz Frame Difference (FD)z Track Color (TC)z Track Window (TW)z We choose to exclusively use TCz FD only works when the camera is stationaryz TW only tracks colors in the center of the frameT PacketsT 84 132 4 1 172 250 255 12Indicates a