Embedded System DesignStephen A. EdwardsColumbia UniversitySpring 2011Spot the ComputerEmbedded Systems: Ubiquitous ComputersiPhone Laser Keyboard Nikon D300 Video WatchGPS Playstation 3 PC Keyboard SD CardIs Your Current Embedded Project...Real-timeNetworkedRuggedizedBattery-poweredWireless0% 15% 30% 45% 60% 75%Source: 2009 Embedded Market StudyIs Your Current Embedded Project...Real-timeNetworkedRuggedizedBattery-poweredWireless0% 15% 30% 45% 60% 75%Source: 2009 Embedded Market StudyDigital Camera Block DiagramDesign An Optimal Device that Meets Constraints OnPrice FunctionalityPerformance SizePower Time-to-marketMaintainability SafetyEmbedded System TechnologiesIntegrated CircuitsProcessing elementsDesign toolsIC Technology1947: First transistor (Shockley, Bell Labs)1958: First integrated circuit (Kilby, TI)1971: First microprocessor (4004: Intel)Today: eight wire layers, 45 nm featuresMoore’s Law: Transistors per chip1970 1980 1990 2000 20101k10k100k1M10M100M1G10G≈ 1.5 × /year“The complexity for minimum componentcosts has increased at a rate of roughly afactor of two per year. Certainly over theshort term this rate can be expected tocontinue, if not to increase.”— Gordon Moore, 1965Source: Intel/Wikipedia$1000 Buys You This Many CPSElectromechanicalRelaysTubesTransistorsIntegrated Circuits1900 1920 1940 1960 1980 200010−410−2100102104106108Source: Ray Kurzweil, The Age of Spiritual Machines1918 Sears Roebuck CatalogAbout $100 in today’s dollars.From Donald Norman, The Invisible Computer, 1998.What Percentage of Time Do You Spend...Developing SpecificationsConceptual DesignDetailed DesignSimulationTesting/DebuggingPrototypingSending to productionDocumentation/meetings0% 5% 10% 15% 20% 25%Source: 2009 Embedded Market StudyWhat Percentage of Time Do You Spend...Developing SpecificationsConceptual DesignDetailed DesignSimulationTesting/DebuggingPrototypingSending to productionDocumentation/meetings0% 5% 10% 15% 20% 25%Source: 2009 Embedded Market StudyIf You Could Improve One Thing...Debugging ToolsScheduleProgramming ToolsEngineering TeamInterfacesMicroprocessorOther hardwareOperating SystemIDE0% 5% 10% 15% 20% 25% 30%Source: 2009 Embedded Market StudyIf You Could Improve One Thing...Debugging ToolsScheduleProgramming ToolsEngineering TeamInterfacesMicroprocessorOther hardwareOperating SystemIDE0% 5% 10% 15% 20% 25% 30%Source: 2009 Embedded Market StudyIf Your System Has More Than One Processor,Does It Use...Multiple different chipsMultiple identical chipsSingle chip, identical coresSingle chip, different cores0% 15% 30% 45% 60%Source: 2009 Embedded Market StudyIf Your System Has More Than One Processor,Does It Use...Multiple different chipsMultiple identical chipsSingle chip, identical coresSingle chip, different cores0% 15% 30% 45% 60%Source: 2009 Embedded Market StudyDoes Your Current Project Contain FPGAs?45% Yes55% NoSource: 2009 Embedded Market StudyDoes Your Current Project Contain FPGAs?45% Yes55% NoSource: 2009 Embedded Market StudyWhy Won’t Your Next Project Use FPGAs?Don’t need themToo expensiveToo power-hungryHard to useToo slowToo smallUnreliableDon’t Know0% 15% 30% 45% 60% 75%Source: 2009 Embedded Market StudyWhy Won’t Your Next Project Use FPGAs?Don’t need themToo expensiveToo power-hungryHard to useToo slowToo smallUnreliableDon’t Know0% 15% 30% 45% 60% 75%Source: 2009 Embedded Market StudyYour Nemesis: The Altera DE2DE2 PeripheralsClass StructureThree Introductory Labs: 2 weeks each1. Access, modify, and display memory in VHDL2. An Ethernet chat client (software only)3. EitherÏan FM audio synthesizer; orÏa video bouncing ball.The project: Design-your-ownCustom Project IdeasBroadly: C + VHDL + peripheral(s)Video game (e.g., Pac-Man)Digital Video Project - Stephen Edwards Block DiagramTDA8708VideoADCTDA8708VideoADCLuminance/Composite InChrominance InSAA9051Digital MultiStandardDecoder88SAA9057ALine-LockedClock Generator74MUX7’374s ’374sSAA7192DigitalColorspaceConverter888Bt101VideoDACRed Analog OutGreen Analog OutBlue Analog OutDSP 56001Digital Signal ProcessorDataAddressVRAM256K X 24 X 2244LUMMUVMMUXLUMOUVOLUMLUVLLUMHUVH1212Control BlockI2C BusRS-232 PORTVideo effects processorDigital photo frameVery fancy digital clockMore IdeasDigital tone controlReal-time audio spectrum analyzerInternet radioSpeech SynthesizerProjects from 2004MIDI synthesizerLine-following robot with video visionSAE student vehicle telemetry systemStereo video vision systemInternet video cameraPac-man-like video gameProjects from 2005Scrabble TimerScorched EarthSAE Auto ShifterInternet RadioBroadcaster3D Maze GameVoIP TelephoneJPEG decoderRally-X video gameProjects from 2006Video-guided Lego Robot360◦camera de-warperVideogame with accelerated line-drawingVoice recorderJPEG decoderProjects from 2007Pac-EdwardsButton Hero VideogameDigital Picture Frame: SD card with JPEG to VGANetworked game of ClueConway’s Game of Life (60 gps)Projects from 2008Real-time ray tracerVideo-camera-controlled pool gameReal-time video decryptionWiiMote-controlled maze gameProjects from 2009Lightsaber video overlayNetworked Video PhoneSound-controlled videogameVisual object trackerThe Three Main Challenges of Embedded SystemsÏCoping with Real-world Sensor DataÏAlgorithm DesignÏImplementation DetailsWhat Does this Circuit Do When You Press the Switch?GND100KVccVoutWhat Does this Circuit Do When You Press the Switch?GND100KVccVoutInside a Pushbutton SwitchA – View of moveablecontact mounted onback side of actuatorThermoplasticActuator ButtonThermoplasticCase and CoverStainlessSteel SpringTerminalsMounting PostsShorting BarSource: Cheery CS series data sheetRaw Data from a CCD (zoomed in)Corrected Image (zoomed in)Correcting Data from CCDsRaw−Dark FrameFlat Field−Bias=Color−→Temp.−→Exposure−→Correcting Data from CCDsRaw−Dark FrameFlat Field−Bias=Color−→Temp.−→Exposure−→Where Does This Noise Come From?Digital camera sensors are remarkably sensitive.My high-end Nikon D300 has a 23.6 mm × 15.8 mm12.3 megapixel CMOS sensor whose pixels are 5.5 µmon a side. When each pixel is sampled with the 12-bitA/D converter, the sensor efficiency isISO: LO 200 400 800 1600 3200G 7.1 5.5 2.7 1.3 0.65 0.33B 5.8 4.6 2.3 1.1 0.55 0.27R 4.7 4.5 2.2 1.1 0.54 0.26The units: electrons per ADU (digital unit).Emil Martinec, A comparison of the Nikon D300 and Canon 40D sensors, 2007.Development Plan1. Obtain some representative raw sensor data2. Develop an algorithmic
View Full Document