Computer Vision: CSE 803Computer VisionGoal of computer visionCritical IssuesRoot and soil next to glassImages: 2D projections of 3DImage receives reflectionsSimple objects: simple image?Where is the sun?CCD Camera has discrete eltsCamera + Programs = DisplaySome image format issuesResolution is “pixels per unit of length”Features detected depend on the resolutionHuman eye as a spherical cameraLook at some CV applicationsAerial images & GISMedical imaging is criticalManufacturing caseSimple Hole Counting Alg.Some hot new applicationsImage processing operationsFind regions via thresholdingExample red blood cell imagesign = imread('Images/stopSign.jpg','jpg'); red = (sign(:, :, 1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'Images/stopRed120.jpg', 'jpg')Slide 26Thresholding is usually not trivialCan cluster pixels by color similarity and by adjacencyDetect Motion via SubtractionTwo frames of aerial imageryBest matching blocks between video frames N+1 to N (motion vectors)Gradient from 3x3 neighborhood2 rows of intensity vs differenceBoundaries not always found wellCanny edge operatorMach band effect shows human biasSlide 37Color and shadingImaging Process (review)Factors that Affect PerceptionCV: Perceiving 3D from 2DMany 3D cuesWhat about models for recognitionSome methods: recognizesummaryStockman MSU/CSE Fall 2009Computer Vision: CSE 803A brief introStockman MSU/CSE Fall 2009Computer VisionWhat are the goals of CV?What are the applications?How do humans perceive the 3D world via images?Some methods of processing images.What are the major research areas?Stockman MSU/CSE Fall 2009Goal of computer visionMake useful decisions about real physical objects and scenes based on sensed images.Alternative (Aloimonos and Rosenfeld): goal is the construction of scene descriptions from images. How do you find the door to leave?How do you determine if a person is friendly or hostile? .. an elder? .. a possible mate?Stockman MSU/CSE Fall 2009Critical IssuesSensing: how do sensors obtain images of the world?Information/features: how do we obtain color, texture, shape, motion, etc.?Representations: what representations should/does a computer [or brain] use?Algorithms: what algorithms process image information and construct scene descriptions?Stockman MSU/CSE Fall 2009Root and soil next to glassStockman MSU/CSE Fall 2009Images: 2D projections of 3D3D world has color, texture, surfaces, volumes, light sources, objects, motion, betweeness, adjacency, connections, etc.2D image is a projection of a scene from a specific viewpoint; many 3D features are captured, some are not.Brightness or color = g(x,y) or f(row, column) for a certain instant of timeImages indicate familiar people, moving objects or animals, health of people or machinesStockman MSU/CSE Fall 2009Image receives reflectionsLight reaches surfaces in 3DSurfaces reflectSensor element receives light energyIntensity mattersAngles matterMaterial matersStockman MSU/CSE Fall 2009Simple objects: simple image?Stockman MSU/CSE Fall 2009Where is the sun?Stockman MSU/CSE Fall 2009CCD Camera has discrete eltsLens collects light raysCCD elts replace chemicals of filmNumber of elts less than with film (so far)Stockman MSU/CSE Fall 2009Camera + Programs = DisplayCamera inputs to frame bufferProgram can interpret dataProgram can add graphicsProgram can add imageryStockman MSU/CSE Fall 2009Some image format issuesSpatial resolution; intensity resolution; image file formatStockman MSU/CSE Fall 2009Resolution is “pixels per unit of length”Resolution decreases by one half in cases at leftHuman faces can be recognized at 64 x 64 pixels per faceStockman MSU/CSE Fall 2009Features detected depend on the resolutionCan tell hearts from diamondsCan tell face valueGenerally need 2 pixels across line or small region (such as eye)Stockman MSU/CSE Fall 2009Human eye as a spherical camera100M sensing elts in retinaRods sense intensityCones sense colorFovea has tightly packed elts, more conesPeriphery has more rodsFocal length is about 20mmPupil/iris controls light entry • Eye scans, or saccades to image details on fovea• 100M sensing cells funnel to 1M optic nerve connections to the brainStockman MSU/CSE Fall 2009Look at some CV applicationsGraphics or image retrieval systems; Geographical: GIS;Medical image analysis; manufacturingStockman MSU/CSE Fall 2009Aerial images & GISAerial image of Wenatchie River watershedCan correspond to map; can inventory snow coverageStockman MSU/CSE Fall 2009Medical imaging is criticalVisible human project at NLMAtlas for comparisonTestbed for methodsStockman MSU/CSE Fall 2009Manufacturing case 100 % inspection neededQuality demanded by major buyerAssembly line updated for visual inspection well before today’s powerful computersStockman MSU/CSE Fall 2009Simple Hole Counting Alg.Customer needs 100% inspectionAbout 100 holesBig problem if any hole missingImplementation in the 70’sAlg also good for counting objectsSee auxiliary slidesStockman MSU/CSE Fall 2009Some hot new applicationsPhototourism: from hundreds of overlapping images, maybe some from cell phones, construct a 3D textured model of the landmark[s]Photo-GPS: From a few cell phone images “the web” tells you where you are located [perhaps using the data as above]Stockman MSU/CSE Fall 2009Image processing operationsThresholding;Edge detection;Motion field computationStockman MSU/CSE Fall 2009Find regions via thresholdingRegion has brighter or darker or redder color, etc.If pixel > threshold then pixel = 1 else pixel = 0Stockman MSU/CSE Fall 2009Example red blood cell imageMany blood cells are separate objectsMany touch – bad!Salt and pepper noise from thresholdingHow useable is this data?Stockman MSU/CSE Fall 2009sign = imread('Images/stopSign.jpg','jpg'); red = (sign(:, :, 1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'Images/stopRed120.jpg', 'jpg')Stockman MSU/CSE Fall 2009sign = imread('Images/stopSign.jpg','jpg'); red = (sign(:, :, 1)>120) & (sign(:,:,2)<100) & (sign(:,:,3)<80); out = red*200; imwrite(out, 'Images/stopRed120.jpg', 'jpg')Stockman MSU/CSE Fall 2009Thresholding is usually not trivialStockman MSU/CSE Fall 2009Can cluster pixels by color similarity and by adjacencyOriginal RGB