EE368 PROJECT 1Pictures at an ExhibitionHyukjoon Kwon, and Jongduk BaekStanford University, Stanford, CA 94305Email: {hjkwon, bjd1219}@stanford.eduAbstract— This project is to recognize an input image throughthe database which are learned from a set of training images.The first thing we have done is to extract only the main image.This step requires an adaptive threshold to decide whether theselected region is a painting we want or not. The morphologicaloperation follows the filtering process. The edges of the paintingextracted in the source image are dilated and eroded. As a result,the four corner points of centered images are searched by usinga radon transform. Inverse projective spatial transform leads usto obtain eigenimages, which enable to form a database. Thetest input is calculated the correlation values with eigenimagein database. The result shows that learning process with threeimages recognize the test input with one hundred probability.I. INTRODUCTIONImagine you are in the Canon Art Center, while seeing oneof European classic paintings. If you are not an expert in thearea of European paintings, you might feel you need some helpto explain a painting you are seeing now. The basic idea of anelectronic museum guide is derived from the need of peoplewho wants to appreciate paintings deeply. This is one exampleof typical applications called ”augmented reality” applicationswhich has been researched in computer vision.The purpose of this project is to develop a core part of thetechnique used in this computer vision application, which isthe recognition of paintings taken by a camera installed in thedevice. We are given a set of training images that we shouldrecognize. An example image set is figure 1. Each image isnot guaranteed to be taken properly. Therefore, some imagescan have an extra part of other images or a decoration ofmuseum and other images can be taken with an askew angle.In the procedure of recognition, those impediments are factorsto make it difficult to recognize perfectly.Fig. 1. The Example of Training ImagesThe algorithm we are using in this paper is to extract acentered image, which we want to recognize, by an imagemask using an adaptive threshold for each image and todistinguish one image from others by comparing correlationvalues calculated from an eigenimage method. The reason whythe adaptive threshold is need is that one simple criterion doesnot satisfy all of images’ conditions. Some of them are exposedinto a strong intensity of light. Others are not. In addition,some paintings have distinct colors against walls or frames,but others do not. Hence, the pre-process step with an adaptivethreshold is required in order to extract a proper eigenimage.The outline of this paper is as follows. Section II provides analgorithm used for segmentation based on color distribution.In section III. we perform the morphological processing toobtain the corner points which become input points for spatialtransform used in section IV. Section V shows the eigenimagesextracted from a set of training images and compares thecorrelation among training images. Finally, conclusions aregiven in Section VI.II. COLOR BASED SEGMENTATIONIn order to recognize the centered painting, what we shoulddo first is to distinguish a painting from a non-painting. Thisprocess can be approached with two ways. The first is toextract the frame color based on the fact that all the imagesare enclosed with an wooden frame. They are almost in anrange of green color. The second is to select the color of wallas a criteria to distinguish an image from a non-image.Fig. 2. The CbCr of Frame ColorIn pursuing the goal, we have to decide which color formatis used. Typically, the input color image is in the RGB format.However, RGB components are heavily dependent on theEE368 PROJECT 2light intensity conditions. Therefore, the same painting can berecognized as a different image depending on the light locationand light intensity. So, this project decides to use YCbCrcomponent. In the YCbCr format, the luminance componentY can be seperated with other chrominance components, Cband Cr. Therefore, the reference colors of paintings are chosenin a format of Cb and Cr.Fig. 3. The CbCr of Wall ColorAs investigating the color distribution of frames (figure 2)and walls (figure 3), we conclude that the second methodmentioned above is more proper to distinguish the paintingswith non-paintings.The wall histogram shows that the means of Cb, Cr are125.7468, 131.2772 and the standard deviations of Cb, Cr are1.5714, 1.9351. On the other hand, the frame has a distributionwith larger standard deviations of Cb, Cr, which are 4.0748 and5.4118. Therefore, it is reasonable to select a criteria havingless variation.Fig. 4. The CbCr of Wall ColorThis color segmentation technique has been applied to a setof training images. As a result, the mask binary images canbe obtained in figure 4 The leftmost columns show the maskin the first step of the procedure. The mask region is chosenwith Cb, Cr values within a mean ± default-range-factor ×standard deviation.However, every image does not guarantee the proper maskin the first step. Therefore we consider the adaptive algorithmwhich decides the range of mask region based on the selectedregion in the previous step. The rightmost column is the maskin the final step of an adaptive selection.Table 1. Adaptive Region Selection AlgorithmInitialization:choose default range factor based on its averagemin-cb = mean-cb − std-cb ∗ range-factormax-cb = mean-cb + std-cb ∗ range-factormin-cb < Cb < max-cbmin-cr = mean-cr − std-cr ∗ range-factormax-cr = mean-cr + std-cr ∗ range-factormin-cr < Cr < max-crRecursion:1. the region around the centroid of frame is filledless than a threshold2. the convex hull of selected region to a bounding boxof the region has a smaller ratio than a threshold3. the selected pixel count is less than a thresholdRepeat:change the range-factor with delta and repeat the procedureThe bounding box area of the selected mask is applied toa set of training images. In the middle of the procedure, wecan confirm that the centered image is extracted excluding theside painting or the obstruction.Fig. 5. The Extracted Centered PaintingIII. MORPHOLOGICAL PROCESSINGA. Edge DetectionThe next thing we have to do is to transform an extractedcentered image to a fixed, normalized rectangular image,which enables us to calculate an eigenimage correlation. InEE368 PROJECT 3order to do a