EE368 Project: Visual Code Marker DetectionKahye SongGroup Number: 42Email: [email protected]— A visual marker detection algorithm has been im-plemented and tested with twelve training images. The algorithmfollows the general scheme suggested in [1] but the details havebeen modified to gain more robust estimation. The algorithmperforms successfully for most of images but the result showsthat it does not handle a marker which has been projected toan image plane with an angle deviated from the normal anglesignificantly.I. INTRODUCTIONThe task of the project is to detect and read visual codemarkers through the cell-phone camera. Visual markers can beused as a hyperlink to a database which has the informationof the object described by the marker. In the paper titled”Real-World Interaction with Camera-Phones”, the author,Michael Rohs, describes the use of visual code markers ininteracting with real-world objects.In this project, a visual marker detection algorithm has beenimplemented based on [1] together with some modifications.The algorithm has been tested by a set of twelve trainingimages and the corresponding ground truth data and the resulthas been evaluated using the provided evaluating function.The visual code markers are 2-dimensional arrays. Thearray consists of 11x11 elements. Each element is eitherblack or white. As shown in the figure below, we fix theelements in three of the corners to be black. One verticalguide bar (7 elements long) and one horizontal guide bar (5elements long) are also included. The immediate neighborsof the corner elements and the guide bar elements are fixedto be white. This leaves us with 83 data elements which canbe either black or white.Fig. 1. Structure of visual code markerExploiting the structure of the marker, the algorithm triesto find the guide bar candidates and pair them based onthe length ratio of the bars and the angle between the bars.When the visual marker locations are estimated, the cornersare verified by checking whether the actual correspondingcorners exist. Then all pixels are projected back to the codeplain and decoded by counting the black pixels in the markerimage corresponding each code coordinate.II. METHODOLOGYIn this section, the whole procedure of the algorithm isdescribed in the original sequence.A. Gray scale and binary conversionThe 24bit RGB image is transformed into a gray-scaleimage by taking the average pixel values of the red andgreen channel at each position. The blue channel has beendiscarded due to poor resolution. Then this gray scale imageis thresholded by the adaptive thresholding scheme whichwas given in [1]. The width of moving average s is set toimagewidth16to improve locality of the averaging and t as 20 toavoid blurred outline of features to be set to 1, which falselyconnects regions.Gray scaled training image 9Fig. 2. Gray-scaled training image 9B. Region labeling and find high eccentricity regionsEach 1-bit valued(black) pixels are aggregated and labeledas a region if they are connected by 4-connectivity. Then eachBinary version of image 9Fig. 3. Binary version of training image 9linear correlation of pixels in each region is computed andthose with high absolute correlation are selected as guide barcandidates. For further testing, the direction/angle of the regionand the length along this direction are estimated by finding themaximum and minimum x, y coordinate values of the region.This is because the bars that we are looking for can fit in arectangle formed by max/min x and y coordinates as in Fig.C. Test1: Find normal angle pairsSince the guide bars form a right angle, we can find pairs ofregions which are located in a normal direction to each otherby using the region direction computed in II-B. Pick the pairswith large angle separation near 90 degrees. However, not tomiss the original pairs with poor direction estimates, the pairsof angles between 55 and 90 degrees are saved at this stage.D. Test2: Find pairs satisfying Pythagorean relationsTo check whether each pair of bars are located close enoughand to check if they can form a right triangle, the Pythagoreanrelations of each pair is tested as they do in Fig 1. amongthose selected in II-C. This is done by looking for the smallestd = c2− a2− b2, where c is the side apposite to the right angle.The distance between the centers of the regions should not betoo close below the decodable resolution limit(The smallestdecodable marker can have 11 by 11 pixels.) and the threeestimate side length should also meet the triangle criterionwhich is c < a + b.E. Test3: Find the pairs with 5:7 length and area ratioAmong pairs selected in the first test, the ratio of length andarea of regions are tested if they are close to 7:5. The pairswhich have each ratio close to75by 0.6 are selected. Thenthe common pairs selected by this test and 50 of those whichhave the smallest d values from the test in II-B.F. Test4: Find the bottom right corner of the visual markercandidatesThe pairs passed the tests up to Test3, they are consideredto be guide bars. Bars in each pair are sorted as bar1 andbar2. Bar1 is the 7 bit length bar and Bar2 is the 5 bit one.Then we can find the bottom left corners of each markercandidate by simply finding the closest pair of ending pointsof bar1 and bar2 and the corresponding bar2 ending pointshould be the bottom right corner of the marker. At this stage,those pairs with too large separation of these ending points tocompare with the length of bar1 and bar2 are removed sinceaccording to the structure, they should be 1 bit length away.Then the bar1 direction is set from this corner to upward andthe direction of bar2 is from this corner to left. Refer to thefigure below.Fig. 4. Square fitting of each regionsG. Test5: Outer product testThe two bars should be located in an order to form anegative outer product in image domain, refer to Fig. 1., which means if we rotate bar1 counterclockwise by 90degrees it should overlap with bar2 not the other way around.Therefore those pairs which do not meet this criterion areremoved.H. Test6: Remove not very square like cornersSince we know that the whole mark size is 11 by 11, wecan estimate preliminary positions of the other corners byadding bar1 and bar2 direction vectors with right scaling.Then we can check whether this polygon formed by cornerestimates are actually approximately square by looking atthe ratio of the sides. The angle has been already tested inII-C. Those pairs which do not