Facial Blemish Removal using Canny Edge Detection and Gaussian BlurringECE533 ProjectFall 2005By:Joseph MarinoGregory Yoblin1Problem Statement:The goal of this project was to design a product capable of automatically removing facial blemishes. It would do this by locating “blemishes” within an image via Canny Edge Detection. After locating the “blemishes,” the program would remove them and fill them in with a value that would render the removed “blemish” undetectable to the human eye.Motivation:Every person in the world has days where they don’t look as good as they want to. Usually, those days occur at the least opportune moments: days when pictures are being taken. Currently, there are several methods of removing these “blemishes.” Some people simply take a patch of skin that looks similar to the region where there is a blemish and paste it over the blemish. Some programs allow the user to “erase” the blemish using some sort of blurring function of varying sizes. However these methods are very time consuming for large numbers of blemishes, and they require a very artistic eye to remove them properly. By creating a program that will work with larger areas to remove multiple blemishes at once, we hope to greatly reduce the amount of effort required to clean up images after they have been taken. Our program is not designed to be the only tool, rather it is very useful as a preprocessing step in the touch-up process. Unfortunately cleaning up an image is a very aesthetic process, and there is no way to quantify what an ideal output would be. As such, our program will not produce ideal results all of the time. But it does a very good job at removing batches of blemishes in open areas.Approach:The project was implemented incrementally to allow for isolated testing and debugging, and to maintain a functional code set at all points during the project. This process began with determining a method for “removing” a blemish from a picture. Initial research had revealed two 2potentially effective methods, linear interpolation via Laplace's equation and filling with the regional average value. Both would be smoothed with a Gaussian filter. The next step was selecting a set of edge detection and morphological operators to produce a blemish mask, a binary image with ones over the blemish pixels to be removed, from a manually selected sub-image. Those pieces were combined to form a program capable of removing blemishes from a manually selected region. Finally, the previous components would be modified for full automation in a block-processing or sliding-window type algorithm.Work Performed:This project was implemented in Matlab functions. Coding began with the makeGaussianMask() function, as either blemish removal method would require smoothening afterward. From there, informal experiments were performed on a sampling of target images where blemishes were manually selected and removed by the two methods described above in the approach. Filling with the average value was unanimously chosen as the more aesthetically pleasing option, and this decision allowed moving forward to write the blemish removal code.The next obstacle was selecting a method of edge detection and the subsequent morphological operators to extract a blemish mask from a sub-image containing a blemish. This called for further experiments, which began with a comparison of Sobel, Prewitt, Canny, and Laplacian of Guassian edge detection results when used on typical sub-images. Of these methods, Canny consistently gave “good” results, in which there would be a connected or nearly connected set of pixels around the boundary of the blemish along with some stray lines unrelated to the blemish. Utilizing contrast stretching as a preprocessing step helped improve the Canny edge detection results further.3Finding the right set of morphological operations was a guess and check type procedure. The best result was obtained using closing to connect any nearly connected blemish boundaries, followed by region filling on the background which when inverted gives the interior pixels of the blemish(s), and finished by opening to remove the stray lines. The results of these steps are shown in Figure 1. During this phase, manual sub-image selection was assumed which allowed the region filling algorithm to use the upper left corner as the starting pixel inside the region to fill. However, this choice would require reconsideration during the automation step. To combine the previously completed segments into a manual selection based blemish removal function, a sub-image selection function getLittleOne() was created and some glue code was added into the mblem() function allowing the processed sub-image to be reinserted into the original picture.Steps toward automation began with creating a “no touch” masking option, so features like eyes, nostrils, and lips could be manually marked as non-blemish regions the algorithm would ignore. Next, several methods of block processing were compared, and the poor results showed a need for adjustment to the blemish detection and removal algorithm for use in an automated setting.Results:Blemish removal in manually selected sub-images was successfully implemented. By choosing a sub-image such that no blemishes touch the edge, the manual method works well and can be 4Figure 1: Contrast Stretching, Canny Edges, Closing & Filling, Openingrepeated for removing multiple blemishes in spatially distant regions. Results of such a use can be seen in Figures 2 and 3 where blemishes on the cheek and chin area have been removed separately. However, when used to block process an entire picture the inability to determine blemish from natural feature greatly diminishes the resulting picture. Blemishes are removed along with other features meant to be left untouched.Discussion:After testing, many things became apparent. First and foremost, the program that we designed is not something that can be used as the sole means of removing “blemishes.” However, it can be used as a wonderful preprocessing step. Given more time, many of the limitations that stop this program from being what we envisioned it to be would be able to be fixed. We will discuss several of them here. To start out with, the program was supposed to be a fully automatic process. However, when we attempted to use this program in a block processing style on a full image we ended up with a
View Full Document
Unlocking...