Max-David Ghozlan EE392JDigital Video ProcessingWinter 00Final project reportRestoration Experiments and ObservationsThe ProblemWe here give a brief introduction to the restoration problem, following both historical and thematic logics.• The origins of restoration: astronomical imagingInvestigations related to restoration began primarily with the efforts of space scientists involved in US andformer Soviet Union programs in the 50s and the early 60s. These programs were responsible for producingimages of the Earth and of our solar system. However, the images obtained from the various planetarymissions of the time, such as Ranger, Lunar Orbiter and Mariner missions were subject to manyphotographic degradations.These were the result of sub-standard imaging environments, the vibration of the machinery and thespinning and tumbling of the spacecraft. Pictures of the later manned space where also blurred due to theinability of the astronaut to steady himself in a gravitationless environment while taking photographs.And the degradation of the images was not a small problem considering the expense required to obtain suchpictures in the first place. For example, the 22 pictures produced during the Mariner IV flight to Mars in1964 were estimated to cost almost $10 million just in terms of the number of bits transmitted alone. Anydegradation reduced the scientific value of these images considerably and clearly cost the space agenciesmoney.Recently, the well-publicized problems with the initial Hubble Space Telescope (HST) main mirrorimperfections have provided an inordinate amount of material for the restoration community over the lastfew years.• Other areas where restoration plays an important roleMedical imagingRestoration has been used for filtering of Poisson distributed film-grain noise in chest X-rays,mammograms and digital angiographic images, and for the removal of additive noise in MagneticResonance Imaging (MRI). Some other application are emerging such as quantitative autoradiography(QAR). QAR is a technique which provides a higher resolution than others such as positron emissiontomography (PET), X-Ray computed tomography (CAT) and MRI, but still needs to be improved inresolution in order to study drug diffusion and cellular uptake in the brain.Movies, aging and deteriorated filmsRestoration has also received some attention in the media, and the particularly in the movies of the lastdecade. For example, in 1991, the movie "JFK" made substantial use of the famous Zapruder 8mm film ofthe assassination of President Kennedy, which has been enhanced and restored many times over the years.Similarly, some important work has been done to produce the digital restoration of the film "Snow Whiteand the Seven Dwarfs", which was originally premiered in 1937.Image and video codingAs techniques are developed to improve coding efficiency and reduce the bit rates of coded images,artifacts such as blocking become quite a problem. Already, much has been done to model these types ofartifacts and develop ways of restoring coded images as a post-processing step to be performed afterdecompression.ExperimentsThe structure of this suite of experiments has been derived from [1].• Blurring of an imageWe have used two types of blurs in these experiments: a uniform 2-D blur and a gaussian blur. They can beeasily activated using a flag.Here are the structures of the blurring filter, h:* Uniform filter:h(i,j) = 1/(L2) if -L/2 < i,j < L/20 otherwiseNote that when implementing this filter, we are force to shift the indices:h(i,j) = 1/(L2) if -0 < i,j < L0 otherwiseand therefore lose some symmetry characteristic.* Gaussian filterh is gaussian of dimension L and covariance matrix L/2*I2 where I2 is the 2D identity matrix.In both cases, the blurring function is parametrized by a factor L that is related to its magnitude.In the case of the gaussian blur, L gives the covariance matrix that defines the gaussian function, whereasin the case of the uniform 2-D blur, L gives the size of the blur. To clarify the explanations that will follow,We have called L0 the parameter, as an initial known parameter.Now to obtain the blurred image, there are two ways to proceed.It's possible to work in the space domain and convolve the filter with the image. This leads to some shiftingof the image, given the nature of the convolution numerical operation.y = h*f (1)where: f is the original imagey is the blurred imageIt's also possible to work in the frequency domain: multiply the Fourier transforms and take the resultinginverse Fourier transform.y = T-1(T(h).T(f)) (2)where stands for the Fourier transform.• MetricsAs a common metric to judge of the quality of the restoration technique, we used the psnr metric.Note that this metric assumes the knowledge of the original unblurred image, since it compares that imagewith the newly restored image.• Addition of noiseInitially, we have performed the experiments without adding any noise to the image resulting from theconvolution of the original image with the blurring filter. But then, in a second phase, we have added somewhite gaussian noise, n, parametrized by the value of its standard deviation sigma.n = N(0,sigma^2)y = h*f + n• UnblurringWe have used two different methods for unblurring the image1. Unblurring subject to the knowledge of the original picture (naive unblur)As the title of the paragraph suggests, this technique assumes knowledge of the original picture beyondassuming the knowledge of the blurring structure. Indeed, it consists in inversing blurring functionscorresponding to various values of the parameter L. Then, the value of L that produces the best psnr is keptand is also compared to the parameter L0, characteristic of the initial blurring filter.2. Unblurring that doesn't assume the knowledge of the original picture (unkown_unblur, case of theuniform blur)Here we have worked only with uniform blurs. Indeed, the technique consists in comparing the structures,in the frequency domain, of the blurred image and of several blurring filters to find out which one fits bests.In the case of uniform blurring, the structure of the frequency representation of the blurring function iswell-known: it is a 2-D sinc. This has thus led us to adopt the following method: we have taken a cut of theFourier transforms and then compared the values of their first zeros (a zero being a point where thefunction is equal to zero).Again, we have looped through several values of L. Also,