MAS 131 MAS 531 Separating Transparent Layers in Images Anonymous MIT student Abstract This paper presents an algorithm to separate different transparent layers present in images of a 3 D scene The criterion of minimum mutual information is presented and evaluated We nd that high frequency details are faithfully recovered while the recovery of low frequency details needs improvement I NTRODUCTION Typically when we image a 3 D scene using a camera different 2 D layers of the scene that are at different distances from the camera are superimposed For example consider the images shown in Figure 1 The two images belong to the same scene containing a chair and a le cabinet However the images are captured by focussing at different depths The surface that is in focus is sharp and the surface that is out of focus is blurred a The le cabinet is in focus b The chair is in focus Fig 1 An example illustrating superposition of transparent layers in an Image Such images are often encountered in microscopy while studying the anatomy of various organisms Different slices of cells or tissues are captured by focussing at different depths In order to study any particular 2 D slice from such images it is important to separate the effect of other 2 D layers that are super imposed in the images T HE MODEL Let a 2 D layer f1 x y interfere with a 2 D layer f2 x y We consider the slices ga x y and gb x y in which either layer f1 x y or layer f2 x y respectively is in focus The other layer is blurred Modeling the effect of blur as a convolution with blur kernals we have ga x y f2 x y h2a x y f1 x y 1 gb x y f1 x y h1b x y f2 x y where denotes convolution and denotes multiplication The above model does in fact describes the images shown in Figure 1 For example consider a region in Figure 1 b where the chair has holes In that case we are able to see through a blurred version of the le cabinet The regions where there are no holes in the chair in Figure 1 b we do not see through any version of the le cabinet And hence the multiply after convolution model We assume that the blur kernals are space invariant for constant depth objects Moreover if the system is telecentric then h1b x y h2a x y h x y P RIOR A RT Separation of different layers in images has been studied before In 1 2 the authors propose the use of Independent Component Analysis 3 4 to separate re ections from Images that are caputered with a polarizer Using local features to perceive transparency in images has been proposed in 5 6 The closest to our work is 7 8 where the authors propose separation of transparent layers using a different model for superposition The novelty of our work lies in the formulation of the transparent layer interference as a mutliplication of images as opposed to the standard approach of summation O UR APPROACH The two equations in eq 1 have four unknowns namely f1 x y f2 x y h2a x y and h1b x y And hence this is a highly under determined system To solve the above system of equations we make the assumption that h1b x y h2a x y h x y This is a reasonable assumption especially for telecentric images 7 8 Under this assumption the resulting model is given by ga x y f2 x y h x y f1 x y 2 gb x y f1 x y h x y f2 x y We are still left with three unknowns and two equations Here we leverage the idea of minimum mutual information that was proposed in 7 8 The underlying intuition is that if f 1 x y and f 2 x y are the solutions to the equations 2 then f 1 x y and f 2 x y have the least mutual information I f 1 x y f 2 x y among all possible solutions to equations 2 We use this criterion to nd the optimal h x y that yields minimum mutual information between the recovered images f 1 x y and f 2 x y Even given h x y nding the solution to the equations 2 is challenging Our algorithm for separating f1 x y and f2 x y is given below 1 Assume a blur kernal h x y and solve for f 1 x y and f 2 x y using equations 2 This is not straightforward from the equations themselves We nd f 1 x y and f 2 x y iteratively from equations 2 as follows a Initialize f 1 x y 1 and f 2 x y 1 b Iteratively update f 1 x y and f 1 x y as f 1 x y ga x y f 2 x y h x y gb x y f 2 x y f 1 x y h x y c Return f 1 x y and f 2 x y after ten iterations 2 Calculate the normalized minimum mutual information 1 x y f 2 x y I f In f 1 x y f 2 x y H f 1 x y H f 2 x y 2 where H f 1 x y P f 1 x y log P f 1 x y f 1 x y I f 1 x y f 2 x y P f 1 x y f 2 x y log f 1 x y f 2 x y P f 1 x y f 2 x y P f 1 x y P f 2 x y 3 Change the blur kernal h x y and return to step 1 In our problem we assume that the blur kernels are Gaussian and are parameterized by their standard deviation We iterate over the steps 1 3 mentioned above by changing The nal estimates of the separated layers are the ones that result in the least normalized mutual information across all the runs For more details refer to the M ATLAB code in the appendix E XPERIMENTAL E VALUATION We generated multi layered images according to equations 2 by combining two sharp images as shown in Figure 4 We used 0 7 for the images The calculated normalized mutual information is shown in the Figure 2 We observe that the algorithm correctly estimates to be 0 7 The recovered images are shown in Figure 4 We nd that the image recovery preserves the high frequency information but performes poorly in the low frequency components Shown in Figure 3 are the x gradients for a particular row for both the original image and the recovered image We nd that the gradients are very accurate con rming that the high frequency components are well preserved The loss in the low frequencies is retained even if the iterations in the algorithm mentioned above were initialized with better estimates for f …