Solving Transparency By Timothy BahlsAbstract:A little bit of physics and math with a smattering of computer science are laid out to begin an approach to handling transparency. Introduction:Computer Vision and Computer Graphics are opposites, an thus highly related to each other. The central problem in graphics is to take a high level concept, create the geometry for it, and reduce it to an image. The central problem in vision is taking an image, deducing its geometry and ultimately deducing the high level concept the image embodies.Since the latter is by far the harder of the two, it can at times help to think of vision in terms of graphics—what input could’ve produced the image I’m currently seeing?This paper tries to formalize a simple problem in transparency and tries to understand how light and transparency interact Motivation:If we assume that dealing with transparency will eventually become important to computer vision, it will be useful to have a head start at the field. This paper lies out a little bit of theory as a starting point.Problem Statement:Instead of trying to find out which regions belong to which, I’ll start with the assumption that we already know where the regions are. It’s a very big assumption, butwe have to start somewhere. We will not start out with knowledge of the depth, transparency, or color of each region, however.Here’s what I’m aiming for: given a set of overlapping regions semitransparent regions, provide a relative ordering. We assume that each region is at a single depths—no need to worry about cyclical overlaps yet. We also assume that each region is a single color—just as segmentation does not try to determine which segments are part of the same object, we’ll let some higher level work with that.Related work:The restrictions placed on the problem owe much to the field of Mondrian Motion, a field I explored with D. Scharstein in the summer of 2004.TheoryLet’s formalize what we want, starting without opacity:We start out with some set S of regions and an image. Each region is an ordered triple (S, d, c) where d is the distance from the region to the camera, c is the color and S is some subset of P = {(x, y): 0 <= x < width, 0 <= y < height}. Our image is really a function I from P into the set of all colors. Without transparency, the value of I at some pixel p is the color of the closest region to the camera that contains p.Once we add transparency, our regions become ordered quadruplets (S, d, c, t) that add some measure or transparency. Then or task becomes much more difficult.We will consider two models of transparency—additive and subtractive.Additive:So how is transparency handled in the ubiquitous photo editing programs? The easiest and most intuitive way to handle transparency is a percent. If the object is x percent transparent, then we can see the appearance as weighted average between the foreground and the background--x percent background, (1-x) percent foreground. I demonstrate this below. Circle A is rgb(0, 0, 1). Circle B is rgb(1,0,0). The background is rgb(1, 1, 1). Both circles are 50% transparent. Thus A appears .5 * rgb(0, 0, 1) + .5 * rgb(1, 1, 1) = rgb(.5, .5, 1). Similarly, B appears to be rgb(1, .5, .5).The overlapping section on the left is .5 * rgb(1, .5, .5) + .5 * rgb(0, 0, 1) = rgb (.5, .25, .75). The one on the right is rgb(.75, .25, .5). In the second image, we can see that once again, the central color depends on the ordering of the regions.Let’s run through a little math, okay? Suppose that we have two overlapping regions A and B over the background C. Suppose we can see A over C, B over C, C by itself, and all three stacked on top of each other. Respectively, let’s call the color of theseregions Ra, Rb, Rx, and Rab. Who’s on top—A or B?Let the color of A and B be denoted Ca and Cb. Let their transparencies be denoted Ta and Tb.Suppose that A were on top. Then:Ca = Ta * Cx + (1-Ta) * CaCab = Ta * Cb + (1-Ta) * CaSolve both sides for (1 – Ta) * Ca:Ca - Ta * Cx = (1-Ta) * CaCab - Ta * Cb = (1-Ta) * CaSet the two sides equal and solve:Ca - Ta * Cx = Cab - Ta * CbCa – Cab = (Cx – Cb) * TaTa = (Ca-Cab) / (Cx – Cb)Symmetrically we can find that if B is on top Tb = (Cb-Cab) / (Cx – Ca). But how do weknow which one is correct? Well, this is the time to remember that we have three equations—one for red, one for green, and one for blue. We solve all three equations for each color. Since the transparency value should match over color, we should be able to find a depth ordering based on whichever matches better.Subtractive:The previous results are quite neat, but high school physics might disagree. If we view partially transparent objects as filters, our task becomes much different. Take a lookat the image below:Science teaches us that only red light can pass through a red filter. Similarly, onlyblue light can pass through a blue filter. Thus no light passes through both, so the intersection appears black. It is a bit worrisome that the left and right images look the same, isn’t it?On this model, we can trace each color of light independently. We can count the number of layers that the light must pass through and multiply together the percentage of lights that pass through. Since multiplication is commutative, we have no hope of recovering a depth order. Does that mean science is telling us to give up?Combining the two:Not so. Compare the diagram above to the method we used earlier:The one below looks a bit more realistic, doesn’t it? Here’s the key—both ways of counting are important. The subtractive method only considers light transmitted though the object and erroneously assumes that no light is reflected. The additive method considers both reflected light and transmitted light, but it erroneously assumes that all colors of light are transmitted equally well through an object.Light transmits or reflects the light it can’t absorb. Thus it is probably safe to assume that an object reflects and transmits the same colors of light. Thus a better representation of light may be this one:HSRT where H=hue, S=saturation, R=reflectance, T= transmittance. Reflectance and transparency are both percentage that must add to less than one. This doesn’t includethe distinction between specular and diffuse lighting, but it should work for now.We should define a way to multiply colors together. It is simple in RGB—we convert our range to