ABSTRACTA classical problem of imaging—the matting problem—is sepa-ration of a non-rectangular foreground image from a (usually)rectangular background image—for example, in a film frame,extraction of an actor from a background scene to allow substitu-tion of a different background. Of the several attacks on this diffi-cult and persistent problem, we discuss here only the special caseof separating a desired foreground image from a background of aconstant, or almost constant, backing color. This backing colorhas often been blue, so the problem, and its solution, have beencalled blue screen matting. However, other backing colors, suchas yellow or (increasingly) green, have also been used, so we of-ten generalize to constant color matting. The mathematics of con-stant color matting is presented and proven to be unsolvable asgenerally practiced. This, of course, flies in the face of the factthat the technique is commonly used in film and video, so wedemonstrate constraints on the general problem that lead to solu-tions, or at least significantly prune the search space of solutions.We shall also demonstrate that an algorithmic solution is possibleby allowing the foreground object to be shot against two constantbacking colors—in fact, against two completely arbitrary backingsso long as they differ everywhere.Key Words: Blue screen matte creation, alpha channel,compositing, chromakey, blue spill, flare, backing shadows,backing impurities, separating surfaces, triangulation matting.CR Categories: I.3.3, I.4.6, J.5.DEFINITIONSA matte originally meant a separate strip of monochrome film thatis transparent at places, on a corresponding strip of color film, thatone wishes to preserve and opaque elsewhere. So when placedtogether with the strip of color film and projected, light is allowedto pass through and illuminate those parts desired but is blockedeverywhere else. A holdout matte is the complement: It is opaquein the parts of interest and transparent elsewhere. In both cases,partially dense regions allow some light through. Hence some ofthe color film image that is being matted is partially illuminated.The use of an alpha channel to form arbitrary compositions ofimages is well-known in computer graphics [9]. An alpha channelgives shape and transparency to a color image. It is the digitalequivalent of a holdout matte—a grayscale channel that has fullvalue pixels (for opaque) at corresponding pixels in the colorimage that are to be seen, and zero valued pixels (for transparent)at corresponding color pixels not to be seen. We shall use 1 and 0to represent these two alpha values, respectively, although a typi-cal 8-bit implementation of an alpha channel would use 255 and0. Fractional alphas represent pixels in the color image with par-tial transparency.We shall use “alpha channel” and “matte” interchangeably, itbeing understood that it is really the holdout matte that is theanalog of the alpha channel.The video industry often uses the terms “key” and “keying”—as in “chromakeying”—rather than the “matte” and “matting” ofthe film industry. We shall consistently use the film terminology,after first pointing out that “chromakey” has now taken on a moresophisticated meaning (e.g., [8]) than it originally had (e.g., [19]).We shall assume that the color channels of an image arepremultiplied by the corresponding alpha channel and shall referto this as the premultiplied alpha case (see [9], [14], [15], [2],[3]). Derivations with non-premultiplied alpha are not so elegant.THE PROBLEMThe mixing of several pictures—the elements—to form a singleresulting picture—the composite—is a very general notion. Herewe shall limit the discussion to a special type of composite fre-quently made in film and television, the matte shot. This consistsof at least two elements, one or more foreground objects each shotagainst a special backing color—typically a bright blue or green—and a background. We shall limit ourselves to the case of oneforeground element for ease of presentation.The matting problem can be thought of as a perceptual proc-ess: the analysis of a complex visual scene into the objects thatcomprise it. A matte has been successfully pulled, if it in combi-nation with the given scene correctly isolates what most humanswould agree to be a separate object in reality from the other ob-jects in the scene, that we can collectively refer to as the back-ground. Note that this analysis problem is the reverse of classic3D geometry-based computer graphics that synthesizes both theobject and its matte simultaneously, and hence for which there isno matting problem.There is also no matting problem of the type we are consider-ing in the case of several multi-film matting techniques such as thesodium, infrared, and ultraviolet processes [6], [16]. These recordthe foreground element on one strip of film and its matte simulta-neously on another strip of film.Blue Screen MattingAlvy Ray Smith and James F. Blinn1Microsoft Corporation1 One Microsoft Way, Redmond, WA 98052-6399. [email protected], [email protected] to make digital or hard copies of part or all of this work or personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. © 1996 ACM-0-89791-746-4/96/008...$3.50 2592The problem we address here is that of extracting a matte for aforeground object, given only a composite image containing it.We shall see that, in general, this is an underspecified problem,even in the case where the background consists of a single back-ing color. Note that a composite image contains no explicit infor-mation about what elements comprise it. We use the term“composite” to convey the idea that the given image is in fact arepresentation of several objects seen simultaneously. The prob-lem, of course, is to determine, the objecthood of one or more ofthese objects. In the film (or video) world, the problem is to ex-tract a matte in a single-film process—that is, one with no specialknowledge about the object to be extracted, such as might becontained in a separate piece of film exposed simultaneously in amulti-film process.Now a formal presentation of the problem: The color C = [RG B
View Full Document
Unlocking...