Seam Carving for Content-Aware Image ResizingShai AvidanMitsubishi Electric Research LabsAriel ShamirThe Interdisciplinary Center & MERLFigure 1: A seam is a connected path of low energy pixels in an image. On the left is the original image with one horizontal and one verticalseam. In the middle the energy function used in this example is shown (the magnitude of the gradient), along with the vertical and horizontalpath maps used to calculate the seams. By automatically carving out seams to reduce image size, and inserting seams to extend it, we achievecontent-aware resizing. The example on the top right shows our result of extending in one dimension and reducing in the other, compared tostandard scaling on the bottom right.AbstractEffective resizing of images should not only use geometric con-straints, but consider the image content as well. We present a sim-ple image operator called seam carving that supports content-awareimage resizing for both reduction and expansion. A seam is an op-timal 8-connected path of pixels on a single image from top to bot-tom, or left to right, where optimality is defined by an image energyfunction. By repeatedly carving out or inserting seams in one direc-tion we can change the aspect ratio of an image. By applying theseoperators in both directions we can retarget the image to a new size.The selection and order of seams protect the content of the image,as defined by the energy function. Seam carving can also be usedfor image content enhancement and object removal. We supportvarious visual saliency measures for defining the energy of an im-age, and can also include user input to guide the process. By storingthe order of seams in an image we create multi-size images, that areable to continuously change in real time to fi t a given size.CR Categories: I.3.0 [Computing Methodologies ]: ComputerGraphics—General; I.4.10 [Computing Methodologies ]: ImageProcessing And Computer Vision —Image RepresentationKeywords: Image resizing, Image retargeting, Image seams,Content-aware image manipulation, Display devices1 IntroductionThe diversity and versatility of display devices today imposes newdemands on digital media. For instance, designers must create dif-ferent alternatives for web-content and design different layouts fordifferent devices. Moreover, HTML, as well as other standards, cansupport dynamic changes of page layout and text. Nevertheless, upto date, images, although being one of the key elements in digitalmedia, typically remain rigid in size and cannot deform to fit differ-ent layouts automatically. Other cases in which the size, or aspectratio of an image must change, are to fit into different displays suchas cell phones or PDAs, or to print on a given paper size or resolu-tion.Standard image scaling is not sufficient since it is oblivious to theimage content and typically can be applied only uniformly. Crop-ping is limited since it can only remove pixels from the image pe-riphery. More effective resizing can only be achieved by consider-ing the image content and not only geometric constraints.We propose a simple image operator, we term seam-carving, thatcan change the size of an image by gracefully carving-out or in-serting pixels in different parts of the image. Seam carving usesan energy function defining the importance of pixels. A seam is aconnected path of low energy pixels crossing the image from top tobottom, or from left to right. By successively removing or insert-ing seams we can reduce, as well as enlarge, the size of an imagein both directions (see Figure 1). For image reduction, seam selec-tion ensures that while preserving the image structure, we removemore of the low energy pixels and fewer of the high energy ones.For image enlarging, the order of seam insertion ensures a balancebetween the original image content and the artificially inserted pix-els. These operators produce, in effect, a content-aware resizing ofimages.We illustrate the application of seam carving and insertion for as-pect ratio change, image retargeting, image content enhancement,and object removal. Furthermore, by storing the order of seam re-moval and insertion operations, and carefully interleaving seams inboth vertical and horizontal directions we define multi-size images.Such images can continuously change their size in a content-awaremanner. A designer can author a multi-size image once, and theclient application, depending on the size needed, can resize the im-age in real time to fit the exact layout or the display.Seam carving can support several types of energy functions such asgradient magnitude, entropy, visual saliency, eye-gaze movement,and more. T he removal or insertion processes are parameter free;however, to allow interactive control, we also provide a scribble-based user interface for adding weights to the energy of an imageand guide the desired results. This tool can also be used for author-ing multi-size images. To summarize, our main contributions are asfollows:• Define seam carving and present its properties.• Present algorithm for image enlarging usin g seam insertions.• Use seams for content-aware image size manipulations.• Define multi-size images for continuous image retargeting.2 BackgroundImage resizing is a standard tool in many image processing appli-cations. It works by uniformly resizing the image to a target size.Recently, there is a growing interest in image retargeting that seeksto change the size of the image while maintaining the important fea-tures intact, where these features can be either detected top-downor bottom-up. Top down methods use tools such as face detectors[Viola and Jones 2001] to detect important regions in the image,whereas bottom-up methods rely on visual saliency methods [Ittiet al. 1999] to construct a visual saliency map of the image. Oncethe saliency map is constructed, cropping can be used to displaythe most important region of the image. Suh et al. [2003] pro-posed automatic thumbnail creation, based on either a saliency mapor the output of a face detector. The large image is then croppedto capture the most salient region in the image. Similarly, Chen etal. [2003] considered the problem of adapting images to mobiledevices. In their approach the most important region in the imageis automatically detected and transmitted to the mobile device. Liuet al. [2003] also addressed image retargeting to mobile devices,suggesting to trade time for space. Given a collection of regions ofinterest, they
View Full Document
Unlocking...