NPR TodayProcessing Images and Video for an Impressionist Effect by Peter Litwinowicz, Siggraph 97Problem StatementPrevious workStroke renderingExampleBrush stroke orientationFrame-to-frame coherenceVideoConclusionFuture workSlide 12Painterly RenderingSlide 14Previous WorkThe Algorithm in DetailThe Algorithm cont...Slide 18Putting it all togetherCreative techniquesTechnical considerationsFuture DirectionsQuestionsSlide 24Slide 25Cartoon SmokeRelated WorkSmoke SimulationSlide 29RenderingSlide 31Simulation-Rendering InterfaceResultsConclusionsSlide 3504/04/05 © 2005 University of WisconsinNPR Today•"Processing Images and Video for an Impressionist Effect", Peter Litwinowicz, Proceedings of SIGGRAPH 97•"Painterly Rendering for Animation", Barbara J. Meier, Proceedings of SIGGRAPH 96•"Cartoon Rendering of Smoke Animations", Andrew Selle, Alex Mohr and Stephen Chenney, Proceedings of NPAR 2004April. 4, 2005 © 2005 University of WisconsinFeng LiuDepartment of Computer ScienceUniversity of Wisconsin-MadisonProcessing Images and Video for an Impressionist Effect by Peter Litwinowicz, Siggraph 97April. 4, 2005 © 2005 University of WisconsinProblem Statement•Transform images/video into animation with Impressionist effect, particularly, with hand-painted styleApril. 4, 2005 © 2005 University of WisconsinPrevious work•Interactive computer-assisted techniques [Haeberli 90]•Interactive pen-and-ink illustrations making method [Salisbury 94]–Scale-dependent image reproduction method [Salisbury 96]•2-1/2D animtion method [Hsu 94]. • System for transforming 3D-gemetry into animations [Meier 96]April. 4, 2005 © 2005 University of WisconsinStroke rendering•Stroke generation–Size, position, length –color–Orientation•Random perturbation•Clipping and rendering–Edge preservation•Using brush texturesApril. 4, 2005 © 2005 University of WisconsinExampleWithout clipping With clippingApril. 4, 2005 © 2005 University of WisconsinBrush stroke orientation•Draw stroke in direction of constant color–the normal to the gradient direction•Area with small magnitude of gradient ?–Interpolate surrounding “good” gradientApril. 4, 2005 © 2005 University of WisconsinFrame-to-frame coherence•How to move strokes across frames–Using Optical flow [Bergen 90] as stroke displacement •How to avoid over-sparse and over dense stroke distribution?–Delaunay triangulation–Maximal area –Minimal distance (a) (b) (c) (d) (e)April. 4, 2005 © 2005 University of WisconsinVideoApril. 4, 2005 © 2005 University of WisconsinConclusion•An algorithm for producing painterly animation from video•Highlights–Use optical flow to move strokes across frames to keep temporal coherence–Orient strokes using gradient-based methods–Methods to redistribute strokes –Edge preservation strategy•Drawback–jitteringApril. 4, 2005 © 2005 University of WisconsinFuture work•Other artistic style•Apply methods to 3D objects•Reducing jittering04/04/05 © 2005 University of Wisconsin04/04/2005© 2005 University of WisconsinPainterly Rendering•Goals–Avoid “shower-door” effect–Provide for frame-to-frame coherence•Previous techniques achieved one or the other04/04/2005© 2005 University of WisconsinPainterly Rendering•How to achieve goals:–Use object geometry, color to decide where to place strokes–Distribute particles on object surface–Paint in screen space whereever a particle is placed•Randomness adds character–Store random seed in “particle”–Perturb color, orientation, scale based on user-selectable parameters04/04/2005© 2005 University of WisconsinPrevious Work•"Paint by Numbers", Paul Haeberli, SIGGRAPH 90, 207-214–Similar painterly style, brush strokes stuck to view-plane•"Comprehensible Rendering of 3-D Shapes", Saito and Takahashi, SIGGRAPH 90, 197—206.–Used reference images, G-Buffers to generate illustrative image•“Cellular Texture Generation”, Fleischer, et al. SIGGRAPH 95, 239-248.–Elements rendered in model space•Every oil-painting book ever made...04/04/2005© 2005 University of WisconsinThe Algorithm in DetailStep 1: Create particles to represent geometry04/04/2005© 2005 University of WisconsinThe Algorithm cont...Step 2: For each frame of animation...–create reference pictures using geometry, surface attributes, and lighting04/04/2005© 2005 University of WisconsinThe Algorithm cont...Step 3: Also for each frame of the animation... –transform particles based on animation parameters–sort particles by distance from viewpoint–for each particle, starting with furthest from viewpoint•transform particle to screen space•determine brush stroke attributes from reference pictures or particles and randomly perturb them based on user-selected parameters•composite brush stroke into paint buffer–end (for each particle)04/04/2005© 2005 University of WisconsinPutting it all together04/04/2005© 2005 University of WisconsinCreative techniques•Like real painting, render the scene in layers–Paint each object with multiple layers, each shrunk in more. Outside layers are painted sparsely, inner layers painted thicker.–Isolate highlights, shadows using image processing techniques and paint in a separate layer–Each object or group of objects in a scene can be given its own layer•Painting parameters can be chosen per-layer•Semi-transparent layers allow compositing of styles04/04/2005© 2005 University of WisconsinTechnical considerations•Brush strokes may jitter in size and orientation slightly between frames–So blur the size and orientation reference images before sampling•Rendering of back-facing particles–Useful so previously obscured strokes don't pop in when animating–Can cause visual problems when layering–Their solution culls back-facing particles, but fades them in as they get close to front-facing04/04/2005© 2005 University of WisconsinFuture Directions•Combining painterly look with traditional renderer•Automatically handling changing object size•Improving particle-placement algorithm to cover geometric surfcace and screen space more evenly•Implementing longer, deformable brushes that can follow curves on a surface04/04/2005© 2005 University of WisconsinQuestionsQuestions?04/04/05 © 2005 University of Wisconsin04/04/2005 © 2005 University of WisconsinCartoon Rendering of Smoke