Slide 1Slide 2Slide 3Slide 4Slide 5Slide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Slide 21Slide 22Slide 23Slide 24Slide 25Slide 26Slide 27Slide 28Slide 29Slide 30Slide 31Slide 32Slide 33Slide 34Slide 35Slide 36Slide 37Slide 38Slide 39Slide 40The GPGPU Creation StoryMikola LysenkoUniversity of Wisconsin-MadisonDept. of Mechanical EngineeringSpatial Automation LabOverviewGoing to cover major historical developments in roughly chronological orderTime period: 1970 – presentFocus on:HistoryKey technologiesMarket forcesReally a story about PC gamesThe Early Graphics DisplaysVector graphics (1960s)Based on the oscilloscopeConsist of electron gun + phosphor displayExamples:SketchPadAsteroidsPongDutchTronix (2007)Problems with Vector DisplaysNo colorThough you can put a filter over part of the screenLimited number of objectsCrude imagesWill always look like linesAtari, Inc. (1979)Raster DisplaysRepresent an image with a “framebuffer”A 2D array of pixelsRequires a RAMDAC:Random Access Memory Digital-to-Analog ConverterTranslates the framebuffer into a video signalCan be used to display any image (in theory)Early Raster GamesSprite based rendering (late 1980s-early 1990s)'Blit' rectangular subregions quickly to animate objectsLater games added scrolling backgroundsExamples: Duke Nukem, Commander KeenApogee (1991)id Software (1990)Ray Casting GamesEnabled by raster displays and faster CPUsRender a column of pixels per rayExamples: Wolfenstein (1992) & Doom (1995)id Software (1992)id Software (1993)Problems with Ray Casting3D is an illusionEnemies/objects are spritesWorld is limited to 2D geometryThough Doom had heightApogee (1994)PolygonsPioneered by SGI (1980s)Basic idea is to represent 3D objects by a collection of polygons on their boundaryMade famous in games by Quake (1996)Basis for all modern 3D graphicsid Software (1996)Polygon Rasterization1. Trace edges of polygon into 2 buffers (LEFT, RIGHT)LEFT RIGHTLEFT RIGHTPolygon Rasterization (cont.)2. Traverse LEFT-RIGHT scan lines in frame bufferPolygon Rasterization (final)Fill in the scan linesThat's it!ProblemsWe need to draw a lot of polygons to get a good approximation of a 3D sceneDrawing polygons is expensiveQuake was able to get around this with very aggressive visibility cullingCPU performance was not scaling fast enoughFirst 3D HardwareOriginally developed for Hollywood and industrial/academic applicationsFirst machines built by SGI (Iris) (late 1980s)Brought to mass market by 3Dfx (1996)id Software (1998)How the First GPUs WorkedUse PCI slot, hijacked video signalCould not do 2D and 3D simultaneously!No windowed mode 3DOnly drew 1 triangle at a time3Dfx Interactive, (1996)Further EvolutionOriginally, PCI was fast enoughThe GPU could barely keep up!But triangle rendering got fasterNeeded more bandwidth to use this speedMade new AGP slot ...Then AGP 2X, 4X, 8X ...Then PCI-E, PCI-Ex16 ...Asymptotically, this is a losing battleHardware CachingOn board RAM was getting cheaperTake some of the texture memory and use it to store vertices!Then the CPU can draw many polygons with a single command, saving a lot of bandwidthBUT... only works for static objectsWhat about animated characters?Vertex ProgramsAllow programmers to parameterize animationsGPU runs a small user defined program for each vertexCan generate an arbitrary animated shape from a few parameters and some cached memoryHow Vertex Programs WorkVertices and topology are stored on the GPUEach vertex is a collection of user-defined values called 'attributes'e.g. Position, Color, Texture Coordinates, etc.May also pass 'uniform' values that are constant for all shaderse.g. Transform matrix, animation coordinates, etc.Attributes + Topology + Uniforms + Vertex Program => Triangles!Example: Skinning(Skeletal Animation)Parameterize animations for a character in terms of a collection of 'bones'Vertices form a 'skin'Attributes:Relative positionBone weightsColor/TextureMartinez, “Ontology for Virtual Humans” (2007)Skinning (cont.)To draw a character, call the vertex program with bone positions passed as uniformsVertex program interpolates verticesBandwidth savings:Thousands of vertices, each with >30 bytes of datavs.A handful of bones (16 bytes each)Skinning ResultsMore characters, more polygonsFaster renderingFlexible animationCan do ragdollsWin-win situationCroteam, (2001)Rendering QualityOver time, people began to look for ways to add more detail to scenes, beyond simple texturingAlternate Lighting ModelsBump MappingShadowsEnvironment MappingThese techniques produced higher quality images, but required special GPU featuresProliferation of FeaturesTo keep up, hardware vendors kept adding more gizmosNew updates almost every monthA programmer's nightmare!Impossible to take advantage of all the new featuresBad for codersBad for gamersBad for video card manufacturersFragment ShadersMake per-pixel color calculations programmableReduces interface complexityImplementing features no longer vendor's problemOpened up brand new rendering techniquesFar too many to even begin covering in this talkDramatic improvements in image qualityWhere does the term “Shader” come from?Shaders were invented by Pixar for their Renderman platform (1982)Meant to help artists specify lighting / texturesUsed in Toy StoryPixar, (1995)Fragment Shader ExamplesDecaudin, “Cartoon-LookingRendering of 3D Scenes” (1996)Gosselin, “Phat Lewt: Drawing a Diamond” (2004)Heidrich & Seidel, “Realistic Hardware Accelerated Shading and Lighting” (1999)Tatarchuk, “Practical Dynamic Parallax Occlusion Mapping” (2005)D'Eon, “NVIDIA Human Head Demo” (2007)Lengyel et al. “Realtime Fur Over Arbitrary Surfaces” (2001)GPU Shaders TodayCan create beautiful, lifelike imageryRequires massive computing powerFast memory, many processorsBethesda Softworks, (2007)Exponential Growth of GPU PowerTo keep up with demands from gaming, GPUs grew very powerfulSource: linux-mag.comThe Birth of GPGPUGPUs had become very powerfulParallel execution for processing vertices/fragmentsMassive memory