.............................................................................................................................................. ................ ................ ................ .................. ..........THE GPU COMPUTING ERA.............................................................................................................................................. ................ ................ ................ .................. ..........GPU COMPUTING IS AT A TIPPING POINT, BECOMING MORE WIDELY USED IN DEMANDINGCONSUMER APPLICATIONS AND HIGH-PERFORMANCE COMPUTING.THIS ARTICLEDESCRIBES THE RAPID EVOLUTION OFGPU ARCHITECTURES—FROM GRAPHICSPROCESSORS TO MASSIVELY PARALLEL MANY-CORE MULTIPROCESSORS, RECENTDEVELOPMENTS INGPU COMPUTING ARCHITECTURES, AND HOW THE ENTHUSIASTICADOPTION OFCPUþGPU COPROCESSING IS ACCELERATING PARALLEL APPLICATIONS.......As we enter the era of GPUcomputing, demanding applications withsubstantial parallelism increasingly use themassively parallel computing capabilitiesof GPUs to achieve superior performanceand efficiency. Today GPU computingenables applications that we previouslythought infeasible because of long execu-tion times.With the GPU’s rapid evolution from aconfigurable graphics processor to a pro-grammable parallel processor, the ubiqui-tous GPU in every PC, laptop, desktop,and workstation is a many-core multi-threaded multiprocessor that excels atboth graphics and compu ting applications.Today’s GPUs use hundr eds of parallelprocessor cores executing te ns of thou-sands of parallel threads to rapidly solvelarge p roblems having substantial inherentparallelism. They’re now the most perva-sive massively parallel processing platformever available, as well as the most cost-effective.Using NVIDIA GPUs as examples, thisarticle describes the evolution of GPU com-puting and its parallel computing model, theenabling architecture and software develop-ments, how computing applications useCPUþGPU coprocessing, example applica-tion performance speedups, and trends inGPU computing.GPU computing’s evolutionWhy h ave GPUs evolved to have largenumbers of parallel threads and manycores? The driving force continues to bethe r eal-time graphics performance neededto render complex, high-resolution 3Dscenes at interactive frame rates for games.Rendering high-definition graphics scenesis a problem with tremendous inherent par-allelism. A graphics programmer writes asingle-thread program that draws one pixel, andthe GPU runs multiple instances of thisthread in parallel—drawing multipl e pixelsin parallel. Graphics programs, written inshading languages such as Cg or High-Level Shading Language (HLSL), thus scaletransparently over a wide range of threadand processor parallelism. Also, GPU com-puting programs—written in C or C þþwith the CUDA parallel computingmodel,1,2or using a parallel computingAPI inspired by CUDA such as Direct-Compute3or OpenCL4—scale transparentlyover a wide r ange of parallelism. Softwarescalability, too, has enabled GPUs torapidly increase their parallelism andperformance with increasing transistordensity.GPU technology developmentThe demand for faster and higher-definition graphics continues to drive the[3B2-14] mmi2010020005.3d 23/3/010 15:43 Page 56John NickollsWilliam J. DallyNVIDIA..............................................................56Published by the IEEE Computer Society 0272-1732/10/$26.00c2010 IEEEdevelopment of increasingly parallel GPUs.Table 1 lists significant milestones in NVIDIAGPU technology development that drove theevolution of unified graphics and computingGPUs. GPU transistor counts increased expo-nentially, doubling roughly every 18 monthswith increasing semiconductor density. Sincetheir 2006 introduction, CUDA parallelcomputing cores per GPU also dou blednearly every 18 months.In the early 1990s, there were no GPUs.Video graphics array (VGA) controllers gen-erated 2D graphics displays for PCs to accel-erate graphical user interfaces. In 1997,NVIDIA released the RIVA 128 3D single-chip graphics accelerator for games and 3Dvisualization applications, programmed withMicrosoft Direct3D and OpenGL. Evolvingto modern GPUs involved adding pro-grammability incrementally—from fixedfunction pipelines to microcoded processors,configurable processors, programmable pro-cessors, and scalable parallel processors.Early GPUsThefirstGPUwastheGeForce256,asingle-chip 3D real-time graphics processorintroduced in 1999 that incl uded nearlyevery feature of high-end workstation 3Dgraphics pipelines of that era. It containeda configurable 32-bit floating-point vertextransform and lighting processor, and a con-figurable integer pixel-fragment pipeline,programmed with OpenGL and MicrosoftDirectX 7 (DX7) APIs.GPUs first used floating-point arithmeticto calculate 3D geometry and vertices, thenapplied it to pixel lighting and color valuesto handle high-dynamic-range scenes andto simplify programming. They imple-mented accurate floating-point rounding toeliminate frame-varying artifacts on movingpolygon edges that would otherwise sparkleat real-time frame rates.As programmable shaders emerged, GPUsbecame more flexible and programmable. In2001, the GeForce 3 introduced the first pro-grammable vertex processor that executedvertex shader programs, along with a config-urab le 32-bit f loating-p oint pixel-fra gmentpipeline, programmed with OpenGL andDX8. The ATI Radeon 9700, introducedin 2002, featured a programmable 24-bitfloating-point pixel-fragment processor pro-grammedwithDX9andOpenGL.TheGeForce FX and GeForce 68005featured[3B2-14] mmi2010020005.3d 23/3/010 15:43 Page 57Table 1. NVIDIA GPU technology development.Date Product Transistors CUDA cores Technology1997 RIVA 128 3 million — 3D graphics accelerator1999 GeForce 256 25 million — First GPU, programmed with DX7 and OpenGL2001 GeForce 3 60 million — First programmable shader GPU, programmedwith DX8 and OpenGL2002 GeForce FX 125 million — 32-bit floating-point (FP) programmable GPU withCg programs, DX9, and OpenGL2004 GeForce 6800 222 million — 32-bit FP programmable scalable GPU, GPGPUCg programs, DX9, and OpenGL2006 GeForce 8800 681 million 128 First unified graphics and computing GPU,programmed in C with CUDA2007 Tesla T8, C870 681 million 128 First GPU computing system programmed in Cwith CUDA2008 GeForce GTX 280 1.4 billion 240 Unified graphics and computing GPU, IEEE FP,CUDA C, OpenCL, and DirectCompute2008 Tesla T10, S1070 1.4