Introduction to Multithreading, Superthreading and Hyperthreadingby Jon "Hannibal" StokesConventional multithreadingClarification of terms: "running" vs. "executing," and "front end" vs. "execution core."Preemptive multitasking vs. Cooperative multitaskingEach program has a mind of its ownA few terms: process, context, and threadFooling the processes: context switchesSMP to the rescue? Superthreading with a multithreaded processorHyper-threading: the next stepImplementing hyper-threadingReplicated resourcesPartitioned resourcesShared resourcesCaching and SMTCache conflictsConclusionsBibliographyRevision HistoryIntroduction to Multithreading, Superthreading and Hyperthreadingby Jon "Hannibal" Stokes1 http://www.arstechnica.com/paedia/h/hyperthreading/hyperthreading-1.htmlBack in the dual-Celeron days,when symmetricmultiprocessing(SMP) firstbecame cheapenough to comewithin reach of theaverage PC user,many hardwareenthusiasts eagerto get in on theSMP craze wereasking whatexactly (besideswinning them theadmiration andenvy of theirpeers) a dual-processing rigcould do for them.It was in thiscontext that the PCcrowd startedseriously talkingabout theadvantages ofmultithreading.Years later whenApple broughtdual-processing toits PowerMac line,SMP wasofficiallymainstream, andwith itmultithreadingbecame a concernfor the mainstreamuser as the ensuinground ofbenchmarksbrought out thefact you reallyneededmultithreadedapplications to getthe full benefits oftwo processors. Even though thePC enthusiastSMP craze haslong since dieddown and, in anodd twist of fate,Mac users are nowmany times morelikely to besporting an SMPrig than their x86-using peers,multithreading isonce again aboutto increase inimportance for PCusers. Intel's nextmajor IA-32processor release,codenamedPrescott, willinclude a featurecalledsimultaneousmultithreading(SMT), alsoknown as hyper-threading. To takefull advantage ofSMT, applicationswill need to bemultithreaded; andjust like with SMP,the higher thedegree ofmultithreading themore performancean application canwring out ofPrescott'shardware. Intel actuallyalready uses SMTin a shippingdesign: thePentium 4 Xeon.Near the end ofthis article we'lltake a look at theway the Xeonimplements hyper-threading; thisanalysis shouldgive us a prettygood idea ofwhat's in store forPrescott. Also, it'srumored that thecurrent crop ofPentium 4'sactually has SMThardware built-in,it's just disabled.(If you add this tothe rumor aboutx86-64 supportbeing present butdisabled as well,then you can getsome idea of justhow cautious Intelis when it comesto introducing newfeatures. I'd kill toget my hands on a2.8 GHz P4 withboth SMT andx86-64 supportturned on.) SMT, in a nutshell,allows the CPU todo what mostusers think it'sdoing anyway: runmore than oneprogram at thesame time. Thismight sound odd,so in order tounderstand how itworks this articlewill first look athow the currentcrop of CPUshandlesmultitasking.Then, we'll discussa technique calledsuperthreadingbefore finallymoving on toexplain hyper-threading in thelast section. So ifyou're looking tounderstand moreaboutmultithreading,symmetricmultiprocessingsystems, andhyper-threadingthen this article isfor you. As always, ifyou've read someof my previoustech articles you'llbe well equippedto understand thediscussion thatfollows. From hereon out, I'll assumeyou know thebasics of pipelinedexecution and arefamiliar with thegeneralarchitecturaldivision between aprocessor's frontend and itsexecution core. Ifthese terms aremysterious to you,then you mightwant to reach wayback and checkout my "Into theK7" article, aswell as some ofmy other work onthe P4 and G4e.ConventionalmultithreadingQuite a bit of whata CPU does isillusion. Forinstance, modernout-of-orderprocessorarchitectures don'tactually execute2 http://www.arstechnica.com/paedia/h/hyperthreading/hyperthreading-1.htmlcode sequentiallyin the order inwhich it waswritten. I'vecovered the topicof out-of-orderexecution (OOE)in previousarticles, so I won'trehash all thathere. I'll just notethat an OOEarchitecture takescode that waswritten andcompiled to beexecuted in aspecific order,reschedules thesequence ofinstructions (ifpossible) so thatthey makemaximum use ofthe processorresources,executes them, andthen arranges themback in theiroriginal order sothat the results canbe written out tomemory. To theprogrammer andthe user, it looksas if an ordered,sequential streamof instructionswent into the CPUand identicallyordered, sequentialstream ofcomputationalresults emerged.Only the CPUknows in whatorder theprogram'sinstructions wereactually executed,and in that respectthe processor islike a black box toboth theprogrammer andthe user. The same kind ofsleight-of-handhappens when yourun multipleprograms at once,except this timethe operatingsystem is alsoinvolved in thescam. To the enduser, it appears asif the processor is"running" morethan one programat the same time,and indeed, thereactually aremultiple programsloaded intomemory. But theCPU can executeonly one of theseprograms at atime. The OSmaintains theillusion ofconcurrency byrapidly switchingbetween runningprograms at afixed interval,called a time slice.The time slice hasto be small enoughthat the userdoesn't notice anydegradation in theusability andperformance of therunning programs,and it has to belarge enough thateach program hasa sufficientamount of CPUtime in which toget useful workdone. Mostmodern operatingsystems include away to change thesize of anindividualprogram's timeslice. So aprogram with alarger time slicegets more actualexecution time onthe CPU relativeto its lowerpriority peers, andhence it runsfaster. (On arelated note, thisbrings to mind oneof my favorite .sigfile