Hyperthreading Technology Aleksandar Milenkovic Electrical and Computer Engineering Department University of Alabama in Huntsville milenka ece uah edu www ece uah edu milenka Outline What is hyperthreading Trends in microarchitecture Exploiting thread level parallelism Hyperthreading architecture Microarchitecture choices and trade offs 3 2 2005 A Milenkovic A Milenkovic 2 What is hyperthreading SMT Simultaneous multithreading Make one physical processor appear as multiple logical processors to the OS and software Intel Xeon for the server market early 2002 Pentium4 for the consumer market November 2002 Motivation boost performance for up to 25 at the cost of 5 increase in additional die area 3 2 2005 A Milenkovic 3 Trends in microarchitecture Higher clock speeds To achieve high clock frequency make pipeline deeper superpipelining Events that disrupt pipeline branch mispredictions cache misses etc become very expensive in terms of lost clock cycles ILP Instruction Level Parallelism Extract parallelism in a single program Superscalar processors have multiple execution units working in parallel Challenge to find enough instructions that can be executed concurrently Out of order execution instructions are sent to execution units based on instruction dependencies rather than program order 3 2 2005 A Milenkovic A Milenkovic 4 Trends in microarchitecture Cache hierarchies Processor memory speed gap Use caches to reduce memory latency Multiple levels of caches smaller and faster closer to the processor core Thread level Parallelism Multiple programs execute concurrently Web servers have an abundance of software threads Users surfing the web listening to music encoding decoding video streams etc 3 2 2005 A Milenkovic 5 Exploiting thread level parallelism CMP Chip Multiprocessing Multiple processors each with a full set of architectural resources reside on the same die Processors may share an on chip cache or each can have its own cache Examples HP Mako IBM Power4 Challenges Power Die area cost Time slice multithreading Processor switches between software threads after a predefined time slice Can minimize the effects of long lasting events Still some execution slots are wasted 3 2 2005 A Milenkovic A Milenkovic 6 Exploiting thread level parallelism Switch on event multithreading Processor switches between software threads after an event e g cache miss Works well but still coarse grained parallelism e g data dependences and branch mispredictions are still wasting cycles SMT Simultaneous multithreading Multiple software threads execute on a single processor without switching Have potential to maximize performance relative to the transistor count and power A Milenkovic 3 2 2005 7 Hyperthreading architecture One physical processor appears as multiple logical processors HT implementation on NetBurst microarchitecture has 2 logical processors Architectural Architectural State State Processor execution resources 3 2 2005 A Milenkovic Architectural state general purpose registers control registers APIC advanced programmable interrupt controller A Milenkovic 8 Hyperthreading architecture Main processor resources are shared caches branch predictors execution units buses control logic Duplicated resources register alias tables map the architectural registers to physical rename registers next instruction pointer and associated control logic return stack pointer instruction streaming buffer and trace cache fill buffers 3 2 2005 A Milenkovic 9 Die Size and Complexity 3 2 2005 A Milenkovic A Milenkovic 10 Resources sharing schemes Partition dedicate equal resources to each logical processors Good when expect high utilization and somewhat unpredicatable Threshold flexible resource sharing with a limit on maximum resource usage Good for small resources with bursty utilization and when the micro ops stay in the structure for short predictable periods Full sharing flexible with no limits Good for large structures with variable working set sizes A Milenkovic 3 2 2005 11 Shared vs partitioned queues shared 3 2 2005 A Milenkovic partitioned A Milenkovic 12 NetBurst Pipeline threshold partitioned A Milenkovic 3 2 2005 13 Shared vs partitioned resources Partitioned E g major pipeline queues Threshold Puts a threshold on the number of resource entries a logical processor can have E g scheduler Fully shared resources E g caches Modest interference Benefit if we have shared code and or data 3 2 2005 A Milenkovic A Milenkovic 14 Scheduler occupancy 3 2 2005 A Milenkovic 15 Shared vs partitioned cache 3 2 2005 A Milenkovic A Milenkovic 16 Performance Improvements 3 2 2005 A Milenkovic A Milenkovic 17