Week 8 - Monday4_15Instruction-Level Parallelism (ILP) One approach to improving performance in a microprocessor design Looking for independent instructions within a single thread of execution andtrying to execute those in an overlapped manner Pipelining - we are executing multiple instructions in parallel We're trying to find different instructions to push down the pipeline that don't have hazards between them Allowed to be at different stages at different points of time To increase ILP Deeper pipeline Have even longer pipe depth Less work per stage Shorter clock cycle May result in more hazards (there are tradeoffs) Multiple instructions in the same stage (multiple issue) Superscalar or super pipelining If we can replicate some elements of the pipeline, we can have multiple pipelines Have more instructions to flow through at once CPI < 1, so use instructions per cycle (IPC) - inverse of CPI 4GHz 4-way multiple-issue 16 BIPS, peak CPI = 0.25, peak IPC = 4 But dependences reduce in realityMultiple Issue Number of challenges to be solved Design of hardware to allow simultaneous issue of multiple instructions Need to design IF that can fetch more than one instruction RF that has more ports for larger number of instructions Resource challenge Static multiple issue and dynamic multiple issue Static Compiler is responsible to figure out which instructions are independent that can be issued together Packaged into "issue slots" Compiler detects and avoid hazards Problem: Restricted to a particular type of machine with a particular issue Cannot run on any hardware Have to recompile Dynamic More common in modern computing CPU examines instruction stream and chooses instructions to issue each cycle May be out of order Adds complexity Find ways to ensure precise exceptions Branch mispredictions are handled correctly Comipler can help by reordering instructions CPU resolves hazards using advanced techniques at runtimeSpeculation "Guess" what to do with an instruction Start operation ASAP Validate whether guess was right If so, complete the operation If not, roll-back and do the right thing Different with branch predictions Where we can determine whether a branch was taken or not taken before the impacted state In speculationAllow instructions to impact the state of the processor Need mechanism to rollback to return to the original state of the processor Example: If I speculate that the branch is taken instead of not taken, I start executing a store instruction Store speculatively update memory structure that reflects the state of the memory If wrong, I will undo the effect of the storeCompiler/Hardware Speculation Compiler can reorder instructions E.g. move load before branch Can include "fix-up" instructions to recover from incorrect speculation Advanced loads - move ahead of stores, checkload to ensure if the load was executed correctly Hardware can look ahead Buffer the results of independent instructions that are executing Any new instructions will see the buffered results and make use of it If wrong, flush some or all the buffer to remove incorrect state andrestart speculationSpeculation and Exceptions If there's an exception that occurs, there are different mechanisms to handle whether static/dynamic speculation Static Additional ISA support for deferred exceptions Make exception wait until it's sure "instruction should've been executed" Dynamic Buffering of exceptions until instruction is completed If squashed, exception gets squashedStatic Multiple Issue Compiler bundles into "issue packets" Group of instructions that can be issued on a single cycle Independent instructions Determined by pipeline resources required Compiler will know what types of instruction to group Issue packet Very long instruction Ex: 32-bit instructions bundled into 64-bit packet Simplifies fetch VLIW architecture Very Long Instruction Word Bundles can be of different sizes Finding enough independent instrucitons Noops if there's not enough Pad issue slotsScheduling Static Multiple Issue Compiler responsible for removing hazards that occur Instructions doesn't have depedencies to begin with Compiler optimization Dependencies between packets (in certain ISAs) Compiler knows dependencies exist Portability goes downMIPS with Static Dual Issue Two-issue packets (two instructions at once) One ALU/branch instruction One load/store instruction 64-bit aligned ALU/branch, then load/storePad an unsued instruction with nop Do the whole pipeline with two instructions at one stage Reduce the resources to duplicate in the pipelineHazards in the Dual-Issue MIPS More instructions executing in parallel We can't use ALU result from load/store in the same packet Adding into t0, and then loading based on t0 Forwarding not possible in a packet Any hazards Use stalling Stall 2 instructions Increase penalty in CPI because more instructions stalled More aggresive scheduling is required4_16Scheduling Example Schedule this for dual-issue MIPS Loop: lw $t0, 0($s1) # $t0 = array element addu $t0, $t0, $s2 # add scalar in $s2 sw $t0, 0($s1) # store result addi $s1, $s1, -4 # decrement pointer bne $s1, $zero, Loop # branch $s1 != 0 ALU/branch Load/store cycleLoop: nop lw $t0, 0($s1) 1 addi $s1, $s1, -4 nop 2 addu $t0, $t0, $s2 nop 3 bne $s1,