CPSC 614:Graduate Computer Architecture Lecture 2 Pipelining, Caching, and Benchmarks5 Steps of MIPS DatapathSlide 3Review: Visualizing Pipelining Figure 3.3, Page 133 , CA:AQA 2eLimits to pipeliningExample: One Memory Port/Structural Hazard Figure 3.6, Page 142 , CA:AQA 2eResolving structural hazardsDetecting and Resolving Structural HazardEliminating Structural Hazards at Design TimeRole of Instruction Set Design in Structural Hazard ResolutionData Hazards Figure 3.9, page 147 , CA:AQA 2eThree Generic Data HazardsSlide 13Slide 14Forwarding to Avoid Data Hazard Figure 3.10, Page 149 , CA:AQA 2eHW Change for Forwarding Figure 3.20, Page 161, CA:AQA 2eData Hazard Even with Forwarding Figure 3.12, Page 153 , CA:AQA 2eResolving this load hazardResolving the Load Data HazardSoftware Scheduling to Avoid Load HazardsInstruction Set ConnectionHistorical Perspective: MicroprogrammingAdministrationControl Hazard on Branches => Three Stage StallExample: Branch Stall ImpactPipelined MIPS Datapath Figure 3.22, page 163, CA:AQA 2/eFour Branch Hazard AlternativesSlide 28Delayed BranchRecall:Speed Up Equation for PipeliningExample: Evaluating Branch AlternativesThe Memory AbstractionRelationship of Caches and PipelineExample: Dual-port vs. Single-portMemory Hierarchy: Terminology4 Questions for Memory HierarchySimplest Cache: Direct Mapped1 KB Direct Mapped Cache, 32B blocksTwo-way Set Associative CacheDisadvantage of Set Associative CacheQ1: Where can a block be placed in the upper level?Q2: How is a block found if it is in the upper level?Q3: Which block should be replaced on a miss?Q4: What happens on a write?Write Buffer for Write ThroughA Modern Memory HierarchyBasic Issues in VM System DesignAddress MapImplications of Virtual Memory for Pipeline designPaging OrganizationAddress TranslationTLBsTranslation Look-Aside BuffersReducing Translation TimeOverlapped Cache & TLB AccessProblems With Overlapped TLB AccessAnother Word on PerformanceSPEC: System Performance Evaluation CooperativeSlide 60How to Summarize PerformanceSPEC First RoundPerformance EvaluationSummary #1/4: Pipelining & PerformanceSummary #2/4: CachesSummary #3/4: The Cache Design SpaceReview #4/4: TLB, Virtual MemoryCPSC614Lec 2.1Prof. Lawrence RauchwergerCPSC 614:Graduate Computer ArchitectureLecture 2 Pipelining, Caching, and BenchmarksCPSC614Lec 2.25 Steps of MIPS DatapathMemoryAccessWriteBackInstructionFetchInstr. DecodeReg. FetchExecuteAddr. CalcALUMemoryReg FileMUX MUXMemoryMUXSignExtendZero?IF/IDID/EXMEM/WBEX/MEM4AdderNext SEQ PCNext SEQ PCRD RD RDWB DataNext PCAddressRS1RS2ImmMUXDatapathControl PathCPSC614Lec 2.35 Steps of MIPS DatapathMemoryAccessWriteBackInstructionFetchInstr. DecodeReg. FetchExecuteAddr. CalcALUMemoryReg FileMUX MUXMemoryMUXSignExtendZero?IF/IDID/EXMEM/WBEX/MEM4AdderNext SEQ PCNext SEQ PCRD RD RDWB DataNext PCAddressRS1RS2ImmMUXDatapathControl PathInst 1Inst 1Inst 2Inst 1Inst 2Inst 3CPSC614Lec 2.4Review: Visualizing PipeliningFigure 3.3, Page 133 , CA:AQA 2eInstr.OrderTime (clock cycles)RegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegCycle 1Cycle 2 Cycle 3 Cycle 4 Cycle 6Cycle 7Cycle 5CPSC614Lec 2.5Limits to pipelining•Hazards: circumstances that would cause incorrect execution if next instruction were launched–Structural hazards: Attempting to use the same hardware to do two different things at the same time –Data hazards: Instruction depends on result of prior instruction still in the pipeline–Control hazards: Caused by delay between the fetching of instructions and decisions about changes in control flow (branches and jumps).CPSC614Lec 2.6Example: One Memory Port/Structural HazardFigure 3.6, Page 142 , CA:AQA 2eInstr.OrderTime (clock cycles)LoadInstr 1Instr 2Instr 3Instr 4RegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegCycle 1Cycle 2 Cycle 3 Cycle 4 Cycle 6Cycle 7Cycle 5DMemStructural HazardCPSC614Lec 2.7Resolving structural hazards•Defn: attempt to use same hardware for two different things at the same time•Solution 1: Waitmust detect the hazardmust have mechanism to stall•Solution 2: Throw more hardware at the problemCPSC614Lec 2.8Detecting and Resolving Structural HazardInstr.OrderTime (clock cycles)LoadInstr 1Instr 2StallInstr 3RegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegCycle 1Cycle 2 Cycle 3 Cycle 4 Cycle 6Cycle 7Cycle 5RegALUDMemIfetchRegBubble Bubble Bubble BubbleBubbleCPSC614Lec 2.9Eliminating Structural Hazards at Design TimeALUInstrCacheReg FileMUX MUXDataCacheMUXSignExtendZero?IF/IDID/EXMEM/WBEX/MEM4AdderNext SEQ PCNext SEQ PCRD RD RDWB DataNext PCAddressRS1RS2ImmMUXDatapathControl PathCPSC614Lec 2.10Role of Instruction Set Design in Structural Hazard Resolution•Simple to determine the sequence of resources used by an instruction–opcode tells it all•Uniformity in the resource usage•Compare MIPS to IA32?•MIPS approach => all instructions flow through same 5-stage pipelingCPSC614Lec 2.11Instr.Orderadd r1,r2,r3sub r4,r1,r3and r6,r1,r7or r8,r1,r9xor r10,r1,r11RegALUDMemIfetchRegRegALUDMemIfetchRegData HazardsFigure 3.9, page 147 , CA:AQA 2eTime (clock cycles)IF ID/RFEXMEMWBRegALUDMemIfetchRegRegALUDMemIfetchRegRegALUDMemIfetchRegCPSC614Lec 2.12•Read After Write (RAW) InstrJ tries to read operand before InstrI writes it•Caused by a “Data Dependence” (in compiler nomenclature). This hazard results from an actual need for communication.Three Generic Data HazardsI: add r1,r2,r3J: sub r4,r1,r3CPSC614Lec 2.13•Write After Read (WAR) InstrJ writes operand before InstrI reads it•Called an “anti-dependence” by compiler writers.This results from reuse of the name “r1”.•Can’t happen in MIPS 5 stage pipeline because:– All instructions take 5 stages, and– Reads are always in stage 2, and – Writes are always in stage 5I: sub r4,r1,r3 J: add r1,r2,r3K: mul r6,r1,r7Three Generic Data HazardsCPSC614Lec 2.14Three Generic Data Hazards•Write After Write (WAW) InstrJ writes operand before InstrI writes it.•Called an “output dependence” by compiler writersThis also results from the reuse of name “r1”.•Can’t happen in MIPS 5 stage pipeline because: – All instructions take 5 stages, and – Writes are always in stage 5•Will see WAR and WAW in later more complicated pipesI: sub r1,r4,r3 J: add r1,r2,r3K: mul r6,r1,r7CPSC614Lec 2.15Time (clock cycles)Forwarding to Avoid Data HazardFigure 3.10, Page 149 , CA:AQA 2eInstr.Orderadd r1,r2,r3sub r4,r1,r3and r6,r1,r7or