Heading - System ControllerHeading - Play Games?Heading - System Controller: AthlonHeading - MRO: Memory Request OrganizerBullet - • Request crossbar responsible for scheduling memory read and write requests from BIU, P...Bullet - • Serves as the coherence pointBullet - • Requests are reordered to minimize page conflict and mximize page hitsBullet - • Anti-starvation mechanism by aging of entriesBullet - • Arbitration bypassed during idle conditions to improve latencyHeading - AMD Athlon Controller:Body - Silicon StatBody - Heading - AMD Athlon Controller:Heading - Cache Coherency IHeading - Cache Coherency IIHeading - Cache Coherency IIIaHeading - Cache Coherency IIIbHeading - Why worry about CC?Heading - Multiple Clock Domains IHeading - Multiple Clock Domains IIHeading - Multiple Clock Domains IIIHeading - Multiple Clock Domains IVHeading - Multiple Clock Domains VHeading - Multiple Clock Domains VIHeading - Multiple Clock Domains VIIHeading - GARTHeading - Quick Summary IHeading - Quick Summary IIBullet - • System Controller is a “traffic cop”Bullet - • Traffic cop may have to deal with clock domain synchronization issueBullet - • Handles Cache Coherency for small scale SMP configurationBullet - • Memory Controller must know everything there is to know about DRAM system. Which bank ...DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandSystem ControllerRAMSystem Controller(North Bridge)I/O Controller(Southbridge)LowBandwidthI/OHigh BandwidthI/OCPU CPUSmall scaleSMP system1~4 CPU’sDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandPlay Games?graphicsmemoryEthernet cardethernet packetethernet packetmulti megabyte textureRAMOS/drivers/etc.Z- bufferTextureCollision detection/geometryinformationGame AIHarddiscAGP KeyboardmouseSystem Controller(North Bridge)I/O Controller(Southbridge)3D gfxprocessorCPUHeavy demand placed on memory systemHeavier still in SMP/SMT/CMP systemSystem Controller == System traffic copDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandSystem Controller: Athlon CPUCPUBIU1 BIU0PCIAGPMCTMRO DRAMAPCMRO:Memory Request OrganizerAPC:AGP PCI Controller blockMCT:Memory Controller (SDRAM/DDR/DRDRAM)DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMRO: Memory Request Organizer• Request crossbar responsible for scheduling memory read and write requests from BIU, PCI, AGP• Serves as the coherence point• Requests are reordered to minimize page conflict and mximize page hits• Anti-starvation mechanism by aging of entries• Arbitration bypassed during idle conditions to improve latencyDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandAMD Athlon Controller:Silicon StatChipVersionTech & VoltageMax Core SpeedDie Size(pad limited)No. of pinsSDRAM1P, 2xAGP0.35um,3.3V100 MHz107 mm2492SDRAM, 2P, 2xAGP0.35um,3.3V100 MHz130 mm2656DDR, 1P, 4xAGP0.25um,2.5V133 MHz133 mm2553DDR, 2P, 4xAGP0.25um,2.5V133 MHzRDRAM, 1P, 4xAGP0.25um,2.5V133 MHz107 mm2492DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandAMD Athlon Controller:Die photo: SDRAM, 2P, 2xAGPMROPCI/APCAGPCFGMCTBIU0BIU1FY0FY1PLLApprox. 500K gates11.43x11.43mm2DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandCache Coherency ICPUCPUBIU1 BIU0MCTMRO DRAMRead Request: I would like data for cachline 0x001CA980Read RequestDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandCache Coherency IICPUCPUBIU1 BIU0MCTMRO DRAMSnoop Request: Do you have cachline 0x001CA980 ?Snoop RequestMemory Fetch: Give me data for 0x001CA980.DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandCache Coherency IIIaCPUCPUBIU1 BIU0MCTMRO DRAMSnoop Response: NoSnoop ResponseSDRAM MCT: RAS to rank 2, bank 0, row 0x00842SDRAM MCT: CAS to rank 2, bank 0, col 0x0C3DataSDRAM MCT: Here’s the data.DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandCache Coherency IIIbCPUCPUBIU1 BIU0MCTMRO DRAMSnoop Response: Yes, I have this cache lineSnoop ResponseSDRAM MCT: RAS to rank 2, bank 0, row 0x00842SDRAM MCT: CAS to rank 2, bank 0, col 0x0C3DataMRO: Here’s the data.Stale DatatrashDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandWhy worry about CC?CPUCPU“Point of Synchronization”Distance to data in DRAMDRAMDistance to data in cacheWhat if distance to DRAM is shorter than distance to cache (in another CPU)?DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains IRAMSystem ControllerHighCPUBandwidthI/O Low BandwidthI/OAGP:66*4 MHzcommand:133 MHzaddress:133*2 MHzdata:133*4 MHzPCI: 33 MHzDDR:133*2 MHzDDR:166*2 MHzMost clock domains are integer multiples of each otherDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains IIRAMSystem ControllerHighCPUBandwidthI/O Low BandwidthI/OAGP:PCI: 33 MHzSDRAM: What if clock domains are not integer multiples of each other?66/100/133 MHz Processor bus100/133 MHz66*2 MHz(32 bit)(32 bit)DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains IIID0 D1 D2LatchEnDataFastClk(100MHz)SlowClk(66MHz)D0 D1 D2Latched DataCtlMaskDffDQDffDQLatchEnDffDQFastClkFastClkSlowClkCtlMaskCmbLogicCmbLogicSlow clock domainFast clock domainAMD AthlonChipsetGearboxLogicGear Box ModuleDRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains IVd0d1d2d3d3d2d1d0Data transfer from 100 MHz clock domain to 133 MHz clock domain (Latency Optimal)d0d1d2d3d3d2d1d0Data transfer from 100 MHz clock domain to 133 MHz clock domain (Bandwidth Optimal)DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains Vd0d1d2d3d3d2d1d0Data transfer from 400 MHz clock domain to 800 MHz clock domain (Latency Optimal)d0d1d2d3d3d2d1d0Data transfer from 400 MHz clock domain to 800 MHz clock domain (Bandwidth Optimal)DRAM MemorySystem: Lecure 4Spring 2003Bruce JacobDavid WangUniversity ofMarylandMultiple Clock Domains VI0123456789500 MHzCPU Clock Domain(10:2)Bus Clock - 100 MHz012345678910CPU Clock Domain(11:2)550 MHzBus Clock - 100 MHz01234567891011CPU Clock Domain(12:2)600 MHzBus Clock - MHz(harmonic)(not-harm)(harmonic)Fractional Multipliers could impactperformance, but you may not have a choiceProcessor to Processor