The Memory Hierarchy – Part I The slides of Part I are taken in large part from V. Heuring & H. Jordan, “Computer SyMemory Hierarchy Outline (1)Memory Hierarchy Outline (2)Memory Technology CharacteristicsMemory Performance GapLevels of the Memory HierarchyThe CPU–Memory InterfaceThe CPU–Memory Interface (cont’d.)Memory Performance ParametersMemories: Basic TechnologiesMemory Cell StructureAn 8-Bit Register as a 1-D RAM ArrayA 4 x 8 2-D Memory Cell ArrayA 64 K x 1 Static RAM ChipA 16 K x 4 SRAM ChipMatrix & Tree Decoders6-Transistor Static RAM CellStatic RAM Read OperationStatic RAM Write OperationsExample Commercial ProductDynamic RAM OrganizationDRAM Chip OrganizationDRAM Read and Write CyclesDRAM Refresh & Row AccessDRAM Commercial ProductA 2-D CMOS ROM ChipROM TypesMemory Boards and ModulesGeneral Structure of a Memory ChipWord Assembly from Narrow ChipsIncreasing the Num. of Words by a Factor of 2kChip Using 2 Chip Selects3-Dimensional Dynamic RAM ArrayA Memory Module and Its InterfaceDynamic RAM Module with Refresh ControlTwo Kinds of Memory Module OrganizationsTiming Advantage of Interleaving1SP06 Mark FranklinChapter 7The Memory Hierarchy –Part IThe slides of Part I are taken in large part from V. Heuring & H. Jordan, “Computer Systems Design and Architecture” .2SP06 Mark FranklinMemory Hierarchy Outline (1)• Memory components:– RAM memory cells & cell arrays.– Static RAM—more expensive, but less complex.– Tree and matrix decoders—needed for large RAM chips.– Dynamic RAM—less expensive, but needs “refreshing”• Chip organization• Timing– ROM—Read-only memory.• Memory boards– Arrays of chips give more addresses and/or wider words.– 2-D and 3-D chip arrays.• Memory modules– Large systems can benefit by partitioning memory for:• separate access by system components.• fast access to multiple words.3SP06 Mark FranklinMemory Hierarchy Outline (2)• The Memory Hierarchy: from fast & expensive to slow & cheap:– Registers → Cache → Main Memory → Disk• Consider two adjacent hierarchy levels: Cache Æ Main Memory– Cache: High speed, expensive (1stlevel on-chip, 2ndlevel off-chip)• Design Types: Direct mapped, associative, set associative– Virtual memory: Makes the hierarchy to disk transparent• Translate the address from CPU’s logical address to the physical addresswhere the information is actually stored.• Memory management — how to move information back and forth.• Multiprogramming, multithreading — what to do while we wait.• The “TLB” helps in speeding the address translation process.• Memory as a subsystem: Overall performance.4SP06 Mark FranklinMemory Technology CharacteristicsLevelMemoryTypeAverageAccess TimeTypicalSizeUnit of Transfer(Block Size)1 Cache.25 – 10ns 8KB - 32MB Word16-64bits2MainMemory40 – 200ns 2MB - 32GB Cache line8B-32B3Disk 5 – 10ms > 100Gb Page4KB-16KB4MagneticTape1 – 5sec > 200Gb Record16KB5SP06 Mark FranklinMemory Performance GapProcessor-DRAM Memory Gap (latency)µProc60%/yr.(2X/1.5yr)DRAM9%/yr.(2X/10 yrs)110100100019801981198319841985198619871988198919901991199219931994199519961997199819992000DRAMCPU1982Processor-MemoryPerformance Gap:(grows 50% / year)PerformanceTime“Moore’s Law”6SP06 Mark FranklinLevels of the Memory HierarchyCPU Registers100s Bytes<2s nsCache100s K Bytes.3 - 2 ns1-0.1 cents/bitMain Memory1000s M Bytes100ns- 400ns$.0001-.00001 cents /bitDisk100sG Bytes, 10 ms 10-6 -10-7cents/bitCapacity, Access Time, CostTapeinfinitesec-min10-8 cents/bitRegistersCacheMemoryDiskTapeInstr. OperandsBlocksPagesFilesStagingXfer Unitprog./compiler1-8 bytescache 8-128 bytesOS512-8K bytesuser/operatorMbytesUpper LevelLower LevelfasterLarger7SP06 Mark FranklinThe CPU–Memory InterfaceSequence of events:Read:1. CPU loads MAR, issuesRead, & REQUEST.2. Main memory transmitswords to MDR.3. Main memory assertsCOMPLETE.Write:1. CPU loads MAR & MDR, asserts Write & REQUEST.2. (MDR) Æ (MAR).3. Main memory assertsCOMPLETE.CPUmMain memoryAddress busData bussAddress01232m–1A0–Am–1D0–Db–1R/WREQUESTCOMPLETEMDRRegisterfileControl signalsmwwMARb8SP06 Mark FranklinThe CPU–Memory Interface (cont’d.)Additional points:• If b < w, main memory must make w/b b-bit transfers.• Some CPUs allow reading & writing of word sizes < wExp: Intel 8088: m = 20; 8- and 16-bit values can be read & written• If memory is sufficiently fast, or if its response is predictable,then COMPLETE may be omitted.• Some systems use separate R & W lines, & omit REQUEST.CPUmMain memoryAddress busData bussAddress01232m–1A0–AmÐ1D0–DbÐ1R/WREQUESTCOMPLETEMDRRegisterfileControl signalsmwwMARb9SP06 Mark FranklinMemory Performance ParametersSymbol Definition Units MeaningtaAccess timetime Time to access a memory word.tcCycle time time Time from start of access to startof next access.k Block size words Number of words per block.ω Bandwidth words/time Word transmission rate.tlLatency time Time to access first word of a sequence of words.tbl= Block time Time to access an entire blockof words.tl+ k/ω access time(Information is stored & moved in blocks at the cache & disk level.)10SP06 Mark FranklinMemories: Basic Technologies• SRAM:– value is stored on a pair of inverting gates– very fast but takes up more space than DRAM (4 to 6 transistors)• Cross Coupled gates (more later)• DRAM:– value is stored as a charge on capacitor (must be refreshed)– very small (hence higher density) but slower than SRAM (factor of 5 to 10)Word linePass transi storCapacitorBi t l i ne11SP06 Mark FranklinMemory Cell StructureSelectDataOutDataInR/WRegardless of the technology, all RAM memory cells must providethese four functions: Select, DataIn, DataOut, and R/W.12SP06 Mark FranklinAn 8-Bit Register as a 1-D RAM ArrayThe entire register is selected with one select line, and uses one R/W line.Data bus is bidirectional and buffered. SelectDataInDataOutR/Wd0SelectR/Wd1d2d3d4d5d6d7DD D D D D D D D13SP06 Mark FranklinA 4 x 8 2-D Memory Cell ArrayR/W is commonto all2-bitaddressBidirectional 8-bit buffered data bus2-4 line decoder selects one of the four 8-bit arraysd0R/Wd1d2d3d4d5d6d7DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD2– 4decoderA1A014SP06 Mark FranklinA 64 K x 1 Static RAM Chip~square array fits IC designparadigmSelecting rows separatelyfrom columns means only256 x 2 = 512 circuit elementsinstead of 65536 circuitelements!CS, Chip Select, allows chips in arraysto be selected