Administrivia CMSC 411 Computer Systems Architecture Lecture 19 Storage Systems concl Multiprocessors Homework 5 posted due April 28 Exam 2 next Thursday Thursday April 23 practice exam posted later today Cache simulator project questions Alan Sussman als cs umd edu l d d CMSC 411 19 some from Patterson Sussman others 2 Designing an I O System Price performance and capacity issues Need to choose which I O devices to connect how to connect them Designing an I O System Example The CPU is seldom the limiting factor for I O performance Suppose the CPU can handle 10 000 I O operations i per second d IOPS And suppose the average I O size is 16 KB CMSC 411 19 some from Patterson Sussman others 4 I O Systems I O System Performance The other links in the I O chain are Consider the disk time first the I O controller suppose it adds 1 ms overhead per I O operation ti the I O bus suppose it is a bus that can transfer 20 MB sec 20 KB ms the disk suppose it rotates at 7200 RPM RPM with 8 ms average seek time and 6 MB sec transfer rate 7200 RPM 7200 60 103 12 revolutions per ms 6 MB sec 6 KB ms So the average disk time is seek rotational latency transfer 8 ms 5 12 ms 16 6 14 9 ms So the average time per transfer is I O controller time bus time disk time 1 ms 16 20 ms 14 9 ms 16 7 ms So with one controller one bus and one disk can do at most 1 16 7 10 3 60 IOPS If this is not good enough should analyze to see whether it is better to add more controllers more buses or more disks Another more complex performance analysis in Section 6 7 for the Internet Archive Cluster CMSC 411 19 some from Patterson Sussman others 5 CMSC 411 19 some from Patterson Sussman others Storage Example Internet Archive Internet Archive Cluster Goal of making a historical record of the Internet 1U storage node PetaBox GB2000 from Capricorn Technologies Internet Archive began in 1996 Wayback Machine interface performs time travel to see what the website at a URL looked like in the past 6 Contains 4 500 GB Parallel ATA PATA disk drives 512 MB of DDR266 DRAM one 10 100 1000 Ethernet interface interface and a 1 GHz GH C3 Processor from VIA 80x86 1015 It contains over a petabyte bytes and is growing by 20 terabytes 1012 bytes of new data per month In addition to storing the historical record the same hardware is used to crawl the Web every few months to get snapshots of the Internet Node dissipates 80 watts 40 GB2000s in a standard VME rack 80 TB of raw storage capacity 40 nodes are connected with a 48 port 10 100 or 10 100 1000 Ethernet switch 1 PetaByte P t B t 12 racks k 4 16 2009 some from Patterson Sussman others 7 4 16 2009 some from Patterson Sussman others 8 Estimated Cost Estimated Performance VIA processor 512 MB of DDR266 DRAM ATA disk controller power supply fans and enclosure l 500 7200 RPM Parallel ATA drive holds 500 GB 375 48 port 10 100 1000 Ethernet switch and all cables for a rack 3000 Cost C 84 00 for 84 500 f a 80 TB 80 TB rack k 7200 RPM Parallel ATA drive holds 500 GB has an average time seek of 8 5 ms transfers at 50 MB second from the disk The PATA link speed is 133 MB second disk MB second performance of the VIA processor is 1000 MIPS operating system uses 50 000 CPU instructions for a disk I O network p protocol stack uses 100 000 CPU instructions to transmit a data block between the cluster and the external world ATA controller overhead is 0 1 ms to perform a disk I O Average I O size is 16 KB for accesses to the historical record via the Wayback interface interface and 50 KB when collecting a new snapshot Disks are limit 75 I Os s per disk 300 s per node 12000 s per rack or about 200 to 600 Mbytes sec Bandwidth per rack k 160 Disks are 60 of the cost Switch needs to support 1 6 to 3 8 Gbits second over 40 Gbit sec links 4 16 2009 some from Patterson Sussman others 9 Estimated Reliability some from Patterson Sussman others some from Patterson Sussman others 10 Conclusions Fallacies Disks never fail CPU memory enclosure MTTF is 1 000 000 hours x 40 PATA Disk MTTF is 125 000 hours x 160 PATA controller MTTF is 500 000 hours x 40 Ethernet Switch MTTF is 500 000 hours x 1 Power supply MTTF is 200 000 hours x 40 Fan MTTF is 200 000 hours x 40 PATA cable MTTF is 1 000 000 hours x 40 MTTF for f the th system t i 531 hours is h 3 weeks k 70 of time failures are disks 20 of time failures are fans or power supplies 4 16 2009 4 16 2009 a mean time to failure MTTF for one disk of 1 2M hours or 140 years computed t db by running i th thousands d off di disks k for f a few f months then counting the number that failed but a more useful measure is the of disks that fail in a given period e g g 5 years y computed p as failed disks total time p disks where failed disks disks hours disk MTTF 11 CMSC 411 19 some from Patterson Sussman others 12 Conclusions Fallacies Conclusions Fallacies Computer systems can achieve 99 999 availability Average disk seek is for a seek of 1 3 of the cylinders li d that s 5 minutes per year downtime and highly unlikely in your environment in 2001 2001 well managed servers typically available 99 to 99 99 9 9 of time DRAM will replace disks in desktop and server machines just a rule of thumb for seeking from one random location to another random location on a different cylinder assuming a large number of cylinders problems with that rule are that seek time is not linear in distance mechanical issues there is locality to disk accesses disk manufacturers have pushed the rate of technology improvement in disks to match or exceed that of DRAM instead of DRAMs killing disks disks are killing tapes CMSC 411 19 some from Patterson Sussman others 13 CMSC 411 19 some from Patterson Sussman others Conclusions Fallacies Fig 6 24 Multiprocessors Snooping Caches and Directory Based Directory Based Multiprocessors CMSC 411 19 some from Patterson Sussman others 15 14 Outline Challenges of Parallel Processing Coherence 1 Application parallelism primarily via new algorithms g that have better parallel p performance p Write Consistency 2 Long remote latency impact both by architect and by the programmer Snooping Building Blocks Snooping protocols and examples For example reduce frequency of remote accesses either by Coherence traffic and Performance on MP Caching shared …