New version page

Berkeley COMPSCI 10 - Lecture Notes

Upgrade to remove ads
Upgrade to remove ads
Unformatted text preview:

CS10 The Beauty and Joy of Computing Lecture #8 : Concurrency 2010-09-29 Nvidia CEO Jen-Hsun Huang said: “whatever capability you think you have today, it’s nothing compared to what you’re going to have in a couple of years … due to supercomputers in the cloud”. Now, cloud computing does what you could do on your PC. Imagine 40,000 times that! UC Berkeley EECS Lecturer SOE Dan Garcia www.theregister.co.uk/2010/09/24/huang_muses_at_gtc/UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (2) Garcia, Fall 2010 Knowledge is recognizing what you know and what you don't. I hear and I forget. I see and I remember. I do and I understand. Happy Confucius Day!UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (3) Garcia, Fall 2010 Intra-computer  Today’s lecture  Multiple computing “helpers” are cores within one machine  Aka “multi-core”  Although GPU parallism is also “intra-computer” Inter-computer  Week 12’s lectures  Multiple computing “helpers” are different machines  Aka “distributed computing”  Grid & cluster computing Concurrency & Parallelism, 10 mi up…UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (4) Garcia, Fall 2010 Anatomy: 5 components of any ComputerUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (5) Garcia, Fall 2010 Anatomy: 5 components of any Computer Computer Memory Devices Input Output John von Neumann invented this architecture Processor Control (“brain”) Datapath (“brawn”) What causes the most headaches for SW and HW designers with multi-core computing? a) Control b) Datapath c) Memory d) Input e) OutputUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (6) Garcia, Fall 2010 Processor Control (“brain”) Datapath (“brawn”) But what is INSIDE a Processor?UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (7) Garcia, Fall 2010 But what is INSIDE a Processor? • Primarily Crystalline Silicon • 1 mm – 25 mm on a side • 2009 “feature size” (aka process) ~ 45 nm = 45 x 10-9 m (then 32, 22, and 16 [by yr 2013]) • 100 - 1000M transistors • 3 - 10 conductive layers • “CMOS” (complementary metal oxide semiconductor) - most common • Package provides: • spreading of chip-level signal paths to board-level • heat dissipation. • Ceramic or plastic with gold wires. Chip in Package Bare Processor Die Processor Control (“brain”) Datapath (“brawn”)UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (8) Garcia, Fall 2010 Moore’s Law Predicts: 2X Transistors / chip every 2 years Gordon Moore Intel Cofounder B.S. Cal 1950! Year # of transistors on an integrated circuit (IC) en.wikipedia.org/wiki/Moore's_law What is this “curve”? a) Constant b) Linear c) Quadratic d) Cubic e) ExponentialUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (9) Garcia, Fall 2010 Moore’s Law and related curvesUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (10) Garcia, Fall 2010 Moore’s Law and related curvesUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (11) Garcia, Fall 2010 Power Density Prediction circa 2000 4004"8008"8080"8085"8086"286"386"486"Pentium® proc"P6"1"10"100"1000"10000"1970" 1980" 1990" 2000" 2010"Year"Power Density (W/cm2)"Hot Plate"Nuclear Reactor"Rocket Nozzle"Source:(S.(Borkar((Intel)(Sunʼs Surface"Core 2 "UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (12) Garcia, Fall 2010 Going Multi-core Helps Energy Efficiency  Power of typical integrated circuit ~ C V2 f  C = Capacitance, how well it “stores” a charge  V = Voltage  f = frequency. I.e., how fast clock is (e.g., 3 GHz) William Holt, HOT Chips 2005"Activity Monitor (on the lab Macs) shows how active your cores areUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (13) Garcia, Fall 2010 Energy & Power Considerations Courtesy: Chris Batten"UC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (14) Garcia, Fall 2010 Parallelism again? What’s different this time? “This shift toward increasing parallelism is not a triumphant stride forward based on breakthroughs in novel software and architectures for parallelism; instead, this plunge into parallelism is actually a retreat from even greater challenges that thwart efficient silicon implementation of traditional uniprocessor architectures.” – Berkeley View, December 2006  HW/SW Industry bet its future that breakthroughs will appear before it’s too late view.eecs.berkeley.eduUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (15) Garcia, Fall 2010  A Thread stands for “thread of execution”, is a single stream of instructions  A program / process can split, or fork itself into separate threads, which can (in theory) execute simultaneously.  An easy way to describe/think about parallelism  A single CPU can execute many threads by Time Division Multipexing  Multithreading is running multiple threads through the same hardware CPU Time Thread0 Thread1 Thread2 Background: ThreadsUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (16) Garcia, Fall 2010 • Applications can almost never be completely parallelized; some serial code remains • s is serial fraction of program, P is # of cores (was processors) • Amdahl’s law: Speedup(P) = Time(1) / Time(P) ≤ 1 / ( s + [ (1-s) / P) ], and as P  ∞ ≤ 1 / s • Even if the parallel portion of your application speeds up perfectly, your performance may be limited by the sequential portion Speedup Issues : Amdahl’s Law Time Number of Cores Parallel portion Serial portion 1 2 3 4 5 en.wikipedia.org/wiki/Amdahl's_lawUC Berkeley CS10 “The Beauty and Joy of Computing” : Concurrency (17) Garcia, Fall 2010 Speedup Issues : Overhead  Even assuming no sequential portion, there’s…  Time to think how to divide the problem up  Time to hand out small “work units” to workers  All workers may not work equally fast  Some workers may fail  There may be contention for shared resources  Workers could overwriting each others’ answers  You may have to wait until the last worker returns to


View Full Document
Download Lecture Notes
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture Notes and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture Notes 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?