61A Lecture 32November 16th, 20111Last timeDistributed systems!Architectures• Client-server• Peer-to-peer!Message passing• ProtocolsSystem design principles!Modularity!Interfaces22Today: Parallel ComputationWhy is parallel computation important?What is parallel computation?Some examples in PythonSome problems with parallel computation33TransistorsComputers execute instructions by manipulating the flow of electricity through transistors.Transistors are made from semiconductors, like silicon.More transistors = more power.Transistors are now less than 100 nanometers in size.4MicroprocessorTransistors are arranged into “integrated circuits” on single pieces of hardware. A microprocessor, or processor is a large integrated circuit of transistors where a computer’s instructions are executed. 4Microprocessors5Intel 4000 2300 Transistors1971National Semiconductor NS3008 ~10,00 Transistors19811993Intel Pentium~3 million transistors2000’sAMD 64~243 million transistors5Moore’s lawIn 1965, the co-founder of Intel, Gordon Moore predicted that the number of transistors that could be fit onto a single chip would double every year.46 years later, that prediction is still true. 66More transistors every year77Physical limitsManufacturers are reaching physical limits!Transistors size limits!Instructions speed limits8The solution: multiple microprocessorsInstead of trying to fit more transistors into a single processor, we are turning to multiple processors.8Parallel ComputationA program (a set of instructions, a piece of code)Executed simultaneously by multiple processorsIn a shared memory environment99Parallel computing examplex = 5x = square(x)y = 6y = y+110write 5 -> xread x: 5calculate 5*5: 25 write 25 -> xwrite 6 -> yread y: 6calculate 6+1: 7write y-> 7 10Parallel computing example11read x: 5calculate 5*5: 25 write 25 -> xread y: 6calculate 6+1: 7write y-> 7 x = 5x = square(x)y = 6y = y+111Parallel computing example12P1 P2write 5 -> x write 6 -> y read x: 5 read y: 6calculate 5*5: 25 calculate 6+1: 7write 25 -> x write 7 -> y x = 5x = square(x)y = 6y = y+1x = 25y = 712Shared memory1312P1 P2read x: 5 calculate 5*5: 25 read x: 5write 25 -> x calculate 5+1: 6 write 6 -> y x = square(x) y = x + 1x = 5x = 25y = 613How many different values of x and y can there be?Quiz:How many different values of x and y can there be at the end?1414Shared memory1512P1 P2read x: 5 calculate 5*5: 25 read x: 5write 25 -> x calculate 5+1: 6 write 6 -> x x = square(x) x = x + 1x = 5x = 615How many different values of x can there be?Quiz:How many different values of x can there be at the end?1616Shared memory1712P1 P2 read x: 5read x: 5calculate 5*5: 25 calculate 5+1: 6 write 6 -> xwrite 25 -> x x = square(x) x = x + 1x = 5x = 2517Parallel computing example: bank balance18 def make_withdraw(balance): def withdraw(amount): global balance if amount > balance: print('Insufficient funds') else: balance = balance - amount print(balance) return withdraw w = make_withdraw(10)w(8) w(7)1819Parallel computing example: bank balance20 def make_withdraw(balance): def withdraw(amount): global balance if amount > balance: print('Insufficient funds') else: balance = balance - amount print(balance) return withdraw w = make_withdraw(10)balance = 10w(8) w(7)2 or 3print('Insufficient funds')19Parallel computing example: bank balance20 def make_withdraw(balance): def withdraw(amount): global balance if amount > balance: print('Insufficient funds') else: balance = balance - amount print(balance) return withdraw w = make_withdraw(10)balance = 10w(8) w(7) print 3read global balance: 10read amount: 8 read global balance: 108 > 10: False read amount: 7if False 7 > 10: False10 - 8: 2 if Falsewrite balance -> 2 10 - 7: 3print 2 write balance -> 32320Next time: how to fix these problemsLocks, semaphores,
View Full Document
Unlocking...