Distributed Systems 15-319, spring 2010 11th Lecture, Feb 16th Lecture MotivationLecture OutlineDo you … ?What is a Distributed System?What is a Distributed System?HistoryWhere are they used?Lecture OutlineParallel vs. Distributed SystemsFurther DistinctionsLecture OutlineAdvantages of Distributed Systems (1/2)Advantages (2/2)DisadvantagesLecture OutlineDesign Goals/ CharacteristicsOne more characteristic: HeterogeneityOne more characteristic: HeterogeneityLecture OutlineDistributed Systems HardwareHardware: MIMDHardware: MIMDHardware: Memory Differentiated Hardware: Interconnection NetworkHardware: CouplingLecture OutlineDistributed Systems SoftwareDistributed Systems SoftwareLecture OutlineService ModelsService Models: Centralized Service Models: Client-server Service Models: Peer-to-peerBitTorrent ExampleService Models: Peer-to-peerService Models: Thin and thick clients Service Models: Multi-tier client-server architecturesService Models: Processor-poolLecture OutlineDistributed System TypesTypes: Distributed Computing Systems (1/3)Types: Distributed Computing Systems (2/3)Types: Distributed Computing Systems (3/3)Grid Computing in DetailGrid ComponentsTypes: Distributed Information Systems (1/4)PerformanceLecture OutlinePerformance: MeasuresPerformance: MeasuresLecture OutlineProgramming Distributed SystemsExample with Message PassingThe Hardware:Writing an MPI programMessage Passing Pseudo codeMPI Code (wordcount.c)Running the MPI CodeAnother Example: Matrix-Vector MultiplyDecompositionAggregating the resultProgram PseudocodeMPI Code (m_v_multiply.c)MPI Code (m_v_multiply.c)Closing NotesDo you … ?ReferencesCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingIntroduction to Cloud ComputingMajd F. SakrDistributed Systems15-319, spring 201011thLecture, Feb 16thCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud Computing2Lecture Motivation Understand Distributed Systems Concepts Understand the concepts / ideas and techniques from Distributed Systems which have made way to Cloud ComputingCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingLecture Outline What are Distributed Systems? Distributed vs. Parallel Systems Advantages and Disadvantages Distributed System Design Hardware Software Service Models Distributed System Types Performance of Distributed Systems Programming Distributed SystemsCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingDo you … ?TimeNow I appreciate why Cloud Computing is importantNow I know what Cloud Computing isDistributed Systems End of week twoParallel Processing End of week fiveCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingWhat is a Distributed System? Distributed Computing: a CS field that studies distributed systems Distributed System: a group of independent/autonomous computers that  are networked together appear to the user as a one computer Work together to achieve a common goalCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingWhat is a Distributed System? Distributed Computing: a CS field that studies ideas around designing and building distributed systems and infrastructure to enable such systems Distributed System: a group of independent/autonomous computers that  are networked together appear to the user as a one computer Work together to achieve a common goalClouds can be Distributed SystemsCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingHistory Problems that are larger than what a single machine can handle Computer Networks, Message passing were invented to facilitate distributed systems ARPANET eventually became InternetCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingWhere are they used? Strategic Systems (Defense / Intelligence) Bioinformatics Visualization and Graphics Economics and Finance Scientific ComputingCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingLecture Outline What are Distributed Systems? Distributed vs. Parallel Systems Advantages and Disadvantages Distributed System Design Hardware Software Service Models Distributed System Types Performance of Distributed Systems Programming Distributed SystemsCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingParallel vs. Distributed Systems A concurrent system could be Parallel or Distributed: Two possible Views to make the distinction View 1: Parallel System : a particular tightly-coupled form of distributed computing Distributed System: a loosely-coupled form of parallel computing  View 2: Parallel System: processors access a shared memory to exchange information  Distributed System: uses a “distributed memory”. Massage passing is used to exchange information between the processors as each one has its own private memory.Distributed ComputingCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingFurther Distinctions Granularity Parallel Systems are typically finer-grained Distributed Systems Distributed Systems are typically the most coarse-grained.Distributed SystemsParallel SystemsCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingLecture Outline What are Distributed Systems? Distributed vs. Parallel Systems Advantages and Disadvantages Distributed System Design Hardware Software Service Models Distributed System Types Performance of Distributed Systems Programming Distributed SystemsCarnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingAdvantages of Distributed Systems (1/2) …Over Centralized Systems Economics: lower (price/performance) ratio Speed:  May have a more total computing power than a centralized system Enhanced performance through load distributing. Inherent Distribution: Some applications are inherently distributed Availability and Reliability: No single point of failure.  The system survives even if a small number of machines crash Incremental Growth: Can add computing power on to your existing infrastructurevs.Carnegie MellonSpring 2010 ©15-319 Introduction to Cloud ComputingAdvantages (2/2) …Over Independent PCs Computation: can be shared over multiple machines Shared management of system: backups & maintenance… Data Sharing: many users can access the same common database Resources Sharing: can share