CS 525 Advanced Topics in Distributed Systems Spring 07Two Questions We’ll Try to AnswerExample: Rapid Atmospheric Modeling System, ColoState UDistributed Computing ResourcesAn Application Coded by a PhysicistSlide 6Slide 7Slide 8Slide 9Slide 10Tiered Architecture (OSI 7 layer-like)The Grid TodayGlobus AllianceMoreSome Things Grid Researchers Consider ImportantGrid History – 1990’sTrends: TechnologyTrends: UsersPropheciesSlide 20DefinitionsSlide 22Grid versus P2P - Pick your favoriteApplicationsSlide 25Scale and FailureSlide 27Services and InfrastructureSlide 29Coolness FactorSlide 31Summary: Grid and P2PCrossover IdeasNext Week OnwardsBackup SlidesSlide 36Slide 37Indranil Gupta (Indy)Lecture 6The GridFebruary 1, 2007CS 525 Advanced Topics in Distributed SystemsSpring 07Two Questions We’ll Try to Answer•What is the Grid? Basics, no hype.•What is its relation to p2p?Example: Rapid Atmospheric Modeling System, ColoState U•Hurricane Georges, 17 days in Sept 1998–“RAMS modeled the mesoscale convective complex that dropped so much rain, in good agreement with recorded data”–Used 5 km spacing instead of the usual 10 km–Ran on 256+ processors•Can one run such a program without access to a supercomputer?WisconsinMITNCSADistributed ComputingResourcesAn Application Coded by a PhysicistJob 0Job 2Job 1Job 3Output files of Job 0Input to Job 2Output files of Job 2Input to Job 3Jobs 1 and 2 can be concurrentAn Application Coded by a PhysicistJob 2Output files of Job 0Input to Job 2Output files of Job 2Input to Job 3May take several hours/days4 stages of a jobInitStage inExecuteStage outPublishComputation Intensive, so Massively ParallelSeveral GBsWisconsinMITNCSAJob 0Job 2Job 1Job 3Job 0Job 2Job 1Job 3WisconsinMITCondor ProtocolNCSAGlobus ProtocolJob 0Job 2Job 1Job 3WisconsinMITNCSAGlobus ProtocolInternal structure of differentsites invisible to GlobusExternal Allocation & SchedulingStage in & Stage out of FilesJob 0Job 3WisconsinCondor ProtocolInternal Allocation & SchedulingMonitoringDistribution and Publishing of FilesTiered Architecture (OSI 7 layer-like)Resource discovery,replication, brokeringHigh energy Physics appsGlobus, CondorWorkstations, LANsOpportunity for Crossover ideas from p2p systemsThe Grid TodaySome are 40Gbps links!(The TeraGrid links)“A parallel Internet”Globus Alliance•Alliance involves U. Illinois Chicago, Argonne National Laboratory, USC-ISI, U. Edinburgh, Swedish Center for Parallel Computers•Activities : research, testbeds, software tools, applications•Globus Toolkit (latest ver - GT3) “The Globus Toolkit includes software services and libraries for resource monitoring, discovery, and management, plus security and file management. Its latest version, GT3, is the first full-scale implementation of new Open Grid Services Architecture (OGSA).”More•Entire community, with multiple conferences, get-togethers (GGF), and projects•Grid Projects:http://www-fp.mcs.anl.gov/~foster/grid-projects/•Grid Users: –Today: Core is the physics community (since the Grid originates from the GriPhyN project)–Tomorrow: biologists, large-scale computations (nug30 already)?Some Things Grid Researchers Consider Important•Single sign-on: collective job set should require once-only user authentication•Mapping to local security mechanisms: some sites use Kerberos, others using Unix•Delegation: credentials to access resources inherited by subcomputations, e.g., job 0 to job 1•Community authorization: e.g., third-party authenticationGrid History – 1990’s•CASA network: linked 4 labs in California and New Mexico–Paul Messina: Massively parallel and vector supercomputers for computational chemistry, climate modeling, etc.•Blanca: linked sites in the Midwest–Charlie Catlett, NCSA: multimedia digital libraries and remote visualization•More testbeds in Germany & Europe than in the US•I-way experiment: linked 11 experimental networks–Tom DeFanti, U. Illinois at Chicago and Rick Stevens, ANL:, for a week in Nov 1995, a national high-speed network infrastructure. 60 application demonstrations, from distributed computing to virtual reality collaboration.•I-Soft: secure sign-on, etc.Trends: Technology•Doubling Periods – storage: 12 mos, bandwidth: 9 mos, and (what law is this?) cpu speed: 18 mos•Then and Now Bandwidth–1985: mostly 56Kbps links nationwide–2004: 155 Mbps links widespreadDisk capacity–Today’s PCs have 100GBs, same as a 1990 supercomputerTrends: Users•Then and Now Biologists: –1990: were running small single-molecule simulations –2004: want to calculate structures of complex macromolecules, want to screen thousands of drug candidatesPhysicists–2006: CERN’s Large Hadron Collider produced 10^15 B/year•Trends in Technology and User Requirements: Independent or Symbiotic?PropheciesIn 1965, MIT's Fernando Corbató and the other designers of the Multics operating system envisioned a computer facility operating “like a power company or water company”.Plug your thin client into the computing Utiling and Play your favorite Intensive Compute &Communicate Application–[Will this be a reality with the Grid?]“We must addressscale & failure”“We need infrastructure”P2P GridDefinitions GridP2P•“Infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities” (1998)•“A system that coordinates resources not subject to centralized control, using open, general-purpose protocols to deliver nontrivial QoS” (2002)•“Applications that takes advantage of resources at the edges of the Internet” (2000)•“Decentralized, self-organizing distributed systems, in which all or most communication is symmetric” (2002)Definitions GridP2P•“Infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities” (1998)•“A system that coordinates resources not subject to centralized control, using open, general-purpose protocols to deliver nontrivial QoS” (2002)•“Applications that takes advantage of resources at the edges of the Internet” (2000)•“Decentralized, self-organizing distributed systems, in which all or most communication is symmetric” (2002)525: (good legal applications without intellectual fodder)525: (clever designs without good, legal applications)Grid versus P2P - Pick your favoriteApplicationsGrid•Often complex & involving
View Full Document
Unlocking...