Slide Number 1AcknowledgementAdministrative Administrative – MP3AdministrativePlan for Today Sample Grid ApplicationsExample: Rapid Atmospheric Modeling System, Colorado State UniversitySlide Number 9Slide Number 10Recently: Large Hadron ColliderThe GridDistributed Computing Resources in GridApplication Coded by a MeteorologistApplication Coded by a MeteorologistSlide Number 16Slide Number 17Slide Number 18Slide Number 19Tiered Architecture (OSI 7 layer-like)Trends: TechnologyTrends: UsersGlobus AllianceMorePropheciesRecap: Grid vs. …Slide Number 27Definitions Definitions Definitions Grid versus P2P - Pick your favoriteApplicationsApplicationsScale and FailureScale and FailureSome Things Grid Researchers Consider ImportantServices and InfrastructureServices and InfrastructureSummary: Grid and P2PEXTRAGrid History – 1990’sCS 425: Distributed SystemsLecture 27 “The Grid”Klara NahrstedtAcknowledgement• The slides during this semester are based on ideas and material from the following sources: – Slides prepared by Professors M. Harandi, J. Hou, I. Gupta, N. Vaidya, Y-Ch. Hu, S. Mitra. – Slides from Professor S. Gosh’s course at University o Iowa.Administrative • MP3 posted – Deadline December 7 (Monday) – pre-competition• Top five groups will be selected for final demonstration on Tuesday, December 8– Demonstration Signup Sheets for Monday, 12/7, will be made available this week (Thursday, 12/3 lecture)– Main Demonstration in front of the Qualcomm Representative will be on Tuesday, December 8 afternoon -details will be announced on Thursday and also on the website and newsgroupAdministrative – MP3• Don’t forget versioning of your messages in your protocols between client and server (Google phones are getting quickly obsolete so it will be important to know what version of client software/hardware you are running and synchronize the overall application as we upgrade)– Readme file must include:• Boot-straping routine – how one install your system –developers manuscript• How one use your system – usage prescription for users• Known bugs, what are the issues with your system/application – Tar or zip your source code and upload it to agora wiki• URL Information will be provided on the web/in class/on newsgroup– Fill out project template as specifiedAdministrative• MP3 instructions– Here's the template page for cs425 students to copy and fill out.https://agora.cs.illinois.edu/display/mlc/cs425-TemplateProject• Website only cs425 students and instructors can access to post the template page and also upload attachmentshttps://agora.cs.illinois.edu/display/mlc/cs425-fa09-projectsPlan for Today • Discussion what is “Grid” distributed computing paradigm • Some basic capabilities of Grid and tools/protocols/services that drive Grid•Comparison between Grid and P2PSample Grid Applications• Astronomers: SETI@Home• Physicists: data from particle colliders• Meteorologists: weather prediction• Bio-informaticians•….Example: Rapid Atmospheric Modeling System, Colorado State University• Weather Prediction is inaccurate• Hurricane Georges, 17 days in Sept 1998• Hurricane Georges, 17 days in Sept 1998– “RAMS modeled the mesoscale convective complex that dropped so much rain, in good agreement with recorded data”– Used 5 km spacing instead of the usual 10 km– Ran on 256+ processorsRecently: Large Hadron Collider• http://lcg.web.cern.ch/lcg/• LHC@home“LHC collisions will produce 10 to 15 petabytes of data a year”http://www.techworld.com/mobility/features/index.cfm?featureid=4074&pn=2The GridSome are 40Gbps links!(The TeraGrid links)“A parallel Internet”Each location is a clusterWisconsinMITNCSA/UIUCDistributed ComputingResourcesin GridApplication Coded by a MeteorologistJob 0Job 2Job 1Job 3Output files of Job 0Input to Job 2Output files of Job 2Input to Job 3Jobs 1 and 2 can be concurrentJob 2Output files of Job 0Input to Job 2Output files of Job 2Input to Job 3May take several hours/days4 stages of a jobInitStage inExecuteStage outPublishComputation Intensive, so Massively ParallelSeveral GBsApplication Coded by a MeteorologistWisconsinMITNCSAJob 0Job 2Job 1Job 3Job 0Job 2Job 1Job 3WisconsinMITCondor ProtocolNCSAGlobus ProtocolJob 0Job 2Job 1Job 3WisconsinMITNCSAGlobus ProtocolInternal structure of differentsites transparent to GlobusExternal Allocation & SchedulingStage in & Stage out of FilesJob 0Job 3WisconsinCondor ProtocolInternal Allocation & SchedulingMonitoringDistribution and Publishing of FilesResource Matchmaking ‘ClassAd’ conceptTiered Architecture (OSI 7 layer-like)GlobusHigh energy Physics appse.g., CondorWorkstations, LANsTrends: Technology• Doubling Periods – storage: 12 mos, bandwidth: 9 mos, and (what law is this?) cpu speed/capacity: 18 mos• Then and NowBandwidth– 1985: mostly 56Kbps links nationwide– 2003: 155 Mbps links widespread– 2009: 1 Gbps links wide spreadDisk capacity– Today’s PCs have 100GBs, and clusters –terabytes/petabytes, same as a 1990 supercomputerTrends: Users• Then and NowBiologists: – 1990: were running small single-molecule simulations – 2003: want to calculate structures of complex macromolecules, want to screen thousands of drug candidatesPhysicists– 2006: CERN’s Large Hadron Collider produced about 10^15 B during the year• Trends in Technology and User Requirements: Independent or Symbiotic?Globus Alliance• Alliance involves U. Illinois Chicago, Argonne National Laboratory, USC-ISI, U. Edinburgh, Swedish Center for Parallel Computers, NCSA• Activities : research, testbeds, software tools, applications• Globus Toolkit (latest ver – GT4)“The Globus Toolkit includes software services and libraries for resource monitoring, discovery, and management, plus security and file management. Its latest version, GT3, is the first full-scale implementation of new Open Grid Services Architecture (OGSA).”More• Entire community, with multiple conferences, get-togethers (GGF), and projects• Grid Projects:http://www-fp.mcs.anl.gov/~foster/grid-projects• Grid Users: – Today: Core is the physics community (since the Grid originates from the GriPhyN project)– Tomorrow: biologists, large-scale computations (nug30 already)?PropheciesIn 1965, MIT's Fernando Corbató and the other designers of the Multics operating system envisioned a computer facility operating “like a power company or water company”.Plug your
View Full Document
Unlocking...