Introduction to Grid Computing The Globus Project Argonne National Laboratory USC Information Sciences Institute http www globus org Copyright c 2002 University of Chicago and The University of Southern California All Rights Reserved This presentation is licensed for use under the terms of the Globus Toolkit Public License See http www globus org toolkit download license html for the full text of this license Outline z Introduction to Grid Computing z Some Definitions z Grid Architecture z The Programming Problem z The Globus Toolkit Introduction Security Resource Management Information Services Data Management z Related work z Futures and Conclusions December 4 2002 Introduction to Grid Computing 2 The Grid Problem z Flexible secure coordinated resource sharing among dynamic collections of individuals institutions and resource From The Anatomy of the Grid Enabling Scalable Virtual Organizations z Enable communities virtual organizations to share geographically distributed resources as they pursue common goals assuming the absence of central location central control omniscience existing trust relationships December 4 2002 Introduction to Grid Computing 3 Elements of the Problem z Resource sharing Computers storage sensors networks Sharing always conditional issues of trust policy negotiation payment z Coordinated problem solving Beyond client server distributed data analysis computation collaboration z Dynamic multi institutional virtual orgs Community overlays on classic org structures Large or small static or dynamic December 4 2002 Introduction to Grid Computing 4 Online Access to Scientific Instruments Advanced Photon Source wide area dissemination archival storage real time collection desktop VR clients with shared controls tomographic reconstruction DOE X ray grand challenge ANL USC ISI NIST U Chicago December 4 2002 Introduction to Grid Computing 5 Data Grids for High Energy Physics PBytes sec Online System 100 MBytes sec 20 TIPS There are 100 triggers per second Each triggered event is 1 MByte in size 622 Mbits sec or Air Freight deprecated France Regional Centre SpecInt95 equivalents Offline Processor Farm There is a bunch crossing every 25 nsecs Tier 1 1 TIPS is approximately 25 000 Tier 0 Germany Regional Centre 100 MBytes sec CERN Computer Centre Italy Regional Centre FermiLab 4 TIPS 622 Mbits sec Tier 2 622 Mbits sec Institute Institute Institute 0 25TIPS Physics data cache Institute 1 MBytes sec Tier 4 Caltech 1 TIPS Tier2 Centre Tier2 Centre Tier2 Centre Tier2 Centre 1 TIPS 1 TIPS 1 TIPS 1 TIPS Physicists work on analysis channels Each institute will have 10 physicists working on one or more channels data for these channels should be cached by the institute server Physicist workstations Image courtesy Harvey Newman Caltech December 4 2002 Introduction to Grid Computing 6 Mathematicians Solve NUG30 z z z Looking for the solution to the NUG30 quadratic assignment problem An informal collaboration of mathematicians and computer scientists Condor G delivered 3 46E8 CPU seconds in 7 days peak 1009 processors in U S and Italy 8 sites 14 5 28 24 1 3 16 15 10 9 21 2 4 29 25 22 13 26 17 30 6 20 19 8 18 7 27 12 11 23 MetaNEOS Argonne Iowa Northwestern Wisconsin December 4 2002 Introduction to Grid Computing 7 Network for Earthquake Engineering Simulation z z NEESgrid US national infrastructure to couple earthquake engineers with experimental facilities databases computers each other On demand access to experiments data streams computing archives collaboration NEESgrid Argonne Michigan NCSA UIUC USC December 4 2002 Introduction to Grid Computing 8 Home Computers Evaluate AIDS Drugs z Community 1000s of home computer users Philanthropic computing vendor Entropia Research group Scripps z Common goal advance AIDS research December 4 2002 Introduction to Grid Computing 9 Broader Context z Grid Computing has much in common with major industrial thrusts Business to business Peer to peer Application Service Providers Storage Service Providers Distributed Computing Internet Computing z Sharing issues not adequately addressed by existing technologies Complicated requirements run program X at site Y subject to community policy P providing access to data at Z according to policy Q High performance unique demands of advanced high performance systems December 4 2002 Introduction to Grid Computing 10 Why Now z z z z Moore s law improvements in computing produce highly functional endsystems The Internet and burgeoning wired and wireless provide universal connectivity Changing modes of working and problem solving emphasize teamwork computation Network exponentials produce dramatic changes in geometry and geography December 4 2002 Introduction to Grid Computing 11 Network Exponentials z Network vs computer performance Computer speed doubles every 18 months Network speed doubles every 9 months Difference order of magnitude per 5 years z 1986 to 2000 Computers x 500 Networks x 340 000 z 2001 to 2010 Computers x 60 Networks x 4000 Moore s Law vs storage improvements vs optical improvements Graph from Scientific American Jan2001 by Cleo Vilett source Vined Khoslan Kleiner Caufield and Perkins December 4 2002 Introduction to Grid Computing 12 The Globus Project Making Grid computing a reality z z z z z Close collaboration with real Grid projects in science and industry Development and promotion of standard Grid protocols to enable interoperability and shared infrastructure Development and promotion of standard Grid software APIs and SDKs to enable portability and code sharing The Globus Toolkit Open source reference software base for building grid infrastructure and applications Global Grid Forum Development of standard protocols and APIs for Grid computing December 4 2002 Introduction to Grid Computing 13 Selected Major Grid Projects Name Access Grid BlueGridNew DISCOM URL Sponsors g www mcs anl gov FL accessgrid DOE NSF g IBM g www cs sandia gov Create deploy group collaboration systems using commodity technologies Grid testbed linking IBM laboratories Create operational Grid providing access to resources at three U S DOE weapons discom DOE Defense Programs laboratories DOE Science sciencegrid org Grid g DOE Office of Science New Earth System Grid ESG g European Union EU DataGrid Focus earthsystemgrid org DOE Office of Science g eu datagrid org December 4 2002 European Union Create operational Grid providing access to resources applications at U S DOE science laboratories partner universities