DOC PREVIEW
UH COSC 6340 - Grid Data Management Systems & Services

This preview shows page 1-2-3-4-5-6-7-8-9-10-73-74-75-76-77-78-79-80-81-82-83-147-148-149-150-151-152-153-154-155-156 out of 156 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 156 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

VLDB 2003 BerlinGrid Data Management Systems & ServicesData Grid Management Systems – Part IArun Jagatheesan, Reagan MooreGrid Services for Structured data –Part IIPaul Watson, Norman PatonVLDB TutorialBerlin, 2003VLDB 2003 BerlinPart I:Data Grid Management SystemsArun JagatheesanReagan Moore{arun, moore}@sdsc.eduSan Diego Supercomputer CenterUniversity of California, San DiegoVLDB TutorialBerlin, 2003http://www.npaci.edu/DICE/SRB/VLDB 2003 Berlin3Tutorial Part I Outline• Concepts• Introduction to Grid Computing• Proliferation of Data Grids• Data Grid Concepts• Practice• Real life use cases SDSC Storage Resource Broker (SRB)• Hands on Session• Research• Active Datagrid Collections• Data Grid Management Systems (DGMS)• Open Research IssuesVLDB 2003 Berlin4Distributed Computing© Images courtesy of Computer History MuseumVLDB 2003 Berlin5Distributed Data Management• Data collecting• Sensor systems, object ring buffers and portals• Data organization• Collections, manage data context• Data sharing• Data grids, manage heterogeneity• Data publication• Digital libraries, support discovery• Data preservation• Persistent archives, manage technology evolution• Data analysis• Processing pipelines, manage knowledge extractionVLDB 2003 Berlin6What is a Grid?“Coordinated resource sharing and problem solving in dynamic, multi-institutional virtual organizations”Ian Foster, ANLWhat is Middleware?Software that manages distributed state information for results of remote servicesReagan Moore, SDSCVLDB 2003 Berlin7Data Grids• A datagrid provides the coordinated management mechanisms for data distributed across remote resources.• Data Grid• Coordinated sharing of information storage• Logical name space for location independent identifiers• Abstractions for storage repositories, information repositories, and access APIs• Computing grid and the datagrid part of the Grid.• Data generation versus data managementVLDB 2003 Berlin8Tutorial Part I Outline• Concepts• Introduction to Grid Computing• Proliferation of Data Grids• Data Grid Concepts• Practice• Real life use cases SDSC Storage Resource Broker (SRB)• Hands on Session• Research• Active Datagrid Collections• Data Grid Management Systems (DGMS)• Open Research Issues Are data grids in production use?How are they applied?VLDB 2003 Berlin9Storage Resource Broker at SDSCMore features, 60 Terabytes and countingVLDB 2003 Berlin10NSF Infrastructure Programs• Partnership for Advanced Computational Infrastructure -PACI• Data grid - Storage Resource Broker• Distributed Terascale Facility - DTF/ETF• Compute, storage, network resources• Digital Library Initiative, Phase II - DLI2• Publication, discovery, access• Information Technology Research projects - ITR• SCEC Southern California Earthquake Center• GEON GeoSciences Network• SEEK Science Environment for Ecological Knowledge• GriPhyN Grid Physics Network• NVO National Virtual Observatory• National Middleware Initiative - NMI• Hardening of grid technology (security, job execution, grid services)• National Science Digital Library - NSDL• Support for education curricula modulesVLDB 2003 Berlin11Federal Infrastructure Programs• NASA• Information Power Grid - IPG• Advanced Data Grid - ADG• Data Management System - Data Assimilation Office• Integration of DODS with Storage Resource Broker• Earth Observing Satellite EOS data pools • Consortium of Earth Observing Satellites CEOS data grid• Library of Congress• National Digital Information Infrastructure and Preservation Program -NDIIPP• National Archives and Records Administration (NARA) and National Historical Public Records Commission• Prototype persistent archives• NIH• Biomedical Informatics Research Network data grid• DOE• Particle Physics Data GridVLDB 2003 Berlin12NSF GriPhyN/iVDGL• Petabyte scale Virtual Data Grids • GriPhyN, iVDGL, PPDG – Trillium• Grid Physics Network• International Virtual Data Grid Laboratory• Particle Physics Data Grid• Distributed worldwide • Harness Petascale processing, data resources• DataTAG – Transatlantic with European SideVLDB 2003 Berlin13Tera Grid• Launched in August 2001• SDSC, NCSA, ANL, CACR, PSC• 20 Tera flops of computing power • One peta byte of storage• 40 Gb/sec (FASTEST network on planet)• “Building the Computational Infrastructure for Tomorrow's Scientific Discovery”VLDB 2003 Berlin14European Datagrid• European Union• Different Communities• High Energy Physics• Biology• Earth Science• Collaborate and complement other European and US projectsVLDB 2003 Berlin15NIH BIRN• Biomedical Informatics Research Network• Access and analyze biomedical image data• Data resources distributed throughout the country• Medical schools and research centers across the nation• Stable high performance grid based environment• Coordinate sharing of virtual data collections and data mining• Growing fast!VLDB 2003 Berlin16Commonality in all these projects• Distributed data management• Authenticity• Access controls• Curation• Data sharing across administrative domains• Common name space for all registered digital entities• Data publication • Browsing and discovery of data in collections• Data Preservation• Management of technology evolutionVLDB 2003 Berlin17Data and Requirements• Mostly Unstructured, heterogeneous• Images, Files, Semi-structured, databases, streams, …• File systems, SAN, FTP sites, Web servers, Archives• Community-Based• Shared amongst one or more communities• Meta-data• Different Meta-data schemas for the same data• Different Notations, ontologies• Sensitive to Sharing• Nobel Prizes, Federal Agreements, project dataVLDB 2003 Berlin18Tutorial Part I Outline• Concepts• Introduction to Grid Computing• Proliferation of Data Grids• Data Grid Concepts• Practice• Real life use cases SDSC Storage Resource Broker (SRB)• Hands on Session• Research• Active Datagrid Collections• Data Grid Management Systems (DGMS)• Open Research IssuesVLDB 2003 Berlin19Data GridData GridUsing a Data Grid – in AbstractAsk for data•User asks for data from the data gridData delivered•The data is found and returned•Where & how details are managed by data grid•But access controls are specified by ownerVLDB 2003 Berlin20Data Grid Transparencies• Find data without knowing the


View Full Document

UH COSC 6340 - Grid Data Management Systems & Services

Download Grid Data Management Systems & Services
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Grid Data Management Systems & Services and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Grid Data Management Systems & Services 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?