Building Preservation EnvironmentsTopicsPreservationPreservation CommunitiesSlide 5Digital LibrariesPreservation MandatesInterPARES - DiplomaticsSlide 9Preservation ApproachData GridsSlide 12Preservation EnvironmentInfrastructure IndependenceData Grids Provide a Level of Indirection for Each Naming ConventionDemonstrationSlide 17Storage Resource Broker 3.3Accessing Multiple Types of Storage SystemsStandard Data Access OperationsAccessing Data at Multiple SitesBuilding a Distributed CollectionFederated Server ArchitectureManaging AccessCollection-owned DataProvide Context for DataDatabase OperationsNational Archives and Records Administration - Research Prototype Persistent ArchivePreservation RequirementsReplicationFederationData Grid ZonesPowerPoint PresentationExamples of ExtensibilitySlide 35Slide 36Sites Using the SRBResearch AreasCharacterizing KnowledgeKnowledge ManagementPreservation StrategiesPersistent ObjectsPreservation StandardsContainersSelf-instantiating ArchiveSlide 46Example Web CrawlCollection RequirementsAccessioning Web SitesPersistent Archive CollectionsA Few Statistics on NSDL Content SDSC Crawl (April 03, 4 Links Deep)Encoding Formats Present in ArchiveAutomated Processes: Categorizing the “Space” of all Descriptive PatternsA String Analysis ApproachSlide 55Metadata ValidationRegular ExpressionsSlide 58Slide 59Regular ExpressionLessons LearnedSlide 62For More InformationBuilding Preservation Building Preservation EnvironmentsEnvironmentsReagan W. MooreReagan W. MooreSan Diego Supercomputer CenterSan Diego Supercomputer [email protected]@sdsc.eduhttp://www.sdsc.edu/srb/http://www.sdsc.edu/srb/Storage Resource BrokerStorage Resource BrokerTopicsTopics•Preservation environments•Digital library technology•Data grid technology•Fundamental concepts / future research•Data / information / knowledge•Persistent objects•Knowledge managementPreservationPreservation•Archival processes through which a digital entity is extracted from its creation environment, and then supported in a preservation environment, while maintaining authenticity and integrity information.•Extraction process requires insertion of support infrastructure underneath the digital material•Goal is infrastructure independence, the ability to use any commercial storage system, database, or access mechanismPreservation CommunitiesPreservation Communities•InterPARES - diplomatics•Preservation of records•NARA•Preservation of records from federal agencies•State archives•Preservation of submitted “collections”•Continuum model•Preservation of active data and recordsPreservationPreservation•What differentiates a preservation environment from a digital library?Digital LibrariesDigital Libraries•Support the community vocabulary•Discovery and browse using community relevant terms•Support the community data format•Maintain information on the data format of each item•Support the community access services•Provide services that manipulate and display the community data formatPreservation MandatesPreservation Mandates•Diplomatics•Authenticity•Integrity•NARA•Infrastructure independence•Scalability•State archives•Automation of archival processesInterPARES - DiplomaticsInterPARES - Diplomatics•Authenticity - maintain links to metadata for:•Date record is made•Date record is transmitted•Date record is received•Date record is set aside [i.e. filed]•Name of author (person or organization issuing the record)•Name of addressee (person or organization for whom the record is intended)•Name of writer (entity responsible for the articulation of the record’s content)•Name of originator (electronic address from which record is sent)•Name of recipient(s) (person or organization to whom the record is sent)•Name of creator (entity in whose archival fonds the record exists)•Name of action or matter (the activity for which the record is created)•Name of documentary form (e.g. E-mail, report, memo)•Identification of digital components •Identification of attachments (e.g. digital signature)•Archival bond (e.g. classification code)InterPARES - DiplomaticsInterPARES - Diplomatics•Integrity - maintain links to metadata for•Name(s) of the handling office / officer•Name of office of primary responsibility for keeping the record•Annotations or comments•Actions carried out on the record•Technical modifications due to transformative migration•ValidationPreservation ApproachPreservation Approach•Provide mechanisms to:•Create archival context for the content•Context is preservation metadata (provenance, administrative, descriptive, structural, behavioral)•Content is the submitted digital entity•Assert integrity - the consistency between the context and the content•Track operations done on material and update context•Assert authenticity - that the material represents the original site•Track the chain of custody•Manage technology evolution (encoding standard, storage repository, information repository, access methods)Data GridsData Grids•What is the difference between a preservation environment and a data grid?Data GridsData Grids•Manage shared collections that are distributed in space•Location of item, access controls, checksums•Implement infrastructure independence•Standard operations for interacting with storage repositories•Implement presentation independence•Standard APIs to support porting of user interfacesPreservation EnvironmentPreservation Environment•Digital library infrastructure that supports•Preservation metadata•Arrangement and description of items•Access mechanisms•Data grid infrastructure that supports•Shared collections that are migrated forward in time•Management of technology evolution•Administrative metadata providing status of recordsInfrastructure IndependenceInfrastructure IndependenceStorage Repository• Storage location• User name• File name• File context (creation date,…)• Access constraintsData Access Methods (Web Browser, DSpace, OAI-PMH)Naming conventions provided by storage systemsData Grids Provide a Level of Indirection Data Grids Provide a Level of Indirection for Each Naming Conventionfor Each Naming ConventionStorage Repository• Storage location• User name• File name• File context (creation date,…)• Access constraintsData Grid• Logical resource name space• Logical user name space• Logical file name