Avoiding Technological Quicksand: Finding a Viable Technical Foundation for Digital Preservationby Jeff RothenbergCLIR JanuPart I: BackgroundI. Background - continuedPart II: Overview of ReportPart II: Overview (continued)Digital Longevity ProblemPart II: Overview (continued)Modes of LossPart II: Overview (continued)Preservation in the Digital AgePart II: Overview (continued)Scope of the ProblemPart II: Overview (continued)Technical Dimensions of the ProblemPart II: Overview (continued)4 categories, each with its own problems:Part II: Overview (continued)Criteria for an Ideal SolutionPart II: Overview (continued)The Emulation SolutionPart II: Overview (continued)Emulation: DiagramPart II: Overview (continued)Emulation in a NutshellPart II: Overview (continued)Research Required for the Emulation ApproachPart III: Opinions, Pro & ConPart III: Opinions, Pro & Con (continued)Part III: Opinions, Pro & Con (continued)Part IV: Conclusion20 Jan 2005 CS 891 Seminar/ Joan SmithAvoiding Technological Quicksand: Finding a Viable Technical Foundation for Digital Preservationby Jeff RothenbergCLIR January 1999I. BackgroundII. Overview of ArticleIII. Opinions, Pro & ConIV. Conclusion20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 2Part I: Background• Preservation issues exist in all areas of civilization– Skills [example: Japan’s ‘living treasures’]– Lifestyles [example: Pennsylvania Dutch communities]– Libraries [example: acid deterioration of paper]– Languages [example: Native American tongues]– Technology [example: classic films]• Pace of change has changed…– Old technologies changed communities over course of generations [Check out James Burke’s series “Connections”]– New technologies remake communities overnight [cars, web]• Pace of change has complicated preservation dilemma20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 3I. Background - continued• Digital objects lack inherent self-preservation qualities– Nothing about a CD indicates its use, purpose, contents, etc. to a future discoverer– Compare with furniture, clothing which have obvious possible uses and supporting information (paintings, traditions, etc.)• Huge growth in purely digital data– Grown well beyond our capacity to save even a large portion of it– Much is not humanly interpretable (Mars mission data, e.g.)• Digital information is both larger and smaller– More can be stored in less square footage– It requires supporting infrastructure that is not small nor cheap• Infrastructure itself poses preservation dilemmas– Operating, maintaining (parts), powering, storing…20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 4Part II: Overview of Report“Digital Information lasts forever (or 5 years, whichever comes first)”• Focus is preservation of digital objects – Does not include digital preservation of non-digital objects, but issues apply to these as well• Discusses pitfalls of various common solutions:– Converting to print– Adoption of standards– Software/hardware ‘museums’– Continual migration of systems to new hardware• Proposes emulation as best long-term solution20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 5Part II: Overview (continued)Digital Longevity Problem•Digital media has relatively short lifespan• A Perfect copy does not mean a usable copy• Obsolescence of hardware and software is a major issue•The right to keep, make, share copies is equally problematic• Costs in terms of physical resources - including manpower - are high20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 6Part II: Overview (continued)Modes of Loss• Physical decay of media– Tape: 200 years or 2 rewinds?– DVD/CD: 50 years or 5 scratches?– Disk: 10 years or 10 power surges?• Loss of information about the format, encoding, or compression of files– Byte stream of a bank statement <> bank statement (paper)• Obsolescence of hardware– Limited repair capability if preserved, but old, hardware breaks• Unavailability of software– Interdependence of the application and the OS• Decision to not preserve– Who needs Khomeini’s writings?20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 7Part II: Overview (continued)Preservation in the Digital Age• Digital objects must retain their digital essence– Machine readable, copy-able, distributable, functional, etc.• Converting digital documents to paper does not solve problem– Item may have non-printable, interactive properties– No longer retains digital essence (see above)• What is an original?– For paper, meaning is clear– For digital objects, it does not mean keeping the original hard drive, for example:• Main purpose is to guarantee fidelity to the “First One”.20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 8Part II: Overview (continued)Scope of the Problem• Information loss has become an accepted cost of technical evolution• Migration to new systems often results in ‘orphaning’ of data (the ‘really old’ stuff is not ‘worth the effort’)• Today’s ‘backups’ & archives become tomorrow’s orphans– Long-term survivability is the big problem– Multi-media and hypermedia-based digital items are as vulnerable as text-based documents, possibly more so20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 9Part II: Overview (continued)Technical Dimensions of the Problem• Physical issues: decay, obsolescence, corruption– Media vulnerability examples-> CD examples (commercial, home-made)-> disk & tape formats• Software dependence“digital documents exist only by virtue of software that understands how to access and display them”– Obsolescence a factor – “version” compatibilities, etc.– OS dependence– Co-dependence on hardware factors• Other issues– Cost, IPR, authentication– Anthropological considerationsMarshall McLuhan: “The medium is the message.”20 Jan 2005/Joan Smith CS 791/891 Digital Preservation 10Part II: Overview (continued)4 categories, each with its own problems:1. Hard copy – not meaningful; loss of digital aspects; human vs. machine tolerance of error2. Standards – suffer from ‘value-added’ aspect of commercial profit needs; vendor implementation deviates from strict adherence; Ex:ASCIIÆUNICODE (8 bit to 16 bit)3. Museums – Can PCs run forever? Media durability (and ∴ ongoing physical connectability) issues; best role is as future digi-archeologists (my term); cost