The Question of QualityGoals for this classUnderstanding Quality in a DLGetting the dataWhat are we looking for?PowerPoint PresentationDimensions of QualityWhat information do we need - related to Digital ObjectsInformation need - Digital Objects, continuedSlide 10Information need - Metadata SpecificationInformation - Collection and CatalogInformation about the RepositoryService Information NeedService Information, continuedMaking more concreteDigital Object AccessibilityDigital Object Accessibility - formallyAn illustrationAccessiblity caseWith the numbersSolidifying PertinencePertinencePertinence formulaPreservabilityDigital preservation techniquesPreservability issuesPreservability factorsCalculating ObsolescenceCalculating fidelityA Preservation ScenarioReferencesThe Question of QualityWeek 9Most of this presentation is based on the work of Marcos Goncales as cited in the referencesGoals for this class•Consider quality in digital libraries–How do we define quality–How do we measure quality–How does quality control impact a user?•The role of logging–Helpful information–Privacy issues•The status of DL loggingUnderstanding Quality in a DL•Quality indicators: proposed descriptions of quantities or observable variables that may be related to quality–“measures” = stronger term. Requires validation–Gonçalves et al provide analysis of quality conditions and recommend specific quantities to be used.•Dimensions of quality•Proposed indicators•Application to DL concernsGetting the data•Where does the data come from?–Logging–Surveys–Focus Groups•Know what information is needed, then choose the method most likely to provide the data.–More about the sources of data after we see what we need to know.What are we looking for?•Consider that we are concerned about the quality of the following characteristics of a DL:–Data objects–Metadata–Collection–Catalog–Repository–Services•What characteristics do we want each of those to have?Dimensions of QualityDimensions of Quality•Digital Object–Accessibility–Pertinence–Preservability–Relevance–Similarity–Significance–Timeliness•Metadata Specification–Accuracy–Completeness –Conformance•Collection–Completeness•Catalog–Completeness–Consistency•Repository–Completeness–Consistency•Services–Composability–Efficiency–Effectiveness–Extensibility–Reusability–ReliabilityWhat information do we need - related to Digital Objects •Accessibility–What collection?–# of structured streams–Rights management metadata–Communities to be served•Pertinence–Context–Information content–Information needInformation need - Digital Objects, continued•Preservability–Fidelity (lossiness)–Migration cost–Digital object complexity–Stream formats•Relevance–Feature frequency–Inverse document frequency–Document size–Document structure–Query size–Collection sizeInformation need - Digital Objects, continued•Similarity–All the same features as in relevance–Also: citation/link patterns•Significance–Citation/link patterns•Timeliness–Age–Time of latest citation–Collection freshnessInformation need - Metadata Specification•Accuracy–Accurate attributes–# attributes in the record•Completeness–Missing attributes–Schema size•Conformance–Conformant attributes–Schema sizeInformation - Collection and Catalog•Completeness of the Collection–Collection size–Size of an “ideal” collection•Completeness of the Catalog–# of digital objects with no metadata •Item level metadata–Size of the collection•Catalog Consistency–# of metadata specifications per digital objectInformation about the Repository•Completeness–# of collections•Consistency–# of collections –Catalog/collection match•How well do the catalogs match the collections?•Are the catalogs for all the collections at the same level of detail?Service Information Need•Composability (ability to be combined to form new services)–Extensibility–Reusability•Efficiency–Response time•Effectiveness–Precision/recall (of search)–ClassificationService Information, continued•Extensibility–# extended services–# services in the DL–# lines of code per service manager•Reusability–# reused services–# services in the DL–# lines of code per service manager•Reliability–# service failures–# accessesMaking more concrete•Each of the measures listed gives an idea of the information need•Exactly what do we measure?•How do we combine numbers obtained to get a usable result?•Following pages describe specific measures and formulas for combining those.Digital Object Accessibility•Basic requirement–If a user cannot access the DO, there is little point in having it in the DL–Identified measures:•Collection, # structured streams, rights management metadata, communities–Say it another way:•Is it present in a collection in the repository?•Is there a service that can retrieve and display the content?•Is the rights management open enough for access by this user?Digital Object Accessibility - formallyDefine dox = a specific digital objectAccessibility = Acc(dox, acy) =–0, if there is no collection C in the DL repository R such that dox  C–Otherwise, acc = (∑z  struct_streams(dox) rz(acy))/ |struc_streams(dox)|–where rz(acy)) is a rights management rule defined as •1, if –Z has no access constraints, or –Z has access constraints and acy  cmz, »Where cmz,  Soc(1) is a community that has the right to access z; and •0, otherwiseThis does not deal with accessibilty related to accessing the streamsAn illustration•NDLTD is the Networked Digital Library of Theses and Dissertations–Some institutions requre that all theses and dissertations be stored in this DL–Student chooses how visible to make the document.•Parts of the document may be visible while other parts are not•The document, or parts of it, may be visible to a restricted community.Accessiblity case•etdx is a specific electronic thesis or dissertation of interest•acc(etdx) is–0 if it is not in the collection–Otherwise (∑z  struct_streams(etdx) rz(acy))/ |struc_streams(dox)|•Where rz(acy) = 1 –if etdx is marked “world wide access” or etdx is marked “local institution only” and acy  C where C is defined as identifiable members of the local institution•= 0 otherwiseWith the numbers•An