Data qualityGroup members:Larry SugarbakerSherry PittamKevin GergelyCraig Palmer – presenter Dec 14 PMJulia Jones – scribe, presenter Dec 15Objectives of morning Dec 15 session:I. Refine research issuesPrioritize? Or Categorize? We chose categorize.General problem of data quality in decision-making: How to determine and communicateuncertainty to decision-makers in studies integrating multiple data sources?Overarching research question: Does uncertainty associated with data quality andsynthesis really have an influence on policymaking and plan implementation?Category 1: Issues for individual studies in which diverse data sources are combinedKnowledge of points where error is introduceda. Develop methods of reducing the introduction of error when datasets are(created and) combinedb. Develop methods for error measurement and logging at each stagec. Develop methods for characterizing relationships among errors – additive,multiplicative, averagingAssociating errors with alternative decisions/actionsCategory 2: Issues for sharing of dataCan metadata become a part of the dataset?a. What happens to metadata when multiple sources are integrated?b. How can metadata management be automated once it is created?c. How can data standardization help the process of combining metadata frommultiple sources?d. Can open-source tools be developed for mapping data content standards to oneanother?II. Move onMake a scenario or tell a story, or describe how research cycle can be sustained? Wechose stories. Stories are numbered to correspond to the list of research issues above.1. Story: the SEEK project focuses on extracting knowledge from one study design toapply to another, e.g. workflow diagram approach of the SEEK project. This approachcould serve as the template for examining potential sources and types and magnitudes oferrors. We are not sure whether SEEK is trying to answer questions 1a,b,c.1.a.. Sherry (OSU): Can standardized tools be developed for data entry? Larry(NatureServ): The problem is about developing intuitive data entry tools that are mappedto standardized data models. Larry’s group have a proposal that has been repeatedlysubmitted to NSF but not funded on this. NCEAS (Matt Jones) has tried but has haddisappointing results. Two approaches: one (NCEAS) involves the use of a questionnaireto provide specifications for designing a form, but the technology available to generatestandardized forms is too crude. An alternative approach (Larry’s group) is to develop aset of tools that are mapped to standardized database structures. These design framescould be selected and tailored by users, and would be available as templates on a website.1. NSF could develop and publish metadata standards across all grants, instead of just forcertain programs. By far the most advanced work is being done by the FederalGeographic Data Committee within the USGS, including a biology standard. Metadatastandards are well developed and in use by the LTER information managers, and thesestandards are used in internal reviews of LTER projects. NBII is making a very big pushto require metadata using the FGDC standards for its projects. It would be important topull Valerie Hutchinson into this discussion.2. a.,b, c. Julia Jones. Data harvesters collect existing databases – e.g. the Long-termEcological Research network’s Clim-DB, Hydro-DB. One could ask how existing data-harvesters answer questions 2 a,b,c. Overarching research question: Does uncertainty associated with data synthesis reallyhave an influence on policymaking and plan implementation? Studies could be done ofdecision-makers perceptions of the value of science findings made from synthesized orintegrated data. For example, data harvesters such as Clim-DB and Hydro-DB havegenerated publications from combined datasets, which are (perhaps) being used by landmanagers or decision makers in the Forest Service and NOAA. This work could beextended by examining how syntheses of datasets are used by decision-makers and howapparent and important the errors were to decision-makers. Specifically, the researchquestion is: how is the increase in power associated with data synthesis balanced by theincrease in uncertainty associated with the ways in which the errors were combined? Anextension of this work could examine how synthetic studies stand up in courts of law incomparison with other forms of “expert testimony.” A great example of a possible study would be a follow up on the President’s Forest planand how the data synthesis and uncertainty affected the ability of a plan to beimplemented. Craig Palmer and others are involved in a 10-year review of thePresident’s Forest plan but this does not (yet) include an analysis of the effect ofuncertainty and its consequences.2.d. Larry Sugarbaker. Problem: Automated mapping of various vegetation classificationstandards. XML based products are emerging to create these mappings but the matchinghas to be done by hand. A solution could be creating an exchange tool; it would involvetwo innovations: (1) develop a set of definitions that define the relationships among thecategories that are matched, and (2) publish this as open source code. Researchchallenge: How general can these tools become to be applied to a wide range ofecological