1.Introduction2.Understanding of the Project2.1Evaluation QuestionsRequirementsSystem Support2.2Evaluation Framework2.3.Data Collection2.4.Data Analysis3.Demonstration System: Z-Interop Search Target3.1Evaluation Questions3.2Evaluation Framework3.3Data Collection3.4Data Analysis4.Demonstration System: TexShare Database Search Targets4.1Evaluation Questions4.2Evaluation Framework4.3Data Collection4.4Data Analysis5.Demonstration System: Database Interaction5.1Evaluation Questions5.2Evaluation Framework5.3Data Collection5.4Data AnalysisAppendix A.Technical RequirementsAppendix B.Data Collection Design: Part 1B.1Data Collection InstrumentB.2Instrument Design DetailB.3Requirement NumbersAppendix C.Data Collection Instrument: Part 2Appendix D.Evaluation Test Searches: Z-Interop Search TargetSearch NumberSearch NumberBoolean OperatorORANDNOTORANDNOTORANDNOTORANDNOTSearch NumberSearch NumberSearch NumberSearch NumberSearch NumberScan NumberAppendix E. Data Collection Instrument: Part 3Appendix F.Evaluation Test Searches: TexShare Database Search TargetsORORANDANDNOTNOTORORANDANDNOTNOTORORANDANDNOTNOTORORANDANDNOTNOTSearch NumberSearch NumberSearch NumberSearch NumberAppendix G.Data Collection Design: Part 4G.1Data Collection InstrumentG.2Instrument Design DetailReferencesWilliam E. Moen, Ph.D. Texas Center for Digital Knowledge Principal Investigator University of North Texas Technical Evaluation Plan for RFP Demonstration Systems and RFP Written Responses This report is submitted in partial fulfillment of contract # 306-03-8165. Kathleen R. Murray, Ph.D. and William E. Moen, Ph.D. Texas Center for Digital Knowledge University of North Texas Denton, Texas February 28, 2003 The following research assistants contributed to this report: Alexis Linoski, Fatih Oguz, Barb Schultz-Jones, and Scott Simon.RFP: TECHNICAL EVALUATION PLAN Contents 1. Introduction......................................................................................................................................... 4 2. Understanding of the Project..............................................................................................................6 2.1 Evaluation Questions......................................................................................................................6 2.2 Evaluation Framework....................................................................................................................6 2.3. Data Collection ...............................................................................................................................7 2.4. Data Analysis..................................................................................................................................7 3. Demonstration System: Z-Interop Search Target .............................................................................. 8 3.1 Evaluation Questions......................................................................................................................8 3.2 Evaluation Framework.................................................................................................................... 9 3.3 Data Collection ...............................................................................................................................9 3.4 Data Analysis................................................................................................................................10 4. Demonstration System: TexShare Database Search Targets.........................................................11 4.1 Evaluation Questions....................................................................................................................11 4.2 Evaluation Framework.................................................................................................................. 12 4.3 Data Collection .............................................................................................................................13 4.4 Data Analysis................................................................................................................................14 5. Demonstration System: Database Interaction ................................................................................. 16 5.1 Evaluation Questions....................................................................................................................16 5.2 Evaluation Framework.................................................................................................................. 16 5.3 Data Collection .............................................................................................................................17 5.4 Data Analysis................................................................................................................................18 Appendix A. Technical Requirements .................................................................................................... 19 Appendix B. Data Collection Design: Part 1........................................................................................... 22 B.1 Data Collection Instrument ...........................................................................................................22 B.2 Instrument Design Detail .............................................................................................................. 22 B.3 Requirement Numbers .................................................................................................................24 Appendix C. Data Collection Instrument: Part 2.................................................................................... 25 Appendix D. Evaluation Test Searches: Z-Interop Search Target ........................................................ 28 Appendix E. Data Collection Instrument: Part 3.................................................................................... 31 Appendix F. Evaluation Test Searches: TexShare Database Search Targets...................................... 33 Appendix G. Data Collection Design: Part 4........................................................................................... 37 G.1 Data Collection Instrument ...........................................................................................................37 G.2 Instrument Design Detail