Intelligent ConsumerDiscussion TopicsText PurposeQuantitative CriteriaQuestions for Quantitative StudiesSlide 6Slide 7Slide 8Slide 9Slide 10Slide 11Slide 12Slide 13Slide 14Slide 15Slide 16Slide 17Slide 18Slide 19Slide 20Qualitative CriteriaQuestions for Qualitative StudiesSlide 23Slide 24Slide 25Slide 26Slide 27Slide 28Evaluation of StudiesSlide 30Copyright © Allyn & Bacon 2008Intelligent ConsumerChapter 14This multimedia product and its contents are protected under copyright law. The following are prohibited by law:• Any public performance or display, including transmission of any image over a network;• Preparation of any derivative work, including the extraction, in whole or in part, of any images;• Any rental, lease, or lending of the program.Copyright © Allyn & Bacon 2008Discussion TopicsThe purpose of the textQuestions for quantitative studiesQuestions for qualitative studiesReviewing three studiesCopyright © Allyn & Bacon 2008Text PurposePresent and explain the fundamental principles of educational research–The quality of published educational research varies greatly–There is a need to have the knowledge and skills to critically evaluate educational researchRephrasing the consumer tips from each chapter into questions to provide guidelines for the evaluation of research articlesCopyright © Allyn & Bacon 2008Quantitative CriteriaTopics for evaluation questions for quantitative studies–Research problem–Review of the literature–Research hypotheses–Selection of participants–Instrumentation–Design–Results–Discussion and conclusionsCopyright © Allyn & Bacon 2008Questions for Quantitative StudiesResearch problem–What are the independent and dependent variables?–Is the problem researchable?–Is the problem significant? Will the results have practical or theoretical importance?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–Is the problem stated clearly and succinctly?–Does the problem communicate whether the study is descriptive, experimental, or non-experimental?–Does the problem indicate the population studied?–Does the problem indicate the variables in the study?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesReview of the literature–Does the review of literature seem comprehensive? Are all important previous studies included?–Are primary sources emphasized?–Is the review up-to-date?–Have studies been critically reviewed, flaws noted, and results summarized?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–Does the review emphasize studies directly related to the problem?–Does the review explicitly relate previous studies t the problem?–If appropriate, does the review establish a basis for research hypotheses?–Is the review well organized?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesResearch hypotheses–Is the hypothesis stated in declarative form?–Does the hypothesis follow from the literature?–Does the hypothesis state expected relationships or differences?–Is the hypothesis testable?–Is the hypothesis clear and concise?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesSelection of participants–Are the participants clearly described?–Is the population clearly defined?–Is the method of sampling used? If so, is it proportional or disproportional?–What is the return rate in a survey study?–Are volunteers used?–Is there an adequate number of participants?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesInstrumentation–Is evidence for validity and reliability clearly stated and adequate? Is the instrument appropriate for the participants?–Are the instruments clearly described? If an instrument is designed for a study by the researchers, is there a description of its development?–Are the procedures for gathering data clearly described?–Are norms appropriate if norm-referenced tests are used?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–Are standard-setting procedures appropriate if criterion-referenced tests are used?–Do the scores distort the reality of the findings?–Do response sets or faking influence the results?–Are observers and interviewers adequately trained?–Are there observer and interviewer effects?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesDesign–Non-experimentalIf descriptive, are relationships inferred?Do graphic presentations distort the findings?If comparative, are criteria for identifying different groups clear?Are causative conclusions reached from correlational findings?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesIs the correlation affected by restriction in the range and reliability of the instruments?If predictions are made, are they based on a different sample?Is the size of the correlation large enough?If causal-comparative, has the causal condition already occurred? How comparable are the subjects in the groups being compared?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–ExperimentalIs there direct manipulation of an independent variable?Is the design clearly described?What extraneous variables are not controlled in the design?Are the treatments very different from one another?Is each replication of the treatment independent of the other replications? Is the number of subjects equal to the number of treatment replications?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–Single subjectIs the measurement of the target behavior reliable?Is the target behavior clearly defined?Are there enough measures of the behavior to establish stability?Are procedures, subjects, and settings described in detail?Is there a single treatment?Are there experimenter or observer effects?Copyright © Allyn & Bacon 2008Questions for Quantitative StudiesResults–Is there an appropriately descriptive statistical summary?–Is statistical significance confused with practical significance?–Is statistical significance confused with internal or external validity?–Are appropriate statistical tests used?Copyright © Allyn & Bacon 2008Questions for Quantitative Studies–Are levels of significance interpreted correctly?–How clearly are the results presented?–Is there a sufficient number of subjects to give valid statistical results?–Are data clearly and accurately presented in