Spending Time On Design Does It Hurt Physics Learning Eugenia Etkina Alan Van Heuvelen Anna Karelina Maria Ruibal Villasenor and David Rosengrant Graduate School of Education Rutgers University New Brunswick NJ 08901 Abstract This paper is the first in a series of three describing a controlled study Transfer of scientific abilities The study was conducted in a large enrollment student introductory physics course taught via Investigative Science Learning Environment Its goal was to find whether designing their own experiments in labs affects students approaches to experimental problem solving in new areas of physics and in biology and their learning of physics concepts The theoretical framework for the design of the study was based on transfer theories such as preparation for future learning actor oriented transfer transfer of situated learning and coordination classes In this paper we describe the design of the study and present data concerning the performance of experimental and control groups on multiplechoice and open ended exam questions and on the lab exams that assess student understanding of the physics and the reasoning processes used in the lab experiments We found that the experimental group outperformed the control on labbased and traditional exams and the difference increased as the year progressed The project was supported by NSF grant DRL 0241078 Keywords Reformed instruction design labs PACS 01 40Fk 0140 gb 01 50Qb INTRODUCTION experiment in biology This paper describes the part of the study related to the comparison of the groups on the paper and pencil exam problems and lab based problems The research question in this paper is if students in the labs focus on designing their own experiments without having the right answer and on the elements of the scientific investigation instead of on solving physics problems do they learn less physics than those who have a good experimental design provided for them and more opportunities to engage in physics problem solving Two other submitted papers describe the aspects of the project related to transfer in the physics and biology content This manuscript is the first of three papers that describe an experimental design study in an introductory course whose goal was to investigate the effects of design labs on student learning of physics and their acquisition and transfer of scientific abilities Scientific abilities are approaches and procedures that scientists use when engaging in the construction of knowledge or in solving complex problems 1 In this study all students enrolled in the course attended the same large room meetings and recitations that followed the ISLE curriculum 2 However they were randomly split into two groups in the labs The experimental group designed their own experiments Their scaffolding included questions concerning the scientific abilities needed for the ISLE design labs 3 and on self assessment rubrics 1 In the control group students performed the same experiments but with the design provided in the write up and supported by conceptual questions that helped students work through the physics Throughout the semester the groups were compared on their physics learning At the end all students performed two lab transfer tasks one to design an experiment to investigate a physics problem in new area of physics and the other an MOTIVATION There are two big motivations for this study 1 Recent reports concerning science and engineering education encourage student acquisition of conceptual and quantitative understanding of physics principles and also the acquisition of abilities to design their own experiments reason from the data construct explanatory models solve complex problems work with other people and communicate 4 7 Should we spend time on the development of these latter abilities CREDIT LINE BELOW TO BE INSERTED ON THE FIRST PAGE OF EACH PAPER CP951 2007 Physics Education Research Conference edited by L Hsu C Henderson and L McCullough 2007 American Institute of Physics 978 0 7354 0465 6 07 23 00 88 Downloaded 07 Nov 2008 to 128 138 145 135 Redistribution subject to ASCE license or copyright see http proceedings aip org proceedings cpcr jsp or this will harm students acquisition of physics conceptual learning and ability to solve traditional problems 2 Many experiments indicate that the ability to transfer what is learned in physics to other unstudied physics areas to other academic disciplines and to work after academia is lacking Can students transfer what they learn in our physics design labs to other unstudied areas of physics and to other academic disciplines the subject of our other two papers two midterm exams one final exam and two lab exams All students learned through the same ISLE curriculum 2 in large room meetings and in smaller recitations The lab sections were split into two groups design labs 4 sections and non design labs 4 sections Students registered for the sections in March of the previous academic year In the previous years we found no difference in performance of lab sections on exams thus we can assume that during the experimental year the student group distribution was random During the semester students were not informed about the study At the end we disclosed the procedure and students signed a consent form allowing us to use their work for research To make sure that the design group and non design group were equal in learning ability we administered Lawson s test of hypothetico deductive reasoning in the first lab session 17 Coletta and Philips 18 found that student s learning gains are strongly correlated with their scores on this test Our lab sections were statistically the same To ensure that the treatment was the same too we used the same three TAs to teach the labs Two of the TAs taught one design and one non design section and the third TA taught two of each All TAs were members of the PER group highly skilled in the interactive teaching Design labs In these labs students had to design their own experiments The scaffolding was provided through write up questions that focused their attention on the elements of the scientific process representing the situation deciding on the experiment analyzing experimental uncertainties etc Students used selfassessment rubrics to help them write lab reports 3 A sample write up for one lab experiment is provided in the Appendix The TAs did not help students design experiments and when students had difficulties they asked questions and