View Full Document

Using Randomization in Development



View the full content.
View Full Document
View Full Document

11 views

Unformatted text preview:

Using Randomization in Development Economics Research A Toolkit Esther Duflo Rachel Glennerster and Michael Kremer CID Working Paper No 138 December 2006 Copyright 2006 Esther Duflo Rachel Glennerster Michael Kremer and the President and Fellows of Harvard College Working Papers Center for International Development at Harvard University Using Randomization in Development Economics Research A Toolkit Esther Duflo Rachel Glennerster and Michael Kremer December 12 2006 Abstract This paper is a practical guide a toolkit for researchers students and practitioners wishing to introduce randomization as part of a research design in the field It first covers the rationale for the use of randomization as a solution to selection bias and a partial solution to publication biases Second it discusses various ways in which randomization can be practically introduced in a field settings Third it discusses designs issues such as sample size requirements stratification level of randomization and data collection methods Fourth it discusses how to analyze data from randomized evaluations when there are departures from the basic framework It reviews in particular how to handle imperfect compliance and externalities Finally it discusses some of the issues involved in drawing general conclusions from randomized evaluations including the necessary use of theory as a guide when designing evaluations and interpreting results JEL Classification I0 J0 O0 C93 Keywords Randomized evaluations Experiments Development Program evaluation We thank the editor T Paul Schultz as well Abhijit Banerjee Guido Imbens and Jeffrey Kling for extensive discussions David Clingingsmith Greg Fischer Trang Nguyen and Heidi Williams for outstanding research assistance and Paul Glewwe and Emmanuel Saez whose previous collaboration with us inspired parts of this chapter Department of Economics Massachusetts Institute of Technology and Abdul Latif Jameel Poverty Action Lab Abdul Latif Jameel Poverty Action Lab Department of Economics Harvard University and Abdul Latif Jameel Poverty Action Lab 1 Contents 1 Introduction 3 2 Why Randomize 4 2 1 The Problem of Causal Inference 5 2 2 Randomization Solves the Selection Bias 7 2 3 Other Methods to Control for Selection Bias 10 2 3 1 Controlling for Selection Bias by Controlling for Observables 10 2 3 2 Regression Discontinuity Design Estimates 11 2 3 3 Difference in Differences and Fixed Effects 12 2 4 Comparing Experimental and Non Experimental Estimates 13 2 5 Publication Bias 15 2 5 1 Publication bias in non experimental studies 15 2 5 2 Randomization and publication bias 17 3 Incorporating Randomized Evaluation in a Research Design 19 3 1 Partners 20 3 2 Pilot projects From program evaluations to field experiments 22 3 3 Alternative Methods of Randomization 24 3 3 1 Oversubscription Method 24 3 3 2 Randomized Order of Phase In 25 3 3 3 Within Group Randomization 26 3 3 4 Encouragement Designs 27 4 Sample size design and the power of experiments 28 4 1 Basic Principles 28 4 2 Grouped Errors 31 4 3 Imperfect Compliance 33 4 4 Control Variables 34 4 5 Stratification 35 4 6 Power calculations in practice 38 1 5 Practical Design and Implementation Issues 40 5 1 Level of Randomization 40 5 2 Cross Cutting Designs 42 5 3 Data Collection 45 5 3 1 Conducting Baseline Surveys 45 5 3 2 Using Administrative Data 46 6 Analysis with Departures from Perfect Randomization 47 6 1 The Probability of Selection Depends on the Strata 47 6 2 Partial Compliance 48 6 2 1 From Intention To Treat to Average Treatment Effects 51 6 2 2 When is IV Not Appropriate 55 6 3 Externalities 56 6 4 Attrition 58 7 Inference Issues 61 7 1 Grouped Data 61 7 2 Multiple Outcomes 62 7 3 Subgroups 64 7 4 Covariates 66 8 External Validity and Generalizing Randomized Evaluations 66 8 1 Partial and General Equilibrium Effects 67 8 2 Hawthorne and John Henry Effects 68 8 3 Generalizing Beyond Specific Programs and Samples 70 8 4 Evidence on the Generalizability of Randomized Evaluation Results 71 8 5 Field Experiments and Theoretical Models 73 2 1 Introduction Randomization is now an integral part of a development economist s toolbox Over the last ten years a growing number of randomized evaluations have been conducted by economists or with their input These evaluations on topics as diverse as the effect of school inputs on learning Glewwe and Kremer 2005 the adoption of new technologies in agriculture Duflo Kremer and Robinson 2006 corruption in driving licenses administration Bertrand Djankov Hanna and Mullainathan 2006 or moral hazard and adverse selection in consumer credit markets Karlan and Zinman 2005b have attempted to answer important policy questions and have also been used by economists as a testing ground for their theories Unlike the early social experiments conducted in the United States with their large budgets large teams and complex implementations many of the randomized evaluations that have been conducted in recent years in developing countries have had fairly small budgets making them affordable for development economists Working with local partners on a smaller scale has also given more flexibility to researchers who can often influence program design As a result randomized evaluation has become a powerful research tool While research involving randomization still represents a small proportion of work in development economics there is now a considerable body of theoretical knowledge and practical experience on how to run these projects In this chapter we attempt to draw together in one place the main lessons of this experience and provide a reference for researchers planning to conduct such projects The chapter thus provides practical guidance on how to conduct analyze and interpret randomized evaluations in developing countries and on how to use such evaluations to answer questions about economic behavior This chapter is not a review of research using randomization in development economics 1 Nor is its main purpose to justify the use of randomization as a complement or substitute to other research methods although we touch upon these issues along the way 2 Rather it is a practical guide a toolkit which we hope will be useful to those interested in including 1 Kremer 2003 and Glewwe and Kremer 2005 provide a review of randomized evaluations in education Banerjee and Duflo 2005 review the results from randomized evaluations on ways to improve teacher s and nurse s attendance in developing countries Duflo 2006


Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Using Randomization in Development and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Using Randomization in Development and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?