Legislative Program Evaluation in the American StatesJohn RisleyWestern Michigan UniversityState LPE Units - John Risley (March 13, 2007)Background Basis of knowledge of state LPE Performance Audits and Evaluations What are the states doing Henry and Mark (2003) Research concerning evaluation Metaevaluation (one of six areas) Funkhouser (2005) Examination of performance audit content Difference in audit unit structuresState LPE Units - John Risley (March 13, 2007)LPE Literature Growth, survey, description of units Growth of units Characteristics of staff Organizational structure Limited review of evaluations and audits conducted Case studies Review of multiple reportsState LPE Units - John Risley (March 13, 2007)Metaevaluation Literature Metaevaluation theory Extensive metaevaluation Limited metaevaluation Use of standards and Guiding Principles Metaevaluation of multiple reports Standards-based criteria Limited classification (met/unmet)State LPE Units - John Risley (March 13, 2007)Metaevaluation Literature: Multiple StudiesNoneYesNo2001979Larson, et al.NoneYesYes151992HoogerwerfNoneYesNo2771997Forss & CarlssonNoneYesNo561997BickmanNoneYesNo232002Scott-Little, Haman& JursNoneYesNo~2002005KruseNoneYesNo32005Leeuw & CooksyNoneYesNo872005Cooksy & CaracelliInterviewed evaluator, reviewed documentsNoNo42005ChatterjiNoneYesNo252005Bollen, Paxon, &MorishimaOther Data ExaminedRelied Solely onReportsRandomSampleNumber ofReportsYearAuthor(s)State LPE Units - John Risley (March 13, 2007)Methodology: LPE Units and Reports LPE units selected LPE units under legislative control NLPES members Search of states and statutes for others Reports Other than strictly financial audits Classification issuesState LPE Units - John Risley (March 13, 2007)Methodology: MetaevaluationCriteria Combination of GAGAS for performance auditing, PES, and KEC Concise criteria with three ratings (met, partially met, and failed to meet) Recognition of limitations Limited information in reports Limited ability for confirmation Limited by selected criteriaState LPE Units - John Risley (March 13, 2007)Findings High percentage meeting criteria Performance Audits vs. Evaluations Standards (GAGAS vs. PES) Comparisons Stakeholder identification Lack of information on some criteria Timeliness, Quality Control, Human Subjects’ RightsState LPE Units - John Risley (March 13, 2007)Issues Revision of criteria Drop criteria with little evidence in reports Different categories (poor, fair, etc. vs. met, unmet) Focus on generalizability and comparisons What is being evaluated Agencies/boards vs. programs and policies Unit structure and report content Units with a financial focus vs. nonfinancial Goal-based vs.
or
We will never post anything without your permission.
Don't have an account? Sign up