Design 4 Quasi Experimental Design 10 25 2011 Quasi Experimental Design Recall What makes an experimental design unique o Cause can be determined o Random assignment to groups Quasi Experimental Design No random assignment o Internal validity o Often much easier to implement that random design Nonequivalent group design o Ex O X O o O O o An R represents Random and an N represents Nonequivalent How is this different from experimental design o Notation o Meaning of nonequivalent o Threats to internal validity Selection threat selection bias Methods paper Sampling Measures Design Procedures Specifically who are you studying demographics Technique used for sampling Ethics with the children 1 Know all of your variables 2 Measurements Information on population and how it translates to sample o 2a Existing measure o 2b Your own measure Analysis 1 Conclusion Validity and Power Data Preparation 10 25 2011 Where are we Foundations o Sampling Measurement Design Analysis Conclusion Validity and Power Recall previous types of validity o External o Construct o Internal correct What is conclusion validity o The degree to which conclusions we reach about relationships in our data are reasonable o Did we conclude there is a relationship Is that conclusion How is conclusion validity different from internal validity Threats to conclusion validity o Concluding there is no relationship when in fact there is one Issues with the measurements Issues with the test setting Issues with the respondents Low statistical power What is statistical power o Concluding there is a relationship when in fact there is not Fishing for a significant relationship Power Odds that you will correctly reject the null hypothesis conclude there is a significant relationship result treatment effect Relationships among four quadrants in table o Lower alpha lower power higher alpha higher power o Lower alpha less likely you will make a Type 1 error o Lower alpha more rigorous test o An alpha of 01 compared with 05 or 10 is being more careful limits the chances of rejecting the null hypothesis o Increasing alpha e g increasing from 01 to 05 or 10 Increases chances of Type 1 Error Decreases chance of Type 2 error Decreases rigor of test o The lower the alpha the less likely you are to reject the null Data Preparation After collecting data o Check data for accuracy o Entering data to computer database o Data cleaning transformations Analysis 2 Conclusion Validity Review Descriptive Statistics Z scores 10 25 2011 Conclusion Validity and Power What is conclusion validity o The degree to which conclusions we reach about relationships in our data are reasonable o Did we conclude there is a relationship Is that conclusion correct Threats to conclusion validity o Concluding there is no relationship when in fact there is one Issues with the measurements Issues with the test setting Issues with the respondents Low statistical power What is statistical power o Concluding there is a relationship when in fact there is not Fishing for a significant relationship Descriptive Statistics Two types of statistics usually presented o Descriptive statistics Provide basic summaries of the sample o Inferential statistics Tests that allow you to reach conclusions Univariate analysis Examining one variable at a time and measure regarding the data o Distribution o Central tendency o Dispersion Descriptive Statistics Example I survey 20 mothers about the age at which their babies first said a word in months Distribution Frequency of individual values or ranges of values Central tendency Estimates of the center of the distribution of values o Mean 11 55 o Median 11 o Mode 10 11 Dispersion The spread of values around the central tendency o Range Highest value minus the lower value o Standard Deviation SD or s How the set of scores relates to Highest value 14 months Lowest value 8 months Range 14 8 6 the mean 2 X X n 1 X Each score X hat Mean score n Sample size o Our values in the baby s first word survey X hat mean 11 55 n 20 X values 8 9 9 10 11 11 11 11 11 12 12 12 12 12 13 13 13 13 14 14 o SD calculation 8 11 55 2 9 11 55 2 9 11 55 2 10 11 55 2 11 11 55 2 11 11 55 2 11 11 55 2 11 11 55 2 11 11 55 2 12 11 55 2 12 11 55 2 12 11 55 2 12 11 55 2 12 11 55 2 13 11 55 2 13 11 55 2 13 11 55 2 13 11 55 2 14 11 55 2 14 11 55 2 50 95 50 95 20 1 2 681 sqrt 2 681 1 637 o What we know about the standard deviation Approximately 68 of scores in the sample will fall within one standard deviation of the mean 11 55 1 637 9 913 11 55 1 637 13 187 Approximately 95 of scores in the sample will fall within two standard deviations of the mean Approximately 99 of scores in the sample will fall within three standard deviations of the mean Z scores A measure of how far an individual score falls from the mean in terms of standard deviations Also known as standard scores normal scores Formula o X X bar SD o Individual score minus mean score divided by standard deviation In the baby s first words example o What would be the z score for a baby who spoke his her first word at 8 months 8 11 5 1 637 3 5 1 637 2 138 z score is 2 138 The score is 2 138 standard deviations below the mean o What would be the z score for a baby who spoke his her first word at 12 months 12 11 5 1 637 0 5 1 637 0 305 z score is 0 305 The score is 0 305 standard deviations above the mean Analysis 3 Power and Correlation 10 25 2011 Power Odds of correctly rejecting the null hypothesis Ho Notation 1 B B beta or odds of making a type 2 error failing to correctly reject Ho What factors influence power o Alpha level o Sample size o Effect size o One tailed vs two tailed test Factors that influence power Alpha level o Increasing increases the chances of making a Type I error incorrectly rejecting the null o However it also decreases and increases power o Decreasing makes it more difficult to reject the null decreasing and decreasing power o Usually set at 05 Factors that influence power Sample size o Larger samples give more accurate estimation of population o Sampling distribution becomes more concentrated around the mean o Power is increased Factors that influence power effect size o What is effect size The degree of difference between the null and alternative hypothesis distributions o Gives us an idea of the meaningfulness or importance of o With larger effect size the distributions are farther apart and significant finding power increases o Logically this should make sense Larger effect size means larger