Applying Science Towards Understanding Behavior in OrganizationsResearch Issues in OrganizationsExperiments: A ReviewExperiments: Pros and ConsExperimental DesignValiditySamplingSampling TechniquesPost with no Control GroupPre – Post with no Control GroupControl Group with no PretestPre – Post with Control GroupMeasurementUtilizing Individual DifferencesTypes of AssessmentBiographical DataSlide 17InterviewsWork SamplesAssessment CentersDrug TestingLetters of RecommendationPsychological Test CharacteristicsPsychological Test TypesIntegrity TestsPersonality TestsThe Big Five InventoryCognitive TestsSlide 29Slide 30Reliability and ValidityApplying Science Towards Understanding Behavior in OrganizationsChapters 2 & 3Research Issues in OrganizationsApproaches to collecting dataExperimentalObservational/correlationalData collection issuesSamplingHow should we select participants?What impact does it have on the results?Experimental designControlling potential confoundsAssigning participants to experimental conditionsMeasurement issuesDescribing and interpreting the resultsExperiments: A ReviewExperiments - Do changes in one variable (X) “cause” changes in another variable (Y)?Independent Variable (X)condition or event that is manipulated by experimenterDependent Variable (Y)variable that is affected (hopefully) by manipulating independent variableExtraneous Variable(s)any variable other than independent variable that may influence dependent variableExperiments: Pros and ConsAdvantage: Allows conclusions about direct effects of one variable on anotherDisadvantages: Experimental conditions are artificialresults may not “generalize” to the real worldSome questions can’t be tested in an experimentRequire control that is not always available in the “real” worldExperimental DesignControlling potential confoundsGoal of experiment is to “rule out” alternate explanations of what affected dependent variableConfounds are threats to internal validityCan be controlled through appropriate experimental design and proceduresInternal ValidityHistoryMaturationTestingInstrumentationStatistical RegressionSelectionMortalitySelection-MaturationDiffusion of TreatmentExternal ValiditySampleSetting (e.g., culture)Time (e.g., 60s vs. 90s)Replication (lack of)Do the results of this experiment generalize (apply) to settings other than the experimentIs there another reason (other than the independent variable) that could explain the results of the experiment.ValidityHow participants are selected for a study influences the extent to which the results can be applied to a larger group (external validity).A wide variety of techniques are availableTwo Main types of samplingProbabilitypredetermined chance of any individual in the population being selected for the studyNonprobabilityTypically nonrandom samplingSamplingSampling TechniquesProbability Sampling1. Simple random sampling2. Systematic sampling3. Stratified random sampling4. Cluster sampling5. Multistage samplingNonprobability Sampling1. Convenience sampling2. Quota sampling3. Snowball samplingPost with no Control GroupTraining PosttestPre – Post with no Control GroupPretest Training PosttestControl Group with no PretestExperimentalGroupTraining PosttestControl Group Placebo PosttestGroupDifferencesPre – Post with Control GroupPretestExperimentalTrainingPosttestPretest Control PosttestGroupDifferencesGroupDifferencesMeasurementMeasurement – the process of assigning numbers to objects or events according to rules (Linn & Gronlund, 1995).Psychological Measurement – concerned with evaluating individual differences in psychological traits.Trait – descriptive label applied to a group of behaviors (e.g., friendly; intelligent)Utilizing Individual DifferencesPsychologists assume that most traits are normally distributed in the population.e.g., height, intelligence, KSAsPsychologists study:measuring these differencesusing these differences to predict performanceI/O Psychologists typically primarily rely on these as predictors of job performance:Cognitive abilitiesPersonalityTypes of AssessmentBiographical InformationInterviewsWork SamplesLetters of RecommendationPsychological TestsBiographical DataGood questions are about events that are: historicalexternaldiscretecontrollable (by the individual)verifiableequal accessjob relevantnon-invasive(Mael, 1991)Rationale vs. empirical methodBiographical DataStrong criterion validitydrug use, criminal history predicts dysfunctional police behavior (Sarchione et al., 1998)not redundant with personality (McManus & Kelly, 1999)Measurement issuesGeneralizabilityFakingFairnessPrivacy concernsInterviewsStructured vs. UnstructuredInfo. gathering vs. interpersonal behavior sampleSituational interview“How would you handle a circumstance in which you needed the help of a person you did not like?”Measurement issuesstructured has more criterion related validityvalue of unstructured? Illusion of validityGuidelines for structured interviewsWork Samplesperform a task under standardized conditionshistorically were for blue collar jobse.g. use of tools, demonstrate driving skillswhite collar examplesspeech interview for foreign worker, test of basic chemistry knowledge, Measurement issueshigh criterion validity if skills are similar to jobcostly to administerwork best with mechanical, rather than people-oriented tasksAssessment CentersRealistic tasks done in groupsAssessed by multiple of raters rating multiple domainsMultiple methodsin basket group exerciseleaderless group exerciseStrong criterion validity (e.g., teachers, police)overall scores predict job performanceMeasurement issuescostly to administerdifferent ratings on a task too highly correlateddimension ratings not correlated strongly across tasksfix? focus on behavior checklists and rater trainingDrug Testingopinion?People are more accepting of it if job involves risks to others (Paronto, et al., 2002)Measurement issuesreliability is very high, but not perfectValidity?Normands, Salyards, & Mahoney (1990)over 5000 postal service applicantsthose who tested positive had 59% higher absenteeism, 47% more likely to be firedno