A Statistical Approach to Predicting Switching Behavior from Reward HistoriesThe ExperimentReward CurveModeling User BehaviorFactors involved in making switching decisionTraining weights by discriminant analysisEvaluation paradigmResults9.29 Project 1A Statistical Approach to Predicting Switching Behavior from Reward HistoriesChristine Fry and Alex ParkMarch 30th, 2004The Experiment•Subjects chooses between 2 buttons•Immediately see result of choice•Accumulate fractions of Hershey’s kisses on each experiment•3 tasks with different reward curves•240 trials (choices) for each task•Pilot vs. main experiment — motivationReward CurveAllocation to AReward• Reward given depends on allocation A choices over last 40 presses• Middle is local optimum based on matchingModeling User Behavior•After seeing reward at time t, subject can choose to–press same button (st=0) (NO SWITCH)–press other button (st=1) (SWITCH)ttssrrrrrBABAA44321110ActionRewardSwitch Decision•Assuming users are rational (big assumption)–We can model switching decision with information that subject acts on•What factors are important for decision making?Factors involved in making switching decision•Possible factors involved in deciding–Change in most recent reward–Average change over last n rewards–Amount of reward accumulated so far–Amount of time elapsed in experiment•Each of the above is a linear function of a history vectorwhere the function is a weighted sum of the inputs1tttrrr10,1niitntrnrtiitrR1t][x1tRrrrtntttt tf xw)(xt]00011[w ]000[w11nn ]01000[w ]10000[w Training weights by discriminant analysis•Predicting switching decisions based on history vectors boils down to 2-way classification of vectors xt. •Classification by linear discriminant analysis–Separate xt into two classes (switch or stay)–Compute weight vector, w, which best separates two classes –Use w and decision boundary to determine relative probabilities of switching for a given test inputExample set of history vectorsDistributions of projected pointsSwitch StaySwitch StayProbability ofswitching for projected input, xP(x|Sw)P(x|St) + P(x|Sw)P(Sw|x) =Evaluation paradigm1) Final allocation resting pointPredictedActual 01101100PredictedSwitchesActualSwitches2) Error rate of switch classificationsErrors•Train weight vector on first 120 trials, play game for the next 120 actions•Train weight vector on first 120 trials, classify next 120 history vectors# Trials•The above tasks can be evaluated as followsResults•Evaluated on set of data from 6 users, using 3 different reward profiles•For classification, deterministic assignment used.•Reward history vector extends 5 observations into the pastMethod 1Method 2Method 3•Overall final allocation prediction error** (1 trial each): 0.12•Overall switch classification error rate: 0.2662No
View Full Document