MIT 9 29 - Statistical Approach to Predicting Switching Behavior from Reward Histories

Unformatted text preview:

A Statistical Approach to Predicting Switching Behavior from Reward HistoriesThe ExperimentReward CurveModeling User BehaviorFactors involved in making switching decisionTraining weights by discriminant analysisEvaluation paradigmResults9.29 Project 1A Statistical Approach to Predicting Switching Behavior from Reward HistoriesChristine Fry and Alex ParkMarch 30th, 2004The Experiment•Subjects chooses between 2 buttons•Immediately see result of choice•Accumulate fractions of Hershey’s kisses on each experiment•3 tasks with different reward curves•240 trials (choices) for each task•Pilot vs. main experiment — motivationReward CurveAllocation to AReward• Reward given depends on allocation A choices over last 40 presses• Middle is local optimum based on matchingModeling User Behavior•After seeing reward at time t, subject can choose to–press same button (st=0) (NO SWITCH)–press other button (st=1) (SWITCH)ttssrrrrrBABAA44321110ActionRewardSwitch Decision•Assuming users are rational (big assumption)–We can model switching decision with information that subject acts on•What factors are important for decision making?Factors involved in making switching decision•Possible factors involved in deciding–Change in most recent reward–Average change over last n rewards–Amount of reward accumulated so far–Amount of time elapsed in experiment•Each of the above is a linear function of a history vectorwhere the function is a weighted sum of the inputs1tttrrr10,1niitntrnrtiitrR1t][x1tRrrrtntttt  tf xw)(xt]00011[w ]000[w11nn ]01000[w ]10000[w Training weights by discriminant analysis•Predicting switching decisions based on history vectors boils down to 2-way classification of vectors xt. •Classification by linear discriminant analysis–Separate xt into two classes (switch or stay)–Compute weight vector, w, which best separates two classes –Use w and decision boundary to determine relative probabilities of switching for a given test inputExample set of history vectorsDistributions of projected pointsSwitch StaySwitch StayProbability ofswitching for projected input, xP(x|Sw)P(x|St) + P(x|Sw)P(Sw|x) =Evaluation paradigm1) Final allocation resting pointPredictedActual  01101100PredictedSwitchesActualSwitches2) Error rate of switch classificationsErrors•Train weight vector on first 120 trials, play game for the next 120 actions•Train weight vector on first 120 trials, classify next 120 history vectors# Trials•The above tasks can be evaluated as followsResults•Evaluated on set of data from 6 users, using 3 different reward profiles•For classification, deterministic assignment used.•Reward history vector extends 5 observations into the pastMethod 1Method 2Method 3•Overall final allocation prediction error** (1 trial each): 0.12•Overall switch classification error rate: 0.2662No


View Full Document

MIT 9 29 - Statistical Approach to Predicting Switching Behavior from Reward Histories

Download Statistical Approach to Predicting Switching Behavior from Reward Histories
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Statistical Approach to Predicting Switching Behavior from Reward Histories and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Statistical Approach to Predicting Switching Behavior from Reward Histories 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?