February 14 2003 1 6 Sequential decision theory As previously let the sample space be a measurable space X B and P P a family of laws on it Let X1 X2 be independent and identically distributed with law P Let A be the space of possible speci c actions with a algebra E We have a algebra T on and a loss function L which is a measurable function L from A to 0 A prior may or may not be given on A sequential decision rule will consist of two functions N and as follows Let X be the set of all sequences xn n 1 with xn X for all n For n 1 2 let Bn be the smallest algebra of subsets of X for which X1 Xn are measurable Let B0 be the trivial algebra X Then N is a function from X into N 0 1 2 such that for each k 0 1 2 N k Bk Such a function is called a stopping time or stopping rule The terminal decision rule is a sequence of functions n 0 n where 0 A and for each n 1 n is a measurable function from X n into A The action actually taken will be N N X1 XN Let N If c 0 is the cost of each observation the total loss including costs of observations in a given case is L N N c The risk is then r c EN EL N where the expectations are with respect to P on X Note that if c 0 and N is required to take nite values as in the above de nition then in general optimal rules do not exist Example This is actually not a statistical decision problem as just formulated but it will illustrate some possible di culties Suppose that a gambler can play a sequence of games as follows In the nth game the gambler wagers 1 and wins 100 2n 1 with probability 0 01 2n Thus the expected gain in each game is 2 1 1 So the Bayes or optimal strategy would seem to be to continue playing inde nitely But the probability that the gambler ever wins is n 1 0 01 2n 0 01 If the gambler never wins which occurs with probability 0 99 then the gambler wagers and loses in nitely many dollars The expected or average gain from games won is also in nite if the gambler continues to play so that the overall average gain is unde ned There is actually no Bayes optimal strategy Let fn be the net winnings after n plays Then Efn n while fn a s by the Borel Cantelli lemma We can also consider sequential randomized decision rules de ned as follows For n 1 2 let An En be a measurable space where An is the space of speci c actions which can be taken after n observations Often all An En will be equal to one space A E Assume that for each n 0 An where the action 0 will mean taking another observation Xn 1 while all other actions in An will imply taking no more observations Each n is a measurable function from X n Bn into the space DE of all probability laws on A E So given X1 Xn if no decision to stop has been made earlier we then take another observation with probability n X1 Xn 0 and otherwise stop and take an action chosen from An 0 with distribution n X1 Xn 1 n X1 Xn 0 1 So for a sequential randomized test of P vs Q we can take An A 1 0 1 for all n where as in Sec 1 5 1 means choosing P and 1 means choosing Q PROBLEMS 1 In the example at the end of Sec 1 5 and where RQ P has only values t or 1 t not 1 with t 2 let be a randomized test that does SPRT 1 8 2 or SPRT 1 2 8 with probability 1 2 each Compare the performance of this test to SPRT 1 4 4 in terms of error probabilities and average sample numbers 2 For a sequential test of P vs Q as in Problem 1 suppose that the nth observation costs 1 3n while LP Q LQP 3 and P Q 1 2 A decision rule must reach a decision after a nite number N of observations Is there an optimal Bayes sequential test in this case Why or why not 2