STATISTICS IN MEDICINE Statist Med 18 1575 1585 1999 ESTIMATING THE SAMPLE SIZE FOR A t TEST USING AN INTERNAL PILOT JONATHAN S DENNE AND CHRISTOPHER JENNISON Department of Medical Statistics De Montfort University James Went Building The Gateway Leicester LE5 9BH U K Department of Mathematical Sciences University of Bath Bath BA2 7AY U K SUMMARY If the sample size for a t test is calculated on the basis of a prior estimate of the variance then the power of the test at the treatment di erence of interest is not robust to misspeci cation of the variance We propose a t test for a two treatment comparison based on Stein s two stage test which involves the use of an internal pilot to estimate variance and thus the nal sample size required We evaluate our procedure s performance and show that it controls the type I and II error rates more closely than existing methods for the same problem We also propose a rule for choosing the size of the internal pilot and show that this is reasonable in terms of the e ciency of the procedure Copyright 1999 John Wiley Sons Ltd 1 INTRODUCTION We consider the comparison of two distributions by a two sample t test when the variance of each observation is unknown Suppose that X X 2 and 2 are series of independent normal observations with means h and h respectively and common unknown variance p 6 7 Furthermore suppose that we wish to perform a test of H h h against the two sided 6 7 alternative h Oh with type I error rate a and power 1 b at a speci ed treatment di erence 6 7 h h d It is well known that since the power function of the xed sample size t test depends 6 7 upon h h and p through h h p no xed sample procedure can achieve a particular 6 7 6 7 power when d is speci ed in absolute terms If d is speci ed as a multiple of the standard deviation then it is possible to determine the required sample size using tables for the non central t distribution but with a few notable exceptions for example Whitehead p 64 it is not normally appropriate to specify d in such a way It is common in practice to determine the sample size required by assuming that the true variance is equal to an estimated value p say However a variance estimate obtained from a small pilot trial can be subject to substantial sampling error Alternatively an estimate based on data from previous trials may be subject to bias if those studies took place under di erent conditions with di erent patient populations from the current study Despite the problems in obtaining a good estimate of p the correct type I error rate will be attained as long as p is only Correspondence to Jonathan S Denne Department of Medical Statistics De Montfort University James Went Building The Gateway Leicester LE5 9BH U K E mail jdenne dmu ac uk Contract grant sponsor SERC EPSRC CCC 0277 6715 99 131575 11 17 50 Copyright 1999 John Wiley Sons Ltd Received April 1997 Accepted October 1998 1576 J S DENNE AND C JENNISON used in calculating the sample size and not in implementing the test However the e ects on the type II error rate of even mild misspeci cation of p are liable to be substantial For instance in the case of a xed sample t test with desired error rates a b 0 05 the actual type II error rises as high as 0 12 if p 0 75p and falls as low as 0 01 if p 1 33p Errors in p of this magnitude are likely when the estimate of p has come from a small trial since even an initial estimate of p on 20 degrees of freedom has a probability of 0 37 of falling outside the range 0 75p 1 33p Recently Browne recommended using an upper one sided con dence limit for p in the usual sample size formula so as to ensure that the probability that the nominal power is exceeded is reasonably high but of course this reassurance comes at a potentially high price in terms of total sample size If p is estimated from a larger collection of historical data then Wittes and Brittain note that the literature tends to report more homogeneous series of cases than will be entered in a large multi centre study so it is quite plausible that an estimate p obtained from previous published results might underestimate the true value of p by a factor of 0 75 Our proposed method attempts to overcome these problems by estimating p from an initial sample which forms an integral part of the current study The rst proponent of this approach was Stein in his classic paper published in 1945 and his ideas have received renewed attention recently in the context of clinical trials for instance Gould Wittes and Brittain call this initial sample an internal pilot and discuss the merits of using an internal pilot in a variety of types of clinical trials for estimating parameters that need to be known in order to meet speci c design requirements They also suggest that internal pilot studies may be used for other purposes such as protocol adherence although we shall not consider such issues in this paper We address the problem of adaptively estimating the sample size in order to produce a two sided t test of H h h with type I error rate a and power 1 b at h h d when 6 7 6 7 comparing two groups of normally distributed observations with means h and h respectively 6 7 and common unknown variance p Let N p denote the sample size per group that would be required for a two sided t test of H which meets these requirements for a given value of p since N p must be an integer the power condition will usually be met conservatively The function N p can be numerically calculated for speci c values of d and p using the non central t distribution The procedure proposed by Wittes and Brittain requires a pre study estimate p and sets the planned total sample size per group at the start of the trial to be N p A proportion of these observations say n per group is then taken and s the pooled sample variance from these initial data is calculated Thus s s s 2 where s and s are the respective 6 7 6 7 within group sample variances from this initial sample The required total sample size is recalculated as N s further observations are taken to bring the number of observations per group up to max N p N s and then a xed sample t test is performed The estimate of p in this t test is the usual pooled estimate from the complete set of data which we denote s The random variation in s causes randomness in the total sample size and …