Section 7 Testing hypotheses about parameters of normal distribution. T-tests and F-tests. We will postpone a more systematic approach to hypotheses testing until the following lectures and in this lecture we will describe in an ad hoc way T-tests and F-tests about the parameters of normal distribution, since they are based on a very similar ideas to confidence intervals for parameters of normal distribution - the topic we have just covered. Suppose that we are given an i.i.d. sample from normal distribution N(µ, ν2) with some unknown parameters µ and ν2 . We will need to decide between two hypotheses about these unknown parameters - null hypothesis H0 and alternative hypothesis H1. Hypotheses H0 and H1 will be one of the following: H0 : µ = µ0, H1 : µ = µ0,∞H0 : µ ∼ µ0, H1 : µ < µ0, H0 : µ ≈ µ0, H1 : µ > µ0, where µ0 is a given ’hypothesized’ parameter. We will also consider similar hypotheses about parameter ν2 . We want to construct a decision rule α : X n � {H0, H1} that given an i.i.d. sample (X1, . . . , Xnn either accepts H0 or rejects H0 (accepts H1).) ≥ X Null hypothesis is usually a ’main’ hypothesis in a sense that it is expected or presumed to be true and we need a lot of evidence to the contrary to reject it. To quantify this, we pick a parameter � ≥ [0, 1], called level of significance, and make sure that a decision rule α rejects H0 when it is actually true with probability ≈ �, i.e. P(α = H1|H0) ≈ �. The probability on the left hand side is understood as a worse case scenario given that the null hypothesis is true, i.e. P (α = H1H0) = sup|(µ,�2 )�H0 Pµ,�2 (α = H1). 41Level of significance � is usually small, for example, � = 0.05. Example. Let us consider a Matlab example about normal body temperature from the lecture about confidence intervals. If a vector ’normtemp’ represents body temperature measurements of 130 people then typing the following command in Matlab [H,P,CI,STATS] = ttest(normtemp,98.6,0.05,’both’) produces the following output: H = 1, P = 2.4106e-007, CI = [98.1220, 98.3765] STATS = tstat: -5.4548, df: 129, sd: 0.7332. Here µ0 = 98.6, � = 0.05, ’both’ means that we consider a null hypothesis µ = µ0 in which case the alternative µ = µ0 is a two-sided hypothesis. The alternative µ > µ0 corresponds to ∞parameter ’right’, and µ < µ0 corresponds to parameter ’left’. H = 1 means that we reject null hypothesis and accept H1, P=2.4106e-007 is a p-value that we will discuss below, CI is a 95% confidence interval for µ0 that we constructed in the previous lecture. If we want to test the hypothesis µ ∼ 98.6 then typing [H,P,CI,STATS] = ttest(normtemp(1:130,1),98.6,0.05,’left’) outputs H = 1, P = 1.2053e-007, CI = [-Inf, 98.3558], STATS = tstat: -5.4548, df: 129, sd: 0.7332. Notice that CI and P are different in this case. The fact that (in both cases) we rejected H0 means that there is a significant evidence against it. In fact, we will see below that a p-value quantifies in some sense how unlikely it is to observe this dataset assuming that the null hypothesis is true. p-value of order 10−7 is a strong evidence against the hypothesis that a normal body temperature is µ0 = 98.6. Let us explain how these tests are constructed. They are based on the result that we proved before that for MLE ˆµ = X¯and ˆν2 = X¯2 X¯2 satisfy− A = ≤n(ˆµν − µ) � N(0, 1) and B = nννˆ22 � �2 n−1 and the random variables A and B are independent. Hypotheses about mean of one normal sample. We showed that a random variable ≤n − 1µˆ − µ � tn−1νˆhas tn−1-distribution with n − 1 degrees of freedom. Let us consider a t-statistic T = ≤n − 1µˆ − µ0 . νˆ42� � This statistic behaves differently depending on whether the ’true’ unknown mean µ = µ0, µ < µ0 or µ > µ0. First of all, if µ = µ0 then T � tn−1. If µ < µ0 then we can rewrite = ≤n − 1µˆ − νˆµ + ≤n − 1µ − νˆµ0 � −→ T since the first term has proper distribution tn−1 and the second term goes to infinity. Similarly, when µ > µ0 then T � + . Therefore, we can make a decision about our hypotheses based →on this information about the behavior of T. I. (H0 : µ = µ0.) In this case, the indication that H0 is not true would be if T becomes | |too large, i.e. T � ±→. Therefore, we consider a decision rule α = H0, if − c ≈ T ≈ c H1, if T > c. | | The choice of the threshold c depends on the level of significance �. We would like to have P(α = H1|H0) = P(|T | > c|H0) ≈ �. But given that µ = µ0, we know that T � tn−1 and, therefore, we can choose c from a condition P(|T | > c|H0) = tn−1(|T | > c) = 2tn−1((c, →)) = � using the table of tn−1-distribution. Notice that this decision rule is equivalent to finding the (1 − �)-confidence interval for unknown parameter µ and making a decision based on whether µ0 falls into this interval. II. (H0 : µ ∼ µ0.) In this case, the indication that H0 is not true, i.e. µ < µ0, would be if T � −→. Therefore, we consider a decision rule α = H0, if T ∼ c H1, if T < c. The choice of the threshold c depends on the condition P(α = H1|H0) = P(T < c|H0) ≈ �. Since we know that T ≤ = T −≤n − 1µ − νˆµ0 � tn−1 we can write P(T < c H0) = sup P � T µ − µ0 � = P(T ≈ c) = tn−1((−→, c]) = �| µ�µ0 ≤ ≈ c −≤n − 1νˆ≤ and we can find c using the table of tn−1-distribution. 43PSfrag replacements� III. (H0 : µ ≈ µ0.) Similar to the previous case, the decision rule will be α = H0, if T ≈ c H1, if T > c. and we find c from the condition tn−1([c, + )) = �. →p-value. Figure 7.1 illustrates the definition of p-value for all three cases above. p-value can be understood as a probability, given that the null hypothesis H0 is true, to observe a value of T -statistic equally or less likely than the one that was observed. Thus, the small p-value means that the observed T -statistic is very unlikely under the null hypothesis which provides a strong evidence against H0. The confidence level � defines what we consider as ’unlikely enough’ to reject the null hypothesis. −6 −4 −2 0 2 4 6 0 0.1 0.2 0.3 0.4 0 0.1 0.2 0.3 0.4 µ = µ0 µ ∼ µ0