Yale STAT 619 - A Note on a Method for the Analysis of Significances en masse

Unformatted text preview:

A Note on a Method for the Analysis of Significances en massePaul SeegerTechnometrics, Vol. 10, No. 3. (Aug., 1968), pp. 586-593.Stable URL:http://links.jstor.org/sici?sici=0040-1706%28196808%2910%3A3%3C586%3AANOAMF%3E2.0.CO%3B2-JTechnometrics is currently published by American Statistical Association.Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/about/terms.html. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtainedprior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content inthe JSTOR archive only for your personal, non-commercial use.Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/journals/astata.html.Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.JSTOR is an independent not-for-profit organization dedicated to and preserving a digital archive of scholarly journals. Formore information regarding JSTOR, please contact [email protected]://www.jstor.orgMon Apr 16 20:14:45 2007VOL.10. No. 3 A Note on a Method for the Analysis of Significances en masse PAULSEEGER University of Uppsalu, Sweden This note concerns the derivation of the p-mean significance levels, in the case of independent tests, for a mass-significance method developed by Eklund [I]. The solution is reached by formulating and solving an urn problem. Some comparisons are made with the p-mean significance levela of Duncan's multiple range test. In three seminar papers from 1961-1963 [l] Eklund suggested the following solution to what he called the mass-significance problem: In large exploratory investigations it is desirable to keep the proportion of false significances low, at most equal to a small value k. Consider therefore the variable number of false significances ?i= number of significances where the denominator is observed but the numerator has to be predicted. Both numerator and denominator are functions of the level of significance a', which is supposed to be used for each of N tests. Eklund's method consists in deter- mining a' so that y < k, where k is predetermined. The observed number of significances at the level of a' may be denoted by n(al). Eklund considered three alternatives for the numerator. If the null hypothesis is true for each of the N tests we can predict the number of false significances to be Na'. This is the most conservative of Eklund's alternatives. The method consists in finding a significance level a' for the individual test so that Na' n(a'> 2 Like the technique for making multiple comparisons based on Bonferroni's inequality [5],Eklund's method is only used to determine the level of significance for the individual test; it can be applied to N tests of any kind. One starts making the tests at the level a' = k. If the criterion (1) is not satisfied, a lower value of Received June 1967. 586NOTES FIGURE1 The number of significances n(a')plotted against the individual level of significance a'. a' is tried until it is satisfied; that is until the curve in Fig. 1touches the straight line. If this happens for the first time for a' = a, ,we say that we have obtained n(a,) "mass-significances" ("approved" significances). In fact only a few a'-values need be investigated, as n(al) can not increase with decreasing a'. After a certain a' = a, , where n(a,) < Nal/k (and n(al) < Nal/k for all a' > a,), it is easy to compute the largest a' < a, which could give n(a,) > n(al) 2 Nal/k. Thus we investigate only a' = k, a, and a, for the case of Fig. 1. The individual significance levels of Eklund's method are determined not only by the number of tests, as, for example, in the method based on Bonfer- roni's inequality, but also by the observations themselves. This latter char- acteristic means that including some tests whose null hypotheses have very low prior probability of being true may increase the number of significances that exists among the original tests. However, requirement (1) seems to be a reasonable one, if measures are taken after every significant test (but not after non-significant ones). Only a small fraction of these measures are then un-justified. As a referee has pointed out the properties of Eklund's method would cer- tainly be greatly illuminated by a decision-theoretic discussion. In one of his alternatives Eklund, in a way, gave diierent prior probabilites to different null hypotheses when he assumed a certain (known) number N, 5 N of them to be true. These aspects will not be considered here. Eklund's method has also been discussed in a paper by Eklund and Seeger in 1965 [2] and in a monograph by Seeger 1966 [3]. In these publications the588 NOTES method is compared to some methods for making multiple comparisons. It is found to give about the same results as Duncan's multiple range test in two very large investigations with N = 703 and 1540 respectively. In [3]it is shown that when the N tests are independent, the probability of at least one significant result when all N null hypotheses are true is equal to k. That is, the experiment- wise error rate is k when the null hypothesis is true. It lies in the nature of the method that the probability of rejecting some true null hypotheses will be greater when some of the other hypotheses are not true. In order to characterize the behaviour of the method when some of N independent tests have untrue null hypotheses, we shall now compute the type of p-mean significance levels in- troduced by Duncan [4] and described by Miller [5]. Let us denote the '(over-all" null hypothesis for the N tests by The p-mean significance level is now defined as the maximum probability of falsely rejecting: for any values of pP+, . . . p, . Afore precisely this can be written ap= sup PP +I.. .PN P(D(p1 # 0 up, # 0u. . . up,,# 0 I zJO'P' where D(pl # 0 Up, # 0 U . . . Upp # 0) stands for the decision to reject at least one of the p null hypotheses. The a,of Eklund's method are now computed in the same way as for a, = a in [3]. We assume that the supremum values are taken for such values of pp+~, pp+~... p~ that the corresponding N -p tests


View Full Document

Yale STAT 619 - A Note on a Method for the Analysis of Significances en masse

Download A Note on a Method for the Analysis of Significances en masse
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view A Note on a Method for the Analysis of Significances en masse and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view A Note on a Method for the Analysis of Significances en masse 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?