Poisson Random Variables Bios 662 Michael G Hudgens Ph D mhudgens bios unc edu http www bios unc edu mhudgens 2006 10 23 16 26 BIOS 662 1 Poisson Poisson Chapter 6 5 text Two main applications Modeling counts of discrete events in space or time Approximation to the Binomial distribution for large N and small p BIOS 662 2 Poisson Poisson Examples Number of abnormal cells in a fixed area of a histological slide Count of bacteria surviving treatment in a fixed volume of bacterial suspension Number of white blood cells in a drop of blood Number of new breast cancer cases registered per month by the National Cancer Registry Number of live births in Greater London during the month of January BIOS 662 3 Poisson Poisson Two assumptions required for Poisson distribution to be an appropriate model The number of events occurring in one part of the continuum space time should be statistically independent of the number of events occurring in another part of the continuum The expected number of counts in a given part of the continuum should approach zero as its size approaches zero BIOS 662 4 Poisson Poisson The Poisson distribution is characterized by one parameter Y Poisson probability mass function e k Pr Y k k Y 0 1 2 The parameter is both the mean and variance E Y V Y BIOS 662 5 Poisson BIOS 662 0 0 0 0 0 1 0 1 0 2 0 2 0 4 0 4 0 5 0 5 0 6 0 6 0 0 5 5 10 10 15 15 20 k 6 0 3 Pr Y k 0 3 Pr Y k 0 0 0 0 0 1 0 1 0 2 0 2 0 3 Pr Y k 0 3 Pr Y k 0 4 0 4 0 5 0 5 0 6 0 6 Poisson PMF 0 5 2 20 0 0 5 5 10 k k 5 10 10 15 20 15 20 k Poisson Poisson and Binomial Suppose X Binomial N and Y Poisson with N Then for N large and small Pr X x Pr Y x i e N N x N x e 1 N x x x Rule of thumb 1 and N 20 BIOS 662 7 Poisson Poisson and Binomial Table 6 6 text k 0 1 2 3 4 BIOS 662 N 10 0 20 0 1074 0 2684 0 3020 0 2013 0 0881 Binomial PMF N 20 N 40 N 1000 0 10 0 05 0 002 0 1216 0 1285 0 1351 0 2702 0 2706 0 2707 0 2852 0 2777 0 2709 0 1901 0 1851 0 1806 0 0898 0 0901 0 0902 8 Poisson PMF 2 0 1353 0 2707 0 2707 0 1804 0 0902 Poisson Poisson and Binomial Sketch of proof Suppose on average events expected to occur over some fixed time interval Divide interval into N subintervals small enough such that the probability of two events occurring in the same subinterval is v unlikely Then the N subintervals approximate a sequence of N Bernoulli trials with success prob N BIOS 662 9 Poisson Poisson and Binomial Thus the probability of observe exactly x events in the N subintervals is N N 1 N x 1 x N x 1 1 x N N As N N N 1 N x 1 N x and N x N 1 1 e N N Thus 1 approx equals N x x e x e x N x BIOS 662 10 Poisson Exact Confidence Intervals Cf Note 6 8 text page 195 Given y occurrences an exact 1 100 CI for is BIOS 662 1 2 1 2 2 2y 1 2 2 y 1 2 2 11 Poisson Normal Approximations If Y Poisson and large say 100 then Y N Thus approx 1 CI Y z1 2 Y A better approximation arises from 1 Y N 4 For 30 approx CI for z1 2 Y 2 BIOS 662 12 Poisson Sum of Poisson Random Variables If Y1 Y2 YN iid Poisson then N X Yi Poisson N i 1 Estimator for 1 X Yi N i If L U is 1 CI for N then L N U N is 1 CI for E g sX z1 2 Yi N 2 i BIOS 662 13 Poisson Example 6 20 Number of bacterial colonies per plate 72 69 63 59 59 53 51 Sum 426 mean 60 86 Exact 95 CI for 7 BIOS 662 1 2 1 2 025 2 426 975 2 427 385 50 467 39 2 2 14 Poisson Example 6 20 Normal approximations 426 z 975 426 385 55 466 45 z 975 2 z 975 2 426 426 386 51 467 41 2 2 Divide endpoints by N 7 to get 95 CI for BIOS 662 15 Poisson Rules of Thumb For 0 05 Y z1 2 2 Y 1 implying approx 95 CI 2 2 Y 1 Y 1 If we observe y 0 a two sided 90 CI for is 1 2 0 95 2 0 3 00 2 Thus if we observed 0 events out of N trials the approx upper bound on a two sided 90 CI is 3 N BIOS 662 16 Poisson Homogeneity Test Often observed counts exhibit larger variance than expected under the Poisson model over dispersion This may be caused by heterogeneity in the s Want to test H0 X1 X2 Xk Poisson BIOS 662 17 Poisson Homogeneity Test Construct 2 GOF test using the following result Suppose Xi Poisson i for i 1 2 k Then the conditional distribution of X1 Xk given P i Xi N is multinomial with cell probabilities i for i 1 2 k 1 k BIOS 662 18 Poisson Homogeneity Test Under H0 H0 X1 X2 Xk Poisson the test statistic 2 X X i T i 2k 1 X approx improves as k gets large 50 P Equivalent form n 1 s2 T X Poisson homogeneity heterogeneity dispersion test BIOS 662 19 Poisson Homogeneity Test Simulation Study 1 0 950 10 15 0 8 0 6 0 4 0 0 0 2 Pr T t k 10 lambda 50 100 120 0 0 0 2 0 4 0 6 0 8 1 0 k 100 lambda 5 140 0 5 10 15 t k 1000 lambda 5 k 10 lambda 500 1000 1050 1100 1150 20 t 5 10 15 20 25 30 25 0 20 t t BIOS 662 5 t 900 0 1 0 80 30 0 8 1 0 0 8 0 6 0 4 0 2 0 0 850 25 60 20 0 6 15 0 2 0 0 10 0 0 0 4 0 6 0 8 1 0 5 Pr T t 0 2 Pr T t Pr T t 0 8 0 6 0 4 0 0 0 2 Pr T t EDF X 2 k 1 0 Pr T t k 10 lambda 5 0 4 1 0 k 10 lambda 5 30 25 30 t 20 Poisson Homogeneity Test Example 6 20 k 7 X 60 …