FSU CIS 5930r - Lecture 26 Testing Random Number Generators

Unformatted text preview:

Slide 1Testing Random-Number GeneratorsChi-Square TestChi-Square ExampleFine PrintsKolmogorov-Smirnov TestK-S TestK-S ExampleK-S Test vs. Chi-square TestSerial-Correlation TestAutocovarianceAutocovarianceAutocovarianceTwo-Level TestsTwo-Level Testsk-Dimensional Uniformity or k-DistributivityVisual CheckVisual CheckSerial TestSerial Test vs. Visual CheckSpectral TestSpectral Test ExampleObservationsAnother Spectral Test ExampleWhite SlideTesting Random-Number GeneratorsAndy WangCIS 5930-03Computer SystemsPerformance AnalysisTesting Random-Number Generators•How do you know if your random number streams are good?–Plot histogram, CDF, QQ-plot–Run tests•Necessary but not sufficient to prove the quality of a generator–If it fails a test, the generator is bad–Passing a test is not a guarantee that a generator is good–Tests can also be used to verify distribution matching23Chi-Square Test•Most commonly used test–Tests if a data set matches a distribution•Steps–Prepare a histogram with k bins–Compare the observed and expected frequencies via a formula–If the computed value < 2[1-α; k-1] (Table A.5)•We have a matchChi-Square Example•N = 1000•k = 10•2[1-0.1; 10-1]= 14.68–A match4i observed expected (observed – expected)2 / expected1 100 100 02 96 100 .163 98 100 .044 85 100 2.255 105 100 .256 93 100 .497 97 100 .098 125 100 6.259 107 100 .4910 94 100 .36Total 1000 1000 10.38Fine Prints•When applied to a nonuniform distribution–Watch out for small expected values, which can affect the outcome•Remedy–Use variable bin sizes, so that the expected outcomes are equal•Designed for discrete distribution with large sample sizes56Kolmogorov-Smirnov Test•Tests if a sample of n observations is from a continuous distribution•Observation–If two distributions match, •| observed CDF Fo(x) - expected CDF Fe(x) | should be smallK-S Test•Differences are measured by the maximum observed deviations above and below the expected CDFK+ = n maxx[Fo(xi) - Fe(xi)]K- = n maxx[Fe(xi + 1) - Fo(xi)]•If both K+ and K- < K[1-α; n] (Table A.9)–We have a match7K-S Example•xn = 3xn-1 % 31n = 30x0 = 15•K[0.9, 30]= 1.042•A match8j xjxj/31 sorted xj/31 j/n – xjxj – (j-1)/n1 14 .451 .032 .001 .0322 11 .354 .064 .002 .0313 2 .064 .096 .003 .0304 6 .193 .129 .004 .0295 18 .580 .161 .005 .027…30 15 .483 .967 .032 .001maximum .032 .032K-S Test vs. Chi-square Test•K-S test–Designed for small samples and continuous distributions–Uses differences between CDFs–Tests each sample without grouping–Exact•Chi-square test–Designed for large samples and discrete distributions–Uses differences between pdfs/pmfs–Requires grouping–Approximate910Serial-Correlation Test•Covariance: tests dependence of two random variables–If the covariance is nonzero, variables are dependent–If the covariance is zero, variables can still be dependentAutocovariance•Autocovariance at lag k (>1) Rk: covariance between numbers that are k values apart, 0 < random number U < 1•For large n–Rk is normally distributed, with a mean of 0•Variance of 1/[144(n – k)]11212111kikniikUUknRAutocovariance•Confidence interval•If confidence interval does not include zero, the sequence has a significant correlation•Example from slide 8–xn = 3xn-1 % 31–x0 = 15–Not so good12knzRk12/2/10 2 4 6 8 10 12-0.04-0.03-0.02-0.0100.010.020.030.040.05lag kRkAutocovariance•Another example xn = 75xn-1 % (231 – 1)x0 = 1•A better random-number generator130 2 4 6 8 10 120000000lag kRk14Two-Level Tests•If the sample test is too small, test results only apply locally•If the sample test is large, test results only apply globally15•Solution–Apply a chi-square test on n bins of size k–Then apply a chi-square test on a set of n-bin statistics–Can locate a nonrandom segment of a random sequenceTwo-Level Tests16k-Dimensional Uniformity or k-Distributivity•So far, tests ensure that numbers are uniformly distributed in one dimensionFor 0 < a < u < b < 1P(a < un < b) = b – a•It is known as the 1-distributivity of un•For 2-distributivityP(a1 < un-1 < b1 and a2 < un < b2) = (b1 – a1)(b2 – a2)Visual Check•Plot successive overlapping pairs of numbers in a 2-D space–E.g., (u0, u1), (u1, u2), …•Example of 15-bit random numbers: –Tausworthe generator–x15 + x + 1–Period: 215 - 1–bn+15 bn+1 bn = 0–bn = bn-14bn-15170 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 100.10.20.30.40.50.60.70.80.91ui ui + 1Visual Check•Another example of 15-bit random numbers: –Tausworthe generator–x15 + x4 + 1–bn+15 bn+4 bn = 0–bn = bn-11bn-15180 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 100.10.20.30.40.50.60.70.80.91ui ui + 119Serial Test•Test for uniformity > 2D–Plot nonoverlapping pairs•E.g., (u0, u1), (u2, u3), …–For 2D, divide the space into K2 cells•Each cell is expected to have n/2K2 points•Use the chi-square test•With K2 – 1 degrees of freedom–Can generalize it into k-dimension–Dependency likely to fail high-dimension chi-square testsSerial Test vs. Visual Check•Serial Test–Uses nonoverlapping points•Assumed by the chi-square test–Has n/2 pairs•Visual Check–Uses overlapping points–Has n – 1 pairs2021Spectral Test•Plot successive overlapping pairs of numbers•K-dimension tuples from an LCG fall on a finite number of parallel hyperplanes22•xn = 3xn-1 % 31–Can see three lines with positive slopes–xn = 3xn-1–xn = 3xn-1 - 31–xn = 3xn-1 - 62Spectral Test Example0 5 10 15 20 25 30 3505101520253035nth numberrandom numberObservations•k-tuples from an LCG fall on at most (k!m)1/k parallel hyperplanes, where m is the modulus–M = 232–3-tuples will have < 2953 hyperplanes–4-tuples, < 566 hyperplanes–10-tuples, < 4123Another Spectral Test Example•xn = 3xn-1 % 31–Period: 30–Pass chi-square test–3 lines•xn = 13xn-1 % 31–Period: 30–Pass chi-square test–10 closer lines–Better 2-distributivity240 5 10 15 20 25 30 3505101520253035nth numberrandom number0 5 10 15 20 25 30 3505101520253035nth numberrandom number25White


View Full Document

FSU CIS 5930r - Lecture 26 Testing Random Number Generators

Download Lecture 26 Testing Random Number Generators
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture 26 Testing Random Number Generators and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture 26 Testing Random Number Generators 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?