1 A Note on how to randomly sample from the f x distribution erm Oct 26 2009 Many software packages have commands to draw random samples from the standard normal distribution and the uniform distribution on the 0 to 1 hereafter the unit uniform However commands and software do not generally exist for taking random samples from other distributions 1 1 Assume the problem is take a random sample from the fX x distribution where one has the ability e g Mathematica to take a random sample from the uniform distribution on the 0 1 interval Let FX X denote the cdf of fX X One can show that one can obtain a random draw from fX X by obtaining a random draw from the unit uniform distribution fU U then plugging that draw into FX 1 U where FX 1 is the inverse of FX X That is FX 1 U is h FX X solved for X and then evaluated at U 1 1 1 Example 1 Take a random sample from the negative exponential distribution MGB 3rd edition p 112 If X has a negative exponential distribution where x if if 0 h FX x 1 n for x Solution is x e fX x e 0 x 1 x 0 0 The corresponding cdf is Solve h 1 e x FX 1 u x ln h 1 ln u 1 1 o So ln u 1 In words if u is a random draw from the unit uniform then random draw from the exponential distribution 1 is a One draws a random sample of N observations from the unit uniform then converts each using the formula ln u 1 1 1 2 Example 2 I often use this rule to take to draw a random sample from a population with an Extreme Value distribution The cdf for the Extreme Value distribtuion is FX x Pr X That is h exp e x ln h e x x x exp e x ln h e x ln ln h ln ln h So if if u is a random draw from the unit uniform then x random draw from the Extreme Value distribution 1 2 x ln ln u is a Some Background This method for taking random samples is an application of Theorem 12 sec 5 2 of MGB page 202 It is called the Probability Integral Transform Theorem 12 says If X is a random variable with a continuous cummulative distribution function FX x then U FX X is uniformly distribution over the interval 0 1 Conversely if U is uniformly distributed on the interval 0 1 then X FX 1 U has the cummulative distribution function FX 1 3 Does this technique work if fX x is a discrete distribution Consider the following example Assume f x 2 if x 1 f x 5 if x 2 f x 3 if x 3 and zero otherwise Picture the df and cdf of this discrete distributino 8 x 1 0 if 2 if 1 x 2 7 if 2 x 3 1 if x 3 1 Note that since u is between zero and one ln u 1 0 so 2 ln u 1 0 F x 1 0 75 0 5 0 25 0 0 1 2 3 4 So one way to take a random sample from this density function is to invert the cdf to get 8 x 1 if 0 y 2 x 2 if 2 y 7 x 3 if y 7 y 3 2 5 2 1 5 1 0 0 25 0 5 0 75 1 x So when we take a random sample from the unit uniform density we transform them into 1 s 2 s and 3 s using the above rule What did I just demonstrate 3 That the probability integral transform works at least sometimes even if the density function is discrete 1 4 Does this technique work if fX x is a mixture of a continous and discrete distributions 4