TRUNCATION Weber Kannan Bordeman 11 18 10 In our discussion of random variables we have talked about methods of narrowing our focus within the population of interest One such method that we have discussed in class is censoring In these notes we address another method truncation A QUICK REVIEW ON CENSORING First let s review what we know about censoring To censor a population is to collapse the density over a range of the random variable onto a single value of the random variable For example if we were analyzing the distribution for annual income in a population and we considered all families with annual income below the poverty level say 22 000 as a single group we would collapse the density of the pdf for all values less than 22 000 onto a single value of the random variable the poverty level or 22 000 Therefore a pile up would occur at 22 000 representing the collapsed density of all families with annual income at or below that amount The whole population is still considered in the censored density function but we consider all families earning less than the lower limit as one in the same A graph of this leftcensored probability density function might look like this assuming a income is normally distributed with mean and variance of 50 000 and 12 000 respectively where we left censor at the poverty level of 22 000 0 30 0 25 0 20 0 15 0 10 0 05 4 6 8 10 Note that Mathematica drew the y axis at x 30 000 instead of the origin and that the x axis is scaled by 10 000 The pile up occurs at the poverty level of 22 000 and the height of the pile up is equal to the area of the un censored distribution left of 22 000 In our poverty level example we have a left censored distribution We can also have rightcensored distributions see the insurance example in the following section However censored distributions need not be only right or left censored Any distribution for which we collapse the values of a range of a random variable onto a single value is a censored distribution Consider the following example Assume we have income data for a particular population and that data has some distribution If we decided to round the income data to the nearest 10 000 our distribution would be censored although not necessarily right or leftcensored Specifically the density for all values between 0 and 4 999 would be collapsed at 0 the density for all values between 5 000 and 14 999 would be collapsed at 10 000 and so on Now that we ve reviewed censoring our task for the remainder of these notes is to discuss another method of refining our distribution called truncation TRUNCATION VS CENSORING The literal meaning of truncation is to shorten or cut off something Extending this definition to our world of statistics we can define the truncation of a distribution as a process which results in certain values being cut off thereby resulting in a shortened distribution As with censoring we can have left or right truncated distributions and we can also have distributions that are truncated in the middle such as a distribution where we can observe values of the random variable between one and ten but we truncate to exclude values between two and three see example 2 under Some Examples of Truncated Distributions The density of the pdf at all values that are cut off are omitted from the truncated distribution and the remaining distribution is shifted upward so that the area beneath it is still one Note the difference from censoring where we pile up the CDF of the distribution outside of our limit We use a truncated distribution when certain values within the distribution cannot be observed Unlike censoring truncation allows us to shorten our scope of analysis of the distribution and disregard the data outside of the limit To see the difference between truncation and censoring consider the following example in the insurance industry Insurance companies levy a policy limit on insurance policy holders Suppose this limit is 5 000 Any damages submitted by the policyholder above 5 000 will be reimbursed by the insurance company as 5 000 because of the policy limit So the distribution for the random variable of interest damages paid is obtained by censoring damage amounts This is a case of right censoring of data as the insurer only pays up to the policy limit even if actual damages exceed that limit As a result there is a pile up at the limit since all losses exceeding 5 000 will be reimbursed for 5 000 Now suppose the policyholder is subject to a deductible of 1 000 Any loss incurred below the deductible 1 000 will not be reported by the policyholder Although losses will occur below the deductible limit 1 000 the insurance company is unaware of them since the policyholders will not report them This is a case of left truncation of data as the insurance company does not observe damages below the limit of 1 000 Let s graph a probability density function assuming that damages are normally distributed with mean of 3 000 and variance of 1 500 We see that the graph is truncated on the left at 1 000 since these values are never observed by the insurance company and we see a pile up at 5 000 since the sum of the probabilities of damages exceeding 5 000 are piled up at this limit 0 25 0 20 0 15 2 3 4 5 Note that Mathematica drew the x axis at y 0 12 and the y axis at x 1 rather than at the origin SOME EQUATIONS FOR TRUNCATION 1 If a continuous random variable x has pdf f x then f x x a f x Prob x a Note that the truncated distribution is a conditional distribution 2 If X is a random variable with density f x and cumulative distribution Fx then the density of X truncated on the left at a and on the right at b is given by f x I a b x Fx b Fx a Note I a b x is an indicator function where I a b x 1 if a x b I a b x 0 otherwise MOMENTS OF TRUNCATED DISTRIBUTIONS We are often interested in the mean and variance of a truncated distribution as they are ways to characterize the distribution The moments can be obtained using the following formulae Mean E x x a a xf x x a dx Variance a x E x x a 2 f x x a dx 3rd Moment a x E x x a 3 f x x a dx 4th Moment a x E x x a 4 f x x a dx SOME EXAMPLES OF TRUNCATED DISTRIBUTIONS 1 Truncated uniform distribution Suppose x has a standard uniform distribution U 0 1 then f x 1 0 X 1 Let the distribution be left truncated at x 1 3 The truncated distribution is also …