View Full Document

Shrinking Trees



View the full content.
View Full Document
View Full Document

5 views

Unformatted text preview:

Shrinking Trees Trevor Hastie Daryl Pregibon AT T Bell Laboratories 600 Mountain Avenue Murray Hill NJ 07947 July 30 1990 Abstract Tree based models provide an alternative to linear models for classi cation and regression data They are used primarily for exploratory analysis of complex data or as a diagnostic tool following a linear model analysis They are also used as the end product in certain applications such as speech recognition medical diagnoses and other instances where repeated fast classi cations are required or where decision rules along coordinate axes facilitate understanding and communication of the model by practitioners in the eld Historically the key problem in tree based modeling is deciding on the right size tree This has been addressed by applying various stopping rules in the tree growing process and more recently by applying a pruning procedure to an overly large tree Both approaches are intended to eliminate over tting the data especially as regards using the tree for prediction The approach taken in this paper provides yet another way to protect against over tting As in the pruning case we start with an overly large tree but rather than cut o branches which seem to contribute little to the overall t we simply smooth the tted values using a process called recursive shrinking The shrinking process is parameterized by a scalar which ranges from zero to one A value of zero implies shrinking all tted values to that of the root of the tree whereas a value of one implies no shrinking whatsoever The shrinking parameter must be speci ed or otherwise selected on the basis of the data We have used cross validation to guide the choice in certain of the applications we have examined Shrinking and pruning are qualitatively di erent although they tend to have similar predictive ability We draw on analogies with the usual linear model to emphasize the di erences as well as the similarities between the two methods A comparison of shrinking and pruning on two



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Shrinking Trees and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Shrinking Trees and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?