DOC PREVIEW
CMU CS 10601 - Lecture

This preview shows page 1-2-3-4 out of 11 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 11 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 11 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 11 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 11 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 11 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

9/21/20091Machine Learning - 10601Linear Regression andBias-Variance TradeoffGeoff Gordon, Miroslav Dudík(http://www.cs.cmu.edu/~ggordon/10601/September 21, 2009 Last time: linear regressionGoal:predict a continuous responsefrom continuous/categorical inputsInput:(x1,y1), … , (xN,yN)Model:y wjφj(x)Performance measure:9/21/20092Linear regressionGoal:Input: (x1,y1), … , (xN,yN)Assume: y wjφj(x)9/21/20093Linear regressiony ≈ w0+ w1x + w2x2Linear regressiony ≈ w0+ w1x + w2x2+ w3x3+ w4x4+ … + w10x109/21/20094Training & Test ErrorDon’t Matchdegree of polynomialtest errortrainingerrorTraining & Test ErrorDon’t Match: Why?errortrainerrortrue9/21/20095Training & Test ErrorTraining error:overly optimisticTest error:approximation of prediction erroras long astest data set never touched during trainingSweet spot formodel complexitydegree of polynomialtest errortrainingerror9/21/20096Sweet spot formodel complexityBias-variancetradeoffBias:• faithfulnessto the truthVariance:• sensitivity to randomnessin training data=true error= bias2+ variance + necessary evilBias:• decreases with model complexityVariance:• increases with model complexity• decreases withsize of data setdegree of polynomialtest errortrainingerror9/21/20097true error= bias2+ variance + necessary evilXY = f(X) + εprediction at x0:error(x0) = true error= bias2+ variance + necessary evil9/21/20098Announcements• project proposals duethis Wednesday at 10:30amLeast squares fit= max likelihoodfor Gaussians9/21/20099Beyond max likelihoodfor Gaussians• add a prior over w• replace the Gaussianby a different model– different noise model– different support for YRegression with a Gaussian priorYw9/21/200910Regression with a Gaussian priorYwstrength of regularizationtest errortrainingerrorBias-variance tradeofffor ridge regression9/21/200911strength of regularizationtest errortrainingerrorBias-variance tradeofffor ridge


View Full Document

CMU CS 10601 - Lecture

Documents in this Course
lecture

lecture

40 pages

Problem

Problem

12 pages

lecture

lecture

36 pages

Lecture

Lecture

31 pages

Review

Review

32 pages

Lecture

Lecture

18 pages

Notes

Notes

10 pages

Boosting

Boosting

21 pages

review

review

21 pages

review

review

28 pages

Lecture

Lecture

31 pages

lecture

lecture

52 pages

Review

Review

26 pages

review

review

29 pages

Lecture

Lecture

37 pages

Lecture

Lecture

35 pages

Boosting

Boosting

17 pages

Review

Review

35 pages

lecture

lecture

32 pages

Lecture

Lecture

28 pages

Lecture

Lecture

30 pages

lecture

lecture

29 pages

leecture

leecture

41 pages

lecture

lecture

34 pages

review

review

38 pages

review

review

31 pages

Lecture

Lecture

41 pages

Lecture

Lecture

15 pages

Lecture

Lecture

21 pages

Lecture

Lecture

38 pages

Notes

Notes

37 pages

lecture

lecture

29 pages

Load more
Download Lecture
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Lecture and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Lecture 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?