DOC PREVIEW
CMU CS 10701 - Recitation

This preview shows page 1-2-3-4-5-6 out of 17 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 17 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

10701 RecitationTopicsQuestions?0-1 Error vs Squared ErrorSquared ErrorExample: Intrinsic Error and Modeling ErrorModeling Error = Bias2+VarianceModeling Error = Bias2+VariancePractical Solution to the Bias-Variance ProblemCross-ValidationCross-Validation ExampleNeural NetworksNeural Networks: Sigmoid FunctionNeural Networks: Sigmoid FunctionNeural Networks: Sigmoid FunctionNeural Networks: BackpropagationNeural Networks: Simple Example10701 RecitationDerek HoiemFeb 3, 2005Topics• Questions– Last homework– Next homework– Class material• Where does error come from?– Measuring error– Intrinsic error– Bias and Variance components•Overfitting– Cross-validation• Backpropagation ExampleQuestions?0-1 Error vs Squared ErrorLossConfidenceSquared Error∑=−=NnnnNtwxye121));((Squared ErrorInfinite Data∫∫−= dtdxxtptwxye ),());((2∫∫−+−= dxxpxtExtEdxxpxtEwxye )())|()|(()())|();((222Tricky StuffIntrinsic ErrorModeling ErrorExample: Intrinsic Error and Modeling ErrorModeling Error = Bias2+Variance∫∫−+−= dxxpxtExtEdxxpxtEwxye )())|()|(()())|();((222Modeling error, huh? Sounds suspicious….Spectacled Friend #1There’s nothing I can do….Spectacled Friend #2What is in your circle of concern?Stephen CoveyModeling Error = Bias2+Variance∫+− dxxpxtEwxy )())|();((2∫−= dxxpxtExtEe )())|()|((22Tricky Stuff∫∫−+−= dxxpwxyEwxyEdxxpxtEwxyEeEDDDmD)())];(();([)()]|());(([)(22Bias2VariancePractical Solution to the Bias-Variance Problem1. Sufficiently complex classifier for problem (low bias)• How can we tell if its too complex or not complex enough?2. Cross-validation or validation set (reduce variance)Cross-Validation• How many folds?–K-fold– Leave-One-OutCross-Validation ExampleNeural Networks• Defined by:– Num inputs– Num hidden layers– Hidden nodes per layer– Num outputs– Activation functionsxh11h12h21h22y1 1 1Neural Networks: Sigmoid Function))(exp(11);(0∑+−+=iiixwwwxσxh11h121w0h11w0h12w1h12w1h11xh11h12h21h22y1 1 1Neural Networks: Sigmoid Function))(exp(11);(0∑+−+=iiixwwwxσw0Æ shift kw Æ slopeNeural Networks: Sigmoid Function))(exp(11);(0∑+−+=iiixwwwxσNeural Networks: Backpropagationxh11h12h21h22y1 1 1Neural Networks: Simple ExampleTraining Input:x=1y=1x=0.5y = 01 11-1h


View Full Document

CMU CS 10701 - Recitation

Documents in this Course
lecture

lecture

12 pages

lecture

lecture

17 pages

HMMs

HMMs

40 pages

lecture

lecture

15 pages

lecture

lecture

20 pages

Notes

Notes

10 pages

Notes

Notes

15 pages

Lecture

Lecture

22 pages

Lecture

Lecture

13 pages

Lecture

Lecture

24 pages

Lecture9

Lecture9

38 pages

lecture

lecture

26 pages

lecture

lecture

13 pages

Lecture

Lecture

5 pages

lecture

lecture

18 pages

lecture

lecture

22 pages

Boosting

Boosting

11 pages

lecture

lecture

16 pages

lecture

lecture

20 pages

Lecture

Lecture

20 pages

Lecture

Lecture

39 pages

Lecture

Lecture

14 pages

Lecture

Lecture

18 pages

Lecture

Lecture

13 pages

Exam

Exam

10 pages

Lecture

Lecture

27 pages

Lecture

Lecture

15 pages

Lecture

Lecture

24 pages

Lecture

Lecture

16 pages

Lecture

Lecture

23 pages

Lecture6

Lecture6

28 pages

Notes

Notes

34 pages

lecture

lecture

15 pages

Midterm

Midterm

11 pages

lecture

lecture

11 pages

lecture

lecture

23 pages

Boosting

Boosting

35 pages

Lecture

Lecture

49 pages

Lecture

Lecture

22 pages

Lecture

Lecture

16 pages

Lecture

Lecture

18 pages

Lecture

Lecture

35 pages

lecture

lecture

22 pages

lecture

lecture

24 pages

Midterm

Midterm

17 pages

exam

exam

15 pages

Lecture12

Lecture12

32 pages

lecture

lecture

19 pages

Lecture

Lecture

32 pages

boosting

boosting

11 pages

pca-mdps

pca-mdps

56 pages

bns

bns

45 pages

mdps

mdps

42 pages

svms

svms

10 pages

Notes

Notes

12 pages

lecture

lecture

42 pages

lecture

lecture

29 pages

lecture

lecture

15 pages

Lecture

Lecture

12 pages

Lecture

Lecture

24 pages

Lecture

Lecture

22 pages

Midterm

Midterm

5 pages

mdps-rl

mdps-rl

26 pages

Load more
Download Recitation
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Recitation and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Recitation 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?