45 0 32367 8 92 03 2 3 3 623 0 3 230 0 3236 0 1 23 30620 0 3 32 0 21 1 0 3 7 0 1 23 23 1 3 0 0 0 0 0 3236 1 1 6 2 2 20 7 0 1 300 1 0 21 6 03 6 0 1 23 24 5 2 0 011236 2 2 12 3 1 231 3 0 2 0 1 0 2 0 6 7 0 1 00 0 1 5 0 30 23 1 0 2 h 1 1 2 21 1 23236 1 01 0 0 23 1 23236 0 1 23 h 1 2 1 0 2 21 1 1 h 0 1 3 1 0 0 0 1 0 h 0 1 23 25 2 0 2 1 0 2 1 601 m 1 231 26 1 1 0 2 h 1 1 2 21 1 23236 1 6 1 m 2 2 231 26 1 2 21 601 1 2 1 26 17 1 26 1 0 0 4 h 21 0 1 0 601 30 1 231 26 1 4 h 21 0 1 0 601 m 1 231 26 1 0 1 23 26 1 1 0 0 0 3 2 0 1 0 2 1 1 0 3 2 1031 21 1 23236 1 0 1 23 27 2 0 2 0 30 1 2 1 0 2 4 h 21 0 1 0 601 m 1 231 26 1 0 0 0 k 1 0 2 3 2 1031 21 1 2 0 2 0 30 1 2 30 0 1 23 28 32 3 3 4 5 9 0 1 23 29 2 0 2 0 30 1 2 1 0 2 4 h 21 0 1 0 601 m 1 231 26 1 0 0 0 k 1 0 2 3 2 1031 21 1 2 0 2 0 30 1 2 30 0 1 23 30 0 20 030 2 12 3 0 23 23210 1 0 2 0 0 Theorem 1 0 2 0 H 232107 1 01 D 21 m 2 2 0 7 3 0 30 1 0 2 h 1 1 2 3 2 1031 3 1 0 1 23236 1 0 1 23 31 45 0 32367 8 92 03 2 3 230 0 3236 0 1 23 30620 0 3 32 0 21 1 0 1 7 0 1 23 1 5 2 0 011236 2 2 12 3 1 231 Finite 3 0 2 0 1 0 2 0 6 7 0 1 00 0 1 5 0 30 23 1 0 2 h 1 1 2 consistent 21 1 23236 1 01 0 0 23 1 23236 0 1 23 h 1 2 1 0 2 21 1 1 h 0 1 3 1 0 0 0 1 0 h 0 1 23 2 2 0 2 1 0 2 1 601 m 1 231 26 1 1 0 2 h 1 1 2 consistent 21 1 23236 1 6 1 m 2 2 231 26 1 2 21 601 1 2 1 26 17 1 26 1 0 0 4 h 21 0 1 0 601 30 1 231 26 1 4 h 21 0 1 0 601 m 1 231 26 1 0 1 23 3 1 1 0 0 0 3 2 0 1 0 2 1 1 0 3 2 1031 21 1 23236 1 0 1 23 4 2 0 2 0 30 1 2 1 0 2 4 h 21 0 1 0 601 m 1 231 26 1 0 0 0 k 1 0 2 3 2 1031 21 1 2 0 2 0 30 1 2 30 0 1 23 5 32 3 3 4 5 9 0 1 23 6 2 0 2 0 30 1 2 1 0 2 4 h 21 0 1 0 601 m 1 231 26 1 0 0 0 k 1 0 2 3 2 1031 21 1 2 0 2 0 30 1 2 30 0 1 23 7 0 20 030 2 12 3 0 23 23210 1 0 2 0 0 Theorem 1 0 2 0 H 232107 1 01 D 21 m 2 2 0 7 3 0 30 1 0 2 h 1 1 2 3 2 1031 3 1 0 1 23236 1 0 1 23 8 236 45 3 2 7 0 0 42 3 7 62 0 m 42 3 7 62 0 9 0 1 23 0 20 030 2 12 3 0 23 23210 1 0 2 0 0 Theorem 1 0 2 0 H 232107 1 01 D 21 m 2 2 0 7 3 0 30 1 0 2 h 1 1 2 3 2 1031 3 1 0 1 23236 1 Even if h makes zero errors in training data may make errors in test 0 1 23 10 2 21 12 3 0 3 3 2 1031 2 20 2 0 1 0 2 0 0 1 23 11 1 2 2 20 0 3 1 0 0 0 3 1 0 1 23236 1 5 0 30 21 0 1 23236 0 0 2 1 0 23 10 1 01 1 1 0 30 21 errortrain h 23 1 23236 01 0 1 23 12 2 0 0 12 3 1 1 0 0 0 10 0 1 0 2 0 0 1 0 2 2 2 0 0 12 1236 1 0 010 23 0 3 3 m 2 2 23 2 7 7 7 7 0 0 2 7 0 1 23 13 236 0 3 3 1 0 12 10 0 236 0 1 0 2 0 1 23 14 1 0 0 236 3 1 0 2 Union bound 0 1 0 2 2 1 2 236 1 1 0 2 7 3 0 1 23 15 030 2 12 3 3 1 0 2 Theorem 1 0 2 0 H 232107 1 01 D 21 m 2 2 0 7 3 0 30 1 0 2 h 0 1 23 16 45 3 3 2 8 2 3 0 1 0 or after moving some terms around with probability at least 1 Important PAC bound holds for all h but doesn t guarantee that algorithm finds best h 0 1 23 17 1 1 1 0 2 0 1 0 1 0 2 0 60 2 1 0 1 0 2 0 0 1 23 18 0 3 21 n 23 0 1 0 0 1 23 19 0 0 2 2 3 1 00 0 1 0 2 0 12 3 2 03 n 11 2 10 0 0 2 2 3 1 00 0 1 2 0 1 11 2 10 2 0 0 1 1 00 2 0 26 1 1 00 3 210 6 6 3 6 3 0 1 23 20 45 3 0 2 2 3 1 00 0 1 0 231 2 0 30312 23 0 1 17 m 1 231 7 0 2 2 3 1 00 3 1 601 1 26 …
View Full Document
Unlocking...