1Model Forms and Complexity ConsiderationsRemember Simple ExamplesOutlook:Sunny -> NoOvercast -> YesRainy-> Yes4 Main Families of ML Algorithmsn Within Families: Same form but different search methodn Families¨ Tree Based Algorithms¨ Rule Based Algorithms¨ Bayesian Algorithms¨ Weight Based Modelsn Today we will talk about Tree Based and Weight Based model forms3More Complex Algorithm…n Two simple representations we discussed so far¨ 0R – Predict the majority class¨ 1R – Use the most predictive single featuren Decision Trees¨ Uses more than one feature¨ Can take advantage of contingencies between features* Only makes 2 mistakes on tennis datasetAlgorithm Foreshadowing:How does it workn Split instances based on best feature¨ Metric called information gain – we will talk about details later¨ Achieves the cleanest splitn Add node for split on selected featuren Split the instances at each leaf noden Stop with the leaf node contains a small number of instances* Only makes 2 mistakes on tennis datasetWhy is it better?n Not because it is more complex¨ Sometimes more complexity makes performance worsen What is different in what the three rule representations assume about your data?¨ 0R¨ 1R¨ Treesn The best algorithm for your data will give you exactly the power you needWhy is it better?Let’s say you know the rule you are trying to learn is a circle, and you have these points. What rule would you learn?Why is it better?Let’s say you know the rule you are trying to learn is a circle, and you have these points. What rule would you learn?Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?If you know the shape, you have fewer degreesof freedom – less room to make a mistake.Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?If you know the shape, you have fewer degreesof freedom – less room to make a mistake.Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?If you know the shape, you have fewer degreesof freedom – less room to make a mistake.Why is it better?Let’s say you know the rule you are trying to learnis a circle and you have these points. What rulewould you learn?Now lets say you don’t know the shape, now what would you learn?If you know the shape, you have fewer degreesof freedom – less room to make a mistake.Why is it better?n Not because it is more complex¨ Sometimes more complexity makes performance worsen What is different in what the three rule representations assume about your data?¨ 0R¨ 1R¨ Treesn The best algorithm for your data will give you exactly the power you