Greg Grudic Machine Learning Greg Grudic Artificial Intelligence 2 Machine Learning CSCI 4202 1 Greg Grudic Machine Learning Currently Java Sun C unix VC Please email me with Taking CSC I4202 in the subject line I encourage you to use matlab However you can use any language you want as long as you make sure I can run your program Admin Stuff 2 2 Machine Learning Semi Supervised Transduction learning Active learning Reinforcement Learning Unsupervised Learning Greg Grudic Classification Regression Introduction to the main subfields of machine learning Supervised learning Today s Lecture Goals 3 h1 h2 h K System yM y1 y2 Greg Grudic Machine Learning Output Variables y y1 y2 yK Hidden Variables h h1 h2 hK Input Variables x x1 x2 xN xN x1 x2 A Generic System 4 Greg Grudic Machine Learning Statistics mathematics theoretical computer science physics neuroscience etc These algorithms originate form many fields Machine Learning algorithms discover the relationships between the variables of a system input output and hidden from direct samples of the system Another Definition of Machine Learning 5 1 2 2 P f xP Greg Grudic Machine Learning Predict y f x where x is not in the training set Find f x i e an approximation of some unknown function system y f x 1 x f x x f x x Given Training examples Supervised Learning 6 Greg Grudic Machine Learning Model output is a prediction that the input belongs to some class If the input is an image the output might be chair face dog boat etc Regression y The output has infinitely many values If the input is stock features the output could be a prediction of tomorrow s stock price Classification y c1 c2 cN Two Types of Supervised Learning 7 Greg Grudic Machine Learning High Dimensional Feature Space Collect Training data Build Model happy M feature space Make a prediction Learning Classification Models 8 Greg Grudic Stock Value Machine Learning Feature Space Collect Training data Build Model stock value M feature space Make a prediction Learning Regression Models 9 diagnosis Greg Grudic Machine Learning f x Disease or maybe recommended therapy x Properties of patient symptoms lab tests Disease f x Approve purchase or not x Properties of customer and proposed purchase Credit risk assessment 10 Examples of Supervised Learning Vehicle Driving Greg Grudic Machine Learning f x Throttle break and steering commands x Image of the road Automated f x Name of the person x Image of person s face Face recognition 11 Examples of Supervised Learning continued Greg Grudic Machine Learning f x Ascii code of the character x Bitmap picture of hand written character where humans can perform the task but can t describe how they do it Situations 12 f x Predicted binding strength to AIDS protease molecule x Bond graph for a new molecule Situations where there is no human expert Appropriate Applications for Supervised Learning Greg Grudic Machine Learning f x Importance score for presenting to user or deleting without presenting x Incoming email message where each user needs a customized function f Situations f x Recommended stock transactions x Description of stock prices Situations where the desired function is changing frequently 13 Appropriate Applications for Supervised Learning continued 1 2 2 P f xP Greg Grudic 1 1 2 Machine Learning 2 k k y f x y f x y f x of some unknown function system y f x And examples of inputs that require classification x1 x 2 x k Predict 1 x f x x f x x Given Training examples Semi Supervised Transduction Learning 14 Greg Grudic Machine Learning 15 from Learning with Local and Global Consistency Dengyong Zhou Olivier Bousquet Thomas N Lal Jason Weston Bernhard Schoelkopf NIPS 2003 Transduction 2 Machine Learning High Dimensional Feature Space Where in the feature space do I need to sample next to improve my classifier the most 16 Premise Data is expensive to collect e g most experiments in biology Goal want to get the best possible model with the smallest dataset Active learning starts with a classifier and asks the following questions Greg Grudic Active Learning Greg Grudic Machine Learning Goal maximize infrequent reward Agent learns by stochastically interacting with its environment getting infrequent rewards Autonomous agent learns to act optimally without human intervention Reinforcement Learning RL 17 Greg Grudic Machine Learning TD gammon Tesauro Packing containers Moore Elevator dispatch Crites and Barto Successful RL applications Delayed reward HARD problem Addresses the temporal credit assignment problem Reinforcement Learning 18 Robot Static Navigational Feature Goal Greg Grudic Machine Learning Hit an obstacle get a negative reward Reach goal get a positive reward Reach goal faster get a bigger positive reward Obstacle RL in Robotics 19 Greg Grudic Machine Learning A simple Robotics Problem 20 Greg Grudic Machine Learning Even Simple Robotic tasks are difficult to Program 21 Greg Grudic Machine Learning Does ML work on Actual Robots 22 Greg Grudic Machine Learning Yes 23 Greg Grudic Machine Learning More Complex Example 24 Greg Grudic Machine Learning 25 Studies how input patterns can be represented to reflect the statistical structure of the overall collection of input patterns No outputs are used unlike supervised learning and reinforcement learning unsupervised learner brings to bear prior biases as to what aspects of the structure of the input should be captured in the output Unsupervised Learning Greg Grudic Machine Learning High Dimensional Feature Space Collect Training data e g consumer info Build Model things that a similar M feature space 26 Unsupervised Learning Example Greg Grudic Machine Learning Locally Linear Embedding LLE 27 From Sam T Roweis roweis cs toronto edu www cs toronto edu roweis and Lawrence K Saul lsaul research att com www research att com lsaul