UT PSY 394U - Supervised Machine Learning: Classification Techniques

Unformatted text preview:

Supervised Machine Learning: Classification TechniquesSupervised Machine LearningDecision TreesDecision Trees, an exampleDecision Trees: AssessmentBayesian NetworksBayesian Networks, an exampleBayesian Networks: AssessmentNeural NetworksNeural Networks: Biological BasisFeed-forward Neural NetworkNeural Networks: TrainingBackpropagationSupport Vector MachinesPerceptron Revisited:Which one is the best?Notion of MarginMaximizing MarginKernel TrickNon-linear SVMs: Feature spacesExamples of Kernel TrickSlide 22Advantages and Applications of SVMThe Future of Supervised Learning (1)The Future of Supervised Learning (2)ReferencesSupervised Machine Learning:Classification TechniquesChaleece SandbergChris BradleyKyle WalshSupervised Machine LearningSML: machine performs function (e.g., classification) after training on a data set where inputs and desired outputs are providedFollowing training, SML algorithm is able to generalize to new, unseen dataApplication: “Data Mining” Often, large amounts of data must be handled efficientlyLook for relevant information, patterns in dataDecision TreesLogic-based algorithmSort instances (data) according to feature values…a hierarchy of testsNodes: featuresRoot node: feature that best divides dataAlgorithms exist for determining the best root nodeBranches: values the node can assumeDecision Trees, an exampleINPUT: data (symptom)low RBC countyesnosize of cellslarge smallbleeding?STOPB12 deficient?yesyesstop bleedingnoSTOP gastrin assaypositivenegativeSTOP anemiaOUTPUT: category (condition)Decision Trees: AssessmentAdvantages:Classification of data based on limiting features is intuitiveHandles discrete/categorical features bestLimitations:Danger of “overfitting” the dataNot the best choice for accuracyBayesian NetworksGraphical algorithm that encodes the joint probability distribution of a data setCaptures probabilistic relationships between variablesBased on probability that instances (data) belong in each categoryBayesian Networks, an exampleWikipedia, 2008Bayesian Networks: AssessmentAdvantages:Takes into account prior information regarding relationships among featuresProbabilities can be updated based on outcomesFast!…with respect to learning classificationCan handle incomplete sets of dataAvoids “overfitting” of dataLimitations:Not suitable for data sets with many featuresNot the best choice for accuracyNeural NetworksUsed for:ClassificationNoise reductionPredictionGreat because:Able to learnAble to generalizeKiranPlaut’s (1996) semantic neural network that could be lesioned and retrained – useful for predicting treatment outcomesMikkulainenEvolving neural network that could adapt to the gaming environment – useful learning applicationNeural Networks: Biological BasisFeed-forward Neural NetworkPerceptron:Hidden layerNeural Networks: TrainingPresenting the network with sample data and modifying the weights to better approximate the desired function.Supervised LearningSupply network with inputs and desired outputsInitially, the weights are randomly setWeights modified to reduce difference between actual and desired outputsBackpropagationBackpropagationSupport Vector MachinesPerceptron Revisited:Linear Classifier: y(x) = sign(w.x + b)w.x + b = 0w.x + b < 0w.x + b > 0Which one is the best?Notion of MarginDistance from a data point to the hyperplane:Data points closest to the boundary are called support vectors Margin d is the distance between two classes.wxw || brrdMaximizing MarginMaximizing margin is a quadratic optimization problem.Quadratic optimization problems are a well-known class of mathematical programming problems, and many (rather intricate) algorithms exist for solving them.Kernel TrickWhat if the dataset is non-linearly separable? We use a kernel to map the data to a higher-dimensional space:0x2x0xNon-linear SVMs: Feature spacesGeneral idea: The original space can always be mapped to some higher-dimensional feature space where the training set becomes separable:Φ: x → φ(x)Examples of Kernel TrickFor the example in the previous figure: The non-linear mappingA more commonly used radial basis function (RBF) kernel),()(2xxxx 222/||||),(jieKjixxxxAdvantages and Applications of SVMAdvantages of SVM Unlike neural networks, the class boundaries don’t change as the weights change.Generalizability is high because margin is maximized.No local minima and robustness to outliers.Applications of SVM Used in almost every conceivable situation where automatic classification of data is needed.(example from class) Raymond Mooney and his KRISPER natural language parser.The Future of Supervised Learning (1)Generation of synthetic data A major problem with supervised learning is the necessity of having large amounts of training data to obtain a good result.Why not create synthetic training data from real, labeled data?Example: use a 3D model to generate multiple 2D images of some object (such as a face) under different conditions (such as lighting). Labeling only needs to be done for the 3D model, not for every 2D model.The Future of Supervised Learning (2)Future applications Personal software assistants learning from past usage the evolving interests of their users in order to highlight relevant news (e.g., filtering scientific journals for articles of interest)Houses learning from experience to optimize energy costs based on the particular usage patterns of their occupantsAnalysis of medical records to assess which treatments are more effective for new diseasesEnable robots to better interact with humansReferenceshttp://homepage.psy.utexas.edu/homepage/class/Psy394U/Hayhoe/cognitive%20science%202008/talks:readings/ http://www.ai-junkie.com/ann/evolved/nnt1.html http://galaxy.agh.edu.pl/~vlsi/AI/backp_t_en/backprop.htmlhttp://cbcl.mit.edu/cbcl/people/heisele/huang-blanz-heisele.pdf


View Full Document

UT PSY 394U - Supervised Machine Learning: Classification Techniques

Documents in this Course
Roadmap

Roadmap

6 pages

Load more
Download Supervised Machine Learning: Classification Techniques
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view Supervised Machine Learning: Classification Techniques and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Supervised Machine Learning: Classification Techniques 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?