## 082917.1

Previewing page
*1*
of
actual document.

**View the full content.**View Full Document

## 082917.1

0 0 51 views

- Pages:
- 4
- School:
- University of Illinois - urbana
- Course:
- Cs 446 - Machine Learning

**Unformatted text preview:**

CS446 Machine Learning Fall 2017 Lecture 1 Introduction to Machine Learning Lecturer Sanmi Koyejo Scribe Omar Elabd Aug 29th 2017 Introduction to Machine Learning Machine learning is an application of artificial intelligence that provides systems the ability to automatically learn and improve from experience without being explicitly programmed Table 1 shows a non comprehensive list of machine learning algorithms categorized by their types Probabilistic Supervised Unsupervised Naive Bayes Generative adversarial networks GANs Latent Dirichlet allocation LDA Gaussian mixture model GMM Non Probabilistic Artificial Neural Networks Support Vector Machines SVM Decision Trees Random Forests Support Vector Machines SVM Linear Discriminant Analysis LDA Adaptive Boosting AdaBoost k means clustering Principal Component Analysis PCA t distributed Stochastic Neighbor Embedding t SNE Hierarchical clustering Table 1 Categorized Machine Learning Algorithms Gradient Descent and Expectation Maximization are both techniques for optimizing machine learning algorithms Probabilistic vs Non Probabilistic Probabilistic Built around learning a probability distribution in order to perform machine learning Non Probabilistic Does not involve learning probabilistic distributions directly 1 2 Lecture 1 Introduction to Machine Learning Supervised Learning Supervised learning is a machine learning task of inferring a function from labeled training data Where the training data is defined as xi yi where xi are instances or inputs and yi are the corresponding labels The goal of supervised machine learning is to accurately predict a label y for an unseen x Unsupervised Learning Unsupervised machine learning is a machine learning task of inferring a function to describe hidden structure from unlabeled data Where the training data is defined as xi where xi are instances or inputs However there are no labels associated with the training data The goal of unsupervised machine learning is to accurately model the underlying structure of the xi s Semi supervised Learning Semi supervised learning is a class of supervised learning tasks and techniques that make use of labeled and unlabeled data for training Reinforcement Learning Reinforcement learning is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward Could be considered probabilistic unsupervised Will be covered at the end of the semester time permitting Learning Given a set of n pairs xi yi xn yn where xi X Rd and yi 1 1 Lecture 1 Introduction to Machine Learning 3 Note that yi is binary either 1 or 1 Our goal is to learn a function h such that h xn 1 y n 1 will result in an accurate prediction Approach 1 Use a constant function h x c where c is a constant In this example we can define c to be 1 or 1 if we chose c to be 1 then regardless of our input x the classifier would always return 1 Pros Given new datasets will give consistent results Cons Does not adapt to the problem we are trying to solve Approach 2 Memoization As input points are seen they are stored in a database If the input point had a certain label we return the result of what has previously been seen y x if x has been seen before h x if x is new Pros Training costs nothing as no training is involved Can in some cases perform well Cons Not 100 accurate as we can have the same xi with different yi s No noise tolerance High storage cost High cost for performing a look up This cost can be reduced by using techniques such as hashing Test is expensive Cost depends on the performance of the lookup function Size of the model will scale with the size of the dataset Inductive Bias Assumptions that a machine learning model makes in order to make predictions on new data 4 Lecture 1 Introduction to Machine Learning Nearest Neighbor Classifier The Nearest Neighbor algorithm is a classification algorithm which uses the labels of it s k closest neighbors to determine the label for a new unseen instance x For values of k other than 1 the majority can be used to determine the label of x The Nearest Neighbor Classifier has the implicit assumption inductive bias that nearby points will have similar labels This assumption may not be accurate for all datasets 1 Nearest Neighbor For a single nearest neighbor the classifier h can be expressed as h x N1 X D where N1 X D represents the closest single point to X in a dataset D This can be expressed as follows N1 X D argmin d X Z z D where d is some distance function e g euclidean or mahalanobis k Nearest Neighbors For k nearest neighbors the classifier function h can be expressed as X 1 h X sign Yz k z Nk X D Where Nk X D set of k points closest to X in a dataset D Nearest Neighbor Performance Performance depends on hyperparameters e g the choice of distance function to use and the choice of the number of nearest neighbors Hyperparameters are the parameters that are set before the training process as opposed to parameters derived in training

View Full Document