DOC PREVIEW
CMU CS 10701 - pca-mdps

This preview shows page 1-2-3-4-26-27-28-53-54-55-56 out of 56 pages.

Save
View full document
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
View full document
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience
Premium Document
Do you want full access? Go Premium and unlock all 56 pages.
Access to all documents
Download any document
Ad free experience

Unformatted text preview:

1Dimensionality reduction (cont.)Machine Learning – 10701/15781Carlos GuestrinCarnegie Mellon UniversityApril 26th, 2006Recommended reading:Bishop, Chapters 3.6, 8.6Shlens PCA tutorialWall et al. 2003 (PCA applied to gene expression data)2Lower dimensional projections Rather than picking a subset of the features, we can new features that are combinations of existing features Let’s see this in the unsupervised setting  just X, but no Y3Liner projection and reconstructionx1x2project into1-dimensionz1reconstruction:only know z1, what was (x1,x2)4Principal component analysis –basic idea Project n-dimensional data into k-dimensional space while preserving information: e.g., project space of 10000 words into 3-dimensions e.g., project 3-d into 2-d Choose projection with minimum reconstruction error5Linear projections, a review Project a point into a (lower dimensional) space: point: x = (x1,…,xn)  select a basis – set of basis vectors – (u1,…,uk) we consider orthonormal basis:  ui····ui=1, and ui····uj=0 for i≠j select a center – x, defines offset of space  best coordinates in lower dimensional space defined by dot-products: (z1,…,zk), zi= (x-x)····ui minimum squared error6PCA finds projection that minimizes reconstruction error Given m data points: xi= (x1i,…,xni), i=1…m Will represent each point as a projection: where: and  PCA: Given kn, find (u1,…,uk) minimizing reconstruction error:x1x27Minimizing reconstruction error and eigen vectors Minimizing reconstruction error equivalent to picking orthonormal basis (u1,…,un) minimizing: Eigen vector: Minimizing reconstruction error equivalent to picking (uk+1,…,un) to be eigen vectors with smallest eigen values8Basic PCA algoritm Start from m by n data matrix X Recenter: subtract mean from each row of X Xc←←←← X – X Compute covariance matrix: Σ ← XcTXc Find eigen vectors and values of Σ Principal components: k eigen vectors with highest eigen values New features are linear combination of old features9PCA example10PCA example – reconstruction only used first principal component11Eigenfaces [Turk, Pentland ’91] Input images:  Principal components:12Eigenfaces reconstruction Each image corresponds to adding 8 principal components:13Relationship to Gaussians PCA assumes data is Gaussian x ~ N(x;Σ) Equivalent to weighted sum of simple Gaussians: Selecting top k principal components equivalent to lower dimensional Gaussian approximation: ε~N(0;σ2), where σ2is defined by errorkx1x214Scaling up Covariance matrix can be really big! Σ is n by n 10000 features → |Σ| finding eigenvectors is very slow… Use singular value decomposition (SVD) finds to k eigenvectors great implementations available, e.g., Matlab svd15SVD Write X = U S VT X ← data matrix, one row per datapoint U ← weight matrix, one row per datapoint – coordinate of xiin eigenspace S ← singular value matrix, diagonal matrix in our setting each entry is eigenvalue λj VT← singular vector matrix in our setting each row is eigenvector vj16PCA using SVD algoritm Start from m by n data matrix X Recenter: subtract mean from each row of X Xc←←←← X – X Call SVD algorithm on Xc– ask for k singular vectors Principal components: k singular vectors with highest singular values (rows of VT) Coefficients become:17Using PCA for dimensionality reduction in classification Want to learn f:XaaaaY X=<X1,…,Xn> but some features are more important than others Approach: Use PCA on X to select a few important features18PCA for classification can lead to problems… Direction of maximum variation may be unrelated to “discriminative” directions: PCA often works very well, but sometimes must use more advanced methods e.g., Fisher linear discriminant19What you need to know Dimensionality reduction why and when it’s important Simple feature selection Principal component analysis minimizing reconstruction error relationship to covariance matrix and eigenvectors using SVD problems with PCA20Markov DecisionProcesses (MDPs)Machine Learning – 10701/15781Carlos GuestrinCarnegie Mellon UniversityApril 26th, 2006Reading:Kaelbling et al. 1996 (see class website)21Announcements Project: Poster session: Friday May 5th2-5pm, NSH Atrium  please arrive a little early to set up Paper: Monday May 8thby noon to Monica Hopes –Wean Hall 4616  maximum of 8 pages, NIPS format FCEs!!!! Please, please, please, please, please, please give us your feedback, it helps us improve the class! ☺ http://www.cmu.edu/fce22Thus far this semester Regression: Classification: Density estimation:23Learning to act Reinforcement learning An agent  Makes sensor observations Must select action Receives rewards  positive for “good”states negative for “bad”states[Ng et al. ’05]24Learning to play backgammon [Tesauro ’95] Combines reinforcement learning with neural networks Played 300,000 games against itself Achieved grandmaster level!25Roadmap to learning about reinforcement learning When we learned about Bayes nets: First talked about formal framework: representation  inference Then learning for BNs For reinforcement learning: Formal framework Markov decision processes Then learning26peasantfootmanbuildingReal-time Strategy GamePeasants collect resources and buildFootmen attack enemiesBuildings train peasants and footmen27States and actions State space:  Joint state x of entire system Action space:  Joint action a= {a1,…, an} for all agents28States change over time Like an HMM, state changes over time Next state depends on current state and action selected e.g., action=“build castle” likely to lead to a state where you have a castle Transition model:  Dynamics of the entire system P(x’|x,a)29Some states and actions are better than others Each state x is associated with a reward positive reward for successful attack negative for loss Reward function:  Total reward R(x)30Discounted RewardsAn assistant professor gets paid, say, 20K per year.How much, in total, will the A.P. earn in their life?20 + 20 + 20 + 20 + 20 + … = InfinityWhat’s wrong with this


View Full Document

CMU CS 10701 - pca-mdps

Documents in this Course
lecture

lecture

12 pages

lecture

lecture

17 pages

HMMs

HMMs

40 pages

lecture

lecture

15 pages

lecture

lecture

20 pages

Notes

Notes

10 pages

Notes

Notes

15 pages

Lecture

Lecture

22 pages

Lecture

Lecture

13 pages

Lecture

Lecture

24 pages

Lecture9

Lecture9

38 pages

lecture

lecture

26 pages

lecture

lecture

13 pages

Lecture

Lecture

5 pages

lecture

lecture

18 pages

lecture

lecture

22 pages

Boosting

Boosting

11 pages

lecture

lecture

16 pages

lecture

lecture

20 pages

Lecture

Lecture

20 pages

Lecture

Lecture

39 pages

Lecture

Lecture

14 pages

Lecture

Lecture

18 pages

Lecture

Lecture

13 pages

Exam

Exam

10 pages

Lecture

Lecture

27 pages

Lecture

Lecture

15 pages

Lecture

Lecture

24 pages

Lecture

Lecture

16 pages

Lecture

Lecture

23 pages

Lecture6

Lecture6

28 pages

Notes

Notes

34 pages

lecture

lecture

15 pages

Midterm

Midterm

11 pages

lecture

lecture

11 pages

lecture

lecture

23 pages

Boosting

Boosting

35 pages

Lecture

Lecture

49 pages

Lecture

Lecture

22 pages

Lecture

Lecture

16 pages

Lecture

Lecture

18 pages

Lecture

Lecture

35 pages

lecture

lecture

22 pages

lecture

lecture

24 pages

Midterm

Midterm

17 pages

exam

exam

15 pages

Lecture12

Lecture12

32 pages

lecture

lecture

19 pages

Lecture

Lecture

32 pages

boosting

boosting

11 pages

bns

bns

45 pages

mdps

mdps

42 pages

svms

svms

10 pages

Notes

Notes

12 pages

lecture

lecture

42 pages

lecture

lecture

29 pages

lecture

lecture

15 pages

Lecture

Lecture

12 pages

Lecture

Lecture

24 pages

Lecture

Lecture

22 pages

Midterm

Midterm

5 pages

mdps-rl

mdps-rl

26 pages

Load more
Download pca-mdps
Our administrator received your request to download this document. We will send you the file to your email shortly.
Loading Unlocking...
Login

Join to view pca-mdps and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view pca-mdps 2 2 and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?