# CALTECH CS 155 - Probabilistic Graphical Models (43 pages)

Previewing pages 1, 2, 3, 20, 21, 22, 41, 42, 43 of 43 page document
View Full Document

## Probabilistic Graphical Models

Previewing pages 1, 2, 3, 20, 21, 22, 41, 42, 43 of actual document.

View Full Document
View Full Document

## Probabilistic Graphical Models

48 views

Lecture Notes

Pages:
43
School:
California Institute of Technology
Course:
Cs 155 - Probabilistic Graphical Models
##### Probabilistic Graphical Models Documents
• 5 pages

• 35 pages

• 40 pages

• 31 pages

• 54 pages

• 3 pages

• 32 pages

Unformatted text preview:

Probabilistic Graphical Models Lecture 12 Dynamical Models CS CNS EE 155 Andreas Krause Announcements Homework 3 out tonight Start early Project milestones due today Please email to TAs 2 Parameter learning for log linear models Feature functions i Ci defined over cliques Log linear model over undirected graph G Feature functions 1 C1 k Ck Domains Ci can overlap Joint distribution How do we get weights wi 3 Log linear conditional random field Define log linear model over outputs Y No assumptions about inputs X Feature functions i Ci x defined over cliques and inputs Joint distribution 4 Example CRFs in NLP Y1 Y2 Y3 Y4 Y5 Y6 Y7 Y8 Y9 Y10 Y11 Y12 X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 Mrs Greene spoke today in New York Green chairs the finance committee Classify into Person Location or Other 5 Example CRFs in vision 6 Gradient of conditional log likelihood Partial derivative Requires one inference per Can optimize using conjugate gradient 7 Exponential Family Distributions Distributions for log linear models More generally Exponential family distributions h x Base measure w natural parameters x Sufficient statistics A w log partition function Hereby x can be continuous defined over any set 8 Examples Exp Family Gaussian distribution h x Base measure w natural parameters x Sufficient statistics A w log partition function Other examples Multinomial Poisson Exponential Gamma Weibull chi square Dirichlet Geometric 9 Moments and gradients Correspondence between moments and log partition function just like in log linear models Can compute moments from derivatives and derivatives from moments MLE moment matching 10 Conjugate priors in Exponential Family Any exponential family likelihood has a conjugate prior 11 Exponential family graphical models So far only defined graphical models over discrete variables Can define GMs over continuous distributions For exponential family distributions Can do much of what we discussed VE JT parameter learning etc for such exponential family

View Full Document

Unlocking...