Slide 1Slide 3A toy exampleSlide 5Slide 6Directed graphical modelsSlide 8Undirected graphical modelsUndirected graphical models: cliquesUndirected graphical models: probability factorizationMarkov Random FieldsMRF nodes as pixelsMRF nodes as patchesNetwork joint probabilityIn order to use MRFs:Outline of MRF sectionOutline of MRF sectionDerivation of belief propagationThe posterior factorizesPropagation rulesPropagation rulesPropagation rulesBelief propagation: the nosey neighbor ruleBelief propagation messagesBeliefsBelief, and message updatesOptimal solution in a chain or tree: Belief PropagationNo factorization with loops!Slide 30Results from Bethe free energy analysisReferences on BP and GBPOutline of MRF sectionStereo problemMRF for stereoSlide 36Graph cutsSlide 38Two moves: swap and expansionThe cost of an alpha-beta assignmentFlash of inspiration here…Slide 42Slide 43Slide 44Slide 45Minimum s-t cuts algorithms“Augmenting Paths”“Augmenting Paths”“Augmenting Paths”Slide 50ICCV 2001Slide 52Slide 53Slide 54To Boykov slidesSlide 56OutlineShortest paths on graphs (examples)Shortest paths: Texture synthesis2D Graph cut shortest path on a graphDP / Shortest-pathsStereo exampleSlide 63Examples of Graph Cuts in visions-t graph cuts for video texturesPixel interactions: “convex” vs. “discontinuity-preserving”Pixel interactions: “convex” vs. “discontinuity-preserving”a-expansion movea-expansion movesSlide 70a-expansion move vs. “standard” movesSlide 72Slide 73end of Boykov slidesMiddlebury stereo web pageSlide 76Algorithm comparisonsSlide 78Comparison of graph cuts and belief propagationSlide 80Graph cuts versus belief propagationMAP versus MMSEOutline of MRF sectionVision applications of MRF’sSuper-resolutionSlide 863 approaches to perceptual sharpeningSuper-resolution: other approachesTraining images, ~100,000 image/scene patch pairsSlide 90Slide 91Slide 92Slide 93Slide 94Slide 95Slide 96Example: input image patch, and closest matches from databaseSlide 98Scene-scene compatibility function, Y(xi, xj)Image-scene compatibility function, F(xi, yi)Markov networkSlide 102Zooming 2 octavesSlide 104Slide 105Slide 106Slide 107Slide 108Slide 109Slide 110Slide 111Generic training imagesSlide 113Kodak Imaging Science Technology Lab test.Algorithms comparedSlide 116Slide 117User preference test resultsSlide 119Slide 120Slide 121Training imageProcessed imageVision applications of MRF’sMotion applicationWhat behavior should we see in a motion algorithm?The aperture problemSlide 128Motion analysis: related workSlide 130Slide 131Slide 132Vision applications of MRF’sForming an ImagePainting the SurfaceGoalBasic StepsLearning the ClassifiersSlide 139Some Areas of the Image Are Locally AmbiguousPropagating InformationPropagating InformationSetting CompatibilitiesImprovements Using PropagationSlide 145(More Results)Slide 147Slide 148Outline of MRF sectionLearning MRF parameters, labeled dataSlide 151Initial guess at joint probabilityIPF update equationConvergence of to correct marginals by IPF algorithmSlide 155IPF results for this example: comparison of joint probabilitiesApplication to MRF parameter estimationLearning MRF parameters, labeled dataSlide 159C280, Computer VisionProf. Trevor [email protected] 20: Markov Random Fields3Making probability distributions modular, and therefore tractable:Probabilistic graphical modelsVision is a problem involving the interactions of many variables: things can seem hopelessly complex. Everything is made tractable, or at least, simpler, if we modularize the problem. That’s what probabilistic graphical models do, and let’s examine that.Readings: Jordan and Weiss intro article—fantastic! Kevin Murphy web page—comprehensive and with pointers to many advanced topics4A toy exampleSuppose we have a system of 5 interacting variables, perhaps some are observed and some are not. There’s some probabilistic relationship between the 5 variables, described by their joint probability,P(x1, x2, x3, x4, x5).If we want to find out what the likely state of variable x1 is (say, the position of the hand of some person we are observing), what can we do?Two reasonable choices are: (a) find the value of x1 (and of all the other variables) that gives the maximum of P(x1, x2, x3, x4, x5); that’s the MAP solution.Or (b) marginalize over all the other variables and then take the mean or the maximum of the other variables. Marginalizing, then taking the mean, is equivalent to finding the MMSE solution. Marginalizing, then taking the max, is called the max marginal solution and sometimes a useful thing to do.5To find the marginal probability at x1, we have to take this sum:),,,,(5432,,,54321xxxxxxxxxPIf the system really is high dimensional, that will quickly become intractable. But if there is some modularity inthen things become tractable again.),,,,(54321xxxxxPSuppose the variables form a Markov chain: x1 causes x2 which causes x3, etc. We might draw out this relationship as follows:1x2x3x4x5x6)|,,,()(),,,,(15432154321xxxxxPxPxxxxxP By the chain rule, for any probability distribution, we have:Now our marginalization summations distribute through those terms:),|,,()|()(21543121xxxxxPxxPxP),,|,(),|()|()(32154213121xxxxxPxxxPxxPxP),,,|(),,|(),|()|()(432153214213121xxxxxPxxxxPxxxPxxPxP)|()|()|()|()(453423121xxPxxPxxPxxPxP12 3 4 55432)|()|()|()|()(),,,,(453423121,,,54321xx x x xxxxxxxPxxPxxPxxPxPxxxxxPP(a,b) = P(b|a) P(a)But if we exploit the assumed modularity of the probability distribution over the 5 variables (in this case, the assumed Markov chain structure), then that expression simplifies:1x2x3x4x5xDirected graphical models•A directed, acyclic graph.•Nodes are random variables. Can be scalars or vectors, continuous or discrete.•The direction of the edge tells the parent-child-relation:•With every node i is associated a conditional pdf defined by all the parent nodes of node i. That conditional probability is•The joint distribution depicted by the graph is the product of all those conditional probabilities:parent childiPxi| xiPx1... xn Pxi| xii1n81x2x3x4x5xAnother modular probabilistic structure, more common in vision problems, is an undirected graph:The joint probability for this graph is given by:),(),(),(),(),,,,(5443322154321xxxxxxxxxxxxxP Where is called a