View Full Document

Gibbs Sampling for the Uninitiated



View the full content.
View Full Document
View Full Document

8 views

Unformatted text preview:

Gibbs Sampling for the Uninitiated Eric Hardisty Department of Computer Science and Institute for Advanced Computer Studies University of Maryland College Park MD 20742 USA hardisty cs umd edu Philip Resnik Department of Linguistics and Institute for Advanced Computer Studies University of Maryland College Park MD 20742 USA resnik umd edu October 20 2009 VERSION 0 3 Abstract This document is intended for computer scientists who would like to try out a Markov Chain Monte Carlo MCMC technique particularly in order to do inference with Bayesian models on problems related to text processing We try to keep theory to the absolute minimum needed and we work through the details much more explicitly than you usually see even in introductory explanations That means we ve attempted to be ridiculously explicit in our exposition and notation After providing the reasons and reasoning behind Gibbs sampling and at least nodding our heads in the direction of theory we work through two applications in detail The first is the derivation of a Gibbs sampler for Naive Bayes models which illustrates a simple case where the math works out very cleanly and it s possible to integrate out the model s continuous parameters to build a more efficient algorithm The second application derives the Gibbs sampler for a model that is similar to Naive Bayes but which adds an additional latent variable Having gone through the two examples we discuss some practical implementation issues We conclude with some pointers to literature that we ve found to be somewhat more friendly to uninitiated readers 1 Introduction Markov Chain Monte Carlo MCMC techniques like Gibbs sampling provide a principled way to approximate the value of an integral 1 1 Why integrals Ok stop right there Many computer scientists including a lot of us who focus in natural language processing don t spend a lot of time with integrals We spend most of our time and energy in a world of discrete events The word bank can mean 1 a financial



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view Gibbs Sampling for the Uninitiated and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view Gibbs Sampling for the Uninitiated and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?