View Full Document

PERFORMANCE EVALUATION OF LATENT VARIABLE MODELS WITH SPARSE PRIORS



View the full content.
View Full Document
View Full Document

7 views

Unformatted text preview:

PERFORMANCE EVALUATION OF LATENT VARIABLE MODELS WITH SPARSE PRIORS David Wipf Jason Palmer Bhaskar Rao and Kenneth Kreutz Delgado Department of Electrical and Computer Engineering University of California San Diego La Jolla CA 92093 0407 USA e mail dwipf japalmer ucsd edu brao kreutz ece ucsd edu ABSTRACT A variety of Bayesian methods have recently been introduced for finding sparse representations from overcomplete dictionaries of candidate features These methods often capitalize on latent structure inherent in sparse distributions to perform standard MAP estimation variational Bayes approximation using convex duality or evidence maximization Despite their reliance on sparsity inducing priors however these approaches may or may not actually lead to sparse representations in practice and so it is a challenging task to determine which algorithm and sparse prior is appropriate Rather than justifying prior selections and modelling assumptions based on the credibility of the full Bayesian model as is commonly done this paper bases evaluations on the actual cost functions that emerge from each method Two minimal conditions are postulated that ideally any sparse learning objective should satisfy Out of all possible cost functions that can be obtained from the methods described above using virtually any sparse prior a unique function is derived that satisfies these conditions Both sparse Bayesian learning SBL and basis pursuit BP are special cases Later all methods are shown to be performing MAP estimation using potentially non factorable implicit priors which suggests new sparse learning cost functions Index Terms sparse representations sparse priors latent variable models underdetermined inverse problems Bayesian learning 1 INTRODUCTION Here we will be concerned with the generative model y X 6 1 where 1 C RNNM is a dictionary of unit 2 norm basis vectors or features x is a vector of unknown weights y is the observation vector and e is uncorrelated noise distributed as



Access the best Study Guides, Lecture Notes and Practice Exams

Loading Unlocking...
Login

Join to view PERFORMANCE EVALUATION OF LATENT VARIABLE MODELS WITH SPARSE PRIORS and access 3M+ class-specific study document.

or
We will never post anything without your permission.
Don't have an account?
Sign Up

Join to view PERFORMANCE EVALUATION OF LATENT VARIABLE MODELS WITH SPARSE PRIORS and access 3M+ class-specific study document.

or

By creating an account you agree to our Privacy Policy and Terms Of Use

Already a member?