Material Detail

Application of expectation consistent approximate inference

Application of expectation consistent approximate inference

This video was recorded at Workshop on Optimization and Inference in Machine Learning and Physics, Lavin 2005. I will discuss two types of applications of an approximate inference technique (EC = expectation consistent) recently developed together with Ole Winther. The EC method is an extension of the TAP (Thouless, Anderson & Palmer) approach which originated in the field of disordered materials and which has been further developed to become applicable to a variety of scenarios in probabilistic modelling & machine laerning. My first application (joint work with Doerthe Malzahn) deals with an approximation to resampling methods (such as the bootstrap) which allows to estimate eg generalization errors in supervised learning. While the exact resampling approach requires the drawing of many samples from the training data and a costly repeated retraining of the model, the approximation attempts an analytic average which combines the replica trick and an inference method which can be performed much faster. In the second application (ongoing work) I discuss the scenario of many solutions to the EC framework and the possibility of averaging them using Parisi's hierarchical scheme.

Quality

  • User Rating
  • Comments
  • Learning Exercises
  • Bookmark Collections
  • Course ePortfolios
  • Accessibility Info

More about this material

Comments

Log in to participate in the discussions or sign up if you are not already a MERLOT member.