Short Course at Automatic Control, LiTH, 2008-03-10 -- 2008-03-13.
Speaker: Dr Anthony Quinn, Department of Electronic and Electrical Engineering, University of Dublin, Trinity College, Ireland
Abstract: The vast majority of tasks we seek to address in statistical signal processing are inductive inference tasks. In other words, we are seeking knowledge in conditions of uncertainty. Probability quantifies degrees of belief amid uncertainty, and the calculus of probability is available to us as a consistent framework for manipulating degrees of belief in order to arrive at answers to problems of interest. This is the Bayesian paradigm which I will advocate in this course. It presents some challenges but more rewards, and the aim of this course is to examine these challenges and rewards critically in the context of signal processing.
In terms of challenges, perhaps the greatest is to overcome the frequentist mindset that still dominates the signal processing field. Thereafter, we must elicit probability functions for all unknowns, most notoriously expressed in the need for priors. Finally, we must develop tractable procedures for computing and manipulating probability functions. A main aim of the course will be to present the Variational Bayes method for approximating distributions, and to examine its contrasts and cooperations with stochastic approximations, which dominate Bayesian signal processing at present.
The reward of such effort is, first-and-foremost, the fact that the Bayesian approach is a principled and prescriptive pathway to solving signal processing problems properly. If non-Bayesian solutions are consistent, they can always be characterized as special cases of Bayesian solutions. The unique armoury of the Bayesian includes, of course, the prior, which can be used to regularize an inference, and to exploit external information. This is well known. Less well known, but perhaps more powerful, is the availability of the marginalization operator, conferred uniquely because of the measure nature of probability functions. Among the compelling advantages of marginalization are the automatic embrace of Ockham's Razor, and the ability to compare set hypotheses, including competing model structures.
All these ideas will be explored in this course, and illustrated via important representative problems, such as sinusoidal identification, principal component analysis and nonlinear filtering. Radiotherapy, functional medical imaging and speech processing will be among the applications to be considered.
The main sections of the course will be: