9. Bayesian Statistics I

So far in this module, we have looked at frequentist or classical statistical ideas, such as maximum likelihood estimation, hypothesis testing and p-values. Underlying the frequentist approach is the belief that there is a true state of reality, and that parameters have a fixed and true value. Probabilities are long-run frequencies; for example, the probability of a driver in London having a car accident is a fixed value between 0 and 1. A typical way of estimating this value is to take a sample of observations, construct a likelihood function for these observations, and to obtain the parameter value that maximizes the likelihood. When we take a Bayesian approach, the parameter we wish to estimate is considered to be a random variable, and probabilities may represent a subjective belief about the state of uncertainty, or there may be a data generating distribution underlying the random parameter. For example, you may have a prior belief about the probability of a driver in London having a car accident, and after collecting a sample of data, you combine your prior beliefs with the likelihood for those observations to construct an updated belief - the posterior. Your belief may change in light of the data.

Bayesian methods sometimes require numerical integration, and cheaper computing has made Bayesian approaches more feasible in the last 20 years. Bayesian approaches are likely to be an important part of working in Health Data Science. In the next two sessions, we introduce the fundamental principles.

The current session introduces the basic concepts underlying Bayesian inference and then applies the basic principles to a simple example using proportions.

Intended learning outcomes

By the end of this session you will be able to:

  • compare the notions of probability and likelihood in Bayesian and Frequentist paradigms

  • explain the notions of prior and posterior distributions

  • apply Bayes Theorem in the discrete case

  • Understand and apply the basic principles of Bayesian analysis using proportions, specifically:

    • use the beta distribution as a prior and derive the posterior distribution

    • obtain credible HPD intervals for the parameter

    • obtain prior and posterior predictive distributions

    • explain the concept of conjugate priors

Further resources 9.1 Introduction to Bayesian Inference