15.3 The logistic regression model

Throughout this session we will assume that the outcome Y is binary. Further, we assume that Y takes a value of 0 (“failure”) or 1 (“success”). As we discussed earlier, the terminology of success and failure does not imply success is a good thing; in health applications “success” often refers to a bad outcome such as death.

We will initially consider the simple situation with a single independent variable of interest, X. We assume that conditional on X, the outcome Y follows a Bernoulli distribution:

Y|X=xBernoulli(πx)

Then πx is the conditional probability of sucess, given X=x. It also represents the conditional expectation of the outcome, given X=x.

πx=E[Y|X=x]=P(Y=1|X=x)

Typically, our research question involves relating this probability to the covariate(s).

15.3.1 Components of the model

The logit function

As we have discussed, we do not wish to directly model π, because fitted values can lie outside the possible range of values. Instead, we will first transform π. In other words, we will model a function of π. We want a one-to-one function (so we can back-transform to the original scale, if we wish) that maps a probability π to the whole real line.

The function that we use in logistic regression is called the logit function. Specifically,

logit(π)=log(π1π)

The probability π lies in the interval [0,1] but the transformed value, logit(π) lies in the range (,).

It will also be useful to know how to back-transform. If logit(π)=L then

π=exp(L)1+exp(L)

This relationship will allow us to obtain fitted probabilities from our logistic regression model.

Odds

Suppose we have a binary outcome, where the probability of success is π, i.e. P(Y=1)=π. Then the odds of success are given by

π1π

Therefore, logit(π) is the logarithm of the odds, or the log-odds. We will see below that using the logit function leads to the parameters of the regression model being interpreted in terms of odds and odds ratios.

The linear predictor

Just as for linear regression models, the linear predictor is an additive function of the independent variables. With a single covariate, it is simply:

β0+β1X

15.3.2 The basic logistic regression model

The equation for a logistic regression model with, relating X to a binary outcome Y is:

logit(πx)=β0+β1X

Note that, unlike linear regression, there is no explicit error term in the logistic regression model.

Interpreting the parameters

Suppose that our single covariate X is binary, taking values 1 (exposed, say) and 0 (unexposed). Our model is then:

logit(πx)={β0when X=0 (unexposed group)β0+β1when X=1 (exposed group)

In other words, we have:

β0is the log-odds of the outcome in the unexposed groupβ0+β1is the log-odds of the outcome in the exposed group

Taking the exponential, we have

eβ0is the odds of the outcome in the unexposed groupeβ0+β1is the odds of the outcome in the exposed group

Now we have that eβ0+β1=eβ0×eβ1. Therefore, eβ1 also represents the multiplicative increase in the odds, going from the unexposed group to the exposed group. This multiplicative increase is known as the odds ratio. Therefore, we can also write:

eβ0is the odds of the outcome in the unexposed groupeβ1is the odds ratio of the outcome, comparing the exposed group to the unexposed group

General interpretation

This leads us to the following interpretation of the model:

logit(πx)=β0+β1X
  • The intercept, β0 is the log-odds among those with X=0. This is often called the baseline log-odds. Alternatively, the exponential eβ0 is the odds among those with X=0.

  • The slope, β1, is the difference in the log-odds associated with a one-unit increase in X. Equivalently, eβ1 is the odds ratio associated with a one-unit increase in X.