16.4 Link Functions¶
The link function provides the relationship between the systematic component and the mean of the distribution. There are many commonly used link functions, the table below lists only three examples with their distributions and mean functions. Here we use matrix notation where \(\mu_{i} = \beta_{0} + \beta_{1}X_{i1} + \beta_{2}X_{i2} + … + \beta_{k}X_{ik}\) is represented by \(\mathbf{X}\mathbf{\beta}\).
Distribution |
Data |
Link Name |
Link function |
Mean function |
|---|---|---|---|---|
Normal |
real: (-\(\infty\) , + \(\infty\)) |
Identity \( \) |
\(\mathbf{X}\mathbf{\beta} =\mu\) |
\( \mu = \mathbf{X}\mathbf{\beta} \) \( \) |
Poisson |
integer: 0,1,2,… |
Log |
\(\mathbf{X}\mathbf{\beta} =ln( \mu)\) |
\( \mu = exp(\mathbf{X}\mathbf{\beta} )\) |
Binomial |
integer: 0,1,2,…N |
Logit |
\(\mathbf{X}\mathbf{\beta}=ln(\frac{\mu}{n-\mu})\) |
\( \mu =\frac{exp(\mathbf{X}\mathbf{\beta})}{1 + exp(\mathbf{X}\mathbf{\beta}} \) |
Gamma |
real: (0, + \(\infty\)) |
Negative Inverse |
\(\mathbf{X}\mathbf{\beta} = -\mu^{-1}\) |
\( \mu = - (\mathbf{X}\mathbf{\beta} )^{-1}\) |
It is important to note that both linear regression which is covered in sessions 12 to 14 and logistic regression in sessions 15 can be reproduced through a GLM.
Recall that a linear regression assumes data is normal distributed so using the identity link function for a normal distribution within the GLM framework will give the same estimated regression coefficients. However, the inference (p-values and confidence intervals) is slightly better using ordinary least squares compared to than maximum likelihood estimation thus we prefer to fit linear regression models using OLS.
In logistic regression if you use the logit function for a binomial family (recalling that Bernoulli is a special type of binomial distribution) you will be able to reproduce the same results as obtained through standard logistic regression modelling. For binary outcomes, the GLM has the extra flexibility compared to the logistic regression model. You can also use other link functions, for example the Probit, the Log-Log and the Complementary log-log functions. These will give similar results but adjust for slight differences from data collection situations to improve the transformation of the expectation of the outcome to the systematic component. In this module we only focus on the logit link, however if you wish to explore further, more information can be found here: https://aip.scitation.org/doi/pdf/10.1063/1.5139815