15.13 Further resources¶
Note: resources below are for you to deepen your understanding of the subject if you wish to do so. This is entirely optional. The extensions below (conditional logistic regression, multinomial regression, ordinal logistic regression and neural networks) are not examinable. These comments are provided for your interest.
The following book contains a much more detailed presentation of logistic regression. Chapter 7, 8 and 11 are particularly useful.
Agresti, A., Categorical Data Analysis, 3rd edition, 2012
The following also provide more detail on the subjects we have covered:
Hosmer D. W., Lemeshow S. and Rodney X. S. Applied logistic regression. Wiley, 2013.
Hosmer, D. W., Lemeshow, S. (1980) Goodness of fit tests for the multiple logistic regression model. Communications in Statistics – Theory and Methods
15.13.1 Conditional logistic regression¶
Conditional logistic regression is specifically designed for grouped data and in particular for pair-matched studies where each group is a pair of two matched subjects.
To analyse a pair-matched study, a naïve idea would be to include into a logistic regression model a parameter specific to each data stratum. However, for matched studies, it would mean having as many parameters as pairs. This raised two main issues. First, each of this stratum specific parameter would be estimated only from the information contained in a unique pair, i.e. very little information. Secondly, as the number of pairs increases, the number of parameters would also increase and maximum likelihood theory would fail to provide valid estimation.
The idea of conditional logistic regression is to remove the dependence upon the stratum specific parameters by conditioning the probability on sufficient exposure information. This way, because the stratum specific parameters vanish from the equation, there is no violation of the assumptions underlying maximum likelihood theory and the other model parameters can be estimated consistently using classic techniques.
We note that in some cases (e.g. where the matching was on measurable characteristics, such as age and sex only), standard logistic regression adjusting for the matching variables is valid.
15.13.2 Multinomial logistic regression¶
The multinomial logistic regression model is a generalization of the logistic regression model for outcomes that have more than \(2\) categories, \(Y\in\{1,\dots,J\}\) with \(J\geq 2\) a natural number. In this case, the conditional distribution of \(Y_i\) given the covariates \(X_i\) is the multinomial distribution. Among the \(J\) categories, a reference one is chosen, e.g. the first category, and for \(j=2,\dots,J\)
The model is estimated simultaneously for all values of \(j\). For a fixed \(j\), the interpretation of the parameters is similar to the logistic regression model.
15.13.3 Ordinal logistic regression¶
The ordinal logistic regression is designed for outcomes that have more than \(2\) categories, \(Y\in\{1,\dots,J\}\) with \(J\geq 2\), and whose categories have an explicit ordering. In the ordinal logistic regression, the modelled quantity is \(\mathrm{logit}(P(Y_i\geq j|X_i))\) for \(j\geq 2\). Indeed, when \(j=1\), t \(P(Y\geq 1)=1\) and does not need to be modelled. As the categories are ordered, a fundamental assumption made by ordinal logistic regression is that the effect of the covariates are homogenous between the different categories. Therefore, the model is written
where only the intercept term depends upon the category. However, it is important to carefully check for violations of this assumption.
15.13.4 Neural networks¶
Artificial neural networks are a class of model widely used for data classification in machine learning. Artificial neural networks are at the heart of deep learning methods used to develop computer vision, speech recognition, audio recognition, etc. Actually, the basic logistic regression model happens to be a special case of artificial neural network. If you are interested in this subject and want to have more insight on the relation between these two models, you might want to read the following article:
Dreiseitl, S., & Ohno-Machado, L. (2002). Logistic regression and artificial neural network classification models: a methodology review. Journal of Biomedical Informatics