This part is invariant to the type of model and is the combination of explanatory variables to predict the expected value \(\mu\) (i.e. the mean) of the distribution.
Link Function
The link function\(g(\mu)\) is an invertible function that connects the mean \(\mu\) of the random component with the linear combination of predictors.
\(g(\mu) = \beta_0 + \beta_1x_{1i} + ... + \beta_px_{pi}\). The inverse of the link function \(g^{-1}\) map the linear predictor (\(\eta\)) into the original scale.
Thus, the relationship between \(\mu\) and \(\eta\) is linear only when the link function is applied i.e. \(g(\mu) = \eta\).
Link function
The simplest link function is the identity link where \(g(\mu) = \mu\) and correspond to the standard linear model. In fact, the linear regression is just a GLM with a Gaussian random component and the identity link function.
Main distributions and link functions
Family
Link
Range
gaussian
identity
\[(-\infty,+\infty)\]
gamma
log
\[(0,+\infty)\]
binomial
logit
\[\frac{0, 1, ..., n_{i}}{n_{i}}\]
binomial
probit
\[\frac{0, 1, ..., n_{i}}{n_{i}}\]
poisson
log
\[0, 1, 2, ...\]
Relevant distributions
Bernoulli distribution
A single Bernoulli trial is defined as:
\[
f(x, p) = p^x (1 - p)^{1 - x}
\]
Where \(p\) is the probability of success and \(k\) the two possible results \(0\) and \(1\). The mean is \(p\) and the variance is \(p(1 - p)\)
Binomial distribution
The probability of having \(k\) success (e.g., 0, 1, 2, etc.), out of \(n\) trials with a probability of success \(p\) (and failing \(q = 1 - p\)) is:
\[
f(n, k, p)= \binom{n}{k} p^k(1 - p)^{n - k}
\]
The \(np\) is the mean of the binomial distribution and \(np(1 - p)\) is the variance. The binomial distribution is just the repetition of \(n\) independent Bernoulli trials.
Bernoulli and Binomial
A classical example for a Bernoulli trial is the coin flip. In R:
n<-1p<-0.7rbinom(1, n, p)# a single bernoulli trial
# generate k (success)n<-50# number of trialsp<-0.7# probability of successrbinom(1, n, p)#> [1] 31# let's do several experiments (e.g., participants)rbinom(10, n, p)#> [1] 32 28 30 38 29 36 31 36 31 39# calculate the probability density given k successesn<-50k<-25p<-0.5dbinom(k, n, p)#> [1] 0.1122752# calculate the probability of doing 0, 1, 2, up to k successesn<-50k<-25p<-0.5pbinom(k, n, p)#> [1] 0.5561376
Binomial GLM
The Bernoulli distributions is used as random component when we have a binary dependent variable or the number of successes over the total number of trials:
Accuracy on a cognitive task
Patients recovered or not after a treatment
People passing or not the exam
The Bernoulli or the Binomial distributions can be used as random component when we have a binary dependent variable or the number of successes over the total number of trials.
When fitting a GLM with the binomial distribution we are including linear predictors on the expected value \(\mu\) i.e. the probability of success.
Binomial GLM
Most of the GLM models deal with a mean-variance relationship:
Poisson distribution
The number of events \(k\) during a fixed time interval (e.g., number of new users on a website in 1 week) is:
Where \(k\) is the number of occurrences (\(k = 0, 1, 2, ...\)), \(e\) is Euler’s number (\(e = 2.71828...\)) and \(!\) is the factorial function. The mean and the variance of the Poisson distribution is \(\lambda\).
Poisson distribution
As \(\lambda\) increases, the distribution is well approximated by a Gaussian distribution, but the Poisson is discrete.
The mean-variance relationship can be easily seen with a continuous predictor:
Gamma distribution
The Gamma distribution has several :parametrizations. One of the most common is the shape-scale parametrization:
\[
f(x;k,\theta )={\frac {x^{k-1}e^{-x/\theta }}{\theta ^{k}\Gamma (k)}}
\] Where \(\theta\) is the scale parameter and \(k\) is the shape parameter.
Gamma distribution
ggamma(mean =c(10, 20, 30), sd =c(10, 10, 10), show ="ss")
Gamma \(\mu\) and \(\sigma^2\)
The mean and variance are defined as:
\(\mu = k \theta\) and \(\sigma^2 = k \theta^2\) with the shape-scale parametrization
\(\mu = \frac{\alpha}{\beta}\) and \(\frac{\alpha}{\beta^2}\) with the shape-rate parametrization
Another important quantity is the coefficient of variation defined as \(\frac{\sigma}{\mu}\) or \(\frac{1}{\sqrt{k}}\) (or \(\frac{1}{\sqrt{\alpha}}\)).
Gamma distribution
Again, we can see the mean-variance relationship:
ggamma(shape =c(5, 5), scale =c(10, 20), show ="ss")
Gamma parametrization
To convert between different parametrizations, you can use the gamma_params() function:
gamma_params<-function(shape=NULL, scale=1/rate, rate=1,mean=NULL, sd=NULL,eqs=FALSE){if(eqs){cat(rep("=", 25), "\n")cat(eqs()$gamma, "\n")cat(rep("=", 25), "\n")}else{if(is.null(shape)){var<-sd^2shape<-mean^2/varscale<-mean/shaperate<-1/scale}elseif(is.null(mean)&is.null(sd)){if(is.null(rate)){scale<-1/rate}else{rate<-1/scale}mean<-shape*scalevar<-shape*scale^2sd<-sqrt(var)}else{stop("when shape and scale are provided, mean and sd need to be NULL (and viceversa)")}out<-list(shape =shape, scale =scale, rate =rate, mean =mean, var =var, sd =sd)# coefficient of variationout$cv<-1/sqrt(shape)return(out)}}
Is not always easy to work with link functions. In R we can use the distribution(link = ) function to have several useful information. For example:
fam<-binomial(link ="logit")fam$linkfun()# link function, from probability to etafam$linkinv()# inverse of the link function, from eta to probability
We are going to see the specific arguments, but this tricks works for any family and links, even if you do not remember the specific function or formula.
Faraway, J. J. (2016). Extending the linear model with r: Generalized linear, mixed effects and nonparametric regression models, second edition. Chapman; Hall/CRC. https://doi.org/10.1201/9781315382722
Fox, J. (2015). Applied regression analysis and generalized linear models. SAGE Publications.
