Link functions

Sometimes I forget just how new statistics is. Informal measurement and inference predate history. Measure theory, which laid the mathematical foundations for random variables and distributions came about in response to the formalization of Calculus in the 17th century. But, actual mathematically-based statistics? That really didn't get going until the work of Fisher. He died the year before I was born. On the timeline of human knowledge, this is really new stuff.

I mention that because I attended a colloquium today on Dimension Reduction by Dr. Wen of MS&T (the engineering school formerly known as Rolla) and she threw out the term "Link Function". It wasn't the first time I'd heard it; I knew it had something to do with tying the mean to the predictors in a Generalized Linear Model (GLM), but I decided that I'd better file it under the heading of "Named Results" and actually learn what it was.

Turns out, there's a very good reason I didn't know what it was: the term didn't get widespread use until after I got my MS. I checked my text from Cornell (the rightly acclaimed Mathematical Statistics by Bickel and Docksum) and they mention the function in the context of the General Linear Model, but it's just given the anonymous designation g. The "generalization" of the linear model into the Generalized Linear Model was published in 1972, but it didn't really catch on until after B&D's publication in 1977, when computing power became cheap enough that anybody could afford to run their data through a package like GLIM. The function is still generally written as g, but it has also picked up the moniker of the Link Function.

So, what is the link function? It's actually rather well named - it provides a "link" from the mean to the linear predictor. So, if μ_i is the mean of our observed dependent variable, the link function g(μ_i) then maps that to the real number line and calls that the linear predictor. The linear predictor is modeled as a linear combination of the independent variables plus an error term which follows a normal distribution. It's sort of like a transformation, but not quite. With a transformation, you transform both the predictor and the error term. Here, the link function is only transforming the predictor.