Exponential families

Back to Q stuff.

An exponential family of distributions is a parameterized family with pdf or pmf f(x|theta) such that



where h(x) ≥ 0 and t_i(x) are real-valued and do not depend on θ. Also, c(θ) > 0 and w_i(θ) are real-valued and do not depend on x. The parameter θ may be a scalar or vector.

Prominent continuous distributions in the exponential family include normal, gamma (including, obviously, exponential), and beta. Discrete examples are binomial, Poisson, and negative binomial.

Taxonomy isn't a big priority with mathematicians. The only reason to group things that conform to a model is that one can prove things about the model and then apply them to the elements of the model. Two such results:





Yeah, right, like I'm ever going to use those. Actually, they look worse than they are. Once you actually fill in the formulas, the partial derivatives are typically pretty easy and they allow you to compute the first two moments without summations or integrals. Higher moments follow a similar pattern.

A subset of the exponential families are the curved exponential distributions. These are exponential families where the parameter space theta is restricted. For example, the set of normal distributions having equal mean and variance, N(μ, μ). These are of interest because inferences about one of the parameters automatically tell you something about another, otherwise independent, parameter.