never2old4school: Variance 3.0 (continued)

Saturday, October 22, 2016

Variance 3.0 (continued)

I was not completely joking at the end of yesterday's post. We were having company over in the evening and I was supposed to be helping Kate get things ready. I incurred a bit (but not a lot) of wrath for finishing as much as I did under those circumstances.

Anyway, let's take a look at that integral.

$\sigma^2 = \frac{pb}{a+b}+\int\frac{a(2\lambda^{-2}-2\mu/\lambda+\mu^2)\lambda^{h-1}e^{-s\lambda}}{(a+b)s^{-h}\Gamma(h)}\textrm{d}\lambda$

That's a polynomial times an exponential, so we should be able to grind it out by repeated integration by parts. Before we do that, though, let's back it up one step.

$\begin{align*} \sigma^2 &= \int \frac{pb + q(\lambda)a}{a+b}f(\lambda)\textrm{d}\lambda\\ &=\frac{pb+a\int q(\lambda) f(\lambda)\textrm{d}\lambda}{a+b} \end{align*}$

What's the integrand? It's a function of a random variable times the density of the variable. The integral is over the entire domain of the random variable. That makes the integral the expectation of the function. Let's see where that takes us:

$\begin{align*} E(q(\lambda))&=E(2\lambda^{-2}-2\mu/\lambda+\mu^2)\\ &=2E(1/\lambda^2)-2\mu E(1/\lambda)+\mu^2\\ &=2E(\beta^2)-2\mu E(\beta)+\mu^2\qquad\textrm{where}~\beta=1/\lambda \end{align*}$

Now, since λ ~ Gamma(h,1/s), β ~ InvGamma(h,s). So,

$\begin{align*} E(\beta)&=\frac{s}{(h-1)}\qquad h>1\\ \textrm{and}\\ E(\beta^2)&=\textrm{Var}(\beta)+E(\beta)^2\\ &=\frac{s^2}{(h-1)(h-2)}+\frac{s^2}{(h-1)^2}\qquad h>2\\ &=\frac{s^2(2h-3)}{(h-1)^2(h-2)} \end{align*}$

The constraints on h are not to be taken lightly. The variance of the inverse gamma is not defined until you have at least three observations (if you actually crank through the integration by parts from the original equation, you'll find the same limitation comes into play since you have to repeat the parts twice to get rid of the squared term in the polynomial). These will have to be "provided" in the prior. While that means we will not be able to use a non-informative prior, that's actually a good thing. We really want to bias the variance high until we have significant data to the contrary. Anyway, we now have everything we need to compete the variance. It's a bit messy, but manageable.

$\begin{align*} \sigma^2&=\frac{\mu^2b+E(q(\lambda))a}{a+b}\\ \textrm{where}\\ E(q(\lambda))&=\frac{2s^2(2h-3)}{(h-1)^2(h-2)}-\frac{2s\mu}{(h-1)}+\mu^2 \end{align*}$

never2old4school

Saturday, October 22, 2016

Variance 3.0 (continued)

No comments:

Post a Comment