So, I got a little myopic in my treatment of what we're calling the "uniform" case of the sampling (this is the case where the relevant rows of the query are scattered about the data with no discernible correlation). In those cases, simple block sampling, BLB, and BMH are all comparable in terms of results, so you'd probably just go with the easy one. It's also fairly easy to show that BLB and BMH are substantially worse than simple block sampling once correlation is introduced. That's not really a surprise as both methods state up front that they are designed for iid observations.
It then occurred to me that I wasn't really doing either of those methods in the uniform case. I had to modify them to work with a finite population. I have to modify them a whole lot more to work with a correlated finite population. So, maybe I'm not really using those methods at all, but rather using them as inspiration for methods that work in the finite case.
Ah, patterns! Now we're getting somewhere. There are lots of ways to build estimators, but not nearly as many ways to evaluate the quality of those estimators. That's what we're really after. The simple block estimator is always the best estimator. But, we need some way to know how good it is so we can create a stopping rule for our sampling. So, how do we do that? There are three options: derivation, bootstrapping, and simulation.
Derivation is fairly easy in the uniform case. Not so much in the correlated case. There's just too much stuff we don't know.
Bootstrapping works in both cases if you can make some assumptions about the kernel distribution (which may or may not be valid).
Simulation, particularly Monte Carlo Markov Chain is the most flexible, as in, you can let the data drive the model rather than the other way around, but it also incurs the most cost.
So, simple block sampling using the sample variance is a good example of the derivation method. If the data is iid, the exact distribution of the estimator can be stated. And, it's easy to show that it breaks down in the correlated case. It's also easy to show that if you can make some fairly restrictive assumptions about the data, you can save it. But, it's really a stretch.
I had been using Bag of Little Bootstraps (BLB) as the starting point for the bootstrap method, and that is a good entry. However, by the time we've accounted for correlation, we have something that's quite different from BLB. It's still very much bootstapping, though, so the flow of the paper works just fine if we cite BLB as an inspiration rather than trying to claim that it's what we're actually doing. FWIW, the modified bootstrap works OK with somewhat less restrictive assumptions.
Bootstrap Metropolis-Hastings is also better viewed as an entry point rather than the actual method we are using. In the correlated case, we have a lot more flexibility and that is helpful.
All this makes for a fairly cohesive part 4 (which I'm trying to get written this week, but it's increasingly looking like it will spill into the weekend).
What I'm really excited about is that it also sets up a really obvious use of the Dirichlet process. The upshot of this is an estimator that is derived to be the MVUE and assessed with a bootstrap process driven off results from a MCMC process. It ties it all together in part 5. I think this will make for a really cohesive paper.
No comments:
Post a Comment