Sunday, March 18, 2018

Nothing recognizable

I have a distribution for the number of rows in each block partition. It's a bit messy, so I won't post it here. It's also a recurrence relationship, though I could convert it to closed form if there was a good reason to do that. Anyway, since it wasn't obvious by looking at it what the distribution actually looked like, I plotted the partition sizes for my sample. At only 1000 blocks, I didn't expect it to be a great match, but I was a little surprised by the shape.

The partitions that make up the sample are exponential (well, actually geometric as these are discrete values, but the block size is large enough to use the continuous approximation). That is, when I'm generating them, I simply use a uniform random variable to decide if that should be the last observation. Since the mean partition size is equal to the mean block size, that also means that about 370 of them should be longer than the block (which turns out to be the case). The graph doesn't look very exponential to me though (I've removed the x=1000 point since it has so many more observations than the rest of the distribution).


The second partition is a bit more complicated because the truncation point is dependent on the size of the first partition. So, this one is not only showing the interior of the distribution, but also all the partitions that hit the max row count. Anyway, it's also not particularly recognizable as anything to me. Same goes for three and four, though four actually does look somewhat exponential or, at least, some member of the gamma family.

The fifth and sixth partitions didn't have enough non-zero values to be useful (and no block contained more than six partitions).

So, that exercise was pretty non-enlightening. What is important is that the covariance terms are quite significant (as expected) when looking at things this way. I'm hoping to have all this quantified tomorrow, though none of this has gone as quickly as I would have thought, so I don't know why this piece will be any different.

No comments:

Post a Comment