Common discrete distributions

Back on to stats. Today, we'll look at some essential discrete distributions. As these are really families of distributions, I'll include the parameters of the families in the functions describing them.

Discrete Uniform: P(X = x|N )  = 1/N     x = 1, 2, ..., N

Intuitive description: X is equally likely to be any one of N values. Obviously, any finite set where the outcomes are equally likely can be mapped to a discrete uniform distribution. If the mapping is a linear function, then the mean and variance can be easily transformed. If it's non-numeric data, then only the pdf has much meaning.

Mean: (N + 1)/2
Variance: (N + 1)(N - 1)/12


Hypergeometric:


also note that implicit in the definition is that M ≥ x and N-M ≥ K-x.

Intuitive description: Sampling without replacement. Using the traditional "Urn" parlance, what is the chance of pulling x Red balls in K tries from an Urn filled with N balls, M of which are Red (without putting any back between tries). This comes up in acceptance testing, where you are trying to determine if a batch of items meets a certain quality level. Let me just state from experience that the defect rate of a small sample is an absolutely bogus way to test quality. Maybe I'll elaborate tomorrow.

Mean: KM / N (While this is intuitively obvious, it is non-trivial to prove; in general, discrete distributions are a PITA).
Variance: an even bigger PITA: (KM/N)((N-M)(N-K)/(N(N-1))

Binomial:


Intuitive description: The number of successes in n Bernoulli trials with success probability p. This one comes up a lot.

Mean: np
Variance: np(1 - p)

Side note (Binomial Theorem): For any real numbers x and y and integer n ≥ 0:



Just one of those things to know.

Poisson:


Intuitive description: The number of occurrences of independent events in a unit interval where the rate is λ. For example, the number of people entering a checkout queue in a given period of time. The important thing is that the independence; that is, the waiting time to the next event has nothing to do with when the last event happened. The "sister" distribution is the continuous exponential which gives the waiting time between two such events.

Mean: λ
Variance: λ

Negative Binomial:


Intuitive description: Number of tries needed to get r successful Bernoulli trials with success probability p. This one also comes up a lot and is often used in stopping rules, as in, "if we haven't seen enough success by now, we can give up because the rate is obviously not as good as our threshold." The Negative Binomial is also often stated as the number of failures prior to the rth success. In this parameterization, the pdf becomes:



While these formulations are equivalent, it's important to know which one you're using when recalling the mean. It should be obvious that the total number of trials is just the expected number of failures plus r and that the variance is the same either way.

Mean: r(1-p)/p (failures); r + r(1-p)/p = r/p (total).
Variance: r(1-p)/p²

An important relationship to be aware of is that the limit of the Negative Binomial (parameterized as failures) as r goes to infinity and p goes to 1 such that r(1-p) goes to λ is the Poisson.

Geometric: P(X=x|p) = p(1-p)^x-1   x = 1,2,...

Intuitive description: This is a special case of the negative binomial which is the waiting time for the first success to occur. It is the discrete version of the Exponential, which is the waiting time for a Poisson event. As such, it shares the Exponentials "memoryless" property, that is, how long you should expect to wait has nothing to do with how long you've already waited. Or, more precisely, P(X>s|X>t) = P(X>s-t) for all integers s > t.

Mean: 1/p
Variance: (1-p)/p²