When Probability Meets Complex Analysis: From the Binomial Distribution to the Poisson Distribution

当概率遇上复变：从二项分布到泊松分布 苏剑林 · 2015-01-13 · Mathematics probability generating function

The Poisson distribution is well suited to describing the probability distribution of the number of occurrences of a random event within a unit of time — for example, the number of service requests received by some service facility over a fixed period, or the number of people waiting at a bus stop. [Wikipedia] The Poisson distribution can also be derived as an approximation of the binomial distribution when the probability is small; this derivation is covered in pretty much every standard textbook on probability theory. However, the proofs typically given in textbooks aren't exactly a pleasure to read — for instance, page 98 of Course in Probability Theory and Mathematical Statistics (2nd edition, edited by Mao Shisong et al.) offers such a proof. So which proof deserves the most applause? I'd say it's the one using generating functions.

The generating function of the binomial distribution is

$$\begin{equation}(q+px)^n,\quad q=1-p\end{equation}$$more

When the number of trials is very large while the probability is very small, we can consider its approximation. In this case $\lambda=pn$ is not too large a number, and $p=\frac{\lambda}{n}$ is very small; this is likewise based on the probability formula

$$\begin{equation}(q+px)^n=\left(1+\frac{\lambda}{n}(x-1)\right)^n\end{equation}$$

Since

$$\begin{equation}\lim_{n\to\infty}\left(1+\frac{x}{n}\right)^n=e^x\end{equation}$$

when $n$ is fairly large but not infinite, the expression above differs only slightly from $e^x$, so we obtain the approximation

$$\begin{equation}\left(1+\frac{\lambda}{n}(x-1)\right)^n\approx e^{\lambda x-\lambda}\end{equation}$$

This is the generating function of the Poisson distribution.

It's worth noting that $e^{\lambda x-\lambda}|_{x=1}=1$, which is exactly the condition that a generating function of a probability distribution must satisfy — and this is quite a remarkable coincidence. We made an approximation, and while the function before the approximation was indeed the generating function of a probability distribution, the function after the approximation also happens to be the generating function of another probability distribution (with no need for any correction). This really is a beautiful coincidence.

Expanding it directly as a series gives us the probability of each term

$$\begin{equation}e^{\lambda x-\lambda}=e^{-\lambda}\sum_{k=0}^{\infty}\frac{\lambda^k}{k !}x^k\end{equation}$$

That is to say

$$\begin{equation}P(X=k)=e^{-\lambda}\frac{\lambda^k}{k !}\end{equation}$$

And so we arrive at the probability formula for the Poisson distribution.

Below is a diagram I recently found showing the relationships among various probability distributions, which I'd like to share with readers.

Relationships among various probability distributions

Original image from: http://www.math.wm.edu/~leemis/2008amstat.pdf