Poisson Distribution (Extended)
How do mathematical limits, Taylor series, and continuous point processes establish the rigorous foundations of Poisson mechanics?
How this lesson unfolds.
Begin with Why does Euler's transcendental number e = 2.71828... appear in a discrete formula for counting events? The sequence then connects The Continuum Limit: Visualizing the Transition from Binomial to Poisson and builds toward The Continuous-Discrete Paradox: Bridging Continuum and Integer Counts.
Why does Euler's transcendental number e = 2.71828... appear in a discrete formula for counting events?
Just as continuously compounding interest at 100% yields e, continuously surviving a million infinitesimal danger periods compounds into e^(−λ).
- $$\lim_{n \to \infty} \left(1 - \frac{\lambda}{n}\right)^n = e^{-\lambda}, \quad P(X=0) = e^{-\lambda}$$
- $$Euler's constant e arises naturally from continuous compounding of survival probabilities: \lim_{n \to \infty} (1 - \lambda/n)^n = e^{-\lambda}.$$
- Learning goalDerive e^(−λ) as the infinite product limit of non-occurrence in continuous time.
- Helpful foundationsCompound interest limit · Definition of e · Binomial zero probability · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapEuler's constant e^(−λ) is the exact physical probability that zero events occur.
- Next connectionFollow the algebraic derivation transitioning Binomial to Poisson for all counts.
More context & mechanics
To understand why e appears in Poisson, consider the probability of observing ZERO events in an interval. Divide the interval into n micro-slices. The probability of zero events in one slice is (1 - p) = (1 - λ/n). Assuming independence across slices, the probability of zero events across all n slices is: P(X = 0) = (1 - (λ) / (n))^n. In the calculus limit as n → ∞, this is the classic definition of the exponential function: lim_(n → ∞) (1 - (λ) / (n))^n = e^(−λ). Thus, e^(−λ) is not an arbitrary constant; it is the exact survival probability of a continuous Poisson process with zero arrivals.
In cybersecurity and hash tables, the probability that a hash bucket receives zero collisions under uniform hashing of n items into m buckets (with λ = n/m) is approximately e^(−λ). This formula governs load factor tuning in Redis, Python dicts, and distributed database sharding.
Jacob Bernoulli first discovered the constant e in 1683 while calculating continuous compound interest on monetary debt: lim_(n → ∞) (1 + 1/n)^n. Poisson reversed Bernoulli's logic to model continuous survival under constant hazard.
The Survival Hook: 'Zero arrivals = e to the minus lambda'. Every event you dodge shrinks your survival probability by an exponential decay factor.
How does a discrete Binomial distribution smoothly transform into a Poisson distribution as n grows from 10 to 1,000?
Think of a low-resolution digital photo: as you increase the pixel count n while keeping total brightness λ constant, the blocky pixels merge into a crisp, smooth image.
- $$\left| \binom{n}{x} p^x (1-p)^{n-x} - \frac{\lambda^x e^{-\lambda}}{x!} \right| \le \min\left(2p, \frac{2\lambda^2}{n}\right)$$
- $$As n increases while holding \lambda = np constant, the Binomial histogram stabilizes, discarding its dependence on n and converging to the invariant Poisson curve.$$
- Learning goalQuantify Binomial-to-Poisson convergence rate using Le Cam's inequality bounds.
- Helpful foundationsBinomial distribution · Poisson limit theorem · Approximation rules · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapLarge n alone does not justify Poisson; p must be small so np remains moderate.
- Next connectionDerive Poisson variance using factorial moment algebraic techniques.
More context & mechanics
Examine what happens as we scale an experiment while holding λ = np = 2 fixed:
- At n = 4, p = 0.5: P(X=0) = (0.5)⁴ = 0.0625.
- At n = 20, p = 0.1: P(X=0) = (0.9)^(20) ≈ 0.1216.
- At n = 100, p = 0.02: P(X=0) = (0.98)^(100) ≈ 0.1326.
- At n = 1000, p = 0.002: P(X=0) = (0.998)^(1000) ≈ 0.13506.
- Exact Poisson limit: e^(−2) = 0.135335...
By n = 100, the Binomial distribution has converged to three decimal places of precision to Poisson, proving that n ceases to be an informative parameter.
In semiconductor microchip fabrication (TSMC, Intel), silicon wafers contain millions of transistors (n ≈ 10⁹) with microscopic defect chance per transistor (p ≈ 10^(-9)). Yield models discard n completely and compute chip failure yields purely using Poisson(λ) with defect density λ ≈ 1.0.
The convergence theorem was formulated systematically by Siméon Denis Poisson in 1837 and popularized by Pafnuty Chebyshev and Richard von Mises in the early 20th century.
Rule of Thumb: When n ≥ 100 and p ≤ 0.01 (np ≤ 10), dump the complicated combinations and switch directly to Poisson(λ = np).
How does the algebraic identity x² = x(x-1) + x unlock the rigorous derivation of Poisson variance?
The term x! in the denominator cancels x(x-1) into (x-2)!, shifting the infinite series index cleanly to regenerate e^λ.
- $$E[X(X-1)] = \lambda^2, \quad E[X^2] = \lambda^2 + \lambda, \quad \text{Var}(X) = \lambda$$
- $$Evaluating E[X^2] directly is difficult, but splitting x^2 into x(x-1) + x allows factorial cancellation in the denominator, proving E[X^2] = \lambda^2 + \lambda and Var(X) = \lambda.$$
- Learning goalDerive Poisson second moment E[X²] and variance Var(X) = λ via factorial moments.
- Helpful foundationsFactorial properties · Taylor series of e^x · Variance formula · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapCannot cancel x² directly with x!; rewrite as x(x-1) + x to cancel factors sequentially.
- Next connectionDerive expectation E[X] = λ using series index-shifting methods.
More context & mechanics
To evaluate Var(X) = E[X²] - (E[X])² where E[X] = λ:
Step 1: Write E[X²] = ∑_(x=0)^∞ x² (λ^x e^(−λ)) / (x!).
Step 2: Rewrite x² as x(x-1) + x. This splits the sum into two parts: E[X²] = ∑_(x=2)^∞ x(x-1) (λ^x e^(−λ)) / (x!) + ∑_(x=1)^∞ x (λ^x e^(−λ)) / (x!).
Step 3: In the first sum, cancel x(x-1) with x! to get (x-2)!: e^(−λ) λ² ∑_(x=2)^∞ (λ^(x-2)) / ((x-2)!) = e^(−λ) λ² e^λ = λ².
Step 4: The second sum is simply the expectation E[X] = λ.
Step 5: Thus, E[X²] = λ² + λ.
Step 6: Compute variance: Var(X) = (λ² + λ) - λ² = λ! A triumph of mathematical elegance.
In biostatistics and RNA sequencing (RNA-seq), gene expression counts are tested for Poisson variance σ² = μ. Any excess variance beyond λ signals biological heterogeneity, prompting the use of Negative Binomial dispersion models (DESeq2, edgeR).
The technique of using factorial moments E[X(X-1)...] to tame factorial denominators was developed by Abraham de Moivre and standardized by Pierre-Simon Laplace.
The Factorial Trick: 'x squared is x(x-1) plus x'. E[X(X-1)] gives λ²; E[X] gives λ. Var = (λ² + λ) - λ² = λ.
How does index shifting transform the infinite sum ∑ x λ^x e^(−λ) / x! into exactly lambda?
Expectation is an infinite weighted balance. Factoring out one λ leaves behind the exact definition of 1, pulling the mean straight to λ.
- $$E[X] = e^{-\lambda} \sum_{x=1}^\infty \frac{\lambda^x}{(x-1)!} = \lambda e^{-\lambda} \sum_{k=0}^\infty \frac{\lambda^k}{k!} = \lambda$$
- $$The term for x=0 vanishes, leaving x=1 to \infty where x cancels with x!$$
- $$Factoring out \lambda reconstitutes the Taylor series of e^\lambda, yielding E[X] = \lambda.$$
- Learning goalDerive Poisson expected value E[X] = λ using series re-indexing techniques.
- Helpful foundationsExpected value summation · Factorials · Taylor series of exponential
- Common trapSummation index must shift from x=0 to x=1 after canceling x; (-1)! is undefined.
- Next connectionConnect continuous-time intervals with discrete integer Poisson counts.
More context & mechanics
Let us follow the step-by-step proof that E[X] = λ:
- Definition: E[X] = ∑_(x=0)^∞ x P(X=x) = ∑_(x=0)^∞ x (λ^x e^(−λ)) / (x!).
- Eliminate 0: When x = 0, the term is 0 · ṡ = 0. The sum starts at x = 1: E[X] = e^(−λ) ∑_(x=1)^∞ x (λ^x) / (x!).
- Cancel x: Because x! = x(x-1)!, we have x/x! = 1/(x-1)!, giving: E[X] = e^(−λ) ∑_(x=1)^∞ (λ^x) / ((x-1)!).
- Factor out λ: Pull out one factor of λ: E[X] = λ e^(−λ) ∑_(x=1)^∞ (λ^(x-1)) / ((x-1)!).
- Substitute k = x - 1: As x runs from 1 to ∞, k runs from 0 to ∞: ∑_(k=0)^∞ (λ^k) / (k!) = e^λ.
- Cancel exponentials: E[X] = λ e^(−λ) e^λ = λ · 1 = λ.
In insurance loss reserves, actuarial models calibrate the baseline frequency λ from historical claim counts, knowing that the empirical sample average X̄ is the maximum likelihood estimator for λ.
This proof technique of index substitution k = x - 1 was formalized during the rigorous arithmetization of calculus led by Augustin-Louis Cauchy in Paris in the 1820s.
Step Sequence: 'Drop zero, cancel x, factor lambda, recognize e to the lambda, cancel exponentials to get lambda'.
How can a continuous temporal process give birth to a probability distribution that only outputs discrete whole numbers?
Raindrops fall at continuous positions on a sidewalk, but if you put out a bucket, you count a discrete number of drops: 0, 1, 2... The canvas is continuous; the inventory is discrete.
- $$T_i \stackrel{\text{iid}}{\sim} \text{Exponential}(\lambda) \iff N(t) = \max \{ n : \sum_{i=1}^n T_i \le t \} \sim \text{Poisson}(\lambda t)$$
- $$Events can happen at any continuous real-valued instant t \in \mathbb{R}^+, but the question 'How many happened?' is a counting operation that must yield a discrete integer.$$
- Learning goalConnect continuous Exponential inter-arrival times with discrete Poisson event counts.
- Helpful foundationsExponential distribution · Poisson process · Continuous vs discrete duality
- Common trapPoisson counts are strictly discrete integers despite occurring across continuous time intervals.
- Next connectionExplore sampling from finite populations without replacement via Hypergeometric models.
More context & mechanics
This apparent paradox dissolves when distinguishing between the timing of events and the count of events:
- The inter-arrival times between consecutive Poisson events are continuous random variables following an Exponential distribution T ~ Exp(λ).
- The count of events in any fixed interval [0, t] is a discrete random variable following a Poisson distribution N(t) ~ Poisson(λ t).
Because time is continuous, the probability of an event landing at an exact infinitesimal instant t₀ is 0. Yet over a non-zero interval, these infinitesimal hazard rates accumulate into a positive integer count X ∈ 0, 1, 2, ṡ.
In algorithmic high-frequency trading (HFT), market orders hit the central limit order book at continuous microsecond timestamps (modeled by Hawkes processes / Poisson processes), while the exchange settlement engine reconciles discrete integer shares traded.
Swedish mathematician Filip Lundberg in 1903 established the collective risk model for insurance, proving that continuous ruin theory emerges from discrete claim counts governed by Poisson arrivals.
The Duality Bridge: 'Continuous waiting time ⟺ Exponential. Discrete event count ⟺ Poisson'. They are two sides of the exact same coin.