Negative Binomial Distribution
How does waiting for the r-th success generalize geometric stopping times and model overdispersed count data?
How this lesson unfolds.
Begin with Why does the combination term in the Negative Binomial PMF use \binom{x-1}{r-1} instead of \binom{x}{r}? The sequence then connects The Unimodal Arch: Why Negative Binomial Rises Then Falls and builds toward Trials vs. Failures: Clarifying What the Random Variable Counts.
Why does the combination term in the Negative Binomial PMF use C(x-1, r-1) instead of C(x, r)?
The game stops immediately when the r-th success occurs. That final success has no freedom to move—it is frozen at the finish line!
- $$P(X=x) = \binom{x-1}{r-1} p^r (1-p)^{x-r}, \quad x \in \{r, r+1, r+2, \dots\}$$
- Learning goalExplain why the Negative Binomial combinatorial factor is C(x-1, r-1) from fixed terminal success.
- Helpful foundationsGeometric distribution · Binomial combinations · Stopping rules · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapUsing C(x, r) violates the stopping rule by allowing earlier termination before trial x.
- Next connectionAnalyze why Negative Binomial PMFs peak and decay unlike Geometric distributions.
More context & mechanics
Let X be the total number of trials required to achieve exactly r successes with single-trial success probability p. For the experiment to terminate at exactly trial x, two conditions must be satisfied simultaneously:
- Condition 1: The final trial x MUST be a success. This occurs with probability p, locked at the finish line.
- Condition 2: In the preceding (x - 1) trials, there must have been exactly (r - 1) successes (and consequently (x - r) failures). The number of ways to arrange those (r - 1) successes in the first (x - 1) slots is C(x-1, r-1).
Multiplying combinations by probabilities: C(x-1, r-1) × p^(r-1) (1-p)^(x-r) × p = C(x-1, r-1) p^r (1-p)^(x-r).
That is why the combination is C(x-1, r-1), NOT C(x, r)!
In athletic sports tournaments (such as the MLB World Series or NBA Finals), two teams play best-of-7 games until one team achieves r=4 wins. The probability of the series ending in exactly x=6 or x=7 games is calculated using this exact formula.
Pierre Rémond de Montmort first formulated the distribution in 1713 in his treatise Essay d'analyse sur les jeux de hazard while solving the problem of points for multi-round games.
The Frozen Finale Hook: 'Lock the last win at the end! Choose (r-1) wins from (x-1) slots'. C(x-1, r-1) p^r (1-p)^(x-r).
Why does the Geometric distribution (r=1) decay monotonically, while the Negative Binomial (r > 1) rises to a peak before descending?
For r=1, success on trial 1 is easiest (no failures needed). For r=5, you CANNOT finish in 1 trial! You need time to accumulate successes, creating an uphill ramp before the downhill tail.
- $$\frac{P(X = x+1)}{P(X = x)} = \frac{x}{x - r + 1} (1 - p), \quad \text{Mode} = \left\lfloor \frac{(r-1)(1-p)}{p} \right\rfloor + r \quad (r > 1)$$
- Learning goalAnalyze consecutive probability ratios to establish unimodality and compute the distribution mode.
- Helpful foundationsRatio test for sequences · Unimodal distributions · Geometric distribution mode · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapNegative Binomial curves peak for r > 1, decaying monotonically only when r = 1.
- Next connectionSynthesize all formulas, identities, and moment properties in the master cheat sheet.
More context & mechanics
Look at the ratio of consecutive probabilities: (P(X = x+1)) / (P(X = x)) = (x) / (x - r + 1) (1 - p). For r = 1 (Geometric), this ratio is x/x(1-p) = 1 - p < 1 for all x. The ratio is always less than 1, so the graph decays immediately from x = 1.
For r > 1, when x is close to r, the factor (x) / (x - r + 1) is substantially greater than 1! For example, if r = 5, p = 0.5, at x = 5 the ratio is 5/1(0.5) = 2.5 > 1, so P(X=6) > P(X=5). The probabilities climb until (x) / (x - r + 1) (1 - p) drops below 1. This produces a distinct hump-shaped, unimodal distribution with mode ⌊ ((r-1)(1-p)) / (p) ⌋ + r.
In video game monetization, tracking how many loot boxes a player opens to collect r=3 rare character shards follows this unimodal curve: very few players get all 3 shards in 3 boxes, most cluster around the peak (15 boxes), and a long tail takes 30+ boxes.
Karl Pearson in 1895 categorized this unimodal curve in his Type III Pearson system, showing how the discrete negative binomial approximates the continuous Gamma distribution.
The Growth Tug-of-War: 'Combinations pull up, failure powers pull down'. Early on, combinations win (curve rises). Later, exponential failure wins (curve falls).
Why is the Negative Binomial distribution mathematically equivalent to the sum of r independent Geometric random variables?
If waiting for 1 bus takes 10 minutes on average, waiting for 3 buses takes 3 × 10 = 30 minutes. The expectations and variances simply multiply by r.
- $$X = \sum_{i=1}^r G_i, \quad G_i \sim \operatorname{Geom}(p) \implies E[X] = \frac{r}{p}, \quad \operatorname{Var}(X) = \frac{r(1-p)}{p^2}$$
- $$Waiting for r successes equals waiting for the 1st success, plus waiting for the 2nd, ..., plus waiting for the r-th: X = \sum_{i=1}^r G_i, meaning Mean = r/p and Variance = r(1-p)/p^2.$$
- Learning goalDerive Negative Binomial expectation and variance by convolving r independent Geometric variables.
- Helpful foundationsGeometric distribution moments · Sum of independent random variables · Linearity of expectation · Foundations of the Theory of Probability (Andrey Kolmogorov, 1933) · An Essay towards solving a Problem in the Doctrine of Chances (Thomas Bayes, 1763)
- Common trapDistinguish trial counts (x ≥ r) from failure counts (y ≥ 0) before evaluating formulas.
- Next connectionExplore cumulative forms, tail identities, and the negative binomial theorem.
More context & mechanics
Every Negative Binomial random variable X ~ NegBin(r, p) can be decomposed into the sum of r independent, identically distributed Geometric random variables: X = G₁ + G₂ + ṡ + G_r, where each Gᵢ ~ Geometric(p). Because each Gᵢ has mean E[Gᵢ] = 1/p and variance Var(Gᵢ) = (1-p)/p², we immediately get:
- Mean: E[X] = ∑_(i=1)^r E[Gᵢ] = r (1/p) = r/p.
- Variance: Var(X) = ∑_(i=1)^r Var(Gᵢ) = r ((1-p) / (p²)) = (r(1-p)) / (p²).
- Standard Deviation: SD(X) = (√(r(1-p))) / (p).
No complicated series summations required—the convolution of i.i.d. Geometric variables gives the moments instantly!
In genomics and microbiome data science (e.g. DESeq2, Single-Cell RNA-seq), raw read counts exhibit extreme overdispersion (variance >> mean). Statisticians parameterize the Negative Binomial as a Gamma-Poisson mixture to model biological overdispersion that breaks Poisson models.
Major Greenwood and G. Udny Yule in 1920 proved that when individuals have differing Poisson accident rates distributed according to a Gamma distribution, the aggregate count follows a Negative Binomial distribution (the accident-proneness model).
Sum of Geometrics Hook: 'r successes means r times Geometric'. Mean = r/p, Variance = r(1-p)/p². Special case r=1 is Geometric.
How can a tail probability of the Negative Binomial distribution be converted directly into a Binomial probability?
If you failed to get 4 wins by game 7, it means in those 7 games you got at most 3 wins. The waiting-time tail flips into a fixed-trial count!
- $$P(X > m) = P(Y < r) = \sum_{k=0}^{r-1} \binom{m}{k} p^k (1-p)^{m-k} \quad (Y \sim \operatorname{Binomial}(m, p))$$
- $$Requiring more than m trials to get r successes is logically identical to getting strictly fewer than r successes in the first m trials: P(X > m) = P(Y < r) where Y ~ Binomial(m, p).$$
- Learning goalConvert infinite Negative Binomial tail probabilities into finite Binomial cumulative sums.
- Helpful foundationsBinomial CDF · Negative Binomial PMF · Complementary events
- Common trapAvoid summing infinite tails directly; invert to finite Binomial CDF sums for computation.
- Next connectionStep through the first-principles combinatorial derivation of the PMF.
More context & mechanics
Calculating the survival function P(X > m) for X ~ NegBin(r, p) requires an infinite sum ∑_(x=m+1)^∞ P(X=x). However, there is a brilliant equivalence connecting Negative Binomial and Binomial:
The event X > m means: 'It took more than m trials to achieve r successes'. This happens if and only if: 'In the first m trials, the total number of successes was strictly less than r'. Let Y ~ Binomial(m, p). Then: P(X > m) = P(Y ≤ r - 1) = ∑_(k=0)^(r-1) C(m, k) p^k (1-p)^(m-k) = I_(1-p)(m - r + 1, r). This converts an intractable infinite sum into a finite sum of at most r terms!
In software reliability, finding the probability that a stress testing run of m=10,000 requests encounters fewer than r=5 fatal memory leaks uses this tail equivalence to evaluate service deployment readiness.
This tail equivalence was proven by Abraham de Moivre in The Doctrine of Chances (1718) and provides the mathematical bridge linking incomplete beta functions to both distributions.
The Tail Flip: 'Waiting more than m trials for r wins = Getting fewer than r wins in m trials'. P(X > m) = P(Binomial(m, p) < r).
How do we prove rigorously from foundational probability rules that P(X=x) = C(x-1, r-1) p^r (1-p)^(x-r)?
Divide the timeline into two chapters: Chapter 1 is the first x-1 trials (standard Binomial). Chapter 2 is trial x (must be a success). Multiply them together.
- $$P(X=x) = \binom{x-1}{r-1} p^{r-1} (1-p)^{x-r} \cdot p = \binom{x-1}{r-1} p^r (1-p)^{x-r}$$
- $$By independence, P(A \cap B) = P(A)P(B).$$
- Learning goalProve the Negative Binomial PMF using two-event independence decomposition.
- Helpful foundationsBinomial PMF · Independence multiplication rule · Event partitioning
- Common trapTrial x must be a success; otherwise the r-th success occurred at a different trial.
- Next connectionClarify exactly what random variable X counts across various textbook definitions.
More context & mechanics
Let us execute the rigorous proof:
- Define Event E: 'The r-th success occurs exactly on trial x'.
- Break E into two joint events:
- Event A: 'Exactly (r - 1) successes occur in the first (x - 1) trials'.
- Event B: 'Trial x results in a success'.
- Since trials are independent Bernoulli experiments, Event A and Event B are statistically independent: P(E) = P(A ∩ B) = P(A) × P(B).
- Calculate P(A): In the first (x - 1) trials, the count of successes follows Binomial(x - 1, p). The probability of exactly (r - 1) successes is: P(A) = C(x-1, r-1) p^(r-1) (1-p)^((x-1)-(r-1)) = C(x-1, r-1) p^(r-1) (1-p)^(x-r).
- Calculate P(B): The probability of success on trial x is simply p: P(B) = p.
- Multiply together: P(E) = [ C(x-1, r-1) p^(r-1) (1-p)^(x-r) ] × p = C(x-1, r-1) p^r (1-p)^(x-r).
The derivation is complete and undeniable.
In sales recruitment, an enterprise software company interviews candidates until hiring r=5 senior engineers. HR planners calculate recruitment budget distributions using this exact derivation.
This two-stage partitioning proof was formalized by Blaise Pascal in his 1654 letters to Pierre de Fermat, establishing conditional stopping times in probability.
The 2-Stage Story: 'First (x-1) trials play Binomial for (r-1) wins; last trial seals the deal with p'. Multiply to combine.
Why do different statistics textbooks define the Negative Binomial random variable differently, and how do you navigate between them?
Do you count the total miles driven to see 3 national parks (trials)? Or only the detour miles off the highway (failures)? Just subtract r to switch between them.
- $$Y = X - r \implies E[Y] = \frac{r(1-p)}{p}, \quad \operatorname{Var}(Y) = \frac{r(1-p)}{p^2}$$
- $$Definition 1 counts total trials X \in {r, r+1, ...}; Definition 2 counts only failures Y = X - r \in {0, 1, ...}.$$
- Learning goalConvert between trial-based X and failure-based Y Negative Binomial formulations.
- Helpful foundationsLinear transformation of random variables · Shifted distributions · Scipy and R conventions
- Common trap`scipy.stats.nbinom` parameterizes the count of failures y = x - r, not total trials.
- Next connectionAdvance to memorylessness, total probability, and indicator variable systems.
More context & mechanics
Beware of the two competing textbook conventions for Negative Binomial:
- Convention 1 (Total Trials X): Support is x ∈ r, r+1, r+2, ṡ. PMF is P(X=x) = C(x-1, r-1) p^r (1-p)^(x-r). Mean is E[X] = r/p.
- Convention 2 (Failures Y): Support is y ∈ 0, 1, 2, ṡ where Y = X - r. PMF is P(Y=y) = C(y+r-1, y) p^r (1-p)^y. Mean is E[Y] = r(1-p)/p.
Both models are identical under the simple change of variable Y = X - r. Notice that Var(X) = Var(Y) = r(1-p)/p² because shifting a random variable by a constant r changes its mean by r but leaves its variance completely unchanged!
Software libraries differ: R's `dnbinom(x, size, prob)` and Python's `scipy.stats.nbinom(n, p)` use Convention 2 (counting FAILURES), while many probability engineering textbooks use Convention 1 (counting TOTAL TRIALS). Knowing how to translate between them prevents severe simulation bugs.
The failure-based definition arose from the Taylor series expansion of (1 - t)^(-r) = ∑_(y=0)^∞ C(y+r-1, y) t^y, which uses negative binomial exponents, giving the distribution its modern name.
The Bridge Rule: 'Trials = Failures + r'. E[Trials] = E[Failures] + r = (r(1-p)) / (p) + r = r/p. Variance is identical for both.