Probability Distribution Functions

While discrete random variables use PMFs and continuous ones use PDFs, every random variable without exception possesses a well-defined Cumulative Distribution Function: F(x) = P(X <= x). As x ranges from -inf to +inf, F(x) starts at 0 and monotonically accumulates probability until it reaches 1. For a continuous random variable with PDF f(t), F(x) is the running integral: F(x) = int_(-inf)^x f(t) dt. To find the probability of landing in any interval (a, b], we simply subtract cumulative totals: P(a < X <= b) = F(b) - F(a), eliminating the need to re-integrate from scratch every time.

Financial risk management (Value at Risk - VaR) and cloud service level agreements (SLAs) rely on CDFs to determine percentiles (e.g., '99% of web requests complete in under 200ms').

Pafnuty Chebyshev and his student Andrei Markov developed distribution functions in the late 19th century to prove generalized central limit theorems without relying on smooth densities.

'CDF = Collected Done Fraction' — F(x) answers: 'What fraction of all possibilities has already happened by time/threshold x?'.

Probability theory uses three specialized functions depending on the topology of the sample space:

• PMF (Probability Mass Function, p(x)): For discrete variables. Gives exact probabilities P(X = x). Sums to 1 (Sigma p(x) = 1).
• PDF (Probability Density Function, f(x)): For continuous variables. Represents local concentration of probability per unit x. Integrates to 1 (int f(x)dx = 1). f(x) can exceed 1.
• CDF (Cumulative Distribution Function, F(x)): Universal for all variables. F(x) = P(X <= x). Bounded strictly between 0 and 1, right-continuous and non-decreasing.

In machine learning, classification models output PMFs (softmax probabilities across classes), regression models output PDFs (mean and variance predictions), and thresholding systems evaluate CDFs (ROC curves and precision-recall trade-offs).

The unification of PMF, PDF, and CDF under a single measure-theoretic framework was accomplished by Andrei Kolmogorov in 1933 using Lebesgue-Stieltjes integration.

Mnemonic 'Mass, Density, Cumulative': Mass for points, Density for intervals, Cumulative for totals.

When a medical diagnosis AI analyzes a patient with fever, high white blood cell count, low oxygen, and age 72, it feeds these features through linear weights and non-linear activations to produce raw logit vectors z = [z₁, z₂, ... zₖ]. To convert these logits into a valid PMF over diagnostic classes, the model applies the Softmax function: pᵢ = exp(zᵢ) / sum(exp(zⱼ)). Because exp(z) > 0 and the denominator sums over all classes, the outputs are strictly non-negative and sum to 1, creating a mathematically valid probability distribution that guides clinical intervention.

Large Language Models (LLMs like GPT-4 and Gemini) produce probability distributions over a 32,000+ token vocabulary at every generation step, using temperature sampling to select the next word from the resulting PMF.

Ludwig Boltzmann developed the softmax distribution in 1868 for statistical mechanics to describe particle energy states, which John Hopfield and Geoffrey Hinton adapted into machine learning in the 1980s.

'Features -> Logits -> Softmax -> Probabilities' — raw evidence becomes scores, which exponentiation turns into positive fractions summing to 1.

If a manufacturing machine produces bolts that can have heights of only 150 cm, 160 cm, or 170 cm with probabilities 0.25, 0.50, and 0.25 respectively, the sample space is discrete: S = 150, 160, 170. The PMF is defined as p(x) = P(X = x). At values outside this set (e.g., x = 155), p(x) = 0. The two fundamental requirements of a valid PMF are: (1) p(x) >= 0 for all x, and (2) sum_(x in S) p(x) = 1. Unlike continuous distributions where individual points have zero probability, discrete points have real, measurable mass.

Quality control engineers in semiconductor fabrication count defects per silicon wafer using discrete Poisson and Binomial PMFs to set automated factory line rejection thresholds.

Blaise Pascal and Pierre de Fermat established the mathematics of discrete probability mass in their 1654 correspondence solving the famous 'problem of points' in unfinished gambling games.

'Discrete = Direct' — in a PMF, you read probabilities directly off the height of the spike at each point.

Operational problems dictate their mathematical representation:

• Multi-class classification (e.g., image categorization: Cat 82%, Dog 14%, Bird 4%) uses a PMF because classes are mutually exclusive, discrete categories.
• Continuous regression (e.g., predicting exact human height or stock price volatility) uses a PDF to show how probability density is distributed across continuous ranges.
• Risk evaluation and compliance (e.g., checking if server response time X <= 200ms or structural load X <= tolerance) uses a CDF F(threshold) = P(X <= threshold) to provide a direct, single-number probability of safety or failure.

Autonomous vehicle motion planners use PDFs to model pedestrian trajectory uncertainty, PMFs to choose discrete maneuvers (brake, swerve left, swerve right), and CDFs to guarantee collision risk remains below 10^(-6).

Abraham Wald established Statistical Decision Theory in 1950, formalizing how probability functions combine with loss functions to make optimal minimax decisions under real-world operational constraints.

'Classify with Mass, Spread with Density, Gate with CDF' — PMF chooses categories, PDF shapes continuous variables, CDF guards thresholds.

Let X be the height of a randomly selected person. Because height can be measured to arbitrary precision (169.5 cm, 169.512 cm, ...), there are uncountably infinite possibilities. The probability of measuring exactly 170.0000... cm is 0. However, the probability of falling within an interval [160, 170] is strictly positive, given by the area under the PDF: P(160 <= X <= 170) = int_160¹70 f(x) dx. If human height is modeled across intervals with probabilities [150-160: 0.15], [160-170: 0.40], [170-180: 0.35], and [180-190: 0.10], these interval probabilities sum to 1.0, satisfying total probability conservation.

Civil engineers designing aircraft fuselage seating and cockpit clearances integrate height and reach PDFs of the target population to accommodate 95% of airline passengers comfortably.

Adolphe Quetelet demonstrated in 1846 that physical human characteristics (height, chest circumference) follow continuous Gaussian probability density curves, founding the field of biometrics.

'Density needs Length to make Mass' — f(x) is height; multiply (integrate) across width dx to get actual probability.

The relationship between PDF f(x) and CDF F(x) is the Fundamental Theorem of Calculus applied to probability:

• Moving from PDF to CDF is Integration: F(x) = int_(-inf)^x f(t) dt. You accumulate all probability density from negative infinity up to point x.
• Moving from CDF to PDF is Differentiation: f(x) = dF(x)/dx. The local density at x is the instantaneous rate of change of the accumulated probability.
• Water Analogy: Think of water flowing into a basin. The PDF f(x) represents the instantaneous rate of flow (liters per second) at moment x, while the CDF F(x) represents the total volume of water accumulated in the basin up to moment x. They are two mathematical perspectives describing the exact same underlying distribution.

Generative modeling in AI (such as Normalizing Flows and Continuous Normalizing Flows) computes probability densities of complex images by differentiating tractable CDF mappings via the Jacobian determinant.

Isaac Newton and Gottfried Wilhelm Leibniz formulated the Fundamental Theorem of Calculus in the late 17th century; its application to probability density and distribution functions formed the cornerstone of mathematical statistics in the 20th century.

'Integrate to Accumulate (PDF -> CDF), Differentiate to Locate (CDF -> PDF)'.