Skip to main content

Module posterior

Module posterior 

Source
Expand description

The Bayesian option’s sampler: a posterior’s quantiles by importance sampling.

What it does. Given where a posterior peaks and its curvature there (the least-squares fit’s answer and covariance, the Laplace approximation), it draws points around the peak, weighs each by how much more or less likely the posterior makes it than the draw did, and reads each unknown’s 5%, 50% and 95% points off the weighted draws. Where the posterior is Gaussian this gives what the covariance already says; where it bends or has a heavy side, as a wind told by heights alone does, it gives the posterior’s own shape.

The method. Self-normalized importance sampling (J. Geweke, “Bayesian inference in econometric models using Monte Carlo integration”, Econometrica 57(6), 1989, pp. 1317–1339): draws xᵢ from a proposal q, weights wᵢ = p(xᵢ)/q(xᵢ) with p the posterior known up to a constant, and each expectation as Σ wᵢ f(xᵢ) / Σ wᵢ. The proposal is a multivariate Student-t with ν = DEGREES_OF_FREEDOM, centered on the peak, its scale the Laplace covariance: its tails are heavier than a Gaussian posterior’s, so the weights stay bounded where the posterior is near Gaussian (Geweke on a proposal’s tails). A draw is

x = m + L z / √(u/ν), z ~ N(0, I), u ~ χ²_ν, L Lᵀ = Σ,

and up to a constant ln q(x) = −(ν + N)/2 · ln(1 + δ²/ν) with δ² = |z|²/(u/ν).

How far to trust it. The weighted draws count for (Σ wᵢ)² / Σ wᵢ² independent ones, Kish’s effective sample size (L. Kish, Survey Sampling, Wiley, 1965), a result for a weighted mean read here for quantiles as an approximation. A posterior far from its Laplace approximation leaves few effective draws; the quantiles still come back, and Posterior::enough says they’re too few to read. A draw whose posterior can’t be had (a flight that won’t fly) gets no weight and is counted in Posterior::failed. Tested on posteriors whose quantiles are known in closed form or by quadrature.

In two blocks. A posterior whose unknowns follow some of them along a curved ridge, as a fit’s drag and clock follow the wind on a log of height alone, leaves one Student-t about its peak few effective draws. sample_in_two draws the marked unknowns first, then the others about where the caller says they center given those, and widens its scales where a pilot’s draws reach further (ADR-320). crate::fit::bayesian draws a fit’s posterior so, and cargo xtask fit counts its 90% intervals on the synthetic flights.

Structs§

Posterior
What the weighted draws say of each unknown.

Constants§

DEGREES_OF_FREEDOM
The proposal’s degrees of freedom: 4. Any Student-t has tails heavier than a Gaussian posterior’s, which keeps the weights bounded there (Geweke); a small ν widens them further for a posterior with a heavy side, and ν = 4 still has a finite variance. A judgment, not a measurement.
MAX_WIDENING
The most sample_in_two’s pilot widens a block’s scale along any direction, in variance: 4, twice the standard deviation, so that a pilot ruled by one far draw can’t spread the main draws past use. A judgment, not a measurement.
MIN_EFFECTIVE_DRAWS
The fewest effective draws for a quantile to be read: 100. At 100, the 5% point’s Monte Carlo error is about √(0.05 · 0.95 / 100) / φ(1.645), 0.22 posterior standard deviations rounded up, past which an interval’s ends say more of the draws than of the flight. A judgment, not a measurement.
PROBABILITIES
The probabilities Posterior::quantiles reads: the 90% interval’s ends and the median.

Functions§

sample
The posterior whose log density, up to a constant, is log_target, by draws weighted draws around peak with the Laplace covariance covariance.
sample_in_two
sample in two blocks, adapted on a pilot: for a posterior whose other unknowns follow some of them along a ridge the Laplace approximation can’t bend to, and which may reach further than its curvature says.