Expand description
The Bayesian option’s sampler: a posterior’s quantiles by importance sampling.
What it does. Given where a posterior peaks and its curvature there (the least-squares fit’s answer and covariance, the Laplace approximation), it draws points around the peak, weighs each by how much more or less likely the posterior makes it than the draw did, and reads each unknown’s 5%, 50% and 95% points off the weighted draws. Where the posterior is Gaussian this gives what the covariance already says; where it bends or has a heavy side, as a wind told by heights alone does, it gives the posterior’s own shape.
The method. Self-normalized importance sampling (J. Geweke, “Bayesian inference in
econometric models using Monte Carlo integration”, Econometrica 57(6), 1989, pp. 1317–1339):
draws xᵢ from a proposal q, weights wᵢ = p(xᵢ)/q(xᵢ) with p the posterior known up to a
constant, and each expectation as Σ wᵢ f(xᵢ) / Σ wᵢ. The proposal is a multivariate Student-t
with ν = DEGREES_OF_FREEDOM, centered on the peak, its scale the Laplace covariance: its
tails are heavier than a Gaussian posterior’s, so the weights stay bounded where the posterior is
near Gaussian (Geweke on a proposal’s tails). A draw is
x = m + L z / √(u/ν), z ~ N(0, I), u ~ χ²_ν, L Lᵀ = Σ,
and up to a constant ln q(x) = −(ν + N)/2 · ln(1 + δ²/ν) with δ² = |z|²/(u/ν).
How far to trust it. The weighted draws count for (Σ wᵢ)² / Σ wᵢ² independent ones, Kish’s
effective sample size (L. Kish, Survey Sampling, Wiley, 1965), a result for a weighted mean
read here for quantiles as an approximation. A posterior far from its Laplace approximation
leaves few effective draws; the quantiles still come back, and Posterior::enough says
they’re too few to read. A draw whose posterior can’t be had (a flight that won’t fly) gets no
weight and is counted in Posterior::failed. Tested on posteriors whose quantiles are known
in closed form or by quadrature.
In two blocks. A posterior whose unknowns follow some of them along a curved ridge, as a
fit’s drag and clock follow the wind on a log of height alone, leaves one Student-t about its
peak few effective draws. sample_in_two draws the marked unknowns first, then the others
about where the caller says they center given those, and widens its scales where a pilot’s
draws reach further (ADR-320). crate::fit::bayesian draws a fit’s posterior so,
and cargo xtask fit counts its 90% intervals on the synthetic flights.
Structs§
- Posterior
- What the weighted draws say of each unknown.
Constants§
- DEGREES_
OF_ FREEDOM - The proposal’s degrees of freedom: 4. Any Student-t has tails heavier than a Gaussian posterior’s, which keeps the weights bounded there (Geweke); a small ν widens them further for a posterior with a heavy side, and ν = 4 still has a finite variance. A judgment, not a measurement.
- MAX_
WIDENING - The most
sample_in_two’s pilot widens a block’s scale along any direction, in variance: 4, twice the standard deviation, so that a pilot ruled by one far draw can’t spread the main draws past use. A judgment, not a measurement. - MIN_
EFFECTIVE_ DRAWS - The fewest effective draws for a quantile to be read: 100. At 100, the 5% point’s Monte Carlo
error is about
√(0.05 · 0.95 / 100) / φ(1.645), 0.22 posterior standard deviations rounded up, past which an interval’s ends say more of the draws than of the flight. A judgment, not a measurement. - PROBABILITIES
- The probabilities
Posterior::quantilesreads: the 90% interval’s ends and the median.
Functions§
- sample
- The posterior whose log density, up to a constant, is
log_target, bydrawsweighted draws aroundpeakwith the Laplace covariancecovariance. - sample_
in_ two samplein two blocks, adapted on a pilot: for a posterior whose other unknowns follow some of them along a ridge the Laplace approximation can’t bend to, and which may reach further than its curvature says.