Expand description
What a flight metric measures, tool by tool and version by version, and what becomes of a metric whose event never happened (M2.2d1).
A reference names a quantity with a word: maxvelocity, deploymentvelocity,
groundhitvelocity, optimumdelay. The same word can mean a different quantity in another
tool, or in another version of the same tool (Loft lesson L80). So a metric is compared
only through its Definition, looked up by definition for the Tool and version that
produced the reference. A version whose meaning has not been measured has no definition, and
its value is withheld rather than compared as if it meant what the newest version means.
OpenRocket 24.12’s definitions are measured, not read from its source (which is GPL):
validation/oracles/openrocket/flights.py flies every configuration of the public designs and
records each summary word beside the quantities of OpenRocket’s own time series it could mean.
The record is validation/fixtures/ork/openrocket-flights.json, and a test holds each
definition below to it on every complete flight (ADR-068).
A metric taken at an event that did not happen, such as the deployment speed of a flight whose
parachute never opened, has no value. OpenRocket writes NaN for it. Loft scored it as 0
(Loft lesson L81). compare withholds it when neither flight had the event, and fails
it when only one did; it is never scored.
use hpr_validate::flight_metrics::{
Event, Failure, FlightMetric, MetricOutcome, ReferenceReading, Side, Tool, Withheld, compare,
};
let openrocket = Tool::OpenRocket { version: "24.12".to_owned() };
// OpenRocket's Chute release example with a G40W-7: its last parachute opens at 14.2306 m/s.
let reading = ReferenceReading::Complete(Some(14.2306));
let outcome = compare(&openrocket, FlightMetric::DeploymentSpeed, reading, Some(14.0));
assert!((outcome.difference().unwrap() + 0.2306).abs() < 1e-9);
// No parachute opened in OpenRocket's flight (it wrote `NaN`), nor in hpr's: withheld.
let none = ReferenceReading::Complete(Some(f64::NAN));
assert_eq!(
compare(&openrocket, FlightMetric::DeploymentSpeed, none, None),
MetricOutcome::Withheld(Withheld::NoEventInEither { event: Event::Deployment })
);
// Only hpr's opened: a disagreement, which fails rather than being scored against 0.
assert_eq!(
compare(&openrocket, FlightMetric::DeploymentSpeed, none, Some(9.0)),
MetricOutcome::Failed(Failure::EventOnlyIn { event: Event::Deployment, side: Side::Measured })
);Enums§
- Definition
- What a metric measures, precisely enough to take it from a flight’s time series.
- Event
- An event in a flight that a metric is taken at.
- Failure
- Why a metric fails without being scored: the two flights disagree about what happened, or a value is broken.
- Flight
Metric - A summary quantity of one flight.
- Metric
Outcome - A metric compared, withheld, or failed. Never a zero standing in for a missing value.
- Occurrence
- Which occurrence of an event that can happen more than once.
- Point
- The point on the rocket whose height or speed a metric takes.
- Reference
Reading - A reference flight’s value for one metric.
- Side
- One of the two flights compared.
- Tool
- A tool that produced a reference flight, with its version as the tool writes it.
- Withheld
- Why a metric was not compared, when that is no fault of either flight.
Constants§
- OPENROCKET_
MEASURED - The OpenRocket version whose summary words
flights.pymeasured. - ROCKETPY_
MEASURED - The RocketPy version whose metrics hpr’s whole-flight cases compare against. Its definitions
come from ADR-021, which read what RocketPy computes, not from a probe like
flights.py.
Functions§
- compare
- Compares hpr’s
measuredvalue ofmetricwith areferencereading fromtool. - definition
- What
metricmeasures in a reference fromtool, orNonewhere that has not been measured for the tool’s version.