Skip to main content

hpr_flightdata/
fluctus.rs

1//! Silicdyne's Fluctus flight computer's logs, as its Fluctus Control Center (FCC) exports a
2//! flight: a `sep=;` line, then a semicolon-separated table with each column's unit in its name.
3//!
4//! The Fluctus records its black box at each tick of its flight software, 50 times a second, from
5//! a second before the launch it detects (*Fluctus – Full Documentation*, v1.7b, Silicdyne, 2025,
6//! <https://silicdyne.net/resources/docs/fluctus_fulldoc_v1_7b_EN.pdf>, read 2026-10-13, p26); the
7//! FCC's BlackBox Viewer exports it to a `.csv` file "which can be opened in any spreadsheet
8//! program, such as Excel" (p58). The first line is `sep=;`, the hint that tells a spreadsheet the
9//! separator; the next names the columns, a unit in brackets after most; each row below is one
10//! tick:
11//!
12//! ```text
13//! sep=;
14//! time (ms);deltaTime (ms);status;baro-altitude (m);dedrck-v-speed (m/s);angle (deg);…;gpsSats
15//! 61500;20;3;0;0;3,5;0;0,0;9,8;0;0,0;21;8400;1;1;0;0;0;0;12,345678;-23,456789;150;1;8
16//! ```
17//!
18//! **Recognition.** Within the file's first 16 lines, the first that isn't blank is either `sep=`
19//! and one separator character (`;`, `,`, a tab or `|`), followed by the header on the next line
20//! that isn't blank, or the header itself, then separated by semicolons. The header's first name
21//! is `time`, and it names `deltaTime` and `baro-altitude` too: those are the Fluctus's own names
22//! (manual p27), and no other layout among the project's sample logs has them. Names are compared
23//! with their bracketed unit taken off and without regard to case. An Eggtimer's or an RRC3's
24//! semicolon-separated file has neither name.
25//!
26//! **Numbers.** The FCC's export has a *Decimal point delimiter* setting, as spreadsheets in
27//! different regions read a `.csv` file differently (p58): the sample logs write `12,5` for 12.5. When the separator isn't a comma, a comma in a cell is read as the decimal mark, and a
28//! point as well; a cell with both, or with two commas, isn't a number (`table::number`). The
29//! files bear this out: their latitudes are written `12,345678`, six digits after the comma,
30//! which as a thousands separator would put the logger millions of degrees from the equator, and
31//! their clock, past 1,000 ms on every row, is written with no thousands separator, so `1,234` is
32//! 1.234, never 1,234. When `sep=,` names the comma, numbers take a point only.
33//!
34//! **Columns.** A name is split into the name and the unit in the brackets at its end, as
35//! `baro-altitude (m)`. The name says what the column holds and the unit how to read it; the
36//! units are those of the header, read without regard to case or spaces:
37//!
38//! | Column | Read as | From |
39//! |---|---|---|
40//! | `time` | the clock, the logger's own: its zero isn't stated, and the rows start about a second before the launch | the header (`ms`); manual p27 ("Frame time") |
41//! | `deltaTime` | not a channel: checked against the clock's step, and a note counts the rows where it differs | the header (`ms`); manual p27 ("Time since last tick") |
42//! | `status` | [`Quantity::Number`], and an event at each change, named as the manual names the code (below) | manual p23 |
43//! | `baro-altitude` | [`Quantity::AltitudeAboveGround`]: the primary track | the header (`m`); manual p27 ("Baro altitude from origin") |
44//! | `dedrck-v-speed` | [`Quantity::Speed`], up positive: the vertical speed by dead reckoning, the accelerometers' and gyroscope's integral | the header (`m/s`); manual p23, p27 |
45//! | `angle` | [`Quantity::Angle`]: the tilt from vertical on the rail and in the descent, from the launch axis in the climb | the header (`deg`); manual p24 |
46//! | `roll-rate` | [`Quantity::AngularRate`] | the header (`deg/s`); manual p27 |
47//! | `vert-accel` | [`Quantity::Acceleration`], vertical | the header (`m/s2`); manual p27 |
48//! | `accel` | [`Quantity::Acceleration`]: the magnitude of the acceleration the accelerometers feel, 1 g at rest | the header (`m/s2`); manual p24 |
49//! | `dedrck-alti` | [`Quantity::InertialAltitude`] | the header (`m`); manual p27 |
50//! | `baro-speed` | [`Quantity::Speed`], up positive: the primary track's vertical speed | the header (`m/s`); manual p23 ("barometric altitude derivative") |
51//! | `amb-temp` | [`Quantity::Temperature`]: the board's sensor, which reads the bay | the header (`deg c`); manual p27 |
52//! | `batt-voltage` | [`Quantity::Voltage`]: the battery's | the header (`mV`); manual p27 |
53//! | `P1-state`, `P2-state`, `P3-state` | [`Quantity::Number`], and an event when an output fires (below) | manual p23 |
54//! | `analog1`, `analog2` | [`Quantity::Voltage`] | the header (`mV`); manual p27 |
55//! | `inFreefall` | [`Quantity::Number`], 1 or 0 | manual p27 ("Rocket in free fall (bool)") |
56//! | `gpsLat`, `gpsLng` | [`Quantity::Latitude`], [`Quantity::Longitude`], when the values bear out degrees (below) | manual p38; a note |
57//! | `gpsAltMSL` | [`Quantity::GnssAltitude`] in meters, when the barometric altitude bears it out (below); otherwise kept as written | manual p27 ("GPS altitude (MSL)"); a note |
58//! | `gpsState` | [`Quantity::Number`] as written: the manual gives no codes; Silicdyne's changelog says that from firmware 2.2 it holds the satellite count, the fix status before it moved elsewhere (<https://silicdyne.net/changelog-fluctus/>, update 2.2, read 2026-10-13) | |
59//! | `gpsSats` | [`Quantity::Number`]: the satellites the GPS uses | manual p27 |
60//!
61//! The units the header may state: `ms`, `s` and `us` for a time; `m` and `ft`; `m/s`, `ft/s`
62//! and `km/h`; `m/s2` (or `m/s²`, `m/s^2`) and `ft/s2`; `deg` (or `°`) and `rad`; `deg/s` and
63//! `rad/s`; `deg c` and `deg f` (or `°c`, `°f`) and `k`; `mV` and `V`; `Pa`, `hPa`, `mbar` and
64//! `kPa`. A column the reader knows whose unit is another, or is of the wrong kind, such as an
65//! angle in `m/s`, is kept as written, [`Quantity::Unrecognized`], with a note; so is one the
66//! manual gives a unit and the header writes none for. A column the reader doesn't know is read
67//! by its unit where the unit can mean one quantity only, a speed, an acceleration, an angle, a
68//! rate of turn, a temperature, a voltage or a pressure; a length, a time, no unit or one the
69//! reader doesn't know keeps it as written, with a note. The clock must be in a unit of time and
70//! the barometric altitude in one of length: a file without either is refused.
71//!
72//! *The altitude's zero.* The manual gives `baro-altitude` "from origin" (p27) and doesn't say
73//! where that is; as of 2026-10-13, in all 6 Fluctus files among the project's sample logs, it
74//! reads 0 on the first rows, on the pad before launch, so the logger zeroes it on the pad. It is
75//! written in whole meters there, though the manual gives the barometer a resolution of 0.1 m
76//! (p18), so the format's resolution is 1 m
77//! ([`LogFormat::altitude_resolution_m`](crate::log::LogFormat::altitude_resolution_m)).
78//!
79//! *The position in degrees.* No unit is in the header for `gpsLat` and `gpsLng`; the manual's
80//! picture of the FCC's *Locate* tab shows the position in decimal degrees to six places (p38), as
81//! the file writes it. Each is read in degrees if it holds at least one reading and every reading
82//! lies within ±90° for the latitude or ±180° for the longitude; otherwise it is kept as written,
83//! with a note.
84//!
85//! *The GPS altitude in meters.* No unit is in the header for `gpsAltMSL`, nor in the manual (p27),
86//! and Silicdyne's changelog says only that the firmware logs it from update 1.6. It is read in
87//! meters only if the barometric altitude bears it out, at the GPS's fresh fixes: the rows where
88//! the GPS altitude differs from its last reading before, the first included. A reading that
89//! repeats the last, as the receiver holds its last fix while it has none, and a gap are left out.
90//! Through those fixes a straight line is drawn by least squares, the GPS altitude against the
91//! barometric; in meters it rises 1 for each meter, in feet 3.28. The GPS altitude is read in
92//! meters where all four hold:
93//!
94//! - there are at least 10 fresh fixes, more than the two that fix a line exactly, so that the
95//!   scatter about it is measured too;
96//! - they span at least 120 m of the barometric altitude. The manual's ±3 m for the GPS (p18) is
97//!   its horizontal position; a GPS receiver's vertical error is larger, the U.S. government's
98//!   standard for GPS giving 13 m against 8 m horizontal, 95% of the time, 1.6 times (*GPS
99//!   Standard Positioning Service Performance Standard*, 5th ed., 2020, Table 3.8-3, p58). Allowing
100//!   twice the horizontal, 6 m, two readings that far off in opposite directions at the span's ends
101//!   tilt the slope by 12 m over the span: over 120 m, the 10% margin below;
102//! - they scatter about the line, RMS, by at most 10% of that span. A receiver that lags far
103//!   behind the flight traces a loop rather than a line, and the slope of a loop doesn't tell the
104//!   unit: a feet GPS lagging by a 10 s filter can come out near 1 (a test pins one at 3 times the
105//!   allowed scatter);
106//! - the slope lies within 10% of 1, far from the 3.28 of feet.
107//!
108//! Otherwise it is kept as written, [`Quantity::Unrecognized`], with a note saying which failed.
109//! A GPS that loses its fix in the climb and regains it low in the descent, the fixes spanning
110//! little of the flight, is kept: its highest reading's rise above the pad can match the apogee in
111//! feet by chance, but the span or the slope gives it away. As of 2026-10-13, none of the 6 sample
112//! files passes: in each, the GPS altitude lags the barometric one by hundreds of meters in the
113//! descent, and every slope is below 0.75.
114//!
115//! **Events.** Each change of `status` is an event named as the manual names the code it changes
116//! to (p23): 0 `IDLE`, 1 `ARMED`, 2 `COUNTDOWN ENGAGED`, 3 `WAITING FOR LAUNCH`, 4 `ASCENT` (the
117//! launch detected), 5 `DESCENT` (the apogee detected), 6 `TOUCHDOWN`; another code is named
118//! `status` and its number. A pyro output's state is 0 for no continuity, 1 for continuity and 10
119//! for fired (p23): each change to 10 is an event, the column's name and `ENABLED`, as `P1-state
120//! ENABLED`. A row whose cell is a gap changes nothing.
121//!
122//! **Placeholders.** A cell of 2,147,483,647, the largest signed 32-bit number, or of −2,147,483,648,
123//! the smallest, isn't a reading: as of 2026-10-13, in one of the 6 sample files, both of the
124//! dead-reckoning columns hold 2,147,483,647 from a few rows after the launch to the end, where
125//! their readings would run to many kilometers a second. Each such cell is a gap, counted in a note.
126//!
127//! **Rows.** Every row must have as many cells as the header has names; a row of another width, a
128//! clock cell that isn't a number, or a time that doesn't come after the row above's is refused,
129//! naming its line. Any other cell that is empty or isn't a number, such as `nan`, is a gap,
130//! counted per column in a note. A reading its unit's conversion carries past the largest number
131//! an `f64` holds, as `1e308` in `deg f` or `1.7e308` in `g`, is refused, naming its line
132//! ([ADR-258][adr-258] §3). Blank lines are skipped; lines may end in CR, LF or both, and a
133//! byte-order mark is dropped. A note counts the steps of the clock longer than its commonest. A
134//! header of more than [`MAX_COLUMNS`] names, or naming a column twice, is refused.
135//!
136//! [adr-258]: https://github.com/nrdptel/fusionspace-eridanus/blob/main/docs/decisions/0258-aerospacenus-board-and-a-phone-apps-csv-before-the-rest.md
137
138use std::collections::{HashMap, HashSet};
139
140use crate::error::LogError;
141use crate::log::{Channel, Event, FlightLog, LogFormat, Quantity, Stated};
142use crate::table::{self, Notes, numbered_lines, short};
143
144/// The format's name in messages.
145const FORMAT: &str = "Silicdyne Fluctus CSV";
146
147/// The step between two altitudes the format can tell apart near the ground, m
148/// ([`crate::log::LogFormat::altitude_resolution_m`]): `baro-altitude` is written in whole meters
149/// in every one of the 6 Fluctus files among the project's sample logs as of 2026-10-13, where it
150/// steps by 1 m near the pad, though the manual gives the barometer a resolution of 0.1 m (p18).
151pub(crate) const ALTITUDE_STEP_M: f64 = 1.0;
152
153/// The most names a header may have: the Fluctus writes 24 (manual p27).
154pub const MAX_COLUMNS: usize = 256;
155
156/// How many lines from the file's start the `sep=` line and the header may lie within.
157const HEAD_LINES: usize = 16;
158
159/// How far the slope of the GPS altitude against the barometric altitude, at the GPS's fresh
160/// fixes, may lie from 1 for the GPS altitude to be read in meters; in feet it would be 3.28
161/// (module docs, *The GPS altitude in meters*).
162const GPS_SLOPE_MARGIN: f64 = 0.1;
163
164/// The least span of the barometric altitude, m, across the GPS's fresh fixes for the slope to be
165/// measured: two readings each 6 m off, the vertical error allowed for, in opposite directions at
166/// the span's ends tilt the slope by 12 m over it, which over 120 m is the 10% margin (module
167/// docs, *The GPS altitude in meters*).
168const GPS_SPAN_FLOOR_M: f64 = 120.0;
169
170/// The fewest fresh fixes the slope is measured from: more than the two that fix a line exactly,
171/// so that the scatter about it is measured too.
172const GPS_MIN_FIXES: usize = 10;
173
174/// The most the fresh fixes may scatter about their line, RMS, as a share of the barometric
175/// altitude's span across them: a receiver that lags far behind the flight traces a loop, not a
176/// line, and the slope of a loop doesn't tell the unit (module docs, *The GPS altitude in
177/// meters*).
178const GPS_SCATTER_SHARE: f64 = 0.1;
179
180/// How far, s, the `deltaTime` may lie from the clock's step for the two to agree: a microsecond,
181/// far below the millisecond both are written in.
182const STEP_TOLERANCE_S: f64 = 1e-6;
183
184/// The manual's names for the flight sequencer's codes, 0 to 6 (p23).
185const STATUS_NAMES: [&str; 7] = [
186    "IDLE",
187    "ARMED",
188    "COUNTDOWN ENGAGED",
189    "WAITING FOR LAUNCH",
190    "ASCENT",
191    "DESCENT",
192    "TOUCHDOWN",
193];
194
195/// A pyro output's state when it fires (manual p23).
196const PYRO_FIRED: f64 = 10.0;
197
198/// What a unit measures.
199#[derive(Debug, Clone, Copy, PartialEq, Eq)]
200enum Dimension {
201    Time,
202    Length,
203    Speed,
204    Acceleration,
205    Angle,
206    AngularRate,
207    Temperature,
208    Voltage,
209    Pressure,
210}
211
212/// A unit the header may state: what it measures, and the value in SI (or degrees) as
213/// `written × multiply / divide + add`. Dividing keeps a thousandth exact in the record, as
214/// 1143.492 s for 1143492 ms.
215#[derive(Debug, Clone, Copy, PartialEq)]
216struct Unit {
217    dimension: Dimension,
218    multiply: f64,
219    divide: f64,
220    add: f64,
221}
222
223impl Unit {
224    const fn new(dimension: Dimension, multiply: f64, divide: f64, add: f64) -> Self {
225        Self {
226            dimension,
227            multiply,
228            divide,
229            add,
230        }
231    }
232
233    /// The value in SI (or degrees); infinite for a value too large for an `f64` once
234    /// converted, as `1e308` in `deg f`, where the conversion multiplies by 5 before dividing by 9.
235    fn convert(self, written: f64) -> f64 {
236        written * self.multiply / self.divide + self.add
237    }
238
239    /// The value in SI (or degrees), a value the conversion makes infinite refused on its line
240    /// as a value too large for its unit ([ADR-258][adr-258] §3).
241    ///
242    /// [adr-258]: https://github.com/nrdptel/fusionspace-eridanus/blob/main/docs/decisions/0258-aerospacenus-board-and-a-phone-apps-csv-before-the-rest.md
243    fn convert_on(self, written: f64, line: usize, column: &str) -> Result<f64, LogError> {
244        table::finite(self.convert(written), FORMAT, line, column)
245    }
246}
247
248/// One foot, m.
249const FOOT_M: f64 = 0.3048;
250
251/// Standard gravity, m/s² (CGPM 1901).
252const G0_M_S2: f64 = 9.806_65;
253
254/// A radian, degrees.
255const RADIAN_DEG: f64 = 180.0 / std::f64::consts::PI;
256
257/// The units the reader knows, in lower case with spaces taken out (module docs, *Columns*).
258const UNITS: [(&str, Unit); 33] = [
259    ("ms", Unit::new(Dimension::Time, 1.0, 1000.0, 0.0)),
260    ("s", Unit::new(Dimension::Time, 1.0, 1.0, 0.0)),
261    ("us", Unit::new(Dimension::Time, 1.0, 1e6, 0.0)),
262    ("µs", Unit::new(Dimension::Time, 1.0, 1e6, 0.0)),
263    ("m", Unit::new(Dimension::Length, 1.0, 1.0, 0.0)),
264    ("ft", Unit::new(Dimension::Length, FOOT_M, 1.0, 0.0)),
265    ("m/s", Unit::new(Dimension::Speed, 1.0, 1.0, 0.0)),
266    ("ft/s", Unit::new(Dimension::Speed, FOOT_M, 1.0, 0.0)),
267    ("km/h", Unit::new(Dimension::Speed, 1.0, 3.6, 0.0)),
268    ("m/s2", Unit::new(Dimension::Acceleration, 1.0, 1.0, 0.0)),
269    ("m/s²", Unit::new(Dimension::Acceleration, 1.0, 1.0, 0.0)),
270    ("m/s^2", Unit::new(Dimension::Acceleration, 1.0, 1.0, 0.0)),
271    (
272        "ft/s2",
273        Unit::new(Dimension::Acceleration, FOOT_M, 1.0, 0.0),
274    ),
275    (
276        "ft/s²",
277        Unit::new(Dimension::Acceleration, FOOT_M, 1.0, 0.0),
278    ),
279    ("deg", Unit::new(Dimension::Angle, 1.0, 1.0, 0.0)),
280    ("°", Unit::new(Dimension::Angle, 1.0, 1.0, 0.0)),
281    ("rad", Unit::new(Dimension::Angle, RADIAN_DEG, 1.0, 0.0)),
282    ("deg/s", Unit::new(Dimension::AngularRate, 1.0, 1.0, 0.0)),
283    ("°/s", Unit::new(Dimension::AngularRate, 1.0, 1.0, 0.0)),
284    (
285        "rad/s",
286        Unit::new(Dimension::AngularRate, RADIAN_DEG, 1.0, 0.0),
287    ),
288    ("degc", Unit::new(Dimension::Temperature, 1.0, 1.0, 273.15)),
289    ("°c", Unit::new(Dimension::Temperature, 1.0, 1.0, 273.15)),
290    (
291        "degf",
292        Unit::new(Dimension::Temperature, 5.0, 9.0, 273.15 - 32.0 * 5.0 / 9.0),
293    ),
294    (
295        "°f",
296        Unit::new(Dimension::Temperature, 5.0, 9.0, 273.15 - 32.0 * 5.0 / 9.0),
297    ),
298    ("k", Unit::new(Dimension::Temperature, 1.0, 1.0, 0.0)),
299    ("mv", Unit::new(Dimension::Voltage, 1.0, 1000.0, 0.0)),
300    ("v", Unit::new(Dimension::Voltage, 1.0, 1.0, 0.0)),
301    ("pa", Unit::new(Dimension::Pressure, 1.0, 1.0, 0.0)),
302    ("hpa", Unit::new(Dimension::Pressure, 100.0, 1.0, 0.0)),
303    ("mbar", Unit::new(Dimension::Pressure, 100.0, 1.0, 0.0)),
304    ("kpa", Unit::new(Dimension::Pressure, 1000.0, 1.0, 0.0)),
305    ("g", Unit::new(Dimension::Acceleration, G0_M_S2, 1.0, 0.0)),
306    ("μs", Unit::new(Dimension::Time, 1.0, 1e6, 0.0)),
307];
308
309/// A unit as the header writes it, if the reader knows it.
310fn unit_of(written: &str) -> Option<Unit> {
311    let key: String = written
312        .chars()
313        .filter(|c| !c.is_whitespace())
314        .flat_map(char::to_lowercase)
315        .collect();
316    UNITS
317        .iter()
318        .find(|(name, _)| *name == key)
319        .map(|(_, unit)| *unit)
320}
321
322/// What a known column is to the record beside its channel.
323#[derive(Debug, Clone, Copy, PartialEq, Eq)]
324enum Role {
325    None,
326    Altitude,
327    VerticalSpeed,
328    Temperature,
329    Battery,
330}
331
332/// What a header column holds.
333#[derive(Debug, Clone, Copy, PartialEq, Eq)]
334enum Kind {
335    /// `time`: the clock.
336    Clock,
337    /// `deltaTime`: the clock's step, checked.
338    Step,
339    /// `status`: the flight sequencer's code.
340    Status,
341    /// A pyro output's state.
342    Pyro,
343    /// A count, flag or code without a unit.
344    Number,
345    /// A reading whose unit the header states.
346    Reading(Quantity, Dimension, Role),
347    Latitude,
348    Longitude,
349    GpsAltitude,
350    /// A column the reader doesn't know: read by its unit, if that can mean one quantity only.
351    Unknown,
352}
353
354/// The names the reader knows, in lower case, and what each holds (module docs, *Columns*).
355const KNOWN: [(&str, Kind); 23] = [
356    ("time", Kind::Clock),
357    ("deltatime", Kind::Step),
358    ("status", Kind::Status),
359    (
360        "baro-altitude",
361        Kind::Reading(
362            Quantity::AltitudeAboveGround,
363            Dimension::Length,
364            Role::Altitude,
365        ),
366    ),
367    (
368        "dedrck-v-speed",
369        Kind::Reading(Quantity::Speed, Dimension::Speed, Role::None),
370    ),
371    (
372        "angle",
373        Kind::Reading(Quantity::Angle, Dimension::Angle, Role::None),
374    ),
375    (
376        "roll-rate",
377        Kind::Reading(Quantity::AngularRate, Dimension::AngularRate, Role::None),
378    ),
379    (
380        "vert-accel",
381        Kind::Reading(Quantity::Acceleration, Dimension::Acceleration, Role::None),
382    ),
383    (
384        "accel",
385        Kind::Reading(Quantity::Acceleration, Dimension::Acceleration, Role::None),
386    ),
387    (
388        "dedrck-alti",
389        Kind::Reading(Quantity::InertialAltitude, Dimension::Length, Role::None),
390    ),
391    (
392        "baro-speed",
393        Kind::Reading(Quantity::Speed, Dimension::Speed, Role::VerticalSpeed),
394    ),
395    (
396        "amb-temp",
397        Kind::Reading(
398            Quantity::Temperature,
399            Dimension::Temperature,
400            Role::Temperature,
401        ),
402    ),
403    (
404        "batt-voltage",
405        Kind::Reading(Quantity::Voltage, Dimension::Voltage, Role::Battery),
406    ),
407    ("p1-state", Kind::Pyro),
408    ("p2-state", Kind::Pyro),
409    ("p3-state", Kind::Pyro),
410    (
411        "analog1",
412        Kind::Reading(Quantity::Voltage, Dimension::Voltage, Role::None),
413    ),
414    (
415        "analog2",
416        Kind::Reading(Quantity::Voltage, Dimension::Voltage, Role::None),
417    ),
418    ("infreefall", Kind::Number),
419    ("gpslat", Kind::Latitude),
420    ("gpslng", Kind::Longitude),
421    ("gpsstate", Kind::Number),
422    ("gpssats", Kind::Number),
423];
424
425/// What a column named so holds: `gpsAltMSL`, or the manual's `gpsAlt`, is the GPS altitude.
426fn kind_of(name: &str) -> Kind {
427    if name.eq_ignore_ascii_case("gpsaltmsl") || name.eq_ignore_ascii_case("gpsalt") {
428        return Kind::GpsAltitude;
429    }
430    KNOWN
431        .iter()
432        .find(|(known, _)| name.eq_ignore_ascii_case(known))
433        .map_or(Kind::Unknown, |(_, kind)| *kind)
434}
435
436/// A header cell's name and the unit in the brackets at its end, both trimmed: `baro-altitude
437/// (m)` is `baro-altitude` and `m`.
438fn split_name(cell: &str) -> (&str, Option<&str>) {
439    let cell = cell.trim();
440    if let Some(inner) = cell.strip_suffix(')')
441        && let Some(open) = inner.rfind('(')
442    {
443        let name = inner[..open].trim();
444        if !name.is_empty() {
445            return (name, Some(inner[open + 1..].trim()));
446        }
447    }
448    (cell, None)
449}
450
451/// The header: its line number, its text and the separator.
452struct Header<'a> {
453    line: usize,
454    text: &'a str,
455    separator: char,
456}
457
458/// The header, if the file is in this layout (module docs, *Recognition*).
459fn find_header(text: &str) -> Option<Header<'_>> {
460    let mut lines = numbered_lines(text)
461        .take(HEAD_LINES)
462        .filter(|(_, line)| !line.trim().is_empty());
463    let (first_number, first) = lines.next()?;
464    let first = first.trim_start_matches([' ', '\t']);
465    let declared = first
466        .get(..4)
467        .filter(|prefix| prefix.eq_ignore_ascii_case("sep="))
468        .and_then(|_| first.get(4..));
469    let header = match declared {
470        Some(rest) => {
471            // One character, and nothing after it but spaces.
472            let mut chars = rest.trim_end_matches(' ').chars();
473            let separator = chars.next()?;
474            if chars.next().is_some() || !matches!(separator, ';' | ',' | '\t' | '|') {
475                return None;
476            }
477            let (line, text) = lines.next()?;
478            Header {
479                line,
480                text,
481                separator,
482            }
483        }
484        None => Header {
485            line: first_number,
486            text: first,
487            separator: ';',
488        },
489    };
490    is_header(header.text, header.separator).then_some(header)
491}
492
493/// Whether a line is this layout's header: `time` first, and `deltaTime` and `baro-altitude`
494/// among the names. The cells are looked at one by one, never collected, so a long line costs no
495/// memory.
496fn is_header(line: &str, separator: char) -> bool {
497    let mut names = line.split(separator).map(|cell| split_name(cell).0);
498    if !names
499        .next()
500        .is_some_and(|first| first.eq_ignore_ascii_case("time"))
501    {
502        return false;
503    }
504    let (mut step, mut altitude) = (false, false);
505    for name in names {
506        step |= name.eq_ignore_ascii_case("deltatime");
507        altitude |= name.eq_ignore_ascii_case("baro-altitude");
508        if step && altitude {
509            return true;
510        }
511    }
512    false
513}
514
515/// Whether the text is in this reader's layout, judged from its first lines, never the file's
516/// name.
517pub fn recognizes(text: &str) -> bool {
518    find_header(text).is_some()
519}
520
521/// A header column.
522struct Column {
523    /// The name as written, unit and all, as the channel keeps it.
524    written: String,
525    kind: Kind,
526    /// The unit the header states, if the reader knows it.
527    unit: Option<Unit>,
528    /// The unit as the header writes it, if it writes one.
529    unit_written: Option<String>,
530    /// Cells that are empty or not numbers, and the first of them.
531    gaps: usize,
532    gap_example: Option<String>,
533    /// Cells holding a 32-bit placeholder, and the first one's line.
534    placeholders: usize,
535    placeholder_line: usize,
536}
537
538/// The header's columns (module docs, *Columns*).
539fn columns(header: &Header<'_>) -> Result<Vec<Column>, LogError> {
540    let mut columns: Vec<Column> = Vec::new();
541    let mut seen: HashSet<String> = HashSet::new();
542    for (index, cell) in header.text.split(header.separator).enumerate() {
543        if index >= MAX_COLUMNS {
544            return Err(syntax(
545                header.line,
546                format!("the header names more than {MAX_COLUMNS} columns"),
547            ));
548        }
549        let written = cell.trim();
550        let (name, unit_written) = split_name(written);
551        if written.is_empty() {
552            return Err(syntax(
553                header.line,
554                format!("the header's column {} has no name", index + 1),
555            ));
556        }
557        if !seen.insert(name.to_ascii_lowercase()) {
558            return Err(syntax(
559                header.line,
560                format!("the header names {:?} twice", short(name)),
561            ));
562        }
563        columns.push(Column {
564            written: written.to_owned(),
565            kind: kind_of(name),
566            unit: unit_written.and_then(unit_of),
567            unit_written: unit_written.map(str::to_owned),
568            gaps: 0,
569            gap_example: None,
570            placeholders: 0,
571            placeholder_line: 0,
572        });
573    }
574    // The header was recognized by its first name, `time`, and its `baro-altitude`.
575    let unit_error = |what: &str, wanted: &str| LogError::Unit {
576        format: FORMAT,
577        line: header.line,
578        message: format!("{what} states no unit of {wanted} this reader knows"),
579    };
580    if columns
581        .first()
582        .and_then(|clock| clock.unit)
583        .is_none_or(|unit| unit.dimension != Dimension::Time)
584    {
585        return Err(unit_error("the clock, `time`,", "time"));
586    }
587    let altitude_in_length = columns.iter().any(|column| {
588        matches!(column.kind, Kind::Reading(_, _, Role::Altitude))
589            && column
590                .unit
591                .is_some_and(|unit| unit.dimension == Dimension::Length)
592    });
593    if !altitude_in_length {
594        return Err(unit_error("`baro-altitude`", "length"));
595    }
596    Ok(columns)
597}
598
599/// Whether a value is one of the 32-bit placeholders (module docs, *Placeholders*).
600fn is_placeholder(value: f64) -> bool {
601    value == f64::from(i32::MAX) || value == f64::from(i32::MIN)
602}
603
604/// The rows read: the clock as written, and each column's values as written, gaps `NaN`.
605struct Rows {
606    lines: Vec<usize>,
607    clock: Vec<f64>,
608    values: Vec<Vec<f64>>,
609}
610
611/// Reads the data rows (module docs, *Rows*).
612fn read_rows<'a>(
613    lines: impl Iterator<Item = (usize, &'a str)>,
614    columns: &mut [Column],
615    separator: char,
616) -> Result<Rows, LogError> {
617    let width = columns.len();
618    let decimal_comma = separator != ',';
619    let mut rows = Rows {
620        lines: Vec::new(),
621        clock: Vec::new(),
622        values: columns.iter().map(|_| Vec::new()).collect(),
623    };
624    for (number, line) in lines {
625        if line.trim().is_empty() {
626            continue;
627        }
628        // One more than the header names, so that a wider row is told apart.
629        let cells: Vec<&str> = line
630            .split(separator)
631            .map(str::trim)
632            .take(width + 1)
633            .collect();
634        if cells.len() != width {
635            return Err(syntax(
636                number,
637                format!(
638                    "the row has {} cells, where the header names {width}",
639                    if cells.len() > width {
640                        format!("more than {width}")
641                    } else {
642                        cells.len().to_string()
643                    }
644                ),
645            ));
646        }
647        let clock_cell = cells.first().copied().unwrap_or_default();
648        let Some(time) = table::number(clock_cell, decimal_comma) else {
649            return Err(syntax(
650                number,
651                format!("the clock's cell {:?} isn't a number", short(clock_cell)),
652            ));
653        };
654        if let Some(&last) = rows.clock.last()
655            && time <= last
656        {
657            return Err(syntax(
658                number,
659                format!("the time {time} doesn't come after the one above it, {last}"),
660            ));
661        }
662        for (index, (column, cell)) in columns.iter_mut().zip(&cells).enumerate().skip(1) {
663            let value = match table::number(cell, decimal_comma) {
664                Some(value) if is_placeholder(value) => {
665                    if column.placeholders == 0 {
666                        column.placeholder_line = number;
667                    }
668                    column.placeholders += 1;
669                    f64::NAN
670                }
671                Some(value) => value,
672                None => {
673                    column.gaps += 1;
674                    if column.gap_example.is_none() {
675                        column.gap_example = Some(short(cell));
676                    }
677                    f64::NAN
678                }
679            };
680            if let Some(store) = rows.values.get_mut(index) {
681                store.push(value);
682            }
683        }
684        rows.lines.push(number);
685        rows.clock.push(time);
686    }
687    Ok(rows)
688}
689
690fn syntax(line: usize, message: String) -> LogError {
691    LogError::Syntax {
692        format: FORMAT,
693        line,
694        message,
695    }
696}
697
698/// Reads a log in this layout.
699///
700/// # Errors
701///
702/// [`LogError::NotThisFormat`] if the file's first lines aren't this layout's (module docs,
703/// *Recognition*); [`LogError::Unit`] if the clock isn't in a unit of time or `baro-altitude` in
704/// one of length; [`LogError::Syntax`] for a header of more than [`MAX_COLUMNS`] names, naming a
705/// column twice or leaving a name empty, a row of the wrong width, a clock cell that isn't a
706/// number or doesn't increase, or a reading its unit's conversion makes infinite, naming the
707/// line; [`LogError::NoData`] if no row follows the header.
708pub fn read(text: &str) -> Result<FlightLog, LogError> {
709    let Some(header) = find_header(text) else {
710        return Err(LogError::NotThisFormat {
711            format: FORMAT,
712            message: format!(
713                "within its first {HEAD_LINES} lines there is no Fluctus header, a `sep=` line \
714                 if any, then names starting `time` and naming `deltaTime` and `baro-altitude`"
715            ),
716        });
717    };
718    let mut columns = columns(&header)?;
719    let rows = read_rows(
720        numbered_lines(text).skip_while(|(number, _)| *number <= header.line),
721        &mut columns,
722        header.separator,
723    )?;
724    if rows.clock.is_empty() {
725        return Err(LogError::NoData { format: FORMAT });
726    }
727    build(&columns, &rows)
728}
729
730/// What the reader does with a column, decided from its kind, its unit and, for the position and
731/// the GPS altitude, its values.
732#[derive(Debug, Clone, Copy)]
733enum Reading {
734    /// A channel of this quantity, converted so.
735    Channel(Quantity, Unit),
736    /// Kept as written, [`Quantity::Unrecognized`]; `true` once a note on the evidence says why.
737    Kept(bool),
738    /// `deltaTime`, checked against the clock.
739    Step(Unit),
740}
741
742/// No conversion.
743const AS_WRITTEN: Unit = Unit::new(Dimension::Length, 1.0, 1.0, 0.0);
744
745/// The record from the rows read; a reading its unit's conversion makes infinite is refused on
746/// its line.
747fn build(columns: &[Column], rows: &Rows) -> Result<FlightLog, LogError> {
748    let clock = columns.first();
749    let clock_name = clock.map_or("time", |clock| clock.written.as_str());
750    let clock_unit = clock.and_then(|clock| clock.unit).unwrap_or(AS_WRITTEN);
751    let time_s = convert_column(clock_unit, &rows.clock, &rows.lines, clock_name)?;
752    let mut log = FlightLog {
753        format: LogFormat::FluctusCsv,
754        logger: "Silicdyne Fluctus".to_owned(),
755        serial_number: None,
756        firmware: None,
757        flight_number: None,
758        stated: Stated::default(),
759        time_s: Vec::new(),
760        altitude_m: Vec::new(),
761        vertical_speed_m_s: None,
762        temperature_k: None,
763        battery_v: None,
764        channels: Vec::new(),
765        events: Vec::new(),
766        notes: Vec::new(),
767    };
768    // The notes on columns kept as written, at most one a column, and few.
769    let mut kept = Notes::default();
770    // The barometric altitude, m, which the GPS altitude is measured against.
771    let altitude_m: Vec<f64> = match columns
772        .iter()
773        .zip(&rows.values)
774        .find(|(column, _)| matches!(column.kind, Kind::Reading(_, _, Role::Altitude)))
775    {
776        Some((column, values)) => convert_column(
777            column.unit.unwrap_or(AS_WRITTEN),
778            values,
779            &rows.lines,
780            &column.written,
781        )?,
782        None => Vec::new(),
783    };
784    let mut events: Vec<(usize, Event)> = Vec::new();
785
786    for (index, column) in columns.iter().enumerate().skip(1) {
787        let Some(values) = rows.values.get(index) else {
788            continue;
789        };
790        let name = short(&column.written);
791        let reading = reading_of(column, values, &altitude_m, &mut log.notes, &mut kept);
792        let (quantity, unit) = match reading {
793            Reading::Step(unit) => {
794                step_check(column, unit, values, &time_s, rows, &mut log.notes);
795                continue;
796            }
797            Reading::Channel(quantity, unit) => (quantity, unit),
798            Reading::Kept(_) => (Quantity::Unrecognized, AS_WRITTEN),
799        };
800        match column.kind {
801            Kind::Status => events.extend(changes(values, |code| Some(status_name(code)), &time_s)),
802            Kind::Pyro => events.extend(changes(
803                values,
804                |state| {
805                    (state == PYRO_FIRED)
806                        .then(|| format!("{} ENABLED", split_name(&column.written).0))
807                },
808                &time_s,
809            )),
810            _ => {}
811        }
812        if column.gaps > 0 {
813            log.notes.push(format!(
814                "the column `{name}` holds {} that {} empty or not a number, such as {:?}: they \
815                 are gaps",
816                count(column.gaps, "cell"),
817                if column.gaps == 1 { "is" } else { "are" },
818                column.gap_example.as_deref().unwrap_or_default()
819            ));
820        }
821        if column.placeholders > 0 {
822            log.notes.push(format!(
823                "the column `{name}` writes 2147483647 or -2147483648, the ends of a signed 32-bit \
824                 number, on {} from line {}: no reading, so they are gaps",
825                count(column.placeholders, "row"),
826                column.placeholder_line
827            ));
828        }
829        let converted = convert_column(unit, values, &rows.lines, &column.written)?;
830        if let Kind::Reading(_, _, role) = column.kind
831            && quantity != Quantity::Unrecognized
832        {
833            match role {
834                Role::Altitude if log.altitude_m.is_empty() => log.altitude_m = converted.clone(),
835                Role::VerticalSpeed if log.vertical_speed_m_s.is_none() => {
836                    log.vertical_speed_m_s = Some(converted.clone());
837                }
838                Role::Temperature if log.temperature_k.is_none() => {
839                    log.temperature_k = Some(converted.clone());
840                }
841                Role::Battery if log.battery_v.is_none() => {
842                    log.battery_v = Some(converted.clone());
843                }
844                _ => {}
845            }
846        }
847        log.channels.push(Channel {
848            column: column.written.clone(),
849            quantity,
850            time_s: time_s.clone(),
851            values: converted,
852        });
853    }
854    // In time order; a status change before a pyro event on the same row, as the file's columns.
855    events.sort_by_key(|(row, _)| *row);
856    log.events = events.into_iter().map(|(_, event)| event).collect();
857    clock_steps(&time_s, &rows.lines, &mut log.notes);
858    log.notes.extend(kept.into_vec());
859    log.time_s = time_s;
860    Ok(log)
861}
862
863/// A column's values converted from its unit, each with its row's line: a value the conversion
864/// makes infinite, as `1.7e308` in `hPa` or `g`, is refused on its line ([ADR-258][adr-258] §3).
865///
866/// [adr-258]: https://github.com/nrdptel/fusionspace-eridanus/blob/main/docs/decisions/0258-aerospacenus-board-and-a-phone-apps-csv-before-the-rest.md
867fn convert_column(
868    unit: Unit,
869    values: &[f64],
870    lines: &[usize],
871    column: &str,
872) -> Result<Vec<f64>, LogError> {
873    values
874        .iter()
875        .zip(lines)
876        .map(|(value, line)| unit.convert_on(*value, *line, column))
877        .collect()
878}
879
880/// The note on a column kept as written, saying why.
881fn kept_note(column: &Column) -> String {
882    let name = short(&column.written);
883    let why = match (&column.unit_written, column.unit, column.kind) {
884        (None, _, Kind::Unknown) => {
885            "this reader doesn't know the column, and the header states no unit".to_owned()
886        }
887        (None, _, _) => "the header states no unit for it".to_owned(),
888        (Some(unit), None, _) => {
889            format!("its unit, {:?}, isn't one this reader knows", short(unit))
890        }
891        (Some(_), Some(unit), Kind::Unknown) => format!(
892            "this reader doesn't know the column, and its unit, a {}, could be more than one \
893             quantity",
894            dimension_word(unit.dimension)
895        ),
896        (Some(_), Some(unit), _) => format!(
897            "its unit is a {}, not what the column holds",
898            dimension_word(unit.dimension)
899        ),
900    };
901    format!("`{name}` kept as written: {why}")
902}
903
904fn dimension_word(dimension: Dimension) -> &'static str {
905    match dimension {
906        Dimension::Time => "time",
907        Dimension::Length => "length",
908        Dimension::Speed => "speed",
909        Dimension::Acceleration => "acceleration",
910        Dimension::Angle => "angle",
911        Dimension::AngularRate => "rate of turn",
912        Dimension::Temperature => "temperature",
913        Dimension::Voltage => "voltage",
914        Dimension::Pressure => "pressure",
915    }
916}
917
918/// How a column is read (module docs, *Columns*).
919fn reading_of(
920    column: &Column,
921    values: &[f64],
922    altitude_m: &[f64],
923    notes: &mut Vec<String>,
924    kept: &mut Notes,
925) -> Reading {
926    let reading = decided(column, values, altitude_m, notes);
927    if let Reading::Kept(false) = reading {
928        kept.push(kept_note(column));
929    }
930    reading
931}
932
933/// How a column is read, and the notes on the decisions made from evidence; a column kept as
934/// written for its unit alone is left for [`reading_of`] to note.
935fn decided(
936    column: &Column,
937    values: &[f64],
938    altitude_m: &[f64],
939    notes: &mut Vec<String>,
940) -> Reading {
941    let stated = |dimension: Dimension| column.unit.filter(|unit| unit.dimension == dimension);
942    let kept = Reading::Kept(false);
943    match column.kind {
944        Kind::Clock => kept,
945        Kind::Step => stated(Dimension::Time).map_or(kept, Reading::Step),
946        Kind::Status | Kind::Pyro | Kind::Number => {
947            if column.unit_written.is_some() {
948                kept
949            } else {
950                Reading::Channel(Quantity::Number, AS_WRITTEN)
951            }
952        }
953        Kind::Reading(quantity, dimension, _) => {
954            stated(dimension).map_or(kept, |unit| Reading::Channel(quantity, unit))
955        }
956        Kind::Latitude | Kind::Longitude => {
957            let (quantity, limit) = if column.kind == Kind::Latitude {
958                (Quantity::Latitude, 90.0)
959            } else {
960                (Quantity::Longitude, 180.0)
961            };
962            if column.unit_written.is_some() {
963                return stated(Dimension::Angle)
964                    .map_or(kept, |unit| Reading::Channel(quantity, unit));
965            }
966            match degrees(values, limit) {
967                Degrees::NoReading => {
968                    notes.push(format!(
969                        "`{}` kept as written, in a unit no reading bears out: the header states \
970                         none, and the column holds no reading to check against ±{limit}°",
971                        short(&column.written)
972                    ));
973                    Reading::Kept(true)
974                }
975                Degrees::Within => {
976                    notes.push(format!(
977                        "`{}` read in degrees, from the manual and the values. The header states \
978                         no unit. The manual's picture of the Locate tab shows the position in \
979                         decimal degrees (p38). Every reading lies within ±{limit}°",
980                        short(&column.written)
981                    ));
982                    Reading::Channel(quantity, AS_WRITTEN)
983                }
984                Degrees::Beyond(beyond) => {
985                    notes.push(format!(
986                        "`{}` kept as written, in a unit the values don't bear out. The header \
987                         states none. A reading of {beyond} lies beyond ±{limit}°, so it \
988                         isn't in degrees",
989                        short(&column.written)
990                    ));
991                    Reading::Kept(true)
992                }
993            }
994        }
995        Kind::GpsAltitude => {
996            if column.unit_written.is_some() {
997                let reading = stated(Dimension::Length)
998                    .map_or(kept, |unit| Reading::Channel(Quantity::GnssAltitude, unit));
999                return reading;
1000            }
1001            match gps_check(values, altitude_m) {
1002                GpsCheck::Meters(fit) => {
1003                    notes.push(format!(
1004                        "`{}` read in meters, from the barometric altitude. The header and the \
1005                         manual state no unit. At its {} fresh fixes it rises {:.2} for each meter \
1006                         of the barometric altitude, within {:.0}% of 1. In feet it would be \
1007                         3.28. The fresh fixes span {} m of the barometric altitude \
1008                         and scatter {} about that line",
1009                        short(&column.written),
1010                        fit.fixes,
1011                        fit.slope,
1012                        GPS_SLOPE_MARGIN * 100.0,
1013                        rounded(fit.span_m),
1014                        rounded(fit.scatter)
1015                    ));
1016                    Reading::Channel(Quantity::GnssAltitude, AS_WRITTEN)
1017                }
1018                GpsCheck::Undecided(why) => {
1019                    notes.push(format!(
1020                        "`{}` kept as written, in a unit no source states: {why}",
1021                        short(&column.written)
1022                    ));
1023                    Reading::Kept(true)
1024                }
1025            }
1026        }
1027        Kind::Unknown => match column.unit {
1028            Some(unit) => {
1029                let quantity = match unit.dimension {
1030                    Dimension::Speed => Quantity::Speed,
1031                    Dimension::Acceleration => Quantity::Acceleration,
1032                    Dimension::Angle => Quantity::Angle,
1033                    Dimension::AngularRate => Quantity::AngularRate,
1034                    Dimension::Temperature => Quantity::Temperature,
1035                    Dimension::Voltage => Quantity::Voltage,
1036                    Dimension::Pressure => Quantity::Pressure,
1037                    Dimension::Time | Dimension::Length => return kept,
1038                };
1039                notes.push(format!(
1040                    "the column `{}` isn't one this reader knows: it is read as a {}, as its unit \
1041                     says",
1042                    short(&column.written),
1043                    dimension_word(unit.dimension)
1044                ));
1045                Reading::Channel(quantity, unit)
1046            }
1047            None => kept,
1048        },
1049    }
1050}
1051
1052/// A count and its noun, plural but for one: `1 row`, `2 rows`.
1053fn count(number: usize, noun: &str) -> String {
1054    if number == 1 {
1055        format!("1 {noun}")
1056    } else {
1057        format!("{number} {noun}s")
1058    }
1059}
1060
1061/// A number as a note writes it: to the meter, as the file writes its altitudes.
1062fn rounded(value: f64) -> String {
1063    format!("{value:.0}")
1064}
1065
1066/// What the position's check found (module docs, *The position in degrees*).
1067#[derive(Debug, PartialEq)]
1068enum Degrees {
1069    /// At least one reading, and every one within ±`limit`.
1070    Within,
1071    /// The first reading beyond ±`limit`.
1072    Beyond(f64),
1073    /// No reading at all: every cell a gap.
1074    NoReading,
1075}
1076
1077/// Whether every reading, gaps aside, lies within ±`limit`; a column of gaps alone bears out
1078/// nothing.
1079fn degrees(values: &[f64], limit: f64) -> Degrees {
1080    if !values.iter().any(|value| value.is_finite()) {
1081        return Degrees::NoReading;
1082    }
1083    match values.iter().find(|value| value.abs() > limit) {
1084        Some(beyond) => Degrees::Beyond(*beyond),
1085        None => Degrees::Within,
1086    }
1087}
1088
1089/// The GPS altitude's straight line against the barometric altitude, by least squares at the
1090/// GPS's fresh fixes.
1091#[derive(Debug, Clone, Copy, PartialEq)]
1092struct GpsFit {
1093    /// The fresh fixes the line is drawn through.
1094    fixes: usize,
1095    /// The barometric altitude's span across them, m.
1096    span_m: f64,
1097    /// The GPS altitude's change for each meter of the barometric altitude's.
1098    slope: f64,
1099    /// The fixes' RMS distance from the line, in the GPS altitude's own unit, on `fixes − 2`
1100    /// degrees of freedom.
1101    scatter: f64,
1102}
1103
1104/// What the GPS altitude's check found (module docs, *The GPS altitude in meters*).
1105#[derive(Debug, PartialEq)]
1106enum GpsCheck {
1107    Meters(GpsFit),
1108    /// Why the check can't decide.
1109    Undecided(String),
1110}
1111
1112/// The (barometric altitude, GPS altitude) pairs at the GPS's fresh fixes: the rows where the GPS
1113/// altitude differs from its last reading before, the first reading included. A reading that
1114/// repeats the last, as the receiver holds its last fix while it has none, or a gap, is left out;
1115/// so is a row whose barometric altitude is a gap.
1116fn fresh_fixes(gps: &[f64], altitude_m: &[f64]) -> Vec<(f64, f64)> {
1117    let mut fixes = Vec::new();
1118    let mut last: Option<f64> = None;
1119    for (&reading, &altitude) in gps.iter().zip(altitude_m) {
1120        if !reading.is_finite() {
1121            continue;
1122        }
1123        if last != Some(reading) && altitude.is_finite() {
1124            fixes.push((altitude, reading));
1125        }
1126        last = Some(reading);
1127    }
1128    fixes
1129}
1130
1131/// The least-squares line through the fixes, about their means; `None` for fewer than three
1132/// fixes or none apart in the barometric altitude.
1133fn gps_fit(fixes: &[(f64, f64)]) -> Option<GpsFit> {
1134    if fixes.len() < 3 {
1135        return None;
1136    }
1137    let n = fixes.len() as f64;
1138    let mean_x = fixes.iter().map(|(x, _)| x).sum::<f64>() / n;
1139    let mean_y = fixes.iter().map(|(_, y)| y).sum::<f64>() / n;
1140    let sxx: f64 = fixes.iter().map(|(x, _)| (x - mean_x).powi(2)).sum();
1141    let sxy: f64 = fixes.iter().map(|(x, y)| (x - mean_x) * (y - mean_y)).sum();
1142    let apart = sxx > 0.0;
1143    if !apart {
1144        return None;
1145    }
1146    let slope = sxy / sxx;
1147    let residuals: f64 = fixes
1148        .iter()
1149        .map(|(x, y)| (y - mean_y - slope * (x - mean_x)).powi(2))
1150        .sum();
1151    let (low, high) = fixes
1152        .iter()
1153        .fold((f64::INFINITY, f64::NEG_INFINITY), |(low, high), (x, _)| {
1154            (low.min(*x), high.max(*x))
1155        });
1156    Some(GpsFit {
1157        fixes: fixes.len(),
1158        span_m: high - low,
1159        slope,
1160        scatter: (residuals / (n - 2.0)).sqrt(),
1161    })
1162}
1163
1164/// Measures the GPS altitude against the barometric altitude, m, at the GPS's fresh fixes: read
1165/// in meters only where the fixes are enough, span enough of the flight, lie along a line and
1166/// that line's slope is within [`GPS_SLOPE_MARGIN`] of 1. Every comparison fails closed, so a NaN
1167/// decides nothing.
1168fn gps_check(gps: &[f64], altitude_m: &[f64]) -> GpsCheck {
1169    let fixes = fresh_fixes(gps, altitude_m);
1170    if fixes.is_empty() {
1171        return GpsCheck::Undecided(
1172            "the file has no GPS altitude or no barometric altitude to measure it against"
1173                .to_owned(),
1174        );
1175    }
1176    if fixes.len() < GPS_MIN_FIXES {
1177        return GpsCheck::Undecided(format!(
1178            "it changes on {} where the barometric altitude is read. That is fewer \
1179             than the {GPS_MIN_FIXES} fresh fixes the check needs to measure a line and the \
1180             scatter about it",
1181            count(fixes.len(), "row")
1182        ));
1183    }
1184    let span = |fit: Option<GpsFit>| fit.map_or(0.0, |fit| fit.span_m);
1185    let fit = gps_fit(&fixes);
1186    let fit = match fit {
1187        Some(fit) if fit.span_m >= GPS_SPAN_FLOOR_M => fit,
1188        _ => {
1189            return GpsCheck::Undecided(format!(
1190                "its {} fresh fixes span {} m of the barometric altitude, less than the {} m the \
1191                 check needs to tell meters from feet",
1192                fixes.len(),
1193                rounded(span(fit)),
1194                rounded(GPS_SPAN_FLOOR_M)
1195            ));
1196        }
1197    };
1198    let on_a_line = fit.scatter <= GPS_SCATTER_SHARE * fit.span_m;
1199    if !on_a_line {
1200        return GpsCheck::Undecided(format!(
1201            "its {} fresh fixes scatter {} about their line, more than {:.0}% of the {} m of \
1202             the barometric altitude they span. They trace a loop, as a receiver lagging far \
1203             behind the flight does. The slope of a loop doesn't tell the unit",
1204            fit.fixes,
1205            rounded(fit.scatter),
1206            GPS_SCATTER_SHARE * 100.0,
1207            rounded(fit.span_m)
1208        ));
1209    }
1210    let near_1 = (fit.slope - 1.0).abs() <= GPS_SLOPE_MARGIN;
1211    if !near_1 {
1212        return GpsCheck::Undecided(format!(
1213            "at its {} fresh fixes it rises {:.2} for each meter of the barometric altitude, \
1214             over {} m of it. That is not within {:.0}% of 1 as in meters. In feet it would be \
1215             3.28",
1216            fit.fixes,
1217            fit.slope,
1218            rounded(fit.span_m),
1219            GPS_SLOPE_MARGIN * 100.0
1220        ));
1221    }
1222    GpsCheck::Meters(fit)
1223}
1224
1225/// The manual's name for a flight sequencer's code (p23), or `status` and the number.
1226fn status_name(code: f64) -> String {
1227    let named = (0_u8..=6)
1228        .find(|&k| f64::from(k) == code)
1229        .and_then(|k| STATUS_NAMES.get(usize::from(k)));
1230    named.map_or_else(|| format!("status {code}"), |name| (*name).to_owned())
1231}
1232
1233/// An event at each row where the values change to one that `name` names, with the row's index;
1234/// a gap changes nothing.
1235fn changes(
1236    values: &[f64],
1237    name: impl Fn(f64) -> Option<String>,
1238    time_s: &[f64],
1239) -> Vec<(usize, Event)> {
1240    let mut events = Vec::new();
1241    let mut last: Option<f64> = None;
1242    for (row, (value, time)) in values.iter().zip(time_s).enumerate() {
1243        if value.is_nan() {
1244            continue;
1245        }
1246        if let Some(before) = last
1247            && before != *value
1248            && let Some(name) = name(*value)
1249        {
1250            events.push((
1251                row,
1252                Event {
1253                    time_s: *time,
1254                    name,
1255                },
1256            ));
1257        }
1258        last = Some(*value);
1259    }
1260    events
1261}
1262
1263/// Checks `deltaTime` against the clock's step; a note counts the rows where they differ.
1264fn step_check(
1265    column: &Column,
1266    unit: Unit,
1267    values: &[f64],
1268    time_s: &[f64],
1269    rows: &Rows,
1270    notes: &mut Vec<String>,
1271) {
1272    let mut differ = 0;
1273    let mut first_line = 0;
1274    for (row, value) in values.iter().enumerate().skip(1) {
1275        let (Some(now), Some(before)) = (time_s.get(row), time_s.get(row - 1)) else {
1276            continue;
1277        };
1278        if value.is_nan() {
1279            continue;
1280        }
1281        if (unit.convert(*value) - (now - before)).abs() > STEP_TOLERANCE_S {
1282            if differ == 0 {
1283                first_line = rows.lines.get(row).copied().unwrap_or_default();
1284            }
1285            differ += 1;
1286        }
1287    }
1288    if differ > 0 {
1289        notes.push(format!(
1290            "`{}` differs from the clock's step on {}, the first on line {first_line}: \
1291             the clock is kept, and `{0}` isn't",
1292            short(&column.written),
1293            count(differ, "row")
1294        ));
1295    }
1296    if column.gaps > 0 {
1297        notes.push(format!(
1298            "the column `{}` holds {} that {} empty or not a number: it is only checked against \
1299             the clock, so nothing is lost",
1300            short(&column.written),
1301            count(column.gaps, "cell"),
1302            if column.gaps == 1 { "is" } else { "are" }
1303        ));
1304    }
1305}
1306
1307/// A note counting the clock's steps longer than its commonest, if any.
1308fn clock_steps(time_s: &[f64], lines: &[usize], notes: &mut Vec<String>) {
1309    // Each step in whole microseconds, so that equal steps count together.
1310    #[allow(
1311        clippy::cast_possible_truncation,
1312        reason = "the clock is finite (a `nan` clock is refused when read); only a step of some 292,000 years would saturate"
1313    )]
1314    let steps: Vec<i64> = time_s
1315        .windows(2)
1316        .map(|pair| ((pair[1] - pair[0]) * 1e6).round() as i64)
1317        .collect();
1318    let mut counts: HashMap<i64, usize> = HashMap::new();
1319    for step in &steps {
1320        *counts.entry(*step).or_default() += 1;
1321    }
1322    let Some(commonest) = counts
1323        .iter()
1324        .max_by_key(|(step, count)| (**count, std::cmp::Reverse(**step)))
1325        .map(|(step, _)| *step)
1326    else {
1327        return;
1328    };
1329    let longer: Vec<(usize, i64)> = steps
1330        .iter()
1331        .enumerate()
1332        .filter(|(_, step)| **step > commonest)
1333        .map(|(index, step)| (index, *step))
1334        .collect();
1335    let Some(&(longest_at, longest)) = longer.iter().max_by_key(|(_, step)| *step) else {
1336        return;
1337    };
1338    let in_ms = |microseconds: i64| format!("{} ms", microseconds as f64 / 1000.0);
1339    notes.push(format!(
1340        "the clock steps by {} on most rows, and by more {}: the longest, {}, before line {}; \
1341         the logger wrote no row between",
1342        in_ms(commonest),
1343        if longer.len() == 1 {
1344            "once".to_owned()
1345        } else {
1346            format!("{} times", longer.len())
1347        },
1348        in_ms(longest),
1349        lines.get(longest_at + 1).copied().unwrap_or_default()
1350    ));
1351}
1352
1353#[cfg(test)]
1354mod tests {
1355    use super::*;
1356
1357    const HEADER: &str = "time (ms);deltaTime (ms);status;baro-altitude (m);dedrck-v-speed \
1358                          (m/s);angle (deg);roll-rate (deg/s);vert-accel (m/s2);accel (m/s2);\
1359                          dedrck-alti (m);baro-speed (m/s);amb-temp (deg c);batt-voltage (mV);\
1360                          P1-state;P2-state;P3-state;analog1 (mV);analog2 (mV);inFreefall;gpsLat;\
1361                          gpsLng;gpsAltMSL;gpsState;gpsSats";
1362
1363    /// A row of the full layout: the time, the status, the altitude and the GPS altitude as
1364    /// given, the rest invented constants.
1365    fn row(time_ms: u32, status: u8, altitude_m: i32, gps_m: i32) -> String {
1366        format!(
1367            "{time_ms};20;{status};{altitude_m};0;3,5;0;0,0;9,8;0;0,0;21;8400;1;1;0;0;0;0;\
1368             12,345678;-23,456789;{gps_m};1;8"
1369        )
1370    }
1371
1372    /// A file of the full layout from (time, status, altitude, GPS altitude) rows.
1373    fn file(rows: &[(u32, u8, i32, i32)]) -> String {
1374        let mut text = format!("sep=;\n{HEADER}\n");
1375        for &(time, status, altitude, gps) in rows {
1376            text.push_str(&row(time, status, altitude, gps));
1377            text.push('\n');
1378        }
1379        text
1380    }
1381
1382    fn notes_starting<'a>(log: &'a FlightLog, start: &str) -> Vec<&'a String> {
1383        log.notes.iter().filter(|n| n.starts_with(start)).collect()
1384    }
1385
1386    fn channel<'a>(log: &'a FlightLog, column: &str) -> &'a Channel {
1387        log.channels
1388            .iter()
1389            .find(|c| c.column == column)
1390            .unwrap_or_else(|| panic!("no {column}: {:?}", log.channels))
1391    }
1392
1393    #[test]
1394    fn recognizes_the_sep_line_and_the_fluctus_names() {
1395        assert!(recognizes(&format!("sep=;\n{HEADER}\n")));
1396        assert!(recognizes(&format!(
1397            "\u{feff}\r\nSEP=;\r\n\r\n{HEADER}\r\n"
1398        )));
1399        // Without the hint, a semicolon-separated header of the Fluctus's names.
1400        assert!(recognizes(&format!("{HEADER}\n")));
1401        // The comma named as the separator.
1402        assert!(recognizes(&format!(
1403            "sep=,\n{}\n",
1404            HEADER.replace(';', ",")
1405        )));
1406        // Without `deltaTime`, without `baro-altitude`, or with another first name.
1407        assert!(!recognizes("sep=;\ntime (ms);status;baro-altitude (m)\n"));
1408        assert!(!recognizes(
1409            "sep=;\ntime (ms);deltaTime (ms);altitude (m)\n"
1410        ));
1411        assert!(!recognizes(
1412            "sep=;\nstatus;time (ms);deltaTime (ms);baro-altitude (m)\n"
1413        ));
1414        // A header split by another separator than the one the hint names.
1415        assert!(!recognizes(&format!("sep=,\n{HEADER}\n")));
1416        // A separator that could be part of a number, or more than one character.
1417        assert!(!recognizes("sep=.\ntime.deltaTime.baro-altitude\n"));
1418        assert!(!recognizes(&format!("sep=;;\n{HEADER}\n")));
1419        // The Eggtimer's and an RRC3's semicolon-separated files.
1420        assert!(!recognizes(
1421            "T;Alt;Veloc;FAlt;FVeloc;LDA;LowV;Apogee\n0;0;0;0;0;0;0;0\n"
1422        ));
1423        assert!(!recognizes(
1424            "Time;Altitude;Pressure;Velocity;Temperature;Events;Voltages\n0,00;0;1013,2;0\n"
1425        ));
1426        // The header too far down.
1427        let far = format!("{}sep=;\n{HEADER}\n", "x\n".repeat(HEAD_LINES));
1428        assert!(!recognizes(&far));
1429        assert!(!recognizes(""));
1430    }
1431
1432    #[test]
1433    fn names_split_into_name_and_unit() {
1434        assert_eq!(
1435            split_name(" baro-altitude (m) "),
1436            ("baro-altitude", Some("m"))
1437        );
1438        assert_eq!(split_name("amb-temp (deg c)"), ("amb-temp", Some("deg c")));
1439        assert_eq!(split_name("gpsLat"), ("gpsLat", None));
1440        assert_eq!(split_name("(m)"), ("(m)", None));
1441        assert_eq!(split_name("odd (a(b)"), ("odd (a", Some("b")));
1442    }
1443
1444    #[test]
1445    fn units_convert_to_si() {
1446        let cases = [
1447            ("ms", 1500.0, 1.5),
1448            ("s", 1.5, 1.5),
1449            ("us", 1.5e6, 1.5),
1450            ("m", 12.0, 12.0),
1451            ("ft", 100.0, 30.48),
1452            ("km/h", 36.0, 10.0),
1453            ("m/s2", 9.8, 9.8),
1454            ("M/S²", 9.8, 9.8),
1455            ("ft/s2", 10.0, 3.048),
1456            ("g", 1.0, 9.806_65),
1457            ("rad", std::f64::consts::PI, 180.0),
1458            ("deg c", 20.0, 293.15),
1459            ("°C", 20.0, 293.15),
1460            ("deg f", 212.0, 373.15),
1461            ("k", 300.0, 300.0),
1462            ("mV", 8400.0, 8.4),
1463            ("V", 8.4, 8.4),
1464            ("hPa", 1013.25, 101_325.0),
1465            ("kPa", 101.325, 101_325.0),
1466        ];
1467        for (written, value, si) in cases {
1468            let unit = unit_of(written).unwrap_or_else(|| panic!("{written}"));
1469            assert!((unit.convert(value) - si).abs() < 1e-9, "{written}");
1470        }
1471        // A thousandth stays exact: 1143.492, not 1143.4920000000002.
1472        assert_eq!(
1473            unit_of("ms").map(|u| u.convert(1_143_492.0)),
1474            Some(1143.492)
1475        );
1476        for unknown in ["furlongs", "c", "f", "", "mm"] {
1477            assert_eq!(unit_of(unknown), None, "{unknown}");
1478        }
1479    }
1480
1481    #[test]
1482    fn decimal_commas_are_read_when_the_separator_isnt_a_comma() {
1483        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);angle (deg)\n1000;20;0;1,234\n\
1484                    1020;20;1;1.5\n1040;20;2;1,000.5\n";
1485        let log = read(text).unwrap();
1486        let angle = channel(&log, "angle (deg)");
1487        assert_eq!(angle.values[..2], [1.234, 1.5]);
1488        assert!(angle.values[2].is_nan());
1489        // With the comma as the separator, a point only.
1490        let comma = "sep=,\ntime (ms),deltaTime (ms),baro-altitude (m),angle (deg)\n\
1491                     1000,20,0,1.25\n";
1492        let log = read(comma).unwrap();
1493        assert_eq!(channel(&log, "angle (deg)").values, [1.25]);
1494    }
1495
1496    #[test]
1497    fn every_column_of_the_layout_is_read_as_its_table_says() {
1498        let log = read(&file(&[(1000, 3, 0, 150), (1020, 4, 1, 150)])).unwrap();
1499        let quantities: Vec<(&str, Quantity)> = log
1500            .channels
1501            .iter()
1502            .map(|c| (c.column.as_str(), c.quantity))
1503            .collect();
1504        assert_eq!(
1505            quantities,
1506            [
1507                ("status", Quantity::Number),
1508                ("baro-altitude (m)", Quantity::AltitudeAboveGround),
1509                ("dedrck-v-speed (m/s)", Quantity::Speed),
1510                ("angle (deg)", Quantity::Angle),
1511                ("roll-rate (deg/s)", Quantity::AngularRate),
1512                ("vert-accel (m/s2)", Quantity::Acceleration),
1513                ("accel (m/s2)", Quantity::Acceleration),
1514                ("dedrck-alti (m)", Quantity::InertialAltitude),
1515                ("baro-speed (m/s)", Quantity::Speed),
1516                ("amb-temp (deg c)", Quantity::Temperature),
1517                ("batt-voltage (mV)", Quantity::Voltage),
1518                ("P1-state", Quantity::Number),
1519                ("P2-state", Quantity::Number),
1520                ("P3-state", Quantity::Number),
1521                ("analog1 (mV)", Quantity::Voltage),
1522                ("analog2 (mV)", Quantity::Voltage),
1523                ("inFreefall", Quantity::Number),
1524                ("gpsLat", Quantity::Latitude),
1525                ("gpsLng", Quantity::Longitude),
1526                // A GPS altitude that never changes can't settle its unit.
1527                ("gpsAltMSL", Quantity::Unrecognized),
1528                ("gpsState", Quantity::Number),
1529                ("gpsSats", Quantity::Number),
1530            ]
1531        );
1532        assert_eq!(log.time_s, [1.0, 1.02]);
1533        assert_eq!(log.altitude_m, [0.0, 1.0]);
1534        assert_eq!(log.temperature_k, Some(vec![294.15, 294.15]));
1535        assert_eq!(log.battery_v, Some(vec![8.4, 8.4]));
1536        assert_eq!(log.vertical_speed_m_s, Some(vec![0.0, 0.0]));
1537        assert_eq!(channel(&log, "gpsLat").values, [12.345678, 12.345678]);
1538        assert_eq!(
1539            notes_starting(
1540                &log,
1541                "`gpsLat` read in degrees, from the manual and the values. "
1542            )
1543            .len(),
1544            1
1545        );
1546        assert_eq!(
1547            notes_starting(
1548                &log,
1549                "`gpsLng` read in degrees, from the manual and the values. "
1550            )
1551            .len(),
1552            1
1553        );
1554        assert_eq!(
1555            notes_starting(
1556                &log,
1557                "`gpsAltMSL` kept as written, in a unit no source states: it changes on 1 row \
1558                 where the barometric altitude is read. That is fewer than the 10 fresh fixes"
1559            )
1560            .len(),
1561            1,
1562            "{:?}",
1563            log.notes
1564        );
1565    }
1566
1567    /// A flight of `rows` rows, a GPS fix on each, climbing to `apogee` m and back down in a
1568    /// straight line each way, the GPS altitude `150 + slope × altitude` and half a unit up and
1569    /// down on alternate rows, so that no two rows in a row repeat a reading.
1570    fn gps_line(slope: f64, apogee: f64, rows: usize) -> FlightLog {
1571        let mut text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);gpsAltMSL\n".to_owned();
1572        let last = (rows - 1) as f64;
1573        for row in 0..rows {
1574            let share = 1.0 - (2.0 * row as f64 / last - 1.0).abs();
1575            let altitude = apogee * share;
1576            let wobble = if row % 2 == 0 { 0.5 } else { -0.5 };
1577            let gps = 150.0 + slope * altitude + wobble;
1578            text.push_str(&format!(
1579                "{};1000;{altitude:.3};{gps:.3}\n",
1580                1000 * (row + 1)
1581            ));
1582        }
1583        read(&text).unwrap()
1584    }
1585
1586    fn gps_quantity(log: &FlightLog) -> Quantity {
1587        channel(log, "gpsAltMSL").quantity
1588    }
1589
1590    fn gps_note(log: &FlightLog) -> &String {
1591        let notes: Vec<&String> = log
1592            .notes
1593            .iter()
1594            .filter(|n| n.starts_with("`gpsAltMSL` "))
1595            .collect();
1596        assert_eq!(notes.len(), 1, "{:?}", log.notes);
1597        notes[0]
1598    }
1599
1600    #[test]
1601    fn the_gps_altitude_is_read_in_meters_only_where_its_slope_is_near_1() {
1602        for (slope, meters) in [
1603            (1.0, true),
1604            (0.92, true),
1605            (0.88, false),
1606            (1.08, true),
1607            (1.12, false),
1608            // In feet, the GPS altitude rises 3.28 for each meter.
1609            (1.0 / FOOT_M, false),
1610            (0.5, false),
1611        ] {
1612            let log = gps_line(slope, 300.0, 21);
1613            assert_eq!(
1614                gps_quantity(&log) == Quantity::GnssAltitude,
1615                meters,
1616                "slope {slope}: {:?}",
1617                log.notes
1618            );
1619            let start = if meters {
1620                "`gpsAltMSL` read in meters, from the barometric altitude. "
1621            } else {
1622                "`gpsAltMSL` kept as written, in a unit no source states: at its 21 fresh fixes \
1623                 it rises "
1624            };
1625            assert!(gps_note(&log).starts_with(start), "{:?}", log.notes);
1626        }
1627        let log = gps_line(1.0, 300.0, 21);
1628        assert!(
1629            gps_note(&log).contains(
1630                "At its 21 fresh fixes it rises 1.00 for each meter of the barometric altitude, \
1631                 within 10% of 1. In feet it would be 3.28. The fresh fixes span 300 m"
1632            ),
1633            "{:?}",
1634            log.notes
1635        );
1636        // Read as written, in meters: 150.5 m on the pad.
1637        assert_eq!(channel(&log, "gpsAltMSL").values[0], 150.5);
1638    }
1639
1640    #[test]
1641    fn the_gps_altitude_needs_120_m_of_span_and_10_fresh_fixes() {
1642        let at_floor = gps_line(1.0, 120.0, 21);
1643        assert_eq!(gps_quantity(&at_floor), Quantity::GnssAltitude);
1644        let below = gps_line(1.0, 119.0, 21);
1645        assert_eq!(gps_quantity(&below), Quantity::Unrecognized);
1646        assert!(
1647            gps_note(&below).contains(
1648                "its 21 fresh fixes span 119 m of the barometric altitude, less than the 120 m"
1649            ),
1650            "{:?}",
1651            below.notes
1652        );
1653        assert_eq!(
1654            gps_quantity(&gps_line(1.0, 300.0, 10)),
1655            Quantity::GnssAltitude
1656        );
1657        let few = gps_line(1.0, 300.0, 9);
1658        assert_eq!(gps_quantity(&few), Quantity::Unrecognized);
1659        assert!(
1660            gps_note(&few).contains("it changes on 9 rows where the barometric altitude is read"),
1661            "{:?}",
1662            few.notes
1663        );
1664    }
1665
1666    /// The reviewed failure: a GPS altitude in feet that loses its fix in the climb and regains
1667    /// it low in the descent, its highest fixed reading 295 ft (90 m) above the pad's against a
1668    /// 300 m apogee. Its rise alone is within 10% of the apogee; its fresh fixes span 90 m, too
1669    /// little to tell the unit, and the slope over them is 3.28.
1670    #[test]
1671    fn a_feet_gps_altitude_regaining_its_fix_late_is_never_read_in_meters() {
1672        let mut altitude = vec![0.0, 0.0, 100.0, 200.0, 280.0, 300.0, 290.0, 250.0, 200.0];
1673        let mut gps = vec![
1674            500.0, 501.0, 501.0, 501.0, 501.0, 501.0, 501.0, 501.0, 501.0,
1675        ];
1676        for (row, height_m) in (0..=9).rev().map(|step| f64::from(step) * 10.0).enumerate() {
1677            altitude.push(height_m);
1678            gps.push((500.0 + height_m / FOOT_M + if row % 2 == 0 { 0.4 } else { -0.4 }).round());
1679        }
1680        let check = gps_check(&gps, &altitude);
1681        let GpsCheck::Undecided(why) = check else {
1682            panic!("read in meters: {check:?}");
1683        };
1684        assert!(why.contains("span 90 m"), "{why}");
1685        // Over a wider span, the slope settles it.
1686        let fixes = fresh_fixes(&gps, &altitude);
1687        let fit = gps_fit(&fixes).unwrap();
1688        assert!((fit.slope - 1.0 / FOOT_M).abs() < 0.05, "{fit:?}");
1689    }
1690
1691    /// A receiver that holds its last fix while it has none writes the same reading on row after
1692    /// row: those rows, and gaps, aren't fixes. This GPS holds the pad's reading through the
1693    /// climb, then follows the descent in meters; drawn through every row, the line would bend
1694    /// toward the held reading.
1695    #[test]
1696    fn held_readings_and_gaps_are_not_fresh_fixes() {
1697        let mut altitude = Vec::new();
1698        let mut gps = Vec::new();
1699        for step in 0..=10 {
1700            altitude.push(f64::from(step) * 30.0);
1701            gps.push(if step == 5 { f64::NAN } else { 150.0 });
1702        }
1703        for step in (0..10).rev() {
1704            let height_m = f64::from(step) * 30.0 + 15.0;
1705            altitude.push(height_m);
1706            gps.push(150.0 + height_m + if step % 2 == 0 { 0.5 } else { -0.5 });
1707        }
1708        // A gap in the barometric altitude leaves its row out.
1709        altitude.push(f64::NAN);
1710        gps.push(100.0);
1711        let fixes = fresh_fixes(&gps, &altitude);
1712        assert_eq!(fixes.len(), 11, "{fixes:?}");
1713        assert!(
1714            matches!(gps_check(&gps, &altitude), GpsCheck::Meters(fit) if fit.fixes == 11),
1715            "{:?}",
1716            gps_check(&gps, &altitude)
1717        );
1718        // Every row, held readings included: the slope falls far from 1.
1719        let every: Vec<(f64, f64)> = altitude
1720            .iter()
1721            .zip(&gps)
1722            .filter(|(a, g)| a.is_finite() && g.is_finite())
1723            .map(|(a, g)| (*a, *g))
1724            .collect();
1725        let bent = gps_fit(&every).unwrap();
1726        assert!(bent.slope < 0.8, "{bent:?}");
1727    }
1728
1729    /// A GPS altitude in feet that lags the flight by a first-order filter of 10 s: its slope
1730    /// against the barometric altitude comes out near 1, but its fixes trace a loop that scatters
1731    /// far more than 10% of the span about the line, so it is kept.
1732    #[test]
1733    fn a_lagging_gps_altitude_whose_slope_is_near_1_is_kept_by_its_scatter() {
1734        let (step_s, tau_s) = (0.02, 10.0);
1735        let (mut time_s, mut lagged) = (0.0_f64, 0.0_f64);
1736        let (mut altitude, mut gps) = (Vec::new(), Vec::new());
1737        let mut next_fix_s = 0.0;
1738        while time_s < 53.0 {
1739            let height_m = if time_s < 3.0 {
1740                300.0 * (std::f64::consts::FRAC_PI_2 * time_s / 3.0).sin()
1741            } else {
1742                (300.0 - 15.0 * (time_s - 3.0)).max(0.0)
1743            };
1744            lagged += (height_m / FOOT_M - lagged) * step_s / tau_s;
1745            if time_s >= next_fix_s {
1746                altitude.push(height_m);
1747                gps.push(lagged);
1748                next_fix_s += 1.0;
1749            }
1750            time_s += step_s;
1751        }
1752        let fit = gps_fit(&fresh_fixes(&gps, &altitude)).unwrap();
1753        assert!(
1754            (fit.slope - 1.0).abs() <= GPS_SLOPE_MARGIN,
1755            "the slope alone would pass: {fit:?}"
1756        );
1757        assert!(
1758            fit.scatter > 3.0 * GPS_SCATTER_SHARE * fit.span_m,
1759            "{fit:?}"
1760        );
1761        let check = gps_check(&gps, &altitude);
1762        let GpsCheck::Undecided(why) = check else {
1763            panic!("read in meters: {check:?}");
1764        };
1765        assert!(why.contains("They trace a loop"), "{why}");
1766    }
1767
1768    #[test]
1769    fn no_gps_altitude_or_no_barometric_altitude_decides_nothing() {
1770        let nan = vec![f64::NAN; 20];
1771        let line: Vec<f64> = (0..20).map(|row| f64::from(row) * 20.0).collect();
1772        for (gps, altitude) in [(&nan, &line), (&line, &nan), (&nan, &nan)] {
1773            assert!(
1774                matches!(gps_check(gps, altitude), GpsCheck::Undecided(ref why)
1775                    if why.starts_with("the file has no GPS altitude")),
1776                "{:?}",
1777                gps_check(gps, altitude)
1778            );
1779        }
1780    }
1781
1782    #[test]
1783    fn a_gps_altitude_with_a_stated_unit_is_read_in_it() {
1784        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);gpsAltMSL (ft)\n\
1785                    1000;20;0;1000\n1020;20;5;1000\n";
1786        let log = read(text).unwrap();
1787        let gps = channel(&log, "gpsAltMSL (ft)");
1788        assert_eq!(gps.quantity, Quantity::GnssAltitude);
1789        assert!((gps.values[0] - 304.8).abs() < 1e-9);
1790        assert!(
1791            log.notes.iter().all(|n| !n.contains("gpsAltMSL")),
1792            "{:?}",
1793            log.notes
1794        );
1795    }
1796
1797    #[test]
1798    fn a_position_beyond_its_range_is_kept() {
1799        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);gpsLat;gpsLng\n\
1800                    1000;20;0;90;180\n1020;20;0;-90;-180\n";
1801        let log = read(text).unwrap();
1802        assert_eq!(channel(&log, "gpsLat").quantity, Quantity::Latitude);
1803        assert_eq!(channel(&log, "gpsLng").quantity, Quantity::Longitude);
1804        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);gpsLat;gpsLng\n\
1805                    1000;20;0;90,5;180\n1020;20;0;0;-180,5\n";
1806        let log = read(text).unwrap();
1807        assert_eq!(channel(&log, "gpsLat").quantity, Quantity::Unrecognized);
1808        assert_eq!(channel(&log, "gpsLng").quantity, Quantity::Unrecognized);
1809        assert_eq!(
1810            notes_starting(
1811                &log,
1812                "`gpsLat` kept as written, in a unit the values don't bear out. "
1813            )
1814            .len(),
1815            1
1816        );
1817        assert!(
1818            notes_starting(
1819                &log,
1820                "`gpsLng` kept as written, in a unit the values don't bear out. "
1821            )[0]
1822            .contains("-180.5")
1823        );
1824    }
1825
1826    #[test]
1827    fn units_of_the_wrong_kind_or_unknown_keep_a_column_as_written() {
1828        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (ft);angle (m/s);roll-rate \
1829                    (furlongs);amb-temp;status (s);pitot (kPa);mystery (m);other\n\
1830                    1000;20;100;1;2;3;4;5;6;7\n";
1831        let log = read(text).unwrap();
1832        assert!((log.altitude_m[0] - 30.48).abs() < 1e-9);
1833        let kept = |column: &str, why: &str| {
1834            assert_eq!(
1835                channel(&log, column).quantity,
1836                Quantity::Unrecognized,
1837                "{column}"
1838            );
1839            let note = format!("`{column}` kept as written: {why}");
1840            assert!(log.notes.contains(&note), "{note}: {:?}", log.notes);
1841        };
1842        kept(
1843            "angle (m/s)",
1844            "its unit is a speed, not what the column holds",
1845        );
1846        kept(
1847            "roll-rate (furlongs)",
1848            "its unit, \"furlongs\", isn't one this reader knows",
1849        );
1850        kept("amb-temp", "the header states no unit for it");
1851        kept(
1852            "status (s)",
1853            "its unit is a time, not what the column holds",
1854        );
1855        kept(
1856            "mystery (m)",
1857            "this reader doesn't know the column, and its unit, a length, could be more than one \
1858             quantity",
1859        );
1860        kept(
1861            "other",
1862            "this reader doesn't know the column, and the header states no unit",
1863        );
1864        // A column the reader doesn't know, read by a unit that means one quantity only.
1865        let pitot = channel(&log, "pitot (kPa)");
1866        assert_eq!(
1867            (pitot.quantity, pitot.values[0]),
1868            (Quantity::Pressure, 5000.0)
1869        );
1870        // An unknown temperature column isn't the record's temperature.
1871        assert_eq!(log.temperature_k, None);
1872    }
1873
1874    /// A position column of gaps alone bears out no unit: kept, with no claim about its readings.
1875    #[test]
1876    fn a_position_of_gaps_alone_is_kept() {
1877        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);gpsLat;gpsLng\n\
1878                    1000;20;0;nan;1,5\n1020;20;0;;2,5\n";
1879        let log = read(text).unwrap();
1880        assert_eq!(channel(&log, "gpsLat").quantity, Quantity::Unrecognized);
1881        assert_eq!(channel(&log, "gpsLng").quantity, Quantity::Longitude);
1882        assert_eq!(
1883            notes_starting(
1884                &log,
1885                "`gpsLat` kept as written, in a unit no reading bears out: the header states \
1886                 none, and the column holds no reading"
1887            )
1888            .len(),
1889            1,
1890            "{:?}",
1891            log.notes
1892        );
1893        assert!(
1894            log.notes
1895                .iter()
1896                .all(|n| !n.contains("every reading lies within ±90°"))
1897        );
1898        assert_eq!(degrees(&[f64::NAN, f64::NAN], 90.0), Degrees::NoReading);
1899        assert_eq!(degrees(&[], 90.0), Degrees::NoReading);
1900        assert_eq!(degrees(&[f64::NAN, 45.0], 90.0), Degrees::Within);
1901        assert_eq!(degrees(&[f64::NAN, -91.0], 90.0), Degrees::Beyond(-91.0));
1902    }
1903
1904    /// A reading a conversion carries past the largest `f64` is refused on its line, never an
1905    /// infinity nor a gap (ADR-258 §3): `deg f` multiplies by 5 before dividing by 9, `hPa`
1906    /// multiplies by 100 and `g` by 9.806 65.
1907    #[test]
1908    fn a_value_its_conversion_makes_infinite_is_refused() {
1909        for (unit, written) in [("deg f", 1e308), ("hPa", 1e307), ("kPa", -1e306)] {
1910            let converted = unit_of(unit).map(|u| u.convert(written));
1911            assert!(
1912                converted.is_some_and(f64::is_infinite),
1913                "{unit}: {converted:?}"
1914            );
1915        }
1916        let header = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);amb-temp (deg f);\
1917                      pitot (hPa);accel (g)\n";
1918        let cases = [
1919            ("1e308;1013;1", 3, "amb-temp (deg f)"),
1920            ("70;1.7e308;1", 3, "pitot (hPa)"),
1921            ("70;1013;-1.7e308", 3, "accel (g)"),
1922        ];
1923        for (cells, line, column) in cases {
1924            let text = format!("{header}1000;20;0;{cells}\n1020;20;0;70;1013;1\n");
1925            assert_eq!(
1926                read(&text).unwrap_err().to_string(),
1927                format!(
1928                    "Silicdyne Fluctus CSV line {line}: the column {column} holds a value too \
1929                     large for its unit's conversion to SI"
1930                )
1931            );
1932        }
1933        // On a later row, the refusal names that row's line.
1934        let text = format!("{header}1000;20;0;70;1013;1\n1020;20;0;70;1013;1.7e308\n");
1935        assert_eq!(
1936            read(&text).unwrap_err().to_string(),
1937            "Silicdyne Fluctus CSV line 4: the column accel (g) holds a value too large for its \
1938             unit's conversion to SI"
1939        );
1940    }
1941
1942    #[test]
1943    fn a_clock_or_altitude_without_its_unit_is_refused() {
1944        for (text, what) in [
1945            (
1946                "sep=;\ntime;deltaTime (ms);baro-altitude (m)\n1;20;0\n",
1947                "`time`",
1948            ),
1949            (
1950                "sep=;\ntime (m);deltaTime (ms);baro-altitude (m)\n1;20;0\n",
1951                "`time`",
1952            ),
1953            (
1954                "sep=;\ntime (ms);deltaTime (ms);baro-altitude (s)\n1;20;0\n",
1955                "`baro-altitude`",
1956            ),
1957            (
1958                "sep=;\ntime (ms);deltaTime (ms);baro-altitude\n1;20;0\n",
1959                "`baro-altitude`",
1960            ),
1961        ] {
1962            match read(text) {
1963                Err(LogError::Unit {
1964                    line: 2, message, ..
1965                }) => {
1966                    assert!(message.contains(what), "{message}");
1967                }
1968                other => panic!("{text}: {other:?}"),
1969            }
1970        }
1971    }
1972
1973    #[test]
1974    fn placeholders_and_cells_that_arent_numbers_are_gaps() {
1975        let text = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);dedrck-alti (m);angle \
1976                    (deg)\n1000;20;0;2147483647;nan\n1020;20;1;-2147483648;\n\
1977                    1040;20;2;2147483646;1,5\n";
1978        let log = read(text).unwrap();
1979        let inertial = channel(&log, "dedrck-alti (m)");
1980        assert!(inertial.values[0].is_nan() && inertial.values[1].is_nan());
1981        // One short of the placeholder is a reading.
1982        assert_eq!(inertial.values[2], 2_147_483_646.0);
1983        let angle = channel(&log, "angle (deg)");
1984        assert!(angle.values[0].is_nan() && angle.values[1].is_nan());
1985        assert_eq!(angle.values[2], 1.5);
1986        assert!(log.notes.contains(
1987            &"the column `dedrck-alti (m)` writes 2147483647 or -2147483648, the ends of a signed \
1988              32-bit number, on 2 rows from line 3: no reading, so they are gaps"
1989                .to_owned()
1990        ));
1991        assert!(
1992            log.notes.contains(
1993                &"the column `angle (deg)` holds 2 cells that are empty or not a number, such as \
1994              \"nan\": they are gaps"
1995                    .to_owned()
1996            )
1997        );
1998    }
1999
2000    #[test]
2001    fn status_changes_and_fired_outputs_are_events() {
2002        let text = "sep=;\ntime (ms);deltaTime (ms);status;baro-altitude (m);P1-state;P2-state\n\
2003                    1000;20;3;0;1;1\n1020;20;4;5;1;0\n1040;20;;9;10;1\n1060;20;5;9;10;10\n\
2004                    1080;20;6;0;1;0\n1100;20;9;0;10;0\n";
2005        let log = read(text).unwrap();
2006        let events: Vec<(f64, &str)> = log
2007            .events
2008            .iter()
2009            .map(|e| (e.time_s, e.name.as_str()))
2010            .collect();
2011        assert_eq!(
2012            events,
2013            [
2014                (1.02, "ASCENT"),
2015                (1.04, "P1-state ENABLED"),
2016                (1.06, "DESCENT"),
2017                (1.06, "P2-state ENABLED"),
2018                (1.08, "TOUCHDOWN"),
2019                (1.1, "status 9"),
2020                (1.1, "P1-state ENABLED"),
2021            ]
2022        );
2023        let names: Vec<String> = (0..=6).map(|code| status_name(f64::from(code))).collect();
2024        assert_eq!(names, STATUS_NAMES.map(str::to_owned));
2025        assert_eq!(status_name(2.5), "status 2.5");
2026        assert_eq!(status_name(-1.0), "status -1");
2027    }
2028
2029    #[test]
2030    fn delta_time_is_checked_against_the_clock() {
2031        let agreeing = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m)\n1000;20;0\n1020;20;0\n\
2032                        1088;68;0\n1108;20;0\n";
2033        let log = read(agreeing).unwrap();
2034        assert!(
2035            log.channels
2036                .iter()
2037                .all(|c| !c.column.starts_with("deltaTime"))
2038        );
2039        assert!(
2040            log.notes.iter().all(|n| !n.contains("differs")),
2041            "{:?}",
2042            log.notes
2043        );
2044        assert_eq!(
2045            log.notes,
2046            [
2047                "the clock steps by 20 ms on most rows, and by more once: the longest, 68 ms, before \
2048              line 5; the logger wrote no row between"
2049            ]
2050        );
2051        let differing = agreeing.replace("1088;68", "1088;20");
2052        let log = read(&differing).unwrap();
2053        assert!(log.notes.contains(
2054            &"`deltaTime (ms)` differs from the clock's step on 1 row, the first on line 5: the \
2055              clock is kept, and `deltaTime (ms)` isn't"
2056                .to_owned()
2057        ));
2058    }
2059
2060    #[test]
2061    fn malformed_files_are_refused_naming_the_line() {
2062        let head = "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m)\n";
2063        for (rows, line) in [
2064            ("1000;20\n", 3),
2065            ("1000;20;0;1\n", 3),
2066            ("1000;20;0\n1000;20;0\n", 4),
2067            ("1000;20;0\n990;20;0\n", 4),
2068            ("1000;20;0\nnan;20;0\n", 4),
2069            ("1000;20;0\n;20;0\n", 4),
2070        ] {
2071            match read(&format!("{head}{rows}")) {
2072                Err(LogError::Syntax { line: found, .. }) => assert_eq!(found, line, "{rows}"),
2073                other => panic!("{rows}: {other:?}"),
2074            }
2075        }
2076        assert_eq!(read(head), Err(LogError::NoData { format: FORMAT }));
2077        assert!(matches!(
2078            read("sep=;\ntime (ms);deltaTime (ms);baro-altitude (m);deltatime\n"),
2079            Err(LogError::Syntax { line: 2, .. })
2080        ));
2081        assert!(matches!(
2082            read("sep=;\ntime (ms);deltaTime (ms);;baro-altitude (m)\n"),
2083            Err(LogError::Syntax { line: 2, .. })
2084        ));
2085        let wide = format!(
2086            "sep=;\ntime (ms);deltaTime (ms);baro-altitude (m){}\n",
2087            (0..MAX_COLUMNS)
2088                .map(|i| format!(";c{i}"))
2089                .collect::<String>()
2090        );
2091        assert!(matches!(read(&wide), Err(LogError::Syntax { line: 2, .. })));
2092        assert!(matches!(
2093            read("Time,Altitude\n0,0\n"),
2094            Err(LogError::NotThisFormat { .. })
2095        ));
2096    }
2097
2098    #[test]
2099    fn crlf_blank_lines_and_a_byte_order_mark_are_read() {
2100        let text = "\u{feff}sep=;\r\ntime (ms);deltaTime (ms);baro-altitude (m)\r\n1000;20;0\r\n\
2101                    \r\n1020;20;3\r\n";
2102        let log = read(text).unwrap();
2103        assert_eq!(log.time_s, [1.0, 1.02]);
2104        assert_eq!(log.altitude_m, [0.0, 3.0]);
2105    }
2106
2107    #[test]
2108    fn the_resolution_is_the_whole_meter() {
2109        assert_eq!(ALTITUDE_STEP_M, 1.0);
2110        assert_eq!(LogFormat::FluctusCsv.altitude_resolution_m(), 1.0);
2111    }
2112}