Skip to main content

hpr_flightdata/
entacore.rs

1//! Entacore's AIM flight computers' logs, as the AIM XTRA and AIM BASE software exports a flight
2//! with *Export to Excel…*: a CSV file (*AIM XTRA/BASE v4.x.x Manual*, Entacore, 2 February 2025,
3//! p38).
4//!
5//! The AIM XTRA samples its sensors at different rates: its barometer every 20 ms, its 100 g
6//! accelerometer, 16 g three-axis accelerometer and gyroscope every 10 ms, its magnetometer every
7//! 100 ms, its temperature every 500 ms and its GPS's position every 200 ms (manual p2), each
8//! written at a whole multiple of its period that the owner sets for each phase of the flight
9//! (p21). So the export gives each group of channels its own clock: "there are multiple time
10//! columns. Each time column applies to the columns to its right until another time column is
11//! reached" (p40). One header line names the columns; each row below holds the next sample of
12//! every group, so a row's cells for different groups were taken at different times, and a group
13//! that has run out leaves its cells empty:
14//!
15//! ```text
16//! time,Gyro X,Gyro Y,Gyro Z,time,pressure,Pressure MSL,Pressure AGL,time,acceleration,…
17//! -0.52,0.07,-0.14,0.21,-0.51,97700,302.87,0.17,-0.52,1.0039,…
18//! ```
19//!
20//! The AIM BASE ground station's software writes the same layout of the telemetry it received
21//! (p28), with fewer channels at slower, irregular rates (p21, p38) and the radio's `RSSI`.
22//!
23//! **Recognition.** The first line that isn't blank is comma-separated, starts with a `time`
24//! column, names at least two, and names at least one of the AIM's own channels, such as `Pressure
25//! AGL`, `lat. X accel.` or `eject. bat. voltage` (the reader's `MARKS`); names are compared
26//! trimmed, without trailing points and without regard to case. A Raven's export starts with
27//! `Time@…` columns and names none of them.
28//!
29//! **Channels.** Each name the reader knows, what it is read as, and where its unit comes from.
30//! The column keeps the file's name, trimmed. A header naming a column twice is refused:
31//!
32//! | Column | Quantity | Unit, and from where |
33//! |---|---|---|
34//! | `time` | the clock of the columns to its right (p40) | seconds or milliseconds, from `GPS Time of Week` or the step (below) |
35//! | `pressure` | [`Quantity::Pressure`] | Pa, kPa or inHg, from the values (below); the manual's sample export writes 100182 (Table 2, p40) |
36//! | `Pressure MSL` | [`Quantity::AltitudeMsl`] | m or ft, from the pressure (below): the software's altitude from "Standard Atmospheric table data" (p22) |
37//! | `Pressure AGL` | [`Quantity::AltitudeAboveGround`]: the primary track | as `Pressure MSL` |
38//! | `acceleration`, `lat. X accel.`, `lat. Y accel.` | [`Quantity::Acceleration`] | g, × 9.806 65 m/s² (standard gravity): the manual plots accelerations "in G's" (p23–24) |
39//! | `Gyro X`, `Gyro Y`, `Gyro Z` | [`Quantity::AngularRate`] | degrees per second (p24) |
40//! | `Mag X`, `Mag Y`, `Mag Z` | [`Quantity::Unrecognized`], as written | below |
41//! | `eject. bat. voltage`, `comp. bat. voltage`, `A Voltage` … `D Voltage` | [`Quantity::Voltage`] | volts, from the values (below) |
42//! | `temperature` | [`Quantity::Temperature`] | °C or °F, from the altitudes' unit and the manual (below) |
43//! | `lat`, `long` | [`Quantity::Latitude`], [`Quantity::Longitude`] | decimal degrees, as the manual's sample export writes them (Table 2, p40); a value beyond ±90° or ±180° is a gap |
44//! | `GPS MSL` | [`Quantity::GnssAltitude`] | the altitudes' unit, if it agrees with `Pressure MSL` (below) |
45//! | `GPS Time of Week` | [`Quantity::Unrecognized`], as written | milliseconds, where the clock is decided by it (below) |
46//! | `SVs` | [`Quantity::Number`] | the satellites used in the GPS's solution (p25) |
47//! | `RSSI` | [`Quantity::Number`], as written | dBm, the strength of the radio signal the AIM BASE received (p29), not an SI unit |
48//! | `launch detect`, `apogee detect`, `A cont.` … `D cont.`, `A on.` … `D on` | [`Quantity::Number`], 0 or 1 | flags; see Events |
49//!
50//! A column the reader doesn't know is kept as written, [`Quantity::Unrecognized`], with a note.
51//! Each unit gets a note on the record saying how it was read and on what evidence; a unit no
52//! evidence settles is kept as written, [`Quantity::Unrecognized`], and the note says why.
53//!
54//! *The clock.* The manual states no unit for `time`. `GPS Time of Week` counts milliseconds: its
55//! values stay below a week's 604,800,000, and the manual's sample export counts it up by 5,000
56//! over 5 of its `time` in seconds (Table 2, p40). The reader takes the median, over the
57//! channel's consecutive samples, of its rise over its clock's: within 10% of 1,000, `time` is in
58//! seconds; within 10% of 1, in milliseconds, and every clock is read × 0.001. As of 2026-10-13
59//! the median is 969.8 to 1,002.4 in the 16 AIM files among the project's sample logs (the BASE's
60//! clock, the time a packet came in, is the less regular). Without such a channel, the unit comes
61//! from the step, as the Eggtimer reader's does: the AIM writes each sensor every 10 to 500 ms
62//! (p2) times a whole multiple the owner sets (p21), so the fastest clock's median step reads
63//! 0.009 to 2 in seconds (the 10 ms period less 10%, to 2 s; 0.0097 to 0.773 in the sample
64//! files) and 9 to 2,000 in milliseconds. A step in neither range, or no clock with two samples,
65//! is refused ([`LogError::Unit`]). The time's zero is the software's: the start of the
66//! recording, or the time the owner chose to make zero, such as the launch (p33).
67//!
68//! *The pressure.* The manual's software shows the pressure in kPa or inHg (p25), but its sample
69//! export writes pascals (Table 2, p40). The reader takes the unit from the first value that is
70//! a launch site's pressure in one of them, on the pad, so a placeholder ahead of it is passed
71//! over: pascals within 50,000 to 110,000 (the bounds [`crate::rrc3`] argues), kilopascals
72//! within 50 to 110, inches of mercury (3,386.389 Pa, NIST SP 811) within 14.8 to 32.5;
73//! otherwise the column is kept as written. Every sample file is in whole pascals.
74//!
75//! *The altitudes.* `Pressure MSL` is the pressure's altitude in the standard atmosphere, and
76//! `Pressure AGL` the same less the logger's ground altitude: over the 16 sample files `Pressure
77//! MSL` is within 0.06 m of the U.S. Standard Atmosphere's pressure altitude of `pressure`, and
78//! `Pressure MSL` less `Pressure AGL` is constant to 0.02 m in each. The software offers metric or
79//! imperial units (p33); the reader decides between meters and feet as [`crate::rrc3`] does: from
80//! the pressure's pad sample to the one of lowest pressure, the climb the pressure implies in the
81//! U.S. Standard Atmosphere, 1976 ([`hpr_atmos::ussa76`]), against the column's climb read in
82//! meters and in feet; the unit within 25% is taken. A climb under 2 m (where the pressure's
83//! 1 Pa step, 0.08 m at sea level, is 4% of it), a file without a pressure of known unit beside
84//! the altitudes on their clock, or a climb in neither unit is refused ([`LogError::Unit`]): the
85//! primary track has no unit to be read in.
86//!
87//! *The temperature* is read in °C beside meters and °F beside feet: the software sets one unit
88//! system, metric or imperial, for all its displays (p33), and its sensor reads °C or °F (p25).
89//! Its first value must also be one an electronics bay can have in that unit, −40 to 60 °C
90//! (−40 to 140 °F), or the column is kept as written; the sample files' first values, 27.4 to
91//! 50.6, are in °C.
92//!
93//! *The voltages* are read in volts where every value lies within 0 to 20: the computer's battery
94//! is 3.5 to 8.4 V and the ejection battery at most 10 V (p10), and the lines' readings sit
95//! between, where millivolts would be thousands. A column with a value outside is kept as
96//! written. The sample files' values are 0.05 to 8.33.
97//!
98//! *`GPS MSL`* is read in the altitudes' unit if it agrees with `Pressure MSL`: over the GPS
99//! samples within 1 s of a barometric one, the median difference is within 300 m in that unit
100//! and beyond 300 m in the other. 300 m is the error a day's weather gives a standard-atmosphere
101//! altitude when the sea-level pressure is 36 hPa off the standard 1,013.25 (8.4 m a hPa near sea
102//! level); the manual's own sample export reads 283 m by GPS beside 95.6 m by pressure (Table 2,
103//! p40), and the sample files' medians are 28 to 173 m. Otherwise, as at a site too low for the
104//! units' factor of 3.28 to show, the column is kept as written.
105//!
106//! *The magnetometer.* The manual gives tesla (p24), but the field the sample files' three
107//! columns make, 0.0011 to 0.0022 in the 9 with them, would be 17 to 34 times the Earth's at its
108//! strongest, about 65 µT, so no source bears a unit out; the record has no quantity for a
109//! magnetic field either, and each file's note gives its own range.
110//!
111//! **The primary track.** The record's time and altitude are `Pressure AGL`'s, on its own clock:
112//! the AIM's barometric altitude above the ground it was calibrated to (p22), the altitude above
113//! the pad that the logger itself states. A file without it is refused: working one out of the
114//! pressure is a reading, not this reader's to make. The primary track takes its samples in time
115//! order, the first at each time, leaving out any sample without a time; a note counts the
116//! samples put in order or left out (the channel keeps them all, in the file's order). Debrief's
117//! sample, a thinned copy of a flight with one row out of order at its end, has one. Where the
118//! altitude above ground reads more than 100 m off zero at the track's first sample, as one of the
119//! sample BASE files does where its ground reference came from an altitude far from the pad's, a
120//! note says so. No column is a vertical speed, so [`FlightLog::vertical_speed_m_s`] is `None`;
121//! the temperature and `comp. bat. voltage` fill [`FlightLog::temperature_k`] and
122//! [`FlightLog::battery_v`] only on the altitude's clock, which none of the sample files has.
123//!
124//! **Events.** `launch detect`, `apogee detect` and the four outputs' `A on.` … `D on` are flags
125//! the AIM logs every 100 ms (p2); each time one goes from 0 to 1 is an [`Event`] named by its
126//! column, at the sample's time, in time order. A flag already 1 at its first sample is no event.
127//! The AIM detects the launch some time after leaving the rail, once its filtered velocity has
128//! stayed above 30 m/s for 0.5 s (p36), so `launch detect` is no liftoff time.
129//!
130//! **Rows one cell too wide.** In 11 of the 13 sample files with GPS columns, every row after the
131//! GPS group (`time,lat,long,GPS MSL,SVs`) runs out holds one cell more than the header: the group
132//! is written as six empty cells, not five, so the groups to its right sit one cell right of
133//! their names. A row one cell wider than the header whose GPS group's cells and the one after
134//! them are all empty has that one after dropped, counted in a note, if every group's cells are
135//! then all filled or all empty, as they are in every such row of every sample file; otherwise
136//! the extra cell is somewhere else, and the row is refused, naming its line. Any other row of
137//! another width is refused too, so the record never holds more values than the file has cells.
138//!
139//! **Cells.** A group with any cell filled in a row is a sample: an empty time or value in it is a
140//! gap (`NaN`), counted in a note; a cell that isn't a number is refused, naming its line. Lines may
141//! end in CR, LF or both, blank lines are skipped, and a byte-order mark is dropped.
142//!
143//! **Where this comes from.** The layout follows Debrief's reader (`lib/parsers/entacoreAim.ts`,
144//! MIT, the project owner's own; see `THIRD-PARTY-NOTICES.md`), which reads each channel on its
145//! own clock and recognizes the file by `Pressure MSL` and `Pressure AGL`; the units are the
146//! manual's, and the files', as above. Where this reader departs from Debrief's: it keeps every
147//! column on its own clock where Debrief resamples a few onto the densest; it takes the altitude
148//! above ground from the export's own column where Debrief works one out from the pressure; it
149//! drops the extra cell of a run-out GPS group, which Debrief reads shifted; and it states the
150//! flags' rises as events.
151
152use std::collections::BTreeSet;
153
154use hpr_atmos::ussa76::Ussa76;
155use hpr_core::gravity::STANDARD_GRAVITY_MPS2;
156
157use crate::error::LogError;
158use crate::log::{Channel, Event, FlightLog, LogFormat, Quantity, Stated};
159use crate::perfectflite::FOOT_M;
160use crate::table::{Notes, finite, number, numbered_lines, short};
161
162/// The format's name in errors.
163const FORMAT: &str = "Entacore AIM CSV";
164
165/// The name of a clock column.
166const CLOCK: &str = "time";
167
168/// Names only the AIM's export writes, trimmed, in lower case and without trailing points: a header
169/// must name one of them, as well as two clocks, to be recognized.
170pub(crate) const MARKS: &[&str] = &[
171    "pressure msl",
172    "pressure agl",
173    "lat. x accel",
174    "lat. y accel",
175    "eject. bat. voltage",
176    "comp. bat. voltage",
177    "launch detect",
178    "apogee detect",
179    "gps time of week",
180    "gps msl",
181];
182
183/// A pad's pressure in pascals, the bounds [`crate::rrc3`] argues: the standard atmosphere's about
184/// 5,570 m up, above any launch site, to above the highest sea-level pressure on record.
185const PAD_PA: (f64, f64) = (50_000.0, 110_000.0);
186
187/// Pascals in a kilopascal.
188const KPA_PA: f64 = 1_000.0;
189
190/// Pascals in a conventional inch of mercury (NIST SP 811, 2008, B.8).
191const INHG_PA: f64 = 3_386.389;
192
193/// The least climb, m, from which the pressure settles the altitudes' unit: the pressure's step,
194/// 1 Pa in every sample file, is 0.08 m of altitude at sea level, 4% of it and 1% of an 8 m hop,
195/// inside [`ALTITUDE_MARGIN`]; and the altitude columns are worked out from the same samples, so
196/// the sensor's noise moves both climbs alike.
197const SETTLING_CLIMB_M: f64 = 2.0;
198
199/// How far, as a share, an altitude column's climb may be from the pressure's for its unit to be
200/// taken: feet and meters are 3.28 times apart, so at most one unit can be within 25%.
201const ALTITUDE_MARGIN: f64 = 0.25;
202
203/// `GPS Time of Week`'s rise over a second of `time` if `time` is in seconds: milliseconds.
204const TOW_RATE: f64 = 1_000.0;
205
206/// How far, as a share, the median rise may be from [`TOW_RATE`] (seconds) or 1 (milliseconds)
207/// for `time`'s unit to be taken from it; the BASE's least, as of 2026-10-13, is 3% under 1,000.
208const TOW_MARGIN: f64 = 0.1;
209
210/// A week in milliseconds: a GPS time of week counted in them stays below it.
211const WEEK_MS: f64 = 604_800_000.0;
212
213/// The fastest clock's median step, as written, read as seconds: the AIM's fastest period, 10 ms
214/// (manual p2), less 10%, to 2 s, its sensors' periods times the whole multiples the owner sets
215/// (p21). The 16 sample files' are 0.0097 to 0.773.
216const STEP_SECONDS: (f64, f64) = (0.009, 2.0);
217
218/// The same steps as [`STEP_SECONDS`], written in milliseconds.
219const STEP_MILLISECONDS: (f64, f64) = (9.0, 2_000.0);
220
221/// The most a voltage column may read, in volts, to be read in them: twice the ejection
222/// battery's 10 V cap, the computer's battery being 3.5 to 8.4 V (manual p10).
223const VOLTS_MAX: f64 = 20.0;
224
225/// The temperatures, °C, an electronics bay on a pad can have: a winter launch to a bay in the
226/// desert sun.
227const BAY_C: (f64, f64) = (-40.0, 60.0);
228
229/// How far, m, `GPS MSL` may read from `Pressure MSL` in the same unit: the error of a
230/// standard-atmosphere altitude when a day's sea-level pressure is 36 hPa off the standard
231/// 1,013.25, at 8.4 m a hPa near sea level.
232const GPS_AGREEMENT_M: f64 = 300.0;
233
234/// How far apart in time, s, a GPS and a barometric sample may be to be compared.
235const PAIR_S: f64 = 1.0;
236
237/// The Earth's magnetic field at its strongest, about 65 µT, in tesla.
238const EARTH_FIELD_T: f64 = 65e-6;
239
240/// How far, m, the altitude above ground may read from zero at the track's first sample before a
241/// note says the logger's ground isn't where the record starts.
242const GROUND_OFF_M: f64 = 100.0;
243
244/// The most events a log keeps; past it, a note counts the rest, so a file of flags toggling every
245/// row can't fill the record with them.
246const MAX_EVENTS: usize = 1_000;
247
248/// Whether a role is one a note is about.
249type RoleTest = fn(Role) -> bool;
250
251/// What a column is.
252#[derive(Debug, Clone, Copy, PartialEq, Eq)]
253enum Role {
254    Pressure,
255    AltitudeMsl,
256    AltitudeAgl,
257    GpsAltitude,
258    Temperature,
259    Acceleration,
260    Gyro,
261    Magnetic,
262    Voltage,
263    Latitude,
264    Longitude,
265    TimeOfWeek,
266    /// The satellites used in the GPS's solution.
267    Satellites,
268    /// The radio's received strength at the AIM BASE, dBm.
269    SignalStrength,
270    /// A 0-or-1 flag, and whether its rise is an event.
271    Flag {
272        event: bool,
273    },
274    Unknown,
275}
276
277/// The names the reader knows, as [`key`] gives them, and what each is.
278const KNOWN: [(&str, Role); 31] = [
279    ("pressure", Role::Pressure),
280    ("pressure msl", Role::AltitudeMsl),
281    ("pressure agl", Role::AltitudeAgl),
282    ("gps msl", Role::GpsAltitude),
283    ("temperature", Role::Temperature),
284    ("acceleration", Role::Acceleration),
285    ("lat. x accel", Role::Acceleration),
286    ("lat. y accel", Role::Acceleration),
287    ("gyro x", Role::Gyro),
288    ("gyro y", Role::Gyro),
289    ("gyro z", Role::Gyro),
290    ("mag x", Role::Magnetic),
291    ("mag y", Role::Magnetic),
292    ("mag z", Role::Magnetic),
293    ("eject. bat. voltage", Role::Voltage),
294    ("comp. bat. voltage", Role::Voltage),
295    ("a voltage", Role::Voltage),
296    ("b voltage", Role::Voltage),
297    ("c voltage", Role::Voltage),
298    ("d voltage", Role::Voltage),
299    ("lat", Role::Latitude),
300    ("long", Role::Longitude),
301    ("gps time of week", Role::TimeOfWeek),
302    ("svs", Role::Satellites),
303    ("rssi", Role::SignalStrength),
304    ("launch detect", Role::Flag { event: true }),
305    ("apogee detect", Role::Flag { event: true }),
306    ("a on", Role::Flag { event: true }),
307    ("b on", Role::Flag { event: true }),
308    ("c on", Role::Flag { event: true }),
309    ("d on", Role::Flag { event: true }),
310];
311
312/// A header name trimmed and without trailing points, its case kept.
313fn bare(name: &str) -> &str {
314    name.trim().trim_end_matches('.').trim()
315}
316
317/// A header name as [`KNOWN`] and [`MARKS`] list them: [`bare`], in lower case. The export writes
318/// both `A on.` and `B on`.
319fn key(name: &str) -> String {
320    bare(name).to_ascii_lowercase()
321}
322
323/// What a column is, by its name.
324fn role_of(name: &str) -> Role {
325    let key = key(name);
326    // `A cont.` … `D cont.`: an output's continuity, one letter and `cont`.
327    if let Some(letter) = key.strip_suffix(" cont")
328        && letter.len() == 1
329    {
330        return Role::Flag { event: false };
331    }
332    KNOWN
333        .iter()
334        .find(|(known, _)| *known == key)
335        .map_or(Role::Unknown, |(_, role)| *role)
336}
337
338/// The first line of the text that isn't blank, any byte-order mark dropped.
339fn first_line(text: &str) -> Option<&str> {
340    let text = text.strip_prefix('\u{feff}').unwrap_or(text);
341    text.split(['\r', '\n'])
342        .map(str::trim)
343        .find(|line| !line.is_empty())
344}
345
346/// Whether a header line is an AIM export's: a `time` column first, at least two, and one of the
347/// AIM's own names. The names are compared in place, and the walk stops once both are seen, so a
348/// long first line costs no copy of each cell.
349fn is_header(line: &str) -> bool {
350    let mut names = line.split(',').map(bare);
351    if !names
352        .next()
353        .is_some_and(|first| first.eq_ignore_ascii_case(CLOCK))
354    {
355        return false;
356    }
357    let mut clocks = 1usize;
358    let mut marked = false;
359    for name in names {
360        if name.eq_ignore_ascii_case(CLOCK) {
361            clocks += 1;
362        } else if !marked && MARKS.iter().any(|mark| name.eq_ignore_ascii_case(mark)) {
363            marked = true;
364        }
365        if clocks >= 2 && marked {
366            return true;
367        }
368    }
369    false
370}
371
372/// Whether the text is this layout: its first line that isn't blank is an AIM export's header.
373pub(crate) fn recognizes(text: &str) -> bool {
374    first_line(text).is_some_and(is_header)
375}
376
377/// One data column of a group.
378struct Column {
379    name: String,
380    role: Role,
381    /// The values as written, `NaN` for an empty cell.
382    values: Vec<f64>,
383    /// Empty cells in samples of the group.
384    empty: usize,
385    /// Whether no evidence settled the unit its role would be read in, so the column is kept as
386    /// written, [`Quantity::Unrecognized`].
387    as_written: bool,
388}
389
390/// A clock and the columns to its right, up to the next.
391struct Group {
392    /// The clock's column, 0-based.
393    first: usize,
394    /// The times, as written until the clock's unit is decided, then in seconds.
395    time_s: Vec<f64>,
396    columns: Vec<Column>,
397    /// The line each sample was read from, one to a time.
398    lines: Vec<usize>,
399    /// Samples whose time cell is empty.
400    empty_times: usize,
401}
402
403impl Group {
404    /// Cells the group takes in a row: its clock and its columns.
405    fn width(&self) -> usize {
406        1 + self.columns.len()
407    }
408
409    /// Whether the row holds some of the group's cells filled and some empty.
410    fn part_filled(&self, cells: &[&str]) -> bool {
411        cells
412            .get(self.first..self.first + self.width())
413            .is_some_and(|run| {
414                run.iter().any(|cell| cell.is_empty()) && run.iter().any(|cell| !cell.is_empty())
415            })
416    }
417}
418
419/// Reads an Entacore AIM CSV export from its text.
420///
421/// # Errors
422///
423/// [`LogError::NotThisFormat`] if the first line isn't an AIM export's header;
424/// [`LogError::Syntax`] for a header with an empty name, a name given twice or a clock without
425/// columns, a file without `Pressure AGL`, a row of another width than the header's (but for the
426/// GPS group's extra cell), or a cell that isn't a number; [`LogError::NoData`] if `Pressure AGL`
427/// has no sample with a time; [`LogError::Unit`] if neither `GPS Time of Week` nor the clocks'
428/// step settles the clock's unit, or the pressure doesn't settle the altitudes' (module docs).
429pub fn read(text: &str) -> Result<FlightLog, LogError> {
430    let text = text.strip_prefix('\u{feff}').unwrap_or(text);
431    let mut lines = numbered_lines(text).filter(|(_, line)| !line.trim().is_empty());
432    let Some((header_line, header)) = lines.next() else {
433        return Err(LogError::NotThisFormat {
434            format: FORMAT,
435            message: "the file is empty".to_owned(),
436        });
437    };
438    if !is_header(header) {
439        return Err(LogError::NotThisFormat {
440            format: FORMAT,
441            message: format!(
442                "its first line, {:?}, isn't an AIM export's header: `time` columns and the \
443                 AIM's channels, such as `Pressure AGL`",
444                short(header)
445            ),
446        });
447    }
448    let mut notes = Notes::default();
449    let mut groups = header_groups(header, header_line, &mut notes)?;
450    let width: usize = groups.iter().map(Group::width).sum();
451    // The GPS group: the one whose first column is `lat`.
452    let gps = groups
453        .iter()
454        .find(|group| group.columns.first().is_some_and(|c| key(&c.name) == "lat"))
455        .map(|group| (group.first, group.width()));
456
457    let mut repaired = 0usize;
458    let mut first_repaired = None;
459    for (number, line) in lines {
460        let mut cells: Vec<&str> = line.split(',').map(str::trim).collect();
461        if let Some((first, gps_width)) = gps
462            && cells.len() == width + 1
463            && cells
464                .get(first..=first + gps_width)
465                .is_some_and(|run| run.iter().all(|cell| cell.is_empty()))
466        {
467            cells.remove(first + gps_width);
468            // Dropped there, the cell must leave every group whole, as the export writes them;
469            // otherwise the extra cell is elsewhere, and the row is refused as any row of another
470            // width is.
471            if let Some(group) = groups.iter().find(|group| group.part_filled(&cells)) {
472                return Err(syntax(
473                    number,
474                    format!(
475                        "the row has {} cells where the header names {width}. Dropping the \
476                         cell after its empty GPS group would leave the group in columns {} to {} \
477                         part filled",
478                        width + 1,
479                        group.first + 1,
480                        group.first + group.width()
481                    ),
482                ));
483            }
484            repaired += 1;
485            first_repaired.get_or_insert(number);
486        }
487        if cells.len() != width {
488            return Err(syntax(
489                number,
490                format!(
491                    "the row has {} cells where the header names {width}",
492                    cells.len()
493                ),
494            ));
495        }
496        for group in &mut groups {
497            read_sample(group, &cells, number)?;
498        }
499    }
500    let repair_note = first_repaired.map(|line| {
501        format!(
502            "{repaired} rows, the first on line {line}, write the run-out GPS group, `time,lat,…`, \
503             one empty cell too wide. The cell after it was dropped, so that the groups to its \
504             right are read under their names"
505        )
506    });
507    build(groups, header_line, notes, repair_note)
508}
509
510/// The groups the header lays out: each `time` column and the columns to its right.
511fn header_groups(header: &str, line: usize, notes: &mut Notes) -> Result<Vec<Group>, LogError> {
512    let mut groups: Vec<Group> = Vec::new();
513    let mut seen: BTreeSet<String> = BTreeSet::new();
514    for (index, name) in header.split(',').map(str::trim).enumerate() {
515        if name.is_empty() {
516            return Err(syntax(
517                line,
518                format!("the header's column {} has no name", index + 1),
519            ));
520        }
521        let name_key = key(name);
522        if name_key == CLOCK {
523            if let Some(last) = groups.last()
524                && last.columns.is_empty()
525            {
526                return Err(syntax(
527                    line,
528                    format!(
529                        "the clock in column {} has no column of its own",
530                        last.first + 1
531                    ),
532                ));
533            }
534            groups.push(Group {
535                first: index,
536                time_s: Vec::new(),
537                columns: Vec::new(),
538                lines: Vec::new(),
539                empty_times: 0,
540            });
541            continue;
542        }
543        if !seen.insert(name_key) {
544            return Err(syntax(
545                line,
546                format!(
547                    "the header names {:?} twice, in column {} and before",
548                    short(name),
549                    index + 1
550                ),
551            ));
552        }
553        // Invariant: the header starts with a clock (`is_header`), so a group is open here.
554        let Some(group) = groups.last_mut() else {
555            continue;
556        };
557        let role = role_of(name);
558        if role == Role::Unknown {
559            notes.push(format!(
560                "the column {:?} isn't one this reader knows, and was kept as written",
561                short(name)
562            ));
563        }
564        group.columns.push(Column {
565            name: name.to_owned(),
566            role,
567            values: Vec::new(),
568            empty: 0,
569            as_written: false,
570        });
571    }
572    if let Some(last) = groups.last()
573        && last.columns.is_empty()
574    {
575        return Err(syntax(
576            line,
577            format!(
578                "the clock in column {} has no column of its own",
579                last.first + 1
580            ),
581        ));
582    }
583    Ok(groups)
584}
585
586/// Reads a group's sample from a row as wide as the header, if the row has one: any of its cells
587/// filled.
588fn read_sample(group: &mut Group, cells: &[&str], line: usize) -> Result<(), LogError> {
589    let Some(sample) = cells.get(group.first..group.first + group.width()) else {
590        return Err(syntax(line, "the row ends inside a group".to_owned()));
591    };
592    if sample.iter().all(|cell| cell.is_empty()) {
593        return Ok(());
594    }
595    let cell_value = |at: usize, name: &str| -> Result<f64, LogError> {
596        let cell = sample[at];
597        if cell.is_empty() {
598            return Ok(f64::NAN);
599        }
600        number(cell, false).ok_or_else(|| {
601            syntax(
602                line,
603                format!(
604                    "{:?} in cell {}, under {:?}, isn't a number",
605                    short(cell),
606                    group.first + at + 1,
607                    short(name)
608                ),
609            )
610        })
611    };
612    let time_s = cell_value(0, CLOCK)?;
613    let mut values = Vec::with_capacity(group.columns.len());
614    for (at, column) in group.columns.iter().enumerate() {
615        values.push(cell_value(at + 1, &column.name)?);
616    }
617    if time_s.is_nan() {
618        group.empty_times += 1;
619    }
620    group.time_s.push(time_s);
621    group.lines.push(line);
622    for (column, value) in group.columns.iter_mut().zip(values) {
623        if value.is_nan() {
624            column.empty += 1;
625        }
626        column.values.push(value);
627    }
628    Ok(())
629}
630
631/// The place, group and column, of the first column with the role.
632fn locate(groups: &[Group], role: Role) -> Option<(usize, usize)> {
633    groups.iter().enumerate().find_map(|(at, group)| {
634        group
635            .columns
636            .iter()
637            .position(|c| c.role == role)
638            .map(|column| (at, column))
639    })
640}
641
642/// The record from the groups read: units decided, every channel, the primary track, the events
643/// and the notes.
644fn build(
645    mut groups: Vec<Group>,
646    header_line: usize,
647    notes: Notes,
648    repair_note: Option<String>,
649) -> Result<FlightLog, LogError> {
650    // The primary track's column, and a sample of it with a time, before any unit is decided.
651    let Some((agl_group, _)) = locate(&groups, Role::AltitudeAgl) else {
652        return Err(syntax(
653            header_line,
654            "the file has no `Pressure AGL`, the logger's altitude above the ground, which the \
655             record's altitude is"
656                .to_owned(),
657        ));
658    };
659    if !groups[agl_group].time_s.iter().any(|t| t.is_finite()) {
660        return Err(LogError::NoData { format: FORMAT });
661    }
662
663    let mut decisions: Vec<String> = Vec::new();
664
665    // The clock, applied to every group.
666    let clock = clock(&groups, header_line)?;
667    for group in &mut groups {
668        for time in &mut group.time_s {
669            *time *= clock.to_s;
670        }
671    }
672    decisions.push(clock.note);
673
674    // The pressure, from its first value that is a pad's.
675    let pressure = locate(&groups, Role::Pressure);
676    let mut pressure_pa: Option<(f64, usize)> = None;
677    if let Some((group, at)) = pressure {
678        let column = &groups[group].columns[at];
679        let pad = column
680            .values
681            .iter()
682            .enumerate()
683            .find_map(|(index, &value)| pressure_unit(value).map(|unit| (index, value, unit)));
684        match pad {
685            Some((index, value, (factor, unit))) => {
686                decisions.push(format!(
687                    "`{}` read in {unit}, from the values: its first value that is a launch \
688                     site's pressure, {value}{}, is one in {unit}. A launch site's pressure is \
689                     50,000 to 110,000 Pa, whether written in pascals, kilopascals or inches of \
690                     mercury",
691                    column.name,
692                    if index > 0 {
693                        format!(", after {index} samples that aren't")
694                    } else {
695                        String::new()
696                    }
697                ));
698                pressure_pa = Some((factor, index));
699            }
700            None => decisions.push(format!(
701                "`{}` kept as written, in a unit its values don't settle: none of them is a \
702                 launch site's pressure. A launch site's pressure is 50,000 to 110,000 Pa, whether \
703                 written in pascals, kilopascals or inches of mercury",
704                column.name
705            )),
706        }
707    }
708
709    // The altitudes, against the pressure.
710    let altitude_names = names_of(&groups, |role| {
711        matches!(role, Role::AltitudeMsl | Role::AltitudeAgl)
712    });
713    let (feet, pressure_m, column_m) = altitude_unit(&groups, pressure, pressure_pa, header_line)?;
714    let (unit, scale) = if feet {
715        ("feet", FOOT_M)
716    } else {
717        ("meters", 1.0)
718    };
719    decisions.push(format!(
720        "{} read in {unit}, from the pressure: the standard atmosphere climbs {pressure_m:.1} m \
721         from the pressure's pad sample to the one of lowest pressure. Over the same span the \
722         column climbs {column_m:.1} {unit}",
723        listed(&altitude_names)
724    ));
725    if let Some((note, agrees)) = gps_altitude(&groups, scale, unit) {
726        decisions.push(note);
727        if !agrees {
728            set_as_written(&mut groups, |role| role == Role::GpsAltitude);
729        }
730    }
731    decide_temperatures(&mut groups, feet, unit, &mut decisions);
732    decide_voltages(&mut groups, &mut decisions);
733
734    let fixed: [(RoleTest, &str); 4] = [
735        (
736            |role| role == Role::Acceleration,
737            "read in g, from the manual: it plots the accelerations \"in G's\", the long axis's \
738             with gravity in it (p23–24)",
739        ),
740        (
741            |role| role == Role::Gyro,
742            "read in degrees per second, from the manual: \"degrees/sec\" (p24)",
743        ),
744        (
745            |role| matches!(role, Role::Latitude | Role::Longitude),
746            "read in degrees, from the values: decimal numbers, as the manual's sample export \
747             writes them (Table 2, p40)",
748        ),
749        (
750            |role| role == Role::Satellites,
751            "kept as written, a count, from the manual: the satellites used in the GPS's \
752             solution (p25)",
753        ),
754    ];
755    for (is, how) in fixed {
756        let names = names_of(&groups, is);
757        if !names.is_empty() {
758            decisions.push(format!("{} {how}", listed(&names)));
759        }
760    }
761    let rssi = names_of(&groups, |role| role == Role::SignalStrength);
762    if !rssi.is_empty() {
763        decisions.push(format!(
764            "{} kept as written, in dBm, from the manual: the strength of the radio signal the \
765             AIM BASE received, \"in decibels (dBm)\" (p29). It is a power level in decibels \
766             against a milliwatt, not an SI unit. The record has no quantity for a signal's \
767             strength",
768            listed(&rssi)
769        ));
770    }
771    if let Some(note) = magnetic_note(&groups) {
772        decisions.push(note);
773    }
774    let time_of_week = names_of(&groups, |role| role == Role::TimeOfWeek);
775    if !time_of_week.is_empty() {
776        decisions.push(match clock.tow_rate_per_s {
777            Some(rate) => format!(
778                "{} kept as written, in milliseconds, from the values: they stay below a week's \
779                 604,800,000. They rise by a median of {rate:.1} a second of the clock. The \
780                 record has no quantity for a time of the GPS week",
781                listed(&time_of_week)
782            ),
783            None => format!(
784                "{} kept as written, in a unit its values don't settle: they don't rise by a \
785                 millisecond's per second of `time`. That is so in either of its units. The \
786                 record has no quantity for a time of the GPS week",
787                listed(&time_of_week)
788            ),
789        });
790    }
791    decisions.extend(flag_notes(&groups));
792
793    let base = groups
794        .iter()
795        .flat_map(|group| &group.columns)
796        .any(|column| column.role == Role::SignalStrength);
797
798    // Every channel, in SI.
799    let mut channel_notes = notes;
800    let mut out_of_range = Vec::new();
801    let mut channels = Vec::new();
802    let mut events = Vec::new();
803    let mut events_left_out = 0usize;
804    for group in &groups {
805        if group.empty_times > 0 {
806            channel_notes.push(format!(
807                "the clock in column {} leaves {} samples' times empty: they are gaps",
808                group.first + 1,
809                group.empty_times
810            ));
811        }
812        for column in &group.columns {
813            if column.empty > 0 {
814                channel_notes.push(format!(
815                    "the column {:?} leaves {} samples' values empty: they are gaps",
816                    short(&column.name),
817                    column.empty
818                ));
819            }
820            let pressure_factor = pressure_pa.map(|(factor, _)| factor);
821            let (quantity, written) = si(column, pressure_factor, scale, feet, &mut out_of_range);
822            // A value that its unit's conversion makes infinite, a pressure in kPa or inHg, an
823            // acceleration in g or a temperature in °F, is refused on its row's line (ADR-258 §3).
824            let mut values = Vec::with_capacity(written.len());
825            for (line, value) in group.lines.iter().zip(written) {
826                values.push(finite(value, FORMAT, *line, &column.name)?);
827            }
828            if let Role::Flag { event: true } = column.role {
829                for (pair, time) in column.values.windows(2).zip(group.time_s.iter().skip(1)) {
830                    if pair[0] == 0.0 && pair[1] == 1.0 && time.is_finite() {
831                        if events.len() < MAX_EVENTS {
832                            events.push(Event {
833                                time_s: *time,
834                                name: column.name.clone(),
835                            });
836                        } else {
837                            events_left_out += 1;
838                        }
839                    }
840                }
841            }
842            channels.push(Channel {
843                column: column.name.clone(),
844                quantity,
845                time_s: group.time_s.clone(),
846                values,
847            });
848        }
849    }
850    events.sort_by(|a, b| a.time_s.total_cmp(&b.time_s));
851    for (name, count) in out_of_range {
852        channel_notes.push(format!(
853            "the column {:?} holds {count} values beyond a {}'s range: they are gaps",
854            short(&name),
855            if name.trim().eq_ignore_ascii_case("lat") {
856                "latitude"
857            } else {
858                "longitude"
859            }
860        ));
861    }
862    if events_left_out > 0 {
863        channel_notes.push(format!(
864            "and {events_left_out} more rises of the flags, left out of the events so that the \
865             list stays short"
866        ));
867    }
868
869    // The primary track: `Pressure AGL`'s samples in time order.
870    let Some(primary) = channels
871        .iter()
872        .position(|c| c.quantity == Quantity::AltitudeAboveGround)
873    else {
874        // Invariant: `Pressure AGL` was found above, and its quantity is always this one.
875        return Err(LogError::NoData { format: FORMAT });
876    };
877    let track = &channels[primary];
878    let mut samples: Vec<(f64, f64)> = track
879        .time_s
880        .iter()
881        .zip(&track.values)
882        .filter(|(time, _)| time.is_finite())
883        .map(|(time, value)| (*time, *value))
884        .collect();
885    let untimed = track.time_s.len() - samples.len();
886    let backwards = samples
887        .windows(2)
888        .filter(|pair| pair[1].0 < pair[0].0)
889        .count();
890    samples.sort_by(|a, b| a.0.total_cmp(&b.0));
891    let before = samples.len();
892    samples.dedup_by(|later, earlier| later.0 == earlier.0);
893    let repeats = before - samples.len();
894    if samples.is_empty() {
895        return Err(LogError::NoData { format: FORMAT });
896    }
897    let mut track_notes = Vec::new();
898    if backwards > 0 || repeats > 0 || untimed > 0 {
899        track_notes.push(format!(
900            "the altitude's clock goes back {backwards} times and repeats a time {repeats} \
901             times, and {untimed} samples have no time. The record's track takes the samples in \
902             time order, the first at each time, and leaves out those without one. The channel \
903             keeps them all"
904        ));
905    }
906    if let Some(&(_, ground)) = samples.iter().find(|(_, value)| value.is_finite())
907        && ground.abs() > GROUND_OFF_M
908    {
909        track_notes.push(format!(
910            "`{}` reads {ground:.1} m at the track's first sample: the logger's ground isn't \
911             where the record starts",
912            track.column
913        ));
914    }
915    if base {
916        track_notes.push(
917            "the file holds `RSSI`, the radio's strength at the AIM BASE. It is the telemetry \
918             the ground station received, at the radio's rates, not the flight computer's own log"
919                .to_owned(),
920        );
921    }
922    let (time_s, altitude_m): (Vec<f64>, Vec<f64>) = samples.into_iter().unzip();
923    let on_track = |channel: &&Channel| channel.time_s == time_s;
924    let temperature_k = channels
925        .iter()
926        .filter(|c| c.quantity == Quantity::Temperature)
927        .find(on_track)
928        .map(|c| c.values.clone());
929    let battery_v = channels
930        .iter()
931        .filter(|c| c.quantity == Quantity::Voltage && key(&c.column) == "comp. bat. voltage")
932        .find(on_track)
933        .map(|c| c.values.clone());
934
935    let mut all_notes = decisions;
936    all_notes.extend(repair_note);
937    all_notes.extend(track_notes);
938    all_notes.extend(channel_notes.into_vec());
939    Ok(FlightLog {
940        format: LogFormat::EntacoreAimCsv,
941        logger: "Entacore AIM".to_owned(),
942        serial_number: None,
943        firmware: None,
944        flight_number: None,
945        stated: Stated::default(),
946        time_s,
947        altitude_m,
948        vertical_speed_m_s: None,
949        temperature_k,
950        battery_v,
951        channels,
952        events,
953        notes: all_notes,
954    })
955}
956
957/// Marks every column whose role is as asked as kept as written.
958fn set_as_written(groups: &mut [Group], is: impl Fn(Role) -> bool) {
959    for column in groups.iter_mut().flat_map(|group| &mut group.columns) {
960        if is(column.role) {
961            column.as_written = true;
962        }
963    }
964}
965
966/// A column's quantity and its values in SI. A latitude or longitude out of range is a gap,
967/// counted in `out_of_range` by the column's name.
968fn si(
969    column: &Column,
970    pressure_pa: Option<f64>,
971    altitude_scale: f64,
972    fahrenheit: bool,
973    out_of_range: &mut Vec<(String, usize)>,
974) -> (Quantity, Vec<f64>) {
975    let scaled = |factor: f64| -> Vec<f64> { column.values.iter().map(|v| v * factor).collect() };
976    if column.as_written {
977        return (Quantity::Unrecognized, column.values.clone());
978    }
979    match column.role {
980        Role::Pressure => match pressure_pa {
981            Some(factor) => (Quantity::Pressure, scaled(factor)),
982            None => (Quantity::Unrecognized, column.values.clone()),
983        },
984        Role::AltitudeMsl => (Quantity::AltitudeMsl, scaled(altitude_scale)),
985        Role::AltitudeAgl => (Quantity::AltitudeAboveGround, scaled(altitude_scale)),
986        Role::GpsAltitude => (Quantity::GnssAltitude, scaled(altitude_scale)),
987        Role::Temperature => (
988            Quantity::Temperature,
989            column
990                .values
991                .iter()
992                .map(|v| {
993                    if fahrenheit {
994                        (v - 32.0) * 5.0 / 9.0 + 273.15
995                    } else {
996                        v + 273.15
997                    }
998                })
999                .collect(),
1000        ),
1001        Role::Acceleration => (Quantity::Acceleration, scaled(STANDARD_GRAVITY_MPS2)),
1002        Role::Gyro => (Quantity::AngularRate, column.values.clone()),
1003        Role::Voltage => (Quantity::Voltage, column.values.clone()),
1004        Role::Latitude | Role::Longitude => {
1005            let (quantity, limit) = if column.role == Role::Latitude {
1006                (Quantity::Latitude, 90.0)
1007            } else {
1008                (Quantity::Longitude, 180.0)
1009            };
1010            let mut count = 0;
1011            let values = column
1012                .values
1013                .iter()
1014                .map(|&v| {
1015                    if v.abs() > limit {
1016                        count += 1;
1017                        f64::NAN
1018                    } else {
1019                        v
1020                    }
1021                })
1022                .collect();
1023            if count > 0 {
1024                out_of_range.push((column.name.clone(), count));
1025            }
1026            (quantity, values)
1027        }
1028        Role::Satellites | Role::SignalStrength | Role::Flag { .. } => {
1029            (Quantity::Number, column.values.clone())
1030        }
1031        Role::TimeOfWeek | Role::Magnetic | Role::Unknown => {
1032            (Quantity::Unrecognized, column.values.clone())
1033        }
1034    }
1035}
1036
1037/// The pressure's unit from a value on the pad: the factor to pascals and the unit's name; `None`
1038/// if it is a launch site's pressure in none of them.
1039fn pressure_unit(pad: f64) -> Option<(f64, &'static str)> {
1040    [
1041        (1.0, "pascals"),
1042        (KPA_PA, "kilopascals"),
1043        (INHG_PA, "inches of mercury"),
1044    ]
1045    .into_iter()
1046    .find(|(factor, _)| (PAD_PA.0..=PAD_PA.1).contains(&(pad * factor)))
1047}
1048
1049/// The altitudes' unit, `true` for feet, with the pressure's climb in meters and the column's as
1050/// written: the climb of `Pressure MSL`, or else `Pressure AGL`, in the pressure's group, against
1051/// the pressure's in the U.S. Standard Atmosphere, from the pressure's pad sample to the one of
1052/// lowest pressure. A file where that can't be measured is refused.
1053fn altitude_unit(
1054    groups: &[Group],
1055    pressure: Option<(usize, usize)>,
1056    pressure_pa: Option<(f64, usize)>,
1057    header_line: usize,
1058) -> Result<(bool, f64, f64), LogError> {
1059    let refuse = |why: String| LogError::Unit {
1060        format: FORMAT,
1061        line: header_line,
1062        message: format!(
1063            "the altitudes' unit, meters or feet, can't be decided from the pressure: {why}"
1064        ),
1065    };
1066    let Some((group, at)) = pressure else {
1067        return Err(refuse("the file has no `pressure`".to_owned()));
1068    };
1069    let group = &groups[group];
1070    let pressure = &group.columns[at];
1071    let Some((factor, pad)) = pressure_pa else {
1072        return Err(refuse(format!(
1073            "`{}` is in a unit its values don't settle",
1074            pressure.name
1075        )));
1076    };
1077    let Some(altitude) = group
1078        .columns
1079        .iter()
1080        .find(|c| c.role == Role::AltitudeMsl)
1081        .or_else(|| group.columns.iter().find(|c| c.role == Role::AltitudeAgl))
1082    else {
1083        return Err(refuse(
1084            "no altitude column is beside the pressure on its clock".to_owned(),
1085        ));
1086    };
1087    let pairs: Vec<(f64, f64)> = pressure
1088        .values
1089        .iter()
1090        .zip(&altitude.values)
1091        .skip(pad)
1092        .filter(|(p, a)| p.is_finite() && *p * factor > 0.0 && a.is_finite())
1093        .map(|(p, a)| (p * factor, *a))
1094        .collect();
1095    let (Some(&first), Some(&lowest)) = (
1096        pairs.first(),
1097        pairs.iter().min_by(|a, b| a.0.total_cmp(&b.0)),
1098    ) else {
1099        return Err(refuse(
1100            "no sample holds both the pressure and the altitude".to_owned(),
1101        ));
1102    };
1103    let atmosphere = Ussa76::standard();
1104    let climb = atmosphere
1105        .pressure_altitude_m(lowest.0)
1106        .and_then(|low| atmosphere.pressure_altitude_m(first.0).map(|pad| low - pad));
1107    let Ok(pressure_m) = climb else {
1108        return Err(refuse(
1109            "the pressure has no altitude in the standard atmosphere".to_owned(),
1110        ));
1111    };
1112    if pressure_m < SETTLING_CLIMB_M {
1113        return Err(refuse(format!(
1114            "the pressure climbs {pressure_m:.2} m, under the {SETTLING_CLIMB_M} m from which \
1115             meters and feet can be told apart"
1116        )));
1117    }
1118    let column = lowest.1 - first.1;
1119    let within = |climb_m: f64| (climb_m / pressure_m - 1.0).abs() <= ALTITUDE_MARGIN;
1120    if within(column) {
1121        Ok((false, pressure_m, column))
1122    } else if within(column * FOOT_M) {
1123        Ok((true, pressure_m, column))
1124    } else {
1125        Err(LogError::Unit {
1126            format: FORMAT,
1127            line: header_line,
1128            message: format!(
1129                "`{}` climbs {column:.1} from the pressure's pad sample to the one of lowest \
1130                 pressure. The pressure's climb in the standard atmosphere is \
1131                 {pressure_m:.1} m. The column's climb is neither meters nor feet within 25%",
1132                altitude.name
1133            ),
1134        })
1135    }
1136}
1137
1138/// How `time` is read.
1139struct Clock {
1140    /// The factor from `time` as written to seconds.
1141    to_s: f64,
1142    /// The note saying how, and on what evidence.
1143    note: String,
1144    /// `GPS Time of Week`'s median rise a second of the clock, where it decided the unit.
1145    tow_rate_per_s: Option<f64>,
1146}
1147
1148/// The median of the values, `None` if there are none.
1149fn median(mut values: Vec<f64>) -> Option<f64> {
1150    values.sort_by(f64::total_cmp);
1151    values.get(values.len() / 2).copied()
1152}
1153
1154/// The clock's unit: from `GPS Time of Week`'s median rise over its clock's, a millisecond's per
1155/// second (seconds) or per millisecond (milliseconds); else from the fastest clock's median step
1156/// against the AIM's periods; else refused.
1157fn clock(groups: &[Group], header_line: usize) -> Result<Clock, LogError> {
1158    let tow = groups.iter().find_map(|group| {
1159        group
1160            .columns
1161            .iter()
1162            .find(|c| c.role == Role::TimeOfWeek)
1163            .map(|c| (group, c))
1164    });
1165    let why = match tow {
1166        None => "the file has no `GPS Time of Week` to decide it by".to_owned(),
1167        Some((group, column)) => {
1168            let samples: Vec<(f64, f64)> = group
1169                .time_s
1170                .iter()
1171                .zip(&column.values)
1172                .filter(|(t, v)| t.is_finite() && v.is_finite())
1173                .map(|(t, v)| (*t, *v))
1174                .collect();
1175            let rates: Vec<f64> = samples
1176                .windows(2)
1177                .filter(|pair| pair[1].0 > pair[0].0)
1178                .map(|pair| (pair[1].1 - pair[0].1) / (pair[1].0 - pair[0].0))
1179                .collect();
1180            let below_a_week = samples.iter().all(|s| s.1 < WEEK_MS);
1181            let near = |rate: f64, target: f64| (rate / target - 1.0).abs() <= TOW_MARGIN;
1182            match median(rates) {
1183                None => format!("`{}` has too few samples to decide it by", column.name),
1184                Some(_) if !below_a_week => format!(
1185                    "`{}` passes a week's 604,800,000 milliseconds, so it doesn't count them",
1186                    column.name
1187                ),
1188                Some(rate) if near(rate, TOW_RATE) => {
1189                    return Ok(Clock {
1190                        to_s: 1.0,
1191                        note: format!(
1192                            "`time` read in seconds, from `{}`: it counts milliseconds, below a \
1193                             week's 604,800,000. It climbs by a median of {rate:.1} per unit of \
1194                             its `time`, a millisecond's per second",
1195                            column.name
1196                        ),
1197                        tow_rate_per_s: Some(rate),
1198                    });
1199                }
1200                Some(rate) if near(rate, 1.0) => {
1201                    return Ok(Clock {
1202                        to_s: 0.001,
1203                        note: format!(
1204                            "`time` read in milliseconds, from `{}`: it counts milliseconds, \
1205                             below a week's 604,800,000. It climbs by a median of {rate:.3} per \
1206                             unit of its `time`, a millisecond's per millisecond",
1207                            column.name
1208                        ),
1209                        tow_rate_per_s: Some(rate * 1_000.0),
1210                    });
1211                }
1212                Some(rate) => format!(
1213                    "`{}` climbs by a median of {rate:.1} per unit of its `time`, neither the \
1214                     1,000 of milliseconds per second nor the 1 per millisecond",
1215                    column.name
1216                ),
1217            }
1218        }
1219    };
1220    // The fastest clock's median step, as written.
1221    let step = groups
1222        .iter()
1223        .filter_map(|group| {
1224            let times: Vec<f64> = group
1225                .time_s
1226                .iter()
1227                .copied()
1228                .filter(|t| t.is_finite())
1229                .collect();
1230            median(
1231                times
1232                    .windows(2)
1233                    .map(|pair| pair[1] - pair[0])
1234                    .filter(|step| *step > 0.0)
1235                    .collect(),
1236            )
1237        })
1238        .min_by(f64::total_cmp);
1239    let periods = "the AIM writes its sensors every 10 to 500 ms (manual p2) times a whole \
1240                   multiple the owner sets (p21)";
1241    match step {
1242        Some(step) if (STEP_SECONDS.0..=STEP_SECONDS.1).contains(&step) => Ok(Clock {
1243            to_s: 1.0,
1244            note: format!(
1245                "`time` read in seconds, from the step: {why}. The fastest clock's median step, \
1246                 {step} as written, is a step of 0.009 to 2 s. That matches how {periods}. \
1247                 Milliseconds would be 9 to 2,000"
1248            ),
1249            tow_rate_per_s: None,
1250        }),
1251        Some(step) if (STEP_MILLISECONDS.0..=STEP_MILLISECONDS.1).contains(&step) => Ok(Clock {
1252            to_s: 0.001,
1253            note: format!(
1254                "`time` read in milliseconds, from the step: {why}. The fastest clock's median \
1255                 step, {step} as written, is a step of 9 to 2,000 ms. That matches how \
1256                 {periods}. Seconds would be 0.009 to 2"
1257            ),
1258            tow_rate_per_s: None,
1259        }),
1260        Some(step) => Err(LogError::Unit {
1261            format: FORMAT,
1262            line: header_line,
1263            message: format!(
1264                "the clock's unit can't be decided: {why}. The fastest clock's median step, \
1265                 {step}, is a step in neither seconds (0.009 to 2) nor milliseconds (9 to \
1266                 2,000). Both ranges follow from how {periods}"
1267            ),
1268        }),
1269        None => Err(LogError::Unit {
1270            format: FORMAT,
1271            line: header_line,
1272            message: format!(
1273                "the clock's unit can't be decided: {why}, and no clock has two samples to \
1274                 measure a step by"
1275            ),
1276        }),
1277    }
1278}
1279
1280/// The note on `GPS MSL`'s unit, and whether it agrees with the altitudes': the median
1281/// difference, over the GPS samples within [`PAIR_S`] of a `Pressure MSL` sample, within
1282/// [`GPS_AGREEMENT_M`] in the altitudes' unit and beyond it in the other. `None` without the
1283/// column.
1284fn gps_altitude(groups: &[Group], scale: f64, unit: &str) -> Option<(String, bool)> {
1285    let (gps_group, gps_at) = locate(groups, Role::GpsAltitude)?;
1286    let gps = &groups[gps_group].columns[gps_at];
1287    let kept = |why: String| {
1288        Some((
1289            format!(
1290                "`{}` kept as written, in a unit `Pressure MSL` doesn't settle: {why}",
1291                gps.name
1292            ),
1293            false,
1294        ))
1295    };
1296    let Some((baro_group, baro_at)) = locate(groups, Role::AltitudeMsl) else {
1297        return kept("the file has no `Pressure MSL` to compare it with".to_owned());
1298    };
1299    let mut barometric: Vec<(f64, f64)> = groups[baro_group]
1300        .time_s
1301        .iter()
1302        .zip(&groups[baro_group].columns[baro_at].values)
1303        .filter(|(t, v)| t.is_finite() && v.is_finite())
1304        .map(|(t, v)| (*t, *v))
1305        .collect();
1306    barometric.sort_by(|a, b| a.0.total_cmp(&b.0));
1307    let other = if scale == 1.0 { FOOT_M } else { 1.0 };
1308    let other_unit = if scale == 1.0 { "feet" } else { "meters" };
1309    let mut same_m = Vec::new();
1310    let mut other_m = Vec::new();
1311    for (time, value) in groups[gps_group].time_s.iter().zip(&gps.values) {
1312        if !(time.is_finite() && value.is_finite()) {
1313            continue;
1314        }
1315        let after = barometric.partition_point(|(t, _)| t < time);
1316        let nearest = [after.checked_sub(1), Some(after)]
1317            .into_iter()
1318            .flatten()
1319            .filter_map(|at| barometric.get(at))
1320            .min_by(|a, b| (a.0 - time).abs().total_cmp(&(b.0 - time).abs()));
1321        if let Some(&(t, baro)) = nearest
1322            && (t - time).abs() <= PAIR_S
1323        {
1324            same_m.push((value - baro) * scale);
1325            other_m.push(value * other - baro * scale);
1326        }
1327    }
1328    let pairs = same_m.len();
1329    let (Some(same), Some(differs)) = (median(same_m), median(other_m)) else {
1330        return kept("no GPS sample is within 1 s of a `Pressure MSL` sample".to_owned());
1331    };
1332    let measured = format!(
1333        "over {pairs} GPS samples within 1 s of one of it, the median difference is {same:.1} m \
1334         read in {unit} and {differs:.1} m in {other_unit}"
1335    );
1336    if same.abs() <= GPS_AGREEMENT_M && differs.abs() > GPS_AGREEMENT_M {
1337        Some((
1338            format!(
1339                "`{}` read in {unit}, from `Pressure MSL`: {measured}, within the 300 m a day's \
1340                 weather moves a standard-atmosphere altitude only in {unit}",
1341                gps.name
1342            ),
1343            true,
1344        ))
1345    } else {
1346        kept(format!(
1347            "{measured}, so the 300 m a day's weather moves a standard-atmosphere altitude \
1348             doesn't tell the units apart"
1349        ))
1350    }
1351}
1352
1353/// Decides each temperature column's unit from the altitudes', and its first value's range.
1354fn decide_temperatures(groups: &mut [Group], feet: bool, unit: &str, decisions: &mut Vec<String>) {
1355    let (degrees, low, high) = if feet {
1356        (
1357            "Fahrenheit",
1358            BAY_C.0 * 9.0 / 5.0 + 32.0,
1359            BAY_C.1 * 9.0 / 5.0 + 32.0,
1360        )
1361    } else {
1362        ("Celsius", BAY_C.0, BAY_C.1)
1363    };
1364    let system = if feet { "imperial" } else { "metric" };
1365    for column in groups.iter_mut().flat_map(|group| &mut group.columns) {
1366        if column.role != Role::Temperature {
1367            continue;
1368        }
1369        let first = column.values.iter().copied().find(|v| v.is_finite());
1370        match first {
1371            Some(value) if (low..=high).contains(&value) => decisions.push(format!(
1372                "`{}` read in degrees {degrees}, from the heights' unit and the manual, p33: the \
1373                 heights are in {unit}, decided from the pressure. The software sets one \
1374                 unit system for all its displays (p33), here {system}, its sensor reading °C or \
1375                 °F (p25). The column's first value, {value}, is a bay's temperature in degrees \
1376                 {degrees}, {low} to {high}",
1377                column.name
1378            )),
1379            _ => {
1380                column.as_written = true;
1381                decisions.push(format!(
1382                    "`{}` kept as written, in a unit its values don't settle: the heights are in \
1383                     {unit}, so the displays are {system} (p33). The column's first value, {}, \
1384                     isn't a bay's temperature in degrees {degrees}, {low} to {high}",
1385                    column.name,
1386                    first.map_or("none".to_owned(), |v| v.to_string())
1387                ));
1388            }
1389        }
1390    }
1391}
1392
1393/// Decides each voltage column's unit from its values: volts if every one lies within 0 to
1394/// [`VOLTS_MAX`].
1395fn decide_voltages(groups: &mut [Group], decisions: &mut Vec<String>) {
1396    for column in groups.iter_mut().flat_map(|group| &mut group.columns) {
1397        if column.role != Role::Voltage {
1398            continue;
1399        }
1400        let (low, high) = column
1401            .values
1402            .iter()
1403            .filter(|v| v.is_finite())
1404            .fold((f64::INFINITY, f64::NEG_INFINITY), |(low, high), &v| {
1405                (low.min(v), high.max(v))
1406            });
1407        if low <= high && low >= 0.0 && high <= VOLTS_MAX {
1408            decisions.push(format!(
1409                "`{}` read in volts, from the values: they lie within {low} to {high}. That is \
1410                 inside the 0 to 20 V of a battery or an output line. The computer's battery is \
1411                 3.5 to 8.4 V and the ejection battery at most 10 V (manual p10). In millivolts \
1412                 they would be thousands",
1413                column.name
1414            ));
1415        } else {
1416            column.as_written = true;
1417            decisions.push(format!(
1418                "`{}` kept as written, in a unit its values don't settle: {}",
1419                column.name,
1420                if low <= high {
1421                    format!(
1422                        "they lie within {low} to {high}, not all inside the 0 to 20 V of a \
1423                         battery or an output line (manual p10)"
1424                    )
1425                } else {
1426                    "it holds no value".to_owned()
1427                }
1428            ));
1429        }
1430    }
1431}
1432
1433/// The note on the magnetometer's columns, kept as written: the field they would make in the
1434/// manual's tesla, against the Earth's. `None` without them.
1435fn magnetic_note(groups: &[Group]) -> Option<String> {
1436    let names = names_of(groups, |role| role == Role::Magnetic);
1437    if names.is_empty() {
1438        return None;
1439    }
1440    // The field from the magnetic columns of the first group holding one, sample by sample.
1441    let group = groups
1442        .iter()
1443        .find(|group| group.columns.iter().any(|c| c.role == Role::Magnetic))?;
1444    let axes: Vec<&Column> = group
1445        .columns
1446        .iter()
1447        .filter(|c| c.role == Role::Magnetic)
1448        .collect();
1449    let (low, high) = (0..group.time_s.len())
1450        .filter_map(|at| {
1451            let squares: Option<f64> = axes
1452                .iter()
1453                .map(|axis| axis.values.get(at).copied().filter(|v| v.is_finite()))
1454                .try_fold(0.0, |sum, v| v.map(|v| sum + v * v));
1455            squares.map(f64::sqrt)
1456        })
1457        .fold((f64::INFINITY, f64::NEG_INFINITY), |(low, high), v| {
1458            (low.min(v), high.max(v))
1459        });
1460    let field = if low <= high {
1461        format!(
1462            "Read so, the field {} make{}, {low:.5} to {high:.5}, would be {:.0} to {:.0} times \
1463             the Earth's at its strongest, about 65 µT",
1464            if axes.len() == 1 {
1465                "its column"
1466            } else {
1467                "the columns"
1468            },
1469            if axes.len() == 1 { "s" } else { "" },
1470            low / EARTH_FIELD_T,
1471            high / EARTH_FIELD_T
1472        )
1473    } else {
1474        "The file holds no sample of them all to check it by".to_owned()
1475    };
1476    Some(format!(
1477        "{} kept as written, in a unit no source bears out: the manual gives tesla (p24). \
1478         {field}. The record has no quantity for a magnetic field",
1479        listed(&names)
1480    ))
1481}
1482
1483/// The notes on the flags: kept as written, 0 or 1 by their values, or else in a unit no source
1484/// states.
1485fn flag_notes(groups: &[Group]) -> Vec<String> {
1486    let mut flags = Vec::new();
1487    let mut others = Vec::new();
1488    for column in groups.iter().flat_map(|group| &group.columns) {
1489        if let Role::Flag { .. } = column.role {
1490            if column
1491                .values
1492                .iter()
1493                .all(|v| v.is_nan() || *v == 0.0 || *v == 1.0)
1494            {
1495                flags.push(column.name.clone());
1496            } else {
1497                others.push(column.name.clone());
1498            }
1499        }
1500    }
1501    let mut notes = Vec::new();
1502    if !flags.is_empty() {
1503        notes.push(format!(
1504            "{} kept as written, flags, from the values: each is 0 or 1, the AIM's flight event \
1505             status and its lines' continuity (manual p2, p36)",
1506            listed(&flags)
1507        ));
1508    }
1509    if !others.is_empty() {
1510        notes.push(format!(
1511            "{} kept as written, in a unit no source states: they hold values other than 0 and 1, \
1512             so they are no flags",
1513            listed(&others)
1514        ));
1515    }
1516    notes
1517}
1518
1519/// The names of the columns whose role is as asked, in the order of [`KNOWN`], so the same
1520/// columns give the same note whatever their order (a header names each once).
1521fn names_of(groups: &[Group], is: impl Fn(Role) -> bool) -> Vec<String> {
1522    let mut names: Vec<(usize, String)> = groups
1523        .iter()
1524        .flat_map(|group| &group.columns)
1525        .filter(|column| is(column.role))
1526        .filter_map(|column| {
1527            let key = key(&column.name);
1528            KNOWN
1529                .iter()
1530                .position(|(known, _)| *known == key)
1531                .map(|place| (place, column.name.clone()))
1532        })
1533        .collect();
1534    names.sort_by_key(|(place, _)| *place);
1535    names.into_iter().map(|(_, name)| name).collect()
1536}
1537
1538/// Names in backticks, as a list: `` `a` ``, `` `a` and `b` ``, `` `a`, `b` and `c` ``.
1539fn listed(names: &[String]) -> String {
1540    let quoted: Vec<String> = names.iter().map(|name| format!("`{name}`")).collect();
1541    match quoted.as_slice() {
1542        [] => String::new(),
1543        [one] => one.clone(),
1544        [rest @ .., last] => format!("{} and {last}", rest.join(", ")),
1545    }
1546}
1547
1548fn syntax(line: usize, message: String) -> LogError {
1549    LogError::Syntax {
1550        format: FORMAT,
1551        line,
1552        message,
1553    }
1554}
1555
1556#[cfg(test)]
1557mod tests {
1558    use super::*;
1559
1560    const HEADER: &str = "time,Gyro X,time,pressure,Pressure MSL,Pressure AGL,time,launch detect,\
1561                          apogee detect,A on.,time,GPS Time of Week";
1562
1563    /// The pressure, in pascals, at an altitude in the standard atmosphere's troposphere: the
1564    /// altimeter formula inverted.
1565    fn pressure_at(altitude_m: f64) -> f64 {
1566        101_325.0 * (1.0 - altitude_m / 44_330.8).powf(1.0 / 0.190_263)
1567    }
1568
1569    /// The standard atmosphere's altitude of a pressure, in the troposphere.
1570    fn altitude_of(pressure_pa: f64) -> f64 {
1571        44_330.8 * (1.0 - (pressure_pa / 101_325.0).powf(0.190_263))
1572    }
1573
1574    fn close(a: f64, b: f64) -> bool {
1575        (a - b).abs() <= 1e-9 * b.abs().max(1.0)
1576    }
1577
1578    /// An invented flight: a pad 300 m up, a climb to 400 m above it, each group on its clock.
1579    fn flight(altitude_unit: f64) -> String {
1580        let mut text = format!("{HEADER}\r\n");
1581        let heights = [0.0, 50.0, 200.0, 400.0, 300.0, 100.0];
1582        for (row, height) in heights.iter().enumerate() {
1583            let t = row as f64 - 1.0;
1584            let p = pressure_at(300.0 + height).round();
1585            let msl = altitude_of(p);
1586            let agl = msl - 300.0;
1587            let launch = u8::from(row >= 1);
1588            let apogee = u8::from(row >= 4);
1589            text.push_str(&format!(
1590                "{:.2},{},{:.2},{p},{:.3},{:.3},{:.2},{launch},{apogee},{apogee},{:.2},{}\r\n",
1591                t - 0.01,
1592                row as f64 * 0.5,
1593                t,
1594                msl / altitude_unit,
1595                agl / altitude_unit,
1596                t + 0.01,
1597                t + 0.02,
1598                500_000_000.0 + (t + 0.02) * 1_000.0
1599            ));
1600        }
1601        text
1602    }
1603
1604    /// A two-row file of the pressure, its altitude above a pad 300 m up and one more group, the
1605    /// times 0 and 1 s: a climb of 100 m.
1606    fn small(extra_header: &str, extra_rows: [&str; 2]) -> String {
1607        let mut text = format!("time,pressure,Pressure AGL,{extra_header}\n");
1608        for (row, height) in [0.0, 100.0].iter().enumerate() {
1609            let p = pressure_at(300.0 + height).round();
1610            let agl = altitude_of(p) - altitude_of(pressure_at(300.0).round());
1611            text.push_str(&format!("{row},{p},{agl:.3},{}\n", extra_rows[row]));
1612        }
1613        text
1614    }
1615
1616    fn note<'a>(log: &'a FlightLog, start: &str) -> &'a str {
1617        let matching: Vec<&String> = log.notes.iter().filter(|n| n.starts_with(start)).collect();
1618        assert_eq!(matching.len(), 1, "{start}: {:?}", log.notes);
1619        matching[0]
1620    }
1621
1622    /// The survey's reading of a note (`xtask logs`' `unit_decision`): the text before the first
1623    /// `: ` of a note opening with a name in backticks that is `read in` or `kept as written`.
1624    fn counted(note: &str) -> Option<&str> {
1625        let (decision, _) = note.split_once(": ")?;
1626        (decision.starts_with('`')
1627            && (decision.contains("` read in ") || decision.contains("` kept as written")))
1628        .then_some(decision)
1629    }
1630
1631    fn unit_error(text: &str) -> String {
1632        match read(text) {
1633            Err(LogError::Unit { message, .. }) => message,
1634            other => panic!("{other:?}"),
1635        }
1636    }
1637
1638    #[test]
1639    fn recognizes_the_aim_header_and_nothing_like_it() {
1640        assert!(recognizes(&format!("\u{feff}\n{HEADER}\n")));
1641        assert!(recognizes(
1642            "TIME,lat. X accel.,lat. Y accel.,Time,acceleration\n"
1643        ));
1644        for other in [
1645            // One clock only.
1646            "time,Pressure AGL\n",
1647            // Two clocks, but none of the AIM's names.
1648            "time,pressure,time,altitude\n",
1649            "time,state,state_name,source,pressure,altitude,height,acceleration,speed\n",
1650            // A Raven's export.
1651            "Time@Axial Accel (Gs),Axial Accel (Gs),Time@Baro (Atm),Baro (Atm)\n",
1652            // Not first.
1653            "Gyro X,time,Pressure AGL,time,pressure\n",
1654            "",
1655        ] {
1656            assert!(!recognizes(other), "{other:?}");
1657        }
1658        // A long first line is walked, not copied, and stops at the first answer.
1659        let long = format!("time,Pressure AGL,time,{}\n", "x,".repeat(200_000));
1660        assert!(recognizes(&long));
1661        assert!(!recognizes(&format!("time,{}\n", "x,".repeat(200_000))));
1662    }
1663
1664    /// Each group on its own clock, in SI; the primary track `Pressure AGL`'s; meters from the
1665    /// pressure, seconds from `GPS Time of Week`; the flags' rises as events; every column's unit
1666    /// in a note the survey counts.
1667    #[test]
1668    fn reads_each_group_on_its_clock_in_si() {
1669        let log = read(&flight(1.0)).unwrap();
1670        assert_eq!(log.format, LogFormat::EntacoreAimCsv);
1671        let columns: Vec<(&str, Quantity)> = log
1672            .channels
1673            .iter()
1674            .map(|c| (c.column.as_str(), c.quantity))
1675            .collect();
1676        assert_eq!(
1677            columns,
1678            [
1679                ("Gyro X", Quantity::AngularRate),
1680                ("pressure", Quantity::Pressure),
1681                ("Pressure MSL", Quantity::AltitudeMsl),
1682                ("Pressure AGL", Quantity::AltitudeAboveGround),
1683                ("launch detect", Quantity::Number),
1684                ("apogee detect", Quantity::Number),
1685                ("A on.", Quantity::Number),
1686                ("GPS Time of Week", Quantity::Unrecognized),
1687            ]
1688        );
1689        assert_eq!(log.time_s, [-1.0, 0.0, 1.0, 2.0, 3.0, 4.0]);
1690        assert!(close(log.channels[0].time_s[0], -1.01));
1691        assert!(log.altitude_m[3] > 399.0 && log.altitude_m[3] < 401.0);
1692        let events: Vec<(f64, &str)> = log
1693            .events
1694            .iter()
1695            .map(|e| (e.time_s, e.name.as_str()))
1696            .collect();
1697        assert_eq!(
1698            events,
1699            [
1700                (0.01, "launch detect"),
1701                (3.01, "apogee detect"),
1702                (3.01, "A on.")
1703            ]
1704        );
1705        assert!(
1706            note(
1707                &log,
1708                "`Pressure MSL` and `Pressure AGL` read in meters, from the pressure: "
1709            )
1710            .contains("climbs 400.0 m")
1711        );
1712        note(&log, "`time` read in seconds, from `GPS Time of Week`: ");
1713        note(&log, "`pressure` read in pascals, from the values: ");
1714        note(
1715            &log,
1716            "`Gyro X` read in degrees per second, from the manual: ",
1717        );
1718        note(
1719            &log,
1720            "`launch detect`, `apogee detect` and `A on.` kept as written, flags, from the \
1721             values: ",
1722        );
1723        note(
1724            &log,
1725            "`GPS Time of Week` kept as written, in milliseconds, from the values: ",
1726        );
1727        assert!(log.temperature_k.is_none() && log.battery_v.is_none());
1728        // Every column is named in a note the survey counts, and none is assumed.
1729        let decisions: Vec<&str> = log.notes.iter().filter_map(|n| counted(n)).collect();
1730        for channel in &log.channels {
1731            let named = format!("`{}`", channel.column);
1732            assert!(
1733                decisions.iter().any(|d| d.contains(&named)),
1734                "{named}: {decisions:?}"
1735            );
1736        }
1737        assert!(log.notes.iter().all(|n| !n.contains("assumed")));
1738    }
1739
1740    /// The same flight with its altitudes in feet reads the same heights; one in neither unit is
1741    /// refused.
1742    #[test]
1743    fn the_altitudes_unit_is_decided_from_the_pressure_both_ways() {
1744        let meters = read(&flight(1.0)).unwrap();
1745        let feet = read(&flight(FOOT_M)).unwrap();
1746        for (m, f) in meters.altitude_m.iter().zip(&feet.altitude_m) {
1747            assert!((m - f).abs() < 1e-3, "{m} {f}");
1748        }
1749        note(
1750            &feet,
1751            "`Pressure MSL` and `Pressure AGL` read in feet, from the pressure: ",
1752        );
1753        assert!(unit_error(&flight(0.5)).contains("neither meters nor feet"));
1754    }
1755
1756    /// A hop of 8 m, written in feet, is told from meters: the pressure's 1 Pa step is about 1%
1757    /// of the climb. Under 2 m the file is refused, never assumed.
1758    #[test]
1759    fn a_short_hop_in_feet_is_decided_and_a_flat_file_refused() {
1760        let hop = |height_m: f64| {
1761            let mut text = "time,pressure,Pressure AGL,time,launch detect\n".to_owned();
1762            for (row, height) in [0.0, height_m / 2.0, height_m, 0.0].iter().enumerate() {
1763                let p = pressure_at(100.0 + height).round();
1764                let agl = altitude_of(p) - altitude_of(pressure_at(100.0).round());
1765                text.push_str(&format!("{row},{p},{:.2},{row},0\n", agl / FOOT_M));
1766            }
1767            text
1768        };
1769        let log = read(&hop(8.0)).unwrap();
1770        note(&log, "`Pressure AGL` read in feet, from the pressure: ");
1771        let top = log.altitude_m.iter().copied().fold(0.0, f64::max);
1772        assert!((top - 8.0).abs() < 0.2, "{top}");
1773        assert!(unit_error(&hop(1.0)).contains("under the 2 m"));
1774    }
1775
1776    #[test]
1777    fn a_pressure_unit_comes_from_the_pad_or_the_file_is_refused() {
1778        assert_eq!(pressure_unit(97_000.0), Some((1.0, "pascals")));
1779        assert_eq!(pressure_unit(97.0), Some((KPA_PA, "kilopascals")));
1780        assert_eq!(pressure_unit(28.6), Some((INHG_PA, "inches of mercury")));
1781        for pad in [970.0, 5.0, 300_000.0, 40.0, 0.0, f64::NAN] {
1782            assert_eq!(pressure_unit(pad), None, "{pad}");
1783        }
1784        // A placeholder ahead of the pad's pressure is passed over, and the climb measured from
1785        // the pad's sample.
1786        let text = "time,pressure,Pressure AGL,time,launch detect\n\
1787                    0,0,0,0,0\n1,97000,0,1,0\n2,96000,85.2,2,1\n";
1788        let log = read(text).unwrap();
1789        assert!(
1790            note(&log, "`pressure` read in pascals, from the values: ")
1791                .contains("97000, after 1 samples that aren't")
1792        );
1793        note(&log, "`Pressure AGL` read in meters, from the pressure: ");
1794        // A pressure in no unit leaves the altitudes with none: refused.
1795        let text = "time,pressure,Pressure AGL,time,launch detect\n0,970,0,0,0\n1,960,90,1,1\n";
1796        assert!(unit_error(text).contains("`pressure` is in a unit its values don't settle"));
1797        let text = "time,Pressure AGL,time,launch detect\n0,0,0,0\n1,90,1,1\n";
1798        assert!(unit_error(text).contains("the file has no `pressure`"));
1799    }
1800
1801    /// `time` in seconds or milliseconds by `GPS Time of Week`, else by the step, else refused.
1802    #[test]
1803    fn the_clock_is_decided_by_the_time_of_week_or_the_step() {
1804        let by_tow = read(&flight(1.0)).unwrap();
1805        assert!(note(&by_tow, "`time` read in seconds, from ").contains("median of 1000.0"));
1806        // The same flight with every clock written in milliseconds reads the same times.
1807        let millis: String = flight(1.0)
1808            .lines()
1809            .enumerate()
1810            .map(|(row, line)| {
1811                if row == 0 {
1812                    return format!("{line}\n");
1813                }
1814                let cells: Vec<String> = line
1815                    .split(',')
1816                    .enumerate()
1817                    .map(|(at, cell)| {
1818                        if [0, 2, 6, 10].contains(&at) {
1819                            format!("{}", (cell.parse::<f64>().unwrap() * 1_000.0).round())
1820                        } else {
1821                            cell.to_owned()
1822                        }
1823                    })
1824                    .collect();
1825                format!("{}\n", cells.join(","))
1826            })
1827            .collect();
1828        let log = read(&millis).unwrap();
1829        assert!(
1830            note(
1831                &log,
1832                "`time` read in milliseconds, from `GPS Time of Week`: "
1833            )
1834            .contains("median of 1.000")
1835        );
1836        for (a, b) in log.time_s.iter().zip(&by_tow.time_s) {
1837            assert!((a - b).abs() < 1e-9, "{a} {b}");
1838        }
1839        assert!((log.events[0].time_s - 0.01).abs() < 1e-9);
1840        note(
1841            &log,
1842            "`GPS Time of Week` kept as written, in milliseconds, from the values: ",
1843        );
1844        // A time of week rising by 7 per unit of `time`: the step decides, 1 s.
1845        let odd: String = flight(1.0)
1846            .lines()
1847            .enumerate()
1848            .map(|(row, line)| {
1849                if row == 0 {
1850                    return format!("{line}\n");
1851                }
1852                let (head, _) = line.rsplit_once(',').unwrap();
1853                format!("{head},{}\n", 1_000 + 7 * row)
1854            })
1855            .collect();
1856        let log = read(&odd).unwrap();
1857        assert!(
1858            note(&log, "`time` read in seconds, from the step: ")
1859                .contains("neither the 1,000 of milliseconds per second")
1860        );
1861        note(
1862            &log,
1863            "`GPS Time of Week` kept as written, in a unit its values don't settle: ",
1864        );
1865        // No time of week: steps of 1 s, or 1,000 ms.
1866        let seconds = read(&small("time,launch detect", ["0,0", "1,1"])).unwrap();
1867        assert!(
1868            note(&seconds, "`time` read in seconds, from the step: ")
1869                .contains("the file has no `GPS Time of Week`")
1870        );
1871        let ms = small("time,launch detect", ["0,0", "1000,1"]).replace("\n1,", "\n1000,");
1872        let ms = read(&ms).unwrap();
1873        note(&ms, "`time` read in milliseconds, from the step: ");
1874        assert_eq!(ms.time_s, [0.0, 1.0]);
1875        // A step of 5 is neither; a single row has none.
1876        let five = small("time,launch detect", ["0,0", "5,1"]).replace("\n1,", "\n5,");
1877        assert!(unit_error(&five).contains("in neither seconds"));
1878        let one = format!(
1879            "time,pressure,Pressure AGL,time,launch detect\n0,{},0,0,0\n",
1880            pressure_at(300.0)
1881        );
1882        assert!(unit_error(&one).contains("no clock has two samples"));
1883    }
1884
1885    /// The GPS group's extra empty cell is dropped only where the GPS group is empty and every
1886    /// group is then whole; any other row of another width is refused.
1887    #[test]
1888    fn a_run_out_gps_group_s_extra_cell_is_dropped() {
1889        let header = "time,lat,long,GPS MSL,SVs,time,pressure,Pressure AGL";
1890        let text = format!(
1891            "{header}\n0,34.5,-117.25,850,8,0,97000,0\n,,,,,,1,96000,85\n,,,,,,2,95000,170\n"
1892        );
1893        let log = read(&text).unwrap();
1894        assert_eq!(log.time_s, [0.0, 1.0, 2.0]);
1895        assert_eq!(log.channels[0].time_s, [0.0]);
1896        note(
1897            &log,
1898            "2 rows, the first on line 3, write the run-out GPS group",
1899        );
1900        // One cell too many where the GPS group isn't empty.
1901        let wide = format!("{header}\n0,34.5,-117.25,850,8,0,97000,0,\n");
1902        let error = read(&wide).unwrap_err();
1903        assert!(
1904            matches!(error, LogError::Syntax { line: 2, .. }),
1905            "{error:?}"
1906        );
1907        // Without the extra cell, the same rows read the same.
1908        let narrow = format!("{header}\n0,34.5,-117.25,850,8,0,97000,0\n,,,,,1,96000,85\n");
1909        let log = read(&narrow).unwrap();
1910        assert_eq!(log.time_s, [0.0, 1.0]);
1911        assert!(log.notes.iter().all(|n| !n.contains("run-out")));
1912        // The extra cell at the row's end: dropping the one after the GPS group would leave the
1913        // next group part filled and shift the last, so the row is refused.
1914        let header = "time,lat,long,GPS MSL,SVs,time,temperature,time,Pressure AGL";
1915        let elsewhere = format!("{header}\n,,,,,,,1.0,5.0,\n");
1916        match read(&elsewhere) {
1917            Err(LogError::Syntax {
1918                line: 2, message, ..
1919            }) => {
1920                assert!(message.contains("columns 6 to 7 part filled"), "{message}");
1921            }
1922            other => panic!("{other:?}"),
1923        }
1924    }
1925
1926    #[test]
1927    fn the_track_is_put_in_time_order_and_the_channel_kept_as_written() {
1928        let text = "time,pressure,Pressure AGL,time,launch detect\n\
1929                    0,97000,0,0,0\n2,96800,17,1,1\n1,96900,8,,\n2,96800,99,,\n,96700,25,,\n";
1930        let log = read(text).unwrap();
1931        assert_eq!(log.time_s, [0.0, 1.0, 2.0]);
1932        assert_eq!(log.altitude_m, [0.0, 8.0, 17.0]);
1933        let agl = &log.channels[1];
1934        assert_eq!(agl.time_s.len(), 5);
1935        assert!(agl.time_s[4].is_nan());
1936        note(
1937            &log,
1938            "the altitude's clock goes back 1 times and repeats a time 1 times, and 1",
1939        );
1940        note(&log, "the clock in column 1 leaves 1 samples' times empty");
1941    }
1942
1943    #[test]
1944    fn a_ground_far_off_zero_and_a_base_file_are_noted() {
1945        let text =
1946            "time,pressure,Pressure AGL,time,RSSI\n0,97000,-2500,0,-90\n1,96000,-2415,1,-95\n";
1947        let log = read(text).unwrap();
1948        note(
1949            &log,
1950            "`Pressure AGL` reads -2500.0 m at the track's first sample",
1951        );
1952        note(&log, "the file holds `RSSI`");
1953        assert!(note(&log, "`RSSI` kept as written, in dBm, from the manual: ").contains("(p29)"));
1954        assert_eq!(log.channels[2].quantity, Quantity::Number);
1955        let log = read(&small("time,RSSI", ["0,-90", "1,-95"])).unwrap();
1956        assert!(
1957            log.notes
1958                .iter()
1959                .all(|n| !n.contains("track's first sample"))
1960        );
1961    }
1962
1963    #[test]
1964    fn malformed_files_are_refused_naming_the_line() {
1965        let cases = [
1966            ("time,Pressure AGL,time\n", 1),
1967            ("time,Pressure AGL,,time,x\n", 1),
1968            ("time,time,Pressure AGL\n", 1),
1969            ("time,Pressure AGL,time,x\n0,1,0\n", 2),
1970            ("time,Pressure AGL,time,x\n0,1,0,abc\n", 2),
1971            ("time,Gyro X,time,launch detect\n0,1,0,0\n", 1),
1972            // A column named twice, in another case or without its point.
1973            ("time,Pressure AGL,time,pressure agl\n0,1,0,1\n", 1),
1974            ("time,Pressure AGL,A on.,time,A on\n0,1,0,0,0\n", 1),
1975        ];
1976        for (text, line) in cases {
1977            match read(text) {
1978                Err(LogError::Syntax { line: at, .. }) => assert_eq!(at, line, "{text:?}"),
1979                other => panic!("{text:?}: {other:?}"),
1980            }
1981        }
1982        let Err(error) = read("time,Pressure AGL,A on.,time,a on\n") else {
1983            panic!("read");
1984        };
1985        assert!(error.to_string().contains("twice"), "{error}");
1986        assert!(matches!(
1987            read("time,Pressure AGL,time,x\n,,,1\n"),
1988            Err(LogError::NoData { .. })
1989        ));
1990        assert!(matches!(read(""), Err(LogError::NotThisFormat { .. })));
1991    }
1992
1993    /// An acceleration in g, a pressure in kPa and a temperature in °F near the largest `f64` are
1994    /// finite as written and infinite in SI: each is refused on its line (ADR-258 §3).
1995    #[test]
1996    fn a_value_its_conversion_makes_infinite_is_refused() {
1997        let g = small("time,acceleration", ["0,1", "1,-1.7e308"]);
1998        assert_eq!(
1999            read(&g).unwrap_err().to_string(),
2000            "Entacore AIM CSV line 3: the column acceleration holds a value too large for its \
2001             unit's conversion to SI"
2002        );
2003
2004        let mut kpa = String::from("time,pressure,Pressure AGL,time,launch detect\n");
2005        for (row, height) in [0.0, 100.0].iter().enumerate() {
2006            let p = pressure_at(300.0 + height).round();
2007            let agl = altitude_of(p) - altitude_of(pressure_at(300.0).round());
2008            kpa.push_str(&format!("{row},{},{agl:.3},{row},0\n", p / 1_000.0));
2009        }
2010        kpa.push_str("2,1.7e308,50,2,0\n");
2011        assert_eq!(
2012            read(&kpa).unwrap_err().to_string(),
2013            "Entacore AIM CSV line 4: the column pressure holds a value too large for its unit's \
2014             conversion to SI"
2015        );
2016
2017        // In feet, so the temperature is read in °F.
2018        let mut fahrenheit = String::from("time,pressure,Pressure AGL,time,temperature\n");
2019        for (row, height) in [0.0, 100.0].iter().enumerate() {
2020            let p = pressure_at(300.0 + height).round();
2021            let agl = altitude_of(p) - altitude_of(pressure_at(300.0).round());
2022            let cell = if row == 0 { "70" } else { "-1.7e308" };
2023            fahrenheit.push_str(&format!("{row},{p},{:.3},{row},{cell}\n", agl / FOOT_M));
2024        }
2025        assert_eq!(
2026            read(&fahrenheit).unwrap_err().to_string(),
2027            "Entacore AIM CSV line 3: the column temperature holds a value too large for its \
2028             unit's conversion to SI"
2029        );
2030    }
2031
2032    #[test]
2033    fn unknown_columns_and_out_of_range_positions_are_kept_and_noted() {
2034        let text = small("widget,time,lat,long", ["7,0,91,-117", ",1,34,200"]);
2035        let log = read(&text).unwrap();
2036        assert_eq!(log.channels[2].quantity, Quantity::Unrecognized);
2037        assert!(log.channels[2].values[1].is_nan());
2038        assert!(log.channels[3].values[0].is_nan() && log.channels[3].values[1] == 34.0);
2039        assert!(log.channels[4].values[1].is_nan());
2040        note(&log, "the column \"widget\" isn't one this reader knows");
2041        note(&log, "the column \"widget\" leaves 1 samples' values empty");
2042        note(
2043            &log,
2044            "the column \"lat\" holds 1 values beyond a latitude's range",
2045        );
2046        note(
2047            &log,
2048            "the column \"long\" holds 1 values beyond a longitude's range",
2049        );
2050    }
2051
2052    #[test]
2053    fn continuity_flags_are_numbers_and_not_events() {
2054        assert_eq!(role_of("A cont."), Role::Flag { event: false });
2055        assert_eq!(role_of(" B on"), Role::Flag { event: true });
2056        assert_eq!(role_of("cont"), Role::Unknown);
2057        assert_eq!(role_of("lat. X accel."), Role::Acceleration);
2058        let log = read(&small("time,A cont.", ["0,0", "1,1"])).unwrap();
2059        assert!(log.events.is_empty());
2060        note(&log, "`A cont.` kept as written, flags, from the values: ");
2061        let log = read(&small("time,A cont.", ["0,0", "1,7"])).unwrap();
2062        note(
2063            &log,
2064            "`A cont.` kept as written, in a unit no source states: ",
2065        );
2066    }
2067
2068    #[test]
2069    fn the_temperature_follows_the_altitudes_unit_within_a_bay_s_range() {
2070        let header = "time,pressure,Pressure MSL,Pressure AGL,time,temperature";
2071        let mut text = format!("{header}\n");
2072        for (row, height) in [0.0, 200.0].iter().enumerate() {
2073            let p = pressure_at(300.0 + height);
2074            text.push_str(&format!(
2075                "{row},{p},{},{},{row},68\n",
2076                (300.0 + height) / FOOT_M,
2077                height / FOOT_M
2078            ));
2079        }
2080        let log = read(&text).unwrap();
2081        assert!(
2082            note(
2083                &log,
2084                "`temperature` read in degrees Fahrenheit, from the heights' unit and the manual, \
2085                 p33: "
2086            )
2087            .contains("here imperial")
2088        );
2089        assert!(close(log.channels[3].values[0], 293.15));
2090        let celsius = read(&flight(1.0).replace("Gyro X", "temperature")).unwrap();
2091        note(
2092            &celsius,
2093            "`temperature` read in degrees Celsius, from the heights' unit and the manual, p33: ",
2094        );
2095        assert!(close(celsius.channels[0].values[1], 0.5 + 273.15));
2096        // 68 is no bay's temperature in °C: kept as written.
2097        let hot = read(&small("time,temperature", ["0,68", "1,68"])).unwrap();
2098        note(
2099            &hot,
2100            "`temperature` kept as written, in a unit its values don't settle: ",
2101        );
2102        assert_eq!(hot.channels[2].quantity, Quantity::Unrecognized);
2103        assert_eq!(hot.channels[2].values[0], 68.0);
2104    }
2105
2106    #[test]
2107    fn voltages_are_volts_by_their_values_or_kept_as_written() {
2108        let log = read(&small(
2109            "time,comp. bat. voltage,A Voltage",
2110            ["0,3.9,0", "1,3.8,7.5"],
2111        ))
2112        .unwrap();
2113        assert!(
2114            note(
2115                &log,
2116                "`comp. bat. voltage` read in volts, from the values: "
2117            )
2118            .contains("within 3.8 to 3.9")
2119        );
2120        note(&log, "`A Voltage` read in volts, from the values: ");
2121        assert_eq!(log.channels[2].quantity, Quantity::Voltage);
2122        // Millivolts: kept as written.
2123        let log = read(&small("time,comp. bat. voltage", ["0,3900", "1,3850"])).unwrap();
2124        note(
2125            &log,
2126            "`comp. bat. voltage` kept as written, in a unit its values don't settle: ",
2127        );
2128        assert_eq!(log.channels[2].quantity, Quantity::Unrecognized);
2129    }
2130
2131    /// `GPS MSL` in the altitudes' unit where it agrees with `Pressure MSL` and the other unit
2132    /// doesn't; kept as written where it disagrees, or where the site is too low to tell.
2133    #[test]
2134    fn the_gps_altitude_is_decided_against_the_barometric_one() {
2135        let file = |pad_m: f64, gps: &dyn Fn(f64) -> f64| {
2136            let mut text =
2137                "time,pressure,Pressure MSL,Pressure AGL,time,lat,long,GPS MSL,SVs\n".to_owned();
2138            for (row, height) in [0.0, 100.0, 300.0].iter().enumerate() {
2139                let p = pressure_at(pad_m + height).round();
2140                let msl = altitude_of(p);
2141                let agl = msl - altitude_of(pressure_at(pad_m).round());
2142                text.push_str(&format!(
2143                    "{row},{p},{msl:.2},{agl:.2},{row}.3,34.5,-117.25,{:.1},9\n",
2144                    gps(msl)
2145                ));
2146            }
2147            text
2148        };
2149        // 40 m above the barometric altitude, in meters.
2150        let log = read(&file(800.0, &|msl| msl + 40.0)).unwrap();
2151        assert!(
2152            note(&log, "`GPS MSL` read in meters, from `Pressure MSL`: ")
2153                .contains("over 3 GPS samples")
2154        );
2155        assert_eq!(log.channels[5].quantity, Quantity::GnssAltitude);
2156        // The same in feet beside meters: kept as written.
2157        let log = read(&file(800.0, &|msl| (msl + 40.0) / FOOT_M)).unwrap();
2158        note(
2159            &log,
2160            "`GPS MSL` kept as written, in a unit `Pressure MSL` doesn't settle: ",
2161        );
2162        assert_eq!(log.channels[5].quantity, Quantity::Unrecognized);
2163        // A site 50 m up: feet and meters both within 300 m, so neither is taken.
2164        let log = read(&file(50.0, &|msl| msl + 20.0)).unwrap();
2165        assert!(
2166            note(
2167                &log,
2168                "`GPS MSL` kept as written, in a unit `Pressure MSL` doesn't settle: "
2169            )
2170            .contains("doesn't tell the units apart")
2171        );
2172    }
2173
2174    #[test]
2175    fn the_magnetometer_note_gives_the_file_s_field() {
2176        let log = read(&small(
2177            "time,Mag X,Mag Y,Mag Z",
2178            ["0,0.0012,0.0016,0", "1,0.0006,0.0008,0"],
2179        ))
2180        .unwrap();
2181        let mag = note(
2182            &log,
2183            "`Mag X`, `Mag Y` and `Mag Z` kept as written, in a unit no source bears out: ",
2184        );
2185        // 0.001 T and 0.002 T are 15 and 31 times 65 µT.
2186        assert!(
2187            mag.contains("0.00100 to 0.00200, would be 15 to 31 times"),
2188            "{mag}"
2189        );
2190    }
2191
2192    #[test]
2193    fn listed_names_read_as_a_sentence() {
2194        let names = |n: &[&str]| n.iter().map(|s| (*s).to_owned()).collect::<Vec<_>>();
2195        assert_eq!(listed(&names(&["a"])), "`a`");
2196        assert_eq!(listed(&names(&["a", "b"])), "`a` and `b`");
2197        assert_eq!(listed(&names(&["a", "b", "c"])), "`a`, `b` and `c`");
2198    }
2199
2200    proptest::proptest! {
2201        /// Any text is read or refused, never a panic; what is read has a strictly increasing
2202        /// primary track, and every channel as many times as values.
2203        #[test]
2204        fn any_text_is_read_or_refused(
2205            text in "time,pressure,Pressure AGL,(time,lat,long,)?time,[a-z ]{0,6}\n\
2206                     ([-0-9.,e ]{0,24}\n){0,12}"
2207        ) {
2208            if let Ok(log) = read(&text) {
2209                proptest::prop_assert!(log.time_s.windows(2).all(|pair| pair[1] > pair[0]));
2210                proptest::prop_assert_eq!(log.time_s.len(), log.altitude_m.len());
2211                for channel in &log.channels {
2212                    proptest::prop_assert_eq!(channel.time_s.len(), channel.values.len());
2213                }
2214            }
2215        }
2216    }
2217}