Skip to main content

hpr_flightdata/
teensy.rs

1//! A course's Teensy data logger: one sensor's reading a line, each line opening with the
2//! sensor's tag and the logger's clock in microseconds, in two layouts.
3//!
4//! The Harvey Mudd College advanced rocketry course flies a data logger it builds around a Teensy
5//! board, and publishes its flights' files on its [*Flight Data* page][flight-data], which lists two kinds of
6//! `.TXT` file, "Teensy Data Logger" and "New Teensy Data Logger", read by the course's own
7//! scripts. No document describes either layout, so this reader takes them from the files, and
8//! the sample logs' survey (`cargo xtask logs`) holds each part of the reading that isn't stated
9//! to a physical check ([ADR-274][adr-274]).
10//!
11//! **The tab-separated layout** ([`LogFormat::TeensyTabs`]) writes a tag, the clock, then the
12//! values, each field after a tab, shown here as spaces:
13//!
14//! ```text
15//! A  2000  0.25  -0.50  9.75
16//! G  2000  1.50  0.50  -2.00
17//! M  2000  20.10  -3.40  17.80
18//! Alt  3100  P  95000  MSL  540.42
19//! HG  3600  0.4788  -0.9577  10.5337
20//! ADC Pin-35  3700  0.0120000000
21//! ADC Pin-36  3800  0.0150000000
22//! ```
23//!
24//! No line states a unit. Three of the values are read in SI, each held by the survey to the
25//! files:
26//!
27//! | Values | Read as | Held to |
28//! |---|---|---|
29//! | `A`'s three | [`Quantity::Acceleration`], m/s², as written | their magnitude on the pad, the 5 s ending 1 s before liftoff, and over the last 2 s, after landing, each within 3 % of standard gravity |
30//! | `Alt`'s `P` | [`Quantity::Pressure`], Pa, as written | `MSL` worked out from it, below |
31//! | `Alt`'s `MSL` | [`Quantity::AltitudeMsl`], m, as written | on every line, 44,330 × (1 − (P / 101,325)^(1/5.255)), the altitude formula of the Bosch BMP180 barometer's data sheet (BST-BMP180-DS000, §3.6), to within 0.0103 m: its half step, 0.005 m, as it is written to the hundredth, and an allowance of 44,330 × 2⁻²³ m for the logger's own rounding of the formula, whose form no document gives |
32//!
33//! The `MSL` is so a pressure altitude, above where the pressure would be 101,325 Pa, not a
34//! height above the sea. The rest is kept as written, [`Quantity::Unrecognized`], as nothing
35//! states its unit and no check settles one: the gyroscope (`G`), the magnetometer (`M`), a
36//! quantity HPR Sim doesn't hold, the high-g accelerometer (`HG`), whose magnitude during the
37//! boost, while `A` reads from 3 g to 90 % of its greatest, is 1.37 to 2.39 times `A`'s in the
38//! sample logs where both are read as m/s², and the two analog inputs. A
39//! value is named by its tag and its place, counted from 1, such as `A 1` or `HG 3`, or by its tag
40//! and the name the line gives it, such as `Alt P`.
41//!
42//! **The layout with named fields** ([`LogFormat::TeensyNamed`]) opens with a line for each
43//! sensor stating its units, then writes a tag, `t`, the clock, and name–value pairs, comma
44//! separated:
45//!
46//! ```text
47//! ICM data in us, milli-g, deg/s, milli-T, deg C
48//! MS5 data in us, deg C, Pa
49//! ICM, t, 500000500, xa, 12.20, ya, -20.50, za, 998.40, xg, 0.18, yg, -0.12, zg, 0.06, xm, 30.15, ym, 4.20, zm, -18.75, T, 24.50
50//! MS5, t, 500001000, T, 24.10, P, 95000
51//! ```
52//!
53//! The first unit is the clock's, and must be `us`. The others go to the fields by their names:
54//! a name less a leading `x`, `y` or `z` is its group, so `xa`, `ya` and `za` are one, and the
55//! units go to the groups in the order the first line of the tag names them; one unit stated goes to
56//! every field if no name carries an axis's `x`, `y` or `z`, as `T1` and `T2` don't, or if all are
57//! one group. A tag whose units don't fit its groups so, or with no units line, such as a `VOL` line
58//! the sample file writes, is kept as written, and a note says so. The units read are `milli-g` and
59//! `g`, to m/s², `deg/s`, `deg C`, to kelvins, and `Pa`; any other is kept as written, as is
60//! `milli-T`, a magnetic field. The survey holds two of the stated units to the files: the
61//! `milli-g` accelerometer's magnitude on the pad and over the file's last 2 s, after landing,
62//! within 3 % of 1,000 milli-g, as for `A`, and the `Pa` pressure on the pad between 50 and 110
63//! kPa, as no launch site's is outside it. The others rest on the file's statement: in the sample
64//! file, the high-g accelerometer read in its stated `g` is 0.996 times the `milli-g` one over the
65//! boost, which the survey reports. A value
66//! is named by its tag and its name, such as `ICM xa`.
67//!
68//! **The clock**, in both layouts, is the logger's count of microseconds, a whole number, read as
69//! seconds from the logger's own zero. The named layout states its unit; the tab-separated one
70//! doesn't, and the survey holds the reading to the flights: the time from liftoff to apogee,
71//! by the clock so read, is within a factor of 2 of √(2h/g) for the apogee's height h, a coarse
72//! check (the burn adds time and drag takes some away), where a clock read wrongly by a power of
73//! ten is 10 or more times off. Each tag is a channel set on its own clock, as each sensor is read at its own rate.
74//!
75//! **What refuses the file.** A line that isn't a tag, a whole-number clock and values, or whose
76//! tag's layout (its count of values, or its names) differs from the tag's first line, or whose
77//! clock doesn't increase from the tag's line before ([`LogError::Syntax`], naming its line). A
78//! logger's clock of this kind runs over after 2³² microseconds, 71.6 minutes, so a longer
79//! recording is refused there. A file's last line, if the file ends with no line end and the
80//! line can't be read, is a line the logger was writing when it stopped: it is dropped, and a
81//! note says so. At most [`MAX_TAGS`] tags, [`MAX_VALUES`] values on a line and names of
82//! [`MAX_NAME_BYTES`]: a file past them isn't a logger's.
83//!
84//! The record has no barometric track, as neither layout writes an altitude above the pad:
85//! [`FlightLog::time_s`] and [`FlightLog::altitude_m`] are empty, and every value is a channel.
86//!
87//! [adr-274]: https://github.com/nrdptel/fusionspace-eridanus/blob/main/docs/decisions/0274-the-rest-of-the-importers-planned-teensy-logger-first.md
88//! [flight-data]: https://pages.hmc.edu/spjut/AdvRoc/FlightData.md.html
89
90use hpr_core::gravity::STANDARD_GRAVITY_MPS2;
91
92use crate::error::LogError;
93use crate::log::{Channel, FlightLog, LogFormat, Quantity, Stated};
94use crate::table::{finite, number, numbered_lines, short};
95
96/// The two layouts' names in errors.
97const TABS: &str = "Teensy data logger, tab-separated";
98const NAMED: &str = "Teensy data logger, named fields";
99
100/// How many of a file's first lines that aren't blank are searched for its marks.
101const RECOGNITION_LINES: usize = 100;
102
103/// The most tags a file may hold, over four times the sample logs' seven.
104pub const MAX_TAGS: usize = 32;
105
106/// The most values a line may hold, over six times the sample logs' ten.
107pub const MAX_VALUES: usize = 64;
108
109/// The longest a tag, a value's name or a unit may be, bytes, three times the sample logs'
110/// longest, `ADC Pin-35`.
111pub const MAX_NAME_BYTES: usize = 32;
112
113/// The largest clock, µs: a 32-bit count of microseconds.
114const CLOCK_LIMIT_US: u64 = u32::MAX as u64;
115
116/// The value of a field in a line: by its place, or under a name the line gives it.
117#[derive(Debug, Clone, PartialEq)]
118struct Line<'a> {
119    tag: &'a str,
120    clock_us: u64,
121    /// Each value's name, `None` for a value by its place.
122    names: Vec<Option<&'a str>>,
123    values: Vec<f64>,
124}
125
126/// A field as a tag, a name or a unit: not empty, not a number, within [`MAX_NAME_BYTES`].
127fn name(field: &str) -> Result<&str, String> {
128    if field.is_empty() {
129        Err("an empty field where a name should be".to_owned())
130    } else if field.len() > MAX_NAME_BYTES {
131        Err(format!(
132            "a name longer than {MAX_NAME_BYTES} bytes, {}",
133            short(field)
134        ))
135    } else if number(field, false).is_some() {
136        Err(format!("a number, {field}, where a name should be"))
137    } else {
138        Ok(field)
139    }
140}
141
142/// The clock: a whole number of microseconds, below the 32-bit count's limit.
143fn clock(field: &str) -> Result<u64, String> {
144    let parsed =
145        if field.is_empty() || field.len() > 20 || !field.bytes().all(|b| b.is_ascii_digit()) {
146            None
147        } else {
148            field.parse::<u64>().ok()
149        };
150    match parsed {
151        Some(us) if us <= CLOCK_LIMIT_US => Ok(us),
152        Some(_) => Err(format!(
153            "a clock, {field} µs, past a 32-bit count of microseconds"
154        )),
155        None => Err(format!(
156            "a clock, {}, that isn't a whole number of microseconds",
157            short(field)
158        )),
159    }
160}
161
162/// The fields after the clock: all numbers, by place, or name–number pairs.
163fn values<'a>(fields: &[&'a str]) -> Result<(Vec<Option<&'a str>>, Vec<f64>), String> {
164    if fields.len() > 2 * MAX_VALUES {
165        return Err(format!("more than {MAX_VALUES} values"));
166    }
167    let numbers: Option<Vec<f64>> = fields.iter().map(|f| number(f, false)).collect();
168    if let Some(numbers) = numbers {
169        if numbers.is_empty() {
170            return Err("no values after the clock".to_owned());
171        }
172        if numbers.len() > MAX_VALUES {
173            return Err(format!("more than {MAX_VALUES} values"));
174        }
175        return Ok((vec![None; numbers.len()], numbers));
176    }
177    if !fields.len().is_multiple_of(2) {
178        return Err("values neither all numbers nor in name-number pairs".to_owned());
179    }
180    let mut names = Vec::with_capacity(fields.len() / 2);
181    let mut numbers = Vec::with_capacity(fields.len() / 2);
182    let (pairs, _) = fields.as_chunks::<2>();
183    for [field, value] in pairs {
184        let field = name(field)?;
185        if names.contains(&Some(field)) {
186            return Err(format!("the name {field} twice on one line"));
187        }
188        names.push(Some(field));
189        numbers
190            .push(number(value, false).ok_or_else(|| {
191                format!("{field}'s value, {}, that isn't a number", short(value))
192            })?);
193    }
194    Ok((names, numbers))
195}
196
197/// A tab-separated line: a tag, the clock, the values.
198fn tab_line(text: &str) -> Result<Line<'_>, String> {
199    let fields: Vec<&str> = text.split('\t').collect();
200    if fields.len() < 3 {
201        return Err("fewer than a tag, a clock and a value, by tabs".to_owned());
202    }
203    let tag = name(fields[0])?;
204    let clock_us = clock(fields[1])?;
205    let (names, values) = values(&fields[2..])?;
206    Ok(Line {
207        tag,
208        clock_us,
209        names,
210        values,
211    })
212}
213
214/// A line of the named layout: a tag, `t`, the clock, then name–value pairs, by commas.
215fn named_line(text: &str) -> Result<Line<'_>, String> {
216    let fields: Vec<&str> = text.split(',').map(str::trim).collect();
217    if fields.len() < 5 || fields[1] != "t" {
218        return Err("not a tag, `t`, a clock and name-value pairs, by commas".to_owned());
219    }
220    let tag = name(fields[0])?;
221    let clock_us = clock(fields[2])?;
222    let (names, values) = values(&fields[3..])?;
223    if names.iter().any(Option::is_none) {
224        return Err("values with no names in a layout that names them".to_owned());
225    }
226    Ok(Line {
227        tag,
228        clock_us,
229        names,
230        values,
231    })
232}
233
234/// A units line of the named layout, `ICM data in us, milli-g, …`: the tag and its units.
235fn units_line(text: &str) -> Option<(&str, Vec<&str>)> {
236    let (tag, units) = text.split_once(" data in ")?;
237    let units: Vec<&str> = units.split(',').map(str::trim).collect();
238    (name(tag.trim()).is_ok() && units.iter().all(|unit| name(unit).is_ok()))
239        .then_some((tag.trim(), units))
240}
241
242/// Whether the text is the tab-separated layout: its first line that isn't blank is a tag, a
243/// clock and values by tabs, and one of its first 100 is an `Alt` line, `Alt`, the clock, `P`
244/// and a pressure, `MSL` and an altitude.
245pub fn recognizes_tabs(text: &str) -> bool {
246    let mut lines = numbered_lines(text)
247        .map(|(_, line)| line)
248        .filter(|line| !line.trim().is_empty())
249        .take(RECOGNITION_LINES);
250    let Some(first) = lines.next() else {
251        return false;
252    };
253    tab_line(first).is_ok()
254        && std::iter::once(first).chain(lines).any(|line| {
255            tab_line(line)
256                .is_ok_and(|line| line.tag == "Alt" && line.names == [Some("P"), Some("MSL")])
257        })
258}
259
260/// Whether the text is the named layout: its first line that isn't blank states a tag's units,
261/// the clock's `us` first, and one of its first 100 is a line of name–value pairs.
262pub fn recognizes_named(text: &str) -> bool {
263    let mut lines = numbered_lines(text)
264        .map(|(_, line)| line)
265        .filter(|line| !line.trim().is_empty())
266        .take(RECOGNITION_LINES);
267    let Some(first) = lines.next() else {
268        return false;
269    };
270    units_line(first).is_some_and(|(_, units)| units[0] == "us")
271        && lines.any(|line| named_line(line).is_ok())
272}
273
274/// The lines of one tag.
275struct Tag<'a> {
276    tag: &'a str,
277    first_line: usize,
278    names: Vec<Option<&'a str>>,
279    last_us: u64,
280    /// Each line's number, for an error about one of its values.
281    lines: Vec<usize>,
282    time_s: Vec<f64>,
283    values: Vec<Vec<f64>>,
284}
285
286/// The tags' lines, in the order the file first writes them, and what the reading dropped.
287struct Lines<'a> {
288    tags: Vec<Tag<'a>>,
289    units: Vec<(&'a str, Vec<&'a str>, usize)>,
290    dropped_last: bool,
291}
292
293/// Reads every line by `parse`, after any units lines the named layout opens with.
294fn lines<'a>(
295    text: &'a str,
296    format: &'static str,
297    parse: fn(&'a str) -> Result<Line<'a>, String>,
298    with_units: bool,
299) -> Result<Lines<'a>, LogError> {
300    let syntax = |line: usize, message: String| LogError::Syntax {
301        format,
302        line,
303        message,
304    };
305    let whole = text;
306    let mut numbered = numbered_lines(text)
307        .filter(|(_, line)| !line.trim().is_empty())
308        .peekable();
309    let mut tags: Vec<Tag<'a>> = Vec::new();
310    let mut units = Vec::new();
311    let mut dropped_last = false;
312    while let Some((number, text)) = numbered.next() {
313        // A line the logger was cut off in: the last that isn't blank, with no line end after it.
314        let cut = numbered.peek().is_none() && {
315            let end = text.as_ptr() as usize - whole.as_ptr() as usize + text.len();
316            !whole[end..].contains(['\n', '\r'])
317        };
318        if with_units
319            && tags.is_empty()
320            && let Some((tag, stated)) = units_line(text)
321        {
322            if units.iter().any(|(t, _, _)| *t == tag) {
323                return Err(syntax(number, format!("a second units line for {tag}")));
324            }
325            if units.len() == MAX_TAGS {
326                return Err(syntax(number, format!("more than {MAX_TAGS} units lines")));
327            }
328            units.push((tag, stated, number));
329            continue;
330        }
331        let checked =
332            parse(text).and_then(
333                |line| match tags.iter().position(|tag| tag.tag == line.tag) {
334                    Some(index) => {
335                        let tag = &tags[index];
336                        if line.names != tag.names {
337                            return Err(format!(
338                                "a {} line laid out unlike its first, on line {}",
339                                line.tag, tag.first_line
340                            ));
341                        }
342                        if line.clock_us <= tag.last_us {
343                            return Err(format!(
344                                "a {} line whose clock, {} µs, doesn't come after the one before",
345                                line.tag, line.clock_us
346                            ));
347                        }
348                        Ok((Some(index), line))
349                    }
350                    None if tags.len() == MAX_TAGS => Err(format!("more than {MAX_TAGS} tags")),
351                    None => Ok((None, line)),
352                },
353            );
354        let (index, line) = match checked {
355            Ok(found) => found,
356            Err(_) if cut => {
357                dropped_last = true;
358                continue;
359            }
360            Err(message) => return Err(syntax(number, message)),
361        };
362        let index = index.unwrap_or_else(|| {
363            tags.push(Tag {
364                tag: line.tag,
365                first_line: number,
366                names: line.names.clone(),
367                last_us: 0,
368                lines: Vec::new(),
369                time_s: Vec::new(),
370                values: vec![Vec::new(); line.values.len()],
371            });
372            tags.len() - 1
373        });
374        let tag = &mut tags[index];
375        tag.last_us = line.clock_us;
376        tag.lines.push(number);
377        tag.time_s.push(line.clock_us as f64 / 1e6);
378        for (column, value) in tag.values.iter_mut().zip(line.values) {
379            column.push(value);
380        }
381    }
382    if tags.is_empty() {
383        return Err(LogError::NoData { format });
384    }
385    Ok(Lines {
386        tags,
387        units,
388        dropped_last,
389    })
390}
391
392/// A value's column: its tag and its name, or its tag and its place.
393fn column(tag: &str, name: Option<&str>, place: usize) -> String {
394    match name {
395        Some(name) => format!("{tag} {name}"),
396        None => format!("{tag} {}", place + 1),
397    }
398}
399
400fn record(format: LogFormat, channels: Vec<Channel>, notes: Vec<String>) -> FlightLog {
401    FlightLog {
402        format,
403        logger: "Teensy data logger".to_owned(),
404        serial_number: None,
405        firmware: None,
406        flight_number: None,
407        stated: Stated {
408            apogee_m: None,
409            ground_elevation_msl_m: None,
410            samples: None,
411            figures: Vec::new(),
412        },
413        time_s: Vec::new(),
414        altitude_m: Vec::new(),
415        vertical_speed_m_s: None,
416        temperature_k: None,
417        battery_v: None,
418        channels,
419        events: Vec::new(),
420        notes,
421    }
422}
423
424const DROPPED_LAST: &str = "the last line, which ends the file with no line end and can't be \
425                            read, dropped: a line the logger was writing when it stopped";
426
427/// Reads the tab-separated layout.
428///
429/// # Errors
430///
431/// [`LogError::NotThisFormat`] if the text isn't the layout ([`recognizes_tabs`]);
432/// [`LogError::Syntax`] for a line that can't be read (see the module's docs);
433/// [`LogError::NoData`] for a file with no lines.
434pub fn read(text: &str) -> Result<FlightLog, LogError> {
435    if !recognizes_tabs(text) {
436        return Err(LogError::NotThisFormat {
437            format: TABS,
438            message: "the first line isn't a tag, a clock and values by tabs, or no `Alt` line \
439                      with `P` and `MSL` comes within the first 100"
440                .to_owned(),
441        });
442    }
443    let Lines {
444        tags, dropped_last, ..
445    } = lines(text, TABS, tab_line, false)?;
446    let mut channels = Vec::new();
447    let mut kept = Vec::new();
448    for tag in tags {
449        let read_by_place = tag.tag == "A" && tag.names == [None, None, None];
450        for (place, (name, values)) in tag.names.iter().zip(tag.values).enumerate() {
451            let quantity = match (tag.tag, *name) {
452                _ if read_by_place => Quantity::Acceleration,
453                ("Alt", Some("P")) => Quantity::Pressure,
454                ("Alt", Some("MSL")) => Quantity::AltitudeMsl,
455                _ => Quantity::Unrecognized,
456            };
457            let column = column(tag.tag, *name, place);
458            if quantity == Quantity::Unrecognized {
459                kept.push(format!("`{column}`"));
460            }
461            channels.push(Channel {
462                column,
463                quantity,
464                time_s: tag.time_s.clone(),
465                values,
466            });
467        }
468    }
469    let mut notes = vec![
470        "each tag's lines on their own clock, the logger's count of microseconds, read as \
471         seconds from the logger's zero; no line states the unit"
472            .to_owned(),
473        "`A` read in m/s² and `Alt`'s `P` in Pa and `MSL` in m, as written. No line states a \
474         unit, and the sample logs' survey holds these to the files"
475            .to_owned(),
476    ];
477    if !kept.is_empty() {
478        notes.push(format!(
479            "kept as written, as no line states their unit: {}",
480            kept.join(", ")
481        ));
482    }
483    if dropped_last {
484        notes.push(DROPPED_LAST.to_owned());
485    }
486    Ok(record(LogFormat::TeensyTabs, channels, notes))
487}
488
489/// A conversion to SI.
490type ToSi = fn(f64) -> f64;
491
492/// A stated unit HPR Sim reads: its quantity and its conversion to SI.
493fn stated_unit(unit: &str) -> Option<(Quantity, ToSi)> {
494    Some(match unit {
495        "milli-g" => (Quantity::Acceleration, |v| {
496            v * STANDARD_GRAVITY_MPS2 / 1_000.0
497        }),
498        "g" => (Quantity::Acceleration, |v| v * STANDARD_GRAVITY_MPS2),
499        "deg/s" => (Quantity::AngularRate, |v| v),
500        "deg C" => (Quantity::Temperature, |v| v + 273.15),
501        "Pa" => (Quantity::Pressure, |v| v),
502        _ => return None,
503    })
504}
505
506/// A field's group: its name less a leading `x`, `y` or `z`.
507fn group(name: &str) -> &str {
508    match name.strip_prefix(['x', 'y', 'z']) {
509        Some(rest) if !rest.is_empty() => rest,
510        _ => name,
511    }
512}
513
514/// Each field's stated unit, or why the tag's are kept as written.
515fn units_of<'a>(names: &[Option<&str>], units: Option<&[&'a str]>) -> Result<Vec<&'a str>, String> {
516    let Some(units) = units else {
517        return Err("no units line states its units".to_owned());
518    };
519    let mut groups: Vec<&str> = Vec::new();
520    for name in names.iter().flatten() {
521        if !groups.contains(&group(name)) {
522            groups.push(group(name));
523        }
524    }
525    let fields = &units[1..];
526    // One unit for fields of several groups only where no name carries an axis, as `T1` and `T2`
527    // don't: a sensor's `xa` and `xg` can't share one.
528    let unaxed = names.iter().flatten().all(|name| group(name) == *name);
529    if fields.len() == 1 && (groups.len() == 1 || unaxed) {
530        Ok(vec![fields[0]; names.len()])
531    } else if fields.len() == groups.len() {
532        Ok(names
533            .iter()
534            .flatten()
535            .map(|name| {
536                // Every name's group is among `groups`, built from the same names.
537                let at = groups.iter().position(|g| *g == group(name)).unwrap_or(0);
538                fields[at]
539            })
540            .collect())
541    } else {
542        Err(format!(
543            "its units line states {} for {} groups of fields",
544            if fields.len() == 1 {
545                "1 unit".to_owned()
546            } else {
547                format!("{} units", fields.len())
548            },
549            groups.len()
550        ))
551    }
552}
553
554/// Reads the layout with named fields.
555///
556/// # Errors
557///
558/// [`LogError::NotThisFormat`] if the text isn't the layout ([`recognizes_named`]);
559/// [`LogError::Unit`] for a units line whose clock isn't in `us`; [`LogError::Syntax`] for a
560/// line that can't be read (see the module's docs); [`LogError::NoData`] for a file with no
561/// data lines.
562pub fn read_named(text: &str) -> Result<FlightLog, LogError> {
563    if !recognizes_named(text) {
564        return Err(LogError::NotThisFormat {
565            format: NAMED,
566            message: "the first line doesn't state a tag's units, the clock's `us` first, or no \
567                      line of name-value pairs comes within the first 100"
568                .to_owned(),
569        });
570    }
571    let Lines {
572        tags,
573        units,
574        dropped_last,
575    } = lines(text, NAMED, named_line, true)?;
576    if let Some((tag, stated, line)) = units.iter().find(|(_, stated, _)| stated[0] != "us") {
577        return Err(LogError::Unit {
578            format: NAMED,
579            line: *line,
580            message: format!(
581                "{tag}'s clock is stated in {}, not `us`, the one unit of time this reader takes",
582                short(stated[0])
583            ),
584        });
585    }
586    let mut channels = Vec::new();
587    let mut notes = vec![
588        "each tag's lines on their own clock, the logger's count of microseconds as its units \
589         lines state, read as seconds from the logger's zero"
590            .to_owned(),
591    ];
592    let mut unread_units: Vec<String> = Vec::new();
593    for tag in tags {
594        let stated = units
595            .iter()
596            .find(|(t, _, _)| *t == tag.tag)
597            .map(|(_, stated, _)| stated.as_slice());
598        let field_units = match units_of(&tag.names, stated) {
599            Ok(field_units) => Some(field_units),
600            Err(why) => {
601                notes.push(format!("{}'s values kept as written: {why}", tag.tag));
602                None
603            }
604        };
605        for (place, (name, values)) in tag.names.iter().zip(tag.values).enumerate() {
606            let column = column(tag.tag, *name, place);
607            let unit = field_units.as_ref().map(|units| units[place]);
608            let (quantity, values) = match unit.and_then(stated_unit) {
609                Some((quantity, to_si)) => {
610                    let values = values
611                        .into_iter()
612                        .zip(&tag.lines)
613                        .map(|(value, line)| finite(to_si(value), NAMED, *line, &column))
614                        .collect::<Result<Vec<f64>, LogError>>()?;
615                    (quantity, values)
616                }
617                None => {
618                    if let Some(unit) = unit {
619                        unread_units.push(format!("`{column}`, in {unit}"));
620                    }
621                    (Quantity::Unrecognized, values)
622                }
623            };
624            channels.push(Channel {
625                column,
626                quantity,
627                time_s: tag.time_s.clone(),
628                values,
629            });
630        }
631    }
632    if !unread_units.is_empty() {
633        notes.push(format!(
634            "kept as written, in a unit stated that HPR Sim doesn't hold: {}",
635            unread_units.join(", ")
636        ));
637    }
638    if dropped_last {
639        notes.push(DROPPED_LAST.to_owned());
640    }
641    Ok(record(LogFormat::TeensyNamed, channels, notes))
642}
643
644#[cfg(test)]
645mod tests {
646    use super::*;
647
648    const TABS_TEXT: &str = "A\t2000\t0.25\t-0.50\t9.75\r\n\
649                             G\t2000\t1.5\t0.5\t-2.0\r\n\
650                             Alt\t3000\tP\t95000\tMSL\t540.42\r\n\
651                             HG\t3500\t0.48\t-0.96\t10.53\r\n\
652                             A\t25000\t0.30\t-0.45\t9.80\r\n\
653                             G\t25000\t1.4\t0.6\t-2.1\r\n\
654                             Alt\t26000\tP\t94990\tMSL\t541.29\r\n\
655                             HG\t26500\t0.48\t-0.48\t10.05\r\n";
656
657    const NAMED_TEXT: &str = "ICM data in us, milli-g, deg/s, milli-T, deg C\r\n\
658                              MS5 data in us, deg C, Pa\r\n\
659                              ADC data in us, deg C\r\n\
660                              ICM, t, 1000000, xa, 10.00, ya, -20.00, za, 1000.00, xg, 0.5, yg, -0.5, zg, 0.0, xm, 40.0, ym, 5.0, zm, -20.0, T, 25.00\r\n\
661                              MS5, t, 1000500, T, 24.50, P, 95000\r\n\
662                              ADC, t, 1000700, T1, 20.000, T2, 21.500\r\n\
663                              VOL, t, 1000900, V, 4.100\r\n\
664                              ICM, t, 1001000, xa, 11.00, ya, -21.00, za, 999.00, xg, 0.4, yg, -0.4, zg, 0.1, xm, 40.5, ym, 5.5, zm, -20.5, T, 25.25\r\n";
665
666    fn channel<'a>(log: &'a FlightLog, column: &str) -> &'a Channel {
667        log.channels
668            .iter()
669            .find(|c| c.column == column)
670            .unwrap_or_else(|| panic!("no {column}"))
671    }
672
673    #[test]
674    fn each_tag_is_read_on_its_own_clock_in_microseconds() {
675        assert!(recognizes_tabs(TABS_TEXT));
676        assert!(!recognizes_named(TABS_TEXT));
677        let log = read(TABS_TEXT).unwrap();
678        let a = channel(&log, "A 3");
679        assert_eq!(a.quantity, Quantity::Acceleration);
680        assert_eq!(a.time_s, [0.002, 0.025]);
681        assert_eq!(a.values, [9.75, 9.80]);
682        let p = channel(&log, "Alt P");
683        assert_eq!(
684            (p.quantity, p.time_s.as_slice()),
685            (Quantity::Pressure, &[0.003, 0.026][..])
686        );
687        assert_eq!(channel(&log, "Alt MSL").quantity, Quantity::AltitudeMsl);
688        for kept in ["G 1", "G 2", "G 3", "HG 1", "HG 2", "HG 3"] {
689            assert_eq!(
690                channel(&log, kept).quantity,
691                Quantity::Unrecognized,
692                "{kept}"
693            );
694        }
695        assert!(log.time_s.is_empty() && log.altitude_m.is_empty());
696    }
697
698    #[test]
699    fn named_fields_take_their_groups_units() {
700        assert!(recognizes_named(NAMED_TEXT));
701        assert!(!recognizes_tabs(NAMED_TEXT));
702        let log = read_named(NAMED_TEXT).unwrap();
703        let za = channel(&log, "ICM za");
704        assert_eq!(za.quantity, Quantity::Acceleration);
705        assert_eq!(za.values[0], 1_000.0 * STANDARD_GRAVITY_MPS2 / 1_000.0);
706        assert_eq!(za.time_s, [1.0, 1.001]);
707        assert_eq!(channel(&log, "ICM yg").quantity, Quantity::AngularRate);
708        assert_eq!(channel(&log, "ICM xm").quantity, Quantity::Unrecognized);
709        assert_eq!(channel(&log, "ICM xm").values[0], 40.0);
710        assert_eq!(channel(&log, "ICM T").values[0], 25.0 + 273.15);
711        assert_eq!(channel(&log, "MS5 P").values, [95_000.0]);
712        assert_eq!(channel(&log, "MS5 T").values, [24.5 + 273.15]);
713        // One unit for two fields: both take it.
714        assert_eq!(channel(&log, "ADC T2").values, [21.5 + 273.15]);
715        // No units line: kept as written.
716        assert_eq!(channel(&log, "VOL V").quantity, Quantity::Unrecognized);
717        assert!(
718            log.notes
719                .iter()
720                .any(|n| n.starts_with("VOL's values kept as written"))
721        );
722        assert!(log.notes.iter().any(|n| n.contains("`ICM xm`, in milli-T")));
723    }
724
725    #[test]
726    fn units_that_dont_fit_the_groups_are_kept_as_written() {
727        let text = NAMED_TEXT.replace("MS5 data in us, deg C, Pa", "MS5 data in us, deg C, Pa, V");
728        let log = read_named(&text).unwrap();
729        assert_eq!(channel(&log, "MS5 P").quantity, Quantity::Unrecognized);
730        assert!(log.notes.iter().any(|n| n
731            == "MS5's values kept as written: its units line states 3 units for 2 groups of fields"));
732    }
733
734    #[test]
735    fn one_unit_goes_to_every_field_only_without_axes() {
736        let text = NAMED_TEXT.replace(
737            "ICM data in us, milli-g, deg/s, milli-T, deg C",
738            "ICM data in us, milli-g",
739        );
740        let log = read_named(&text).unwrap();
741        assert_eq!(channel(&log, "ICM xa").quantity, Quantity::Unrecognized);
742        assert_eq!(channel(&log, "ICM xg").quantity, Quantity::Unrecognized);
743        assert!(log.notes.iter().any(|n| n
744            == "ICM's values kept as written: its units line states 1 unit for 4 groups of fields"));
745        // `T1` and `T2` carry no axis: the one unit goes to both.
746        assert_eq!(channel(&log, "ADC T1").quantity, Quantity::Temperature);
747    }
748
749    #[test]
750    fn a_clock_not_in_microseconds_is_refused_by_its_line() {
751        let text = NAMED_TEXT.replace("MS5 data in us,", "MS5 data in ms,");
752        let error = read_named(&text).unwrap_err();
753        assert!(matches!(error, LogError::Unit { line: 2, .. }), "{error}");
754    }
755
756    #[test]
757    fn a_clock_that_doesnt_increase_is_refused_by_its_line() {
758        let text = TABS_TEXT.replace("A\t25000", "A\t2000");
759        let error = read(&text).unwrap_err();
760        let LogError::Syntax { line, message, .. } = &error else {
761            panic!("{error}");
762        };
763        assert_eq!(*line, 5);
764        assert!(message.contains("doesn't come after"), "{message}");
765    }
766
767    #[test]
768    fn a_line_laid_out_unlike_its_tags_first_is_refused() {
769        let text = TABS_TEXT.replace("A\t25000\t0.30\t-0.45\t9.80", "A\t25000\t0.30\t-0.45");
770        let error = read(&text).unwrap_err();
771        assert!(
772            matches!(&error, LogError::Syntax { line: 5, message, .. } if message.contains("unlike its first, on line 1")),
773            "{error}"
774        );
775    }
776
777    #[test]
778    fn a_cut_last_line_is_dropped_only_with_no_line_end() {
779        let cut = format!("{TABS_TEXT}A\t48000\t0.31\t-0.4");
780        let log = read(&cut).unwrap();
781        assert_eq!(channel(&log, "A 1").values.len(), 2);
782        assert!(log.notes.iter().any(|n| n == DROPPED_LAST));
783        let ended = format!("{cut}\r\n");
784        let error = read(&ended).unwrap_err();
785        assert!(matches!(error, LogError::Syntax { line: 9, .. }), "{error}");
786        // A bad last line with a line end and blank space after it is refused, not dropped.
787        let trailing = format!("{cut}\r\n \t");
788        let error = read(&trailing).unwrap_err();
789        assert!(matches!(error, LogError::Syntax { line: 9, .. }), "{error}");
790        // A whole last line with no line end is read.
791        let whole = TABS_TEXT.trim_end();
792        let log = read(whole).unwrap();
793        assert_eq!(channel(&log, "HG 1").values.len(), 2);
794        assert!(!log.notes.iter().any(|n| n == DROPPED_LAST));
795    }
796
797    #[test]
798    fn a_clock_past_32_bits_or_not_whole_is_refused() {
799        assert!(clock("4294967295").is_ok());
800        assert!(clock("4294967296").unwrap_err().contains("32-bit"));
801        for bad in ["", "1.5", "-3", "+3", "12a", "99999999999999999999999"] {
802            assert!(clock(bad).is_err(), "{bad}");
803        }
804    }
805
806    #[test]
807    fn the_bounds_hold_at_their_limits() {
808        let at = |count: usize| {
809            let values = vec!["1"; count].join("\t");
810            format!("X\t1\t{values}\r\nAlt\t2\tP\t95000\tMSL\t540.42\r\n")
811        };
812        assert!(read(&at(MAX_VALUES)).is_ok());
813        assert!(!recognizes_tabs(&at(MAX_VALUES + 1)));
814        let long = "N".repeat(MAX_NAME_BYTES);
815        let named = |tag: &str| format!("{tag}\t1\t1\r\nAlt\t2\tP\t95000\tMSL\t540.42\r\n");
816        assert!(read(&named(&long)).is_ok());
817        assert!(!recognizes_tabs(&named(&format!("{long}N"))));
818        let mut many = String::from("Alt\t1\tP\t95000\tMSL\t540.42\r\n");
819        for tag in 1..MAX_TAGS {
820            many.push_str(&format!("T{tag}\t1\t1\r\n"));
821        }
822        assert!(read(&many).is_ok());
823        many.push_str("Last\t1\t1\r\n");
824        let error = read(&many).unwrap_err();
825        assert!(
826            matches!(&error, LogError::Syntax { message, .. } if message.contains("more than 32 tags")),
827            "{error}"
828        );
829    }
830
831    #[test]
832    fn neither_layout_takes_the_others_or_a_table() {
833        for text in [
834            "Time,Altitude\n0,0\n1,5\n",
835            "A\t1\t2\t3\n",
836            "ICM, t, 1, xa, 2\n",
837            "ICM data in ms, g\nICM, t, 1, xa, 2\n",
838        ] {
839            assert!(!recognizes_tabs(text) && !recognizes_named(text), "{text}");
840        }
841    }
842
843    #[test]
844    fn a_value_too_large_for_its_conversion_is_refused_by_its_line() {
845        let text = NAMED_TEXT.replace("za, 999.00", "za, 1.7e308");
846        let error = read_named(&text).unwrap_err();
847        assert!(
848            matches!(&error, LogError::Syntax { line: 8, message, .. } if message.contains("ICM za")),
849            "{error}"
850        );
851    }
852
853    #[test]
854    fn a_name_twice_or_a_value_with_no_name_is_refused() {
855        let twice = NAMED_TEXT.replace("T, 24.50, P, 95000", "P, 24.50, P, 95000");
856        assert!(matches!(
857            read_named(&twice),
858            Err(LogError::Syntax { line: 5, .. })
859        ));
860        let unnamed = NAMED_TEXT.replace("V, 4.100", "4.100, 4.2");
861        assert!(matches!(
862            read_named(&unnamed),
863            Err(LogError::Syntax { line: 7, .. })
864        ));
865    }
866}