Skip to main content

hpr_flightdata/
marsa.rs

1//! A MARSA altimeter's flight download: its settings, then a row a sample of the time, the
2//! barometric altitude and the acceleration, as MARSA Systems' software saves it to a `.dat` file,
3//! or as a spreadsheet re-saves that file.
4//!
5//! MARSA Systems' MARSA4 and MARSA54 flight computers share their firmware and their PC software,
6//! MARSA_Connect (MARSA's home page, 2011), which downloads a flight from the altimeter's memory
7//! and saves it to the computer (*MARSA54L Flight Computer User Manual*, V1.4, 2014, p. 18). No
8//! document describes the file, so this reader takes the layout from the two sample files, and
9//! the sample logs' survey (`cargo xtask logs`) holds each unit to a check ([ADR-274][adr-274]).
10//! An invented download, cut short:
11//!
12//! ```text
13//! <BARO_ZERO 0> 790
14//! <ACCEL_ZERO 0> 498
15//! <LAUNCH TIME 0> 12000
16//! <FIRMWARE VER> 26
17//! <MARSA VERSION> 2
18//! <CHAN 1>
19//! <CHAN APOGEE OPTION> 1
20//! <CHAN ALTITUDE SETTING> 500
21//! -0.2,-58.1,-0.08
22//! -0.1,-58.1,-0.08
23//! 0.0,-58.1,8.20
24//! 0.1,-29.0,8.65
25//! ```
26//!
27//! **The settings.** Each line before the samples is a setting, `<NAME> value`, or a section's
28//! heading, `<NAME>` alone, such as `<CHAN 1>`, which opens a pyro channel's settings. Each setting
29//! is a [`Figure`] of the record's [`Stated::figures`], named as the file names it, after its
30//! section's heading where it has one (`CHAN 1, CHAN APOGEE OPTION`); its value as written, and a
31//! number of [`Quantity::Number`] where it is one, as no document gives the settings' units. The
32//! MARSA4 manual describes two: `Baro0` and `Accel0`, the file's `BARO_ZERO` and `ACCEL_ZERO`, are
33//! "the A/D values of the ground barometric and sensor readings" (*MARSA4 User Manual*, Version
34//! 1.0, 2009, p. 17). `FIRMWARE VER` is the record's [`FlightLog::firmware`]. A setting named twice
35//! in one section is refused, as no reader could tell which value is the altimeter's.
36//!
37//! **The samples.** Each line after the settings is a sample: three numbers by commas, which no
38//! line names. They are read as the time, s, from the logger's zero at liftoff; the barometric
39//! altitude, in feet, above the logger's own zero; and the acceleration, in units of standard
40//! gravity, 9.806 65 m/s², its reading at rest taken out. MARSA's manuals give the altitude in
41//! feet: the altimeter's display reads "898 ft" in the MARSA54L manual's example (p. 12), and its
42//! altitude events are set "in 100 ft increments". The software plots "Baro altitude and
43//! acceleration" (MARSA4 manual, p. 18), the altitude "based on the barometric pressure according
44//! to the standard atmospheric model" (p. 16). The survey holds the units to the files: each
45//! file's apogee against an AltOS flight computer's in the same rocket, the feet's check, and the
46//! barometric climb over the first seconds after liftoff against the acceleration integrated
47//! twice, the g's.
48//!
49//! - The altitude is [`Quantity::AltitudeAboveGround`] and the record's primary track. It moves
50//!   by its barometer's count, not by the tenth of a foot it is written in, and it needn't read 0
51//!   on the pad: the logger's zero is its own ([`FlightLog::altitude_m`]), and in both sample
52//!   files the pad reads several counts below it. The survey reports it.
53//! - The acceleration is [`Quantity::Acceleration`]: it reads about 0 at rest, not 1 g, and so
54//!   it is the rocket's acceleration along its axis while the rocket points straight up, as
55//!   AltOS's is ([`crate::altos`]).
56//!
57//! A sample's time must be later than the one before, or the file is refused at its line: in
58//! both sample files the steps are a tenth of a second early in the flight and grow later. A last
59//! line the file ends in with no line end, which isn't a whole sample, is dropped with a note: a
60//! download cut off as it was written.
61//!
62//! **A spreadsheet's re-save.** One sample file is a download a spreadsheet re-saved as CSV: each
63//! setting's line ends in two empty cells, `<BARO_ZERO 0> 790,,`, and the numbers are rewritten,
64//! `8.2` for `8.20`. Empty cells at a line's end are passed over, so both read the same.
65//!
66//! **Bounds.** A file may hold at most [`MAX_SETTINGS`] settings and headings, a name
67//! [`MAX_NAME_BYTES`] bytes and a value [`MAX_VALUE_CHARS`] characters; past them it is refused,
68//! so a hostile file can't make the record's figures fill memory.
69//!
70//! [adr-274]: https://github.com/nrdptel/fusionspace-eridanus/blob/main/docs/decisions/0274-the-rest-of-the-importers-planned-teensy-logger-first.md
71
72use std::collections::BTreeSet;
73
74use crate::error::LogError;
75use crate::log::{Channel, Figure, FlightLog, LogFormat, Quantity, Stated};
76use crate::table::finite;
77
78/// The format's name in errors.
79const FORMAT: &str = "MARSA download";
80
81/// The lines [`recognizes`] looks through for the `<MARSA VERSION>` setting.
82const RECOGNITION_LINES: usize = 200;
83
84/// The most settings and headings a file may hold.
85pub const MAX_SETTINGS: usize = 256;
86
87/// The longest name a setting may have, in bytes.
88pub const MAX_NAME_BYTES: usize = 64;
89
90/// The longest value a setting may have, in characters.
91pub const MAX_VALUE_CHARS: usize = 64;
92
93/// The setting whose value is the firmware's version.
94const FIRMWARE: &str = "FIRMWARE VER";
95
96/// The altitude's step, m: its barometer's count, 29.3 ft at most within 300 ft of the pad among
97/// the sample files ([`LogFormat::altitude_resolution_m`]). It is measured there, not stated by
98/// any document, and holds near the pad only: a count of pressure spans more feet where the air
99/// is thinner, so the step is larger higher up.
100pub const ALTITUDE_STEP_M: f64 = 29.3 * FOOT_M;
101
102/// A foot, m.
103const FOOT_M: f64 = 0.3048;
104
105/// Standard gravity, m/s² (the 3rd CGPM, 1901).
106const STANDARD_GRAVITY_M_S2: f64 = 9.806_65;
107
108/// The columns, as the record names them: no line of the file names them.
109const COLUMNS: [&str; 2] = ["altitude", "acceleration"];
110
111/// A line with any empty cells at its end, a spreadsheet's re-save's, taken off.
112fn trimmed(line: &str) -> &str {
113    line.trim()
114        .trim_end_matches(|c: char| c == ',' || c.is_whitespace())
115}
116
117/// A setting's or heading's name and value, `<NAME> value` or `<NAME>`: `None` for a line that
118/// isn't one. The name is ASCII letters, digits, spaces, `_`, `-` and `.`, as MARSA's are, such as
119/// `CHAN ALT BITS 0 1`: an XML document's `<?xml …?>` or `<kml>` isn't a setting.
120fn setting(line: &str) -> Option<(&str, &str)> {
121    let rest = line.strip_prefix('<')?;
122    let (name, value) = rest.split_once('>')?;
123    let allowed = |c: char| c.is_ascii_alphanumeric() || matches!(c, ' ' | '_' | '-' | '.');
124    if name.trim().is_empty() || !name.chars().all(allowed) {
125        return None;
126    }
127    Some((name.trim(), value.trim()))
128}
129
130/// Whether a text is a MARSA download: its first line that isn't blank is a setting or a
131/// heading, and one of its first 200 is the `<MARSA VERSION>` setting with a value.
132pub fn recognizes(text: &str) -> bool {
133    let text = text.strip_prefix('\u{feff}').unwrap_or(text);
134    let mut lines = text
135        .split(['\r', '\n'])
136        .map(trimmed)
137        .filter(|line| !line.is_empty());
138    let Some(first) = lines.next() else {
139        return false;
140    };
141    setting(first).is_some()
142        && std::iter::once(first)
143            .chain(lines)
144            .take(RECOGNITION_LINES)
145            .any(
146                |line| matches!(setting(line), Some(("MARSA VERSION", value)) if !value.is_empty()),
147            )
148}
149
150/// A sample's three numbers, or why the line isn't one.
151fn sample(line: &str) -> Result<[f64; 3], String> {
152    let cells: Vec<&str> = line.split(',').map(str::trim).collect();
153    let [time, altitude, acceleration] = cells.as_slice() else {
154        return Err(format!(
155            "a sample is three numbers by commas, and the line has {} cells",
156            cells.len()
157        ));
158    };
159    let number = |cell: &str| {
160        cell.parse::<f64>()
161            .ok()
162            .filter(|value| value.is_finite())
163            .ok_or_else(|| format!("{:?} isn't a number", short(cell)))
164    };
165    Ok([number(time)?, number(altitude)?, number(acceleration)?])
166}
167
168/// Reads a MARSA download from its text.
169///
170/// # Errors
171///
172/// [`LogError::NotThisFormat`] if the text isn't one ([`recognizes`]); [`LogError::Syntax`] for
173/// a setting past the bounds or named twice in its section, a setting among the samples, a line
174/// that isn't three numbers, a time no later than the one before, or a value too large for its
175/// conversion to SI; [`LogError::NoData`] for a file with no samples.
176pub fn read(text: &str) -> Result<FlightLog, LogError> {
177    let text = text.strip_prefix('\u{feff}').unwrap_or(text);
178    if !recognizes(text) {
179        return Err(LogError::NotThisFormat {
180            format: FORMAT,
181            message: "the first line isn't a `<NAME> value` setting, or no `<MARSA VERSION>` \
182                      setting comes within the first 200 lines"
183                .to_owned(),
184        });
185    }
186    let syntax = |line: usize, message: String| LogError::Syntax {
187        format: FORMAT,
188        line,
189        message,
190    };
191    let ends_unended = !text.ends_with(['\r', '\n']);
192    // Lines end in CR, LF or both.
193    let text = text.replace("\r\n", "\n").replace('\r', "\n");
194    let lines: Vec<(usize, &str)> = text
195        .split('\n')
196        .enumerate()
197        .map(|(index, line)| (index + 1, trimmed(line)))
198        .filter(|(_, line)| !line.is_empty())
199        .collect();
200    let last_line = lines.last().map(|&(number, _)| number);
201
202    let mut figures = Vec::new();
203    let mut named = BTreeSet::new();
204    let mut section: Option<&str> = None;
205    let mut settings = 0;
206    let mut firmware = None;
207    let mut time_s: Vec<f64> = Vec::new();
208    let mut altitude_m = Vec::new();
209    let mut acceleration = Vec::new();
210    let mut dropped_last = false;
211    for (number, line) in lines {
212        if let Some((name, value)) = setting(line) {
213            if !time_s.is_empty() {
214                return Err(syntax(
215                    number,
216                    "a setting among the samples: the settings come before them".to_owned(),
217                ));
218            }
219            settings += 1;
220            if settings > MAX_SETTINGS {
221                return Err(syntax(
222                    number,
223                    format!("more than {MAX_SETTINGS} settings and headings"),
224                ));
225            }
226            if name.len() > MAX_NAME_BYTES || value.chars().count() > MAX_VALUE_CHARS {
227                return Err(syntax(
228                    number,
229                    format!(
230                        "a setting's name is at most {MAX_NAME_BYTES} bytes and its value \
231                         {MAX_VALUE_CHARS} characters: {:?}",
232                        short(line)
233                    ),
234                ));
235            }
236            if value.is_empty() {
237                section = Some(name);
238                continue;
239            }
240            let full = match section {
241                Some(heading) => format!("{heading}, {name}"),
242                None => name.to_owned(),
243            };
244            if !named.insert(full.clone()) {
245                return Err(syntax(
246                    number,
247                    format!("the setting `{}` is named twice", short(&full)),
248                ));
249            }
250            if section.is_none() && name == FIRMWARE {
251                firmware = Some(value.to_owned());
252            }
253            let figure = match value.parse::<f64>().ok().filter(|v| v.is_finite()) {
254                Some(number) => Figure::new(full, value, Some(Quantity::Number), vec![number]),
255                None => Figure::new(full, value, None, Vec::new()),
256            };
257            figures.push(figure);
258            continue;
259        }
260        let [time, altitude, accel] = match sample(line) {
261            Ok(values) => values,
262            Err(_) if ends_unended && Some(number) == last_line => {
263                dropped_last = true;
264                continue;
265            }
266            Err(message) => return Err(syntax(number, message)),
267        };
268        if let Some(&before) = time_s.last()
269            && time <= before
270        {
271            return Err(syntax(
272                number,
273                format!("the time {time} s isn't later than the one before, {before} s"),
274            ));
275        }
276        time_s.push(time);
277        altitude_m.push(finite(altitude * FOOT_M, FORMAT, number, COLUMNS[0])?);
278        acceleration.push(finite(
279            accel * STANDARD_GRAVITY_M_S2,
280            FORMAT,
281            number,
282            COLUMNS[1],
283        )?);
284    }
285    if time_s.is_empty() {
286        return Err(LogError::NoData { format: FORMAT });
287    }
288
289    let mut notes = vec![
290        "no line names the samples' three numbers. They are read as the time in s, the \
291         barometric altitude in ft and the acceleration in g, as MARSA's manuals and software \
292         give them. The sample logs' survey holds the feet and the g to the files"
293            .to_owned(),
294        "the acceleration reads about 0 at rest, not 1 g: it is kept so, the rocket's own \
295         acceleration only while it points straight up"
296            .to_owned(),
297        "the settings are kept as written, as no document states their units".to_owned(),
298    ];
299    if dropped_last {
300        notes.push(
301            "the last line, which ends the file with no line end and isn't a whole sample, \
302             dropped: a download cut off as it was written"
303                .to_owned(),
304        );
305    }
306    let channel = |column: &str, quantity, values: &[f64]| Channel {
307        column: column.to_owned(),
308        quantity,
309        time_s: time_s.clone(),
310        values: values.to_vec(),
311    };
312    let channels = vec![
313        channel(COLUMNS[0], Quantity::AltitudeAboveGround, &altitude_m),
314        channel(COLUMNS[1], Quantity::Acceleration, &acceleration),
315    ];
316    Ok(FlightLog {
317        format: LogFormat::Marsa,
318        logger: "MARSA".to_owned(),
319        serial_number: None,
320        firmware,
321        flight_number: None,
322        stated: Stated {
323            apogee_m: None,
324            ground_elevation_msl_m: None,
325            samples: None,
326            figures,
327        },
328        time_s,
329        altitude_m,
330        vertical_speed_m_s: None,
331        temperature_k: None,
332        battery_v: None,
333        channels,
334        events: Vec::new(),
335        notes,
336    })
337}
338
339/// The most characters of the file an error quotes.
340const QUOTED_CHARS: usize = 80;
341
342/// Text from the file as an error quotes it, cut to [`QUOTED_CHARS`] characters.
343fn short(text: &str) -> String {
344    if text.chars().count() <= QUOTED_CHARS {
345        text.to_owned()
346    } else {
347        let mut cut: String = text.chars().take(QUOTED_CHARS - 1).collect();
348        cut.push('…');
349        cut
350    }
351}
352
353#[cfg(test)]
354mod tests {
355    use super::*;
356
357    /// An invented download: settings, a section, and four samples, CRLF and a trailing space as
358    /// the altimeter's software writes them.
359    const DOWNLOAD: &str = "<BARO_ZERO 0> 790\r\n<ACCEL_ZERO 0> 498\r\n<FIRMWARE VER> 26\r\n\
360                            <MARSA VERSION> 2\r\n<CHAN 1>\r\n<CHAN APOGEE OPTION> 1\r\n\
361                            <CHAN 2>\r\n<CHAN APOGEE OPTION> 0\r\n\
362                            -0.1,-58.1,-0.08 \r\n0.0,-58.1,8.20 \r\n0.1,-29.0,8.65 \r\n\
363                            0.2,0.0,9.00 \r\n";
364
365    fn error_line(text: &str) -> (usize, String) {
366        match read(text) {
367            Err(LogError::Syntax { line, message, .. }) => (line, message),
368            other => panic!("expected a syntax error, got {other:?}"),
369        }
370    }
371
372    #[test]
373    fn the_samples_read_in_seconds_feet_and_g() {
374        assert!(recognizes(DOWNLOAD));
375        let log = read(DOWNLOAD).unwrap();
376        assert_eq!(log.format, LogFormat::Marsa);
377        assert_eq!(log.time_s, [-0.1, 0.0, 0.1, 0.2]);
378        assert_eq!(
379            log.altitude_m,
380            [-58.1 * 0.3048, -58.1 * 0.3048, -29.0 * 0.3048, 0.0]
381        );
382        let acceleration = &log.channels[1];
383        assert_eq!(
384            (acceleration.column.as_str(), acceleration.quantity),
385            ("acceleration", Quantity::Acceleration)
386        );
387        assert_eq!(acceleration.values[1], 8.20 * 9.806_65);
388        assert_eq!(log.channels[0].quantity, Quantity::AltitudeAboveGround);
389        assert_eq!(log.firmware.as_deref(), Some("26"));
390    }
391
392    #[test]
393    fn each_setting_is_a_figure_named_after_its_section() {
394        let log = read(DOWNLOAD).unwrap();
395        let names: Vec<&str> = log.stated.figures.iter().map(|f| f.name.as_str()).collect();
396        assert_eq!(
397            names,
398            [
399                "BARO_ZERO 0",
400                "ACCEL_ZERO 0",
401                "FIRMWARE VER",
402                "MARSA VERSION",
403                "CHAN 1, CHAN APOGEE OPTION",
404                "CHAN 2, CHAN APOGEE OPTION",
405            ]
406        );
407        let zero = &log.stated.figures[0];
408        assert_eq!(
409            (zero.written.as_str(), zero.quantity, zero.values.as_slice()),
410            ("790", Some(Quantity::Number), &[790.0][..])
411        );
412    }
413
414    #[test]
415    fn a_spreadsheets_resave_reads_the_same() {
416        let resaved = DOWNLOAD
417            .replace(" \r\n", "\r\n")
418            .replace("> 790\r\n", "> 790,,\r\n")
419            .replace("<CHAN 1>\r\n", "<CHAN 1>,,\r\n")
420            .replace("0.2,0.0,9.00", "0.2,0,9");
421        assert_ne!(resaved, DOWNLOAD);
422        let (a, b) = (read(DOWNLOAD).unwrap(), read(&resaved).unwrap());
423        assert_eq!(a.time_s, b.time_s);
424        assert_eq!(a.altitude_m, b.altitude_m);
425        assert_eq!(a.channels, b.channels);
426        assert_eq!(a.stated, b.stated);
427    }
428
429    #[test]
430    fn a_time_no_later_than_the_one_before_is_refused_at_its_line() {
431        let text = DOWNLOAD.replace("0.1,-29.0", "0.0,-29.0");
432        let (line, message) = error_line(&text);
433        assert_eq!(line, 11);
434        assert!(message.contains("isn't later"), "{message}");
435    }
436
437    #[test]
438    fn a_line_that_isnt_three_numbers_is_refused_at_its_line() {
439        for (row, expected) in [
440            ("0.1,-29.0", "cells"),
441            ("0.1,-29.0,8.65,1", "cells"),
442            ("0.1,x,8.65", "isn't a number"),
443            ("0.1,inf,8.65", "isn't a number"),
444        ] {
445            let text = DOWNLOAD.replace("0.1,-29.0,8.65 ", row);
446            let (line, message) = error_line(&text);
447            assert_eq!(line, 11, "{row}");
448            assert!(message.contains(expected), "{row}: {message}");
449        }
450    }
451
452    #[test]
453    fn a_setting_among_the_samples_or_named_twice_is_refused() {
454        let among = DOWNLOAD.replace("0.1,-29.0,8.65 ", "<CHAN 3>");
455        assert!(error_line(&among).1.contains("among the samples"));
456        let twice = DOWNLOAD.replace("<ACCEL_ZERO 0>", "<BARO_ZERO 0>");
457        let (line, message) = error_line(&twice);
458        assert_eq!(line, 2);
459        assert!(
460            message.contains("`BARO_ZERO 0` is named twice"),
461            "{message}"
462        );
463        // One name in two sections is two settings.
464        assert!(read(DOWNLOAD).is_ok());
465    }
466
467    #[test]
468    fn a_cut_last_line_is_dropped_only_with_no_line_end() {
469        let cut = format!("{DOWNLOAD}0.3,29");
470        let log = read(&cut).unwrap();
471        assert_eq!(log.time_s.len(), 4);
472        assert!(log.notes.iter().any(|n| n.contains("cut off")));
473        let ended = format!("{DOWNLOAD}0.3,29\r\n");
474        assert_eq!(error_line(&ended).0, 13);
475        let whole = format!("{DOWNLOAD}0.3,29.1,9.1");
476        let log = read(&whole).unwrap();
477        assert_eq!(log.time_s.len(), 5);
478        assert!(!log.notes.iter().any(|n| n.contains("cut off")));
479    }
480
481    #[test]
482    fn the_bounds_hold_at_their_limits() {
483        let settings: String = (1..MAX_SETTINGS)
484            .map(|i| format!("<S {i}> {i}\n"))
485            .collect();
486        let text = format!("<MARSA VERSION> 2\n{settings}0,0,0\n");
487        assert_eq!(read(&text).unwrap().stated.figures.len(), MAX_SETTINGS);
488        let over = format!("<S 0> 0\n{text}");
489        assert!(error_line(&over).1.contains("more than 256"));
490        let long_name = "N".repeat(MAX_NAME_BYTES);
491        let at = format!("<{long_name}> 1\n<MARSA VERSION> 2\n0,0,0\n");
492        assert!(read(&at).is_ok());
493        let past = format!("<{long_name}N> 1\n<MARSA VERSION> 2\n0,0,0\n");
494        assert!(error_line(&past).1.contains("at most 64 bytes"));
495        let value = "9".repeat(MAX_VALUE_CHARS);
496        assert!(read(&format!("<V> {value}\n<MARSA VERSION> 2\n0,0,0\n")).is_ok());
497        let past = format!("<V> {value}9\n<MARSA VERSION> 2\n0,0,0\n");
498        assert_eq!(error_line(&past).0, 1);
499    }
500
501    #[test]
502    fn a_value_too_large_for_its_conversion_is_refused() {
503        let text = DOWNLOAD.replace("0.1,-29.0,8.65", "0.1,-29.0,1e308");
504        let (line, message) = error_line(&text);
505        assert_eq!(line, 11);
506        assert!(message.contains("acceleration"), "{message}");
507    }
508
509    /// A byte-order mark, as a spreadsheet's "CSV UTF-8" re-save writes, is passed over.
510    #[test]
511    fn a_byte_order_mark_is_passed_over() {
512        let marked = format!("\u{feff}{DOWNLOAD}");
513        assert!(recognizes(&marked));
514        assert_eq!(read(&marked).unwrap(), read(DOWNLOAD).unwrap());
515    }
516
517    #[test]
518    fn a_file_with_no_version_setting_or_no_samples_isnt_read() {
519        let unversioned = DOWNLOAD.replace("<MARSA VERSION> 2\r\n", "");
520        assert!(!recognizes(&unversioned));
521        assert!(matches!(
522            read(&unversioned),
523            Err(LogError::NotThisFormat { .. })
524        ));
525        // A version with no value is a heading, not the setting.
526        assert!(!recognizes(
527            &DOWNLOAD.replace("<MARSA VERSION> 2", "<MARSA VERSION>")
528        ));
529        let settings_only = "<BARO_ZERO 0> 790\n<MARSA VERSION> 2\n";
530        assert!(matches!(read(settings_only), Err(LogError::NoData { .. })));
531        // An XML document isn't a download, even one holding the setting.
532        assert!(!recognizes(
533            "<?xml version=\"1.0\"?>\n<MARSA VERSION> 2\n0,0,0\n"
534        ));
535        let kml = "<kml xmlns=\"http://www.opengis.net/kml/2.2\">\n<MARSA VERSION> 2\n</kml>\n";
536        assert!(!recognizes(kml));
537    }
538}