Skip to main content

hpr_flightdata/
entacore_raw.rs

1//! Entacore AIM XTRA's own files: a flight as the AIM software saves it to an `.xtra` file, and
2//! the flight computer's memory as the software downloads it, raw, before any conversion.
3//!
4//! The AIM software saves a flight in its own `.xtra` file and exports it to CSV
5//! ([`entacore`](crate::entacore)); the manual names both but describes neither layout (*AIM
6//! XTRA/BASE v4.x.x Manual*, Entacore, 2 February 2025, p38). The layouts here were worked out from
7//! one flight's three files among the sample logs: a raw dump of the computer's memory, the
8//! `.xtra` the software made of it, and an earlier copy of that `.xtra`, cut short. The raw dump's
9//! last recording reads, by the rules below, to the `.xtra`'s values bit for bit, and the short
10//! copy's bytes are the `.xtra`'s first ones. So every rule here is checked against the
11//! maker's own conversion, on one flight.
12//!
13//! **What the records hold.** One record a sample of one group of readings, each group at its own
14//! rate (the manual gives the barometer every 20 ms, the accelerometers and gyroscope every 10 ms,
15//! the magnetometer every 100 ms, the temperature every 500 ms, and the GPS's position every
16//! 200 ms and its time every 500 ms, p2, each written at a multiple of its period the owner sets
17//! for each phase of the flight, p21). Each
18//! group is a kind of record, and each of its fields a channel, on the clock of that kind's
19//! records. The names are the CSV export's, where its column is the same sensor:
20//!
21//! | Kind: `.xtra` class, raw type | Channels | Quantity, unit | Raw value |
22//! |---|---|---|---|
23//! | 2, `0x02` | `acceleration` | [`Quantity::Acceleration`], g × 9.806 65 m/s², the CSV's | i16 / 256 |
24//! | 3, `0x0b` | `lat. X accel.`, `lat. Y accel.` | the same | i16 / 256 each |
25//! | 15, `0x0c` | `Gyro X`, `Gyro Y`, `Gyro Z` | [`Quantity::AngularRate`], °/s (p24) | i16 × 0.07 each |
26//! | 4, `0x03` | `pressure` | [`Quantity::Pressure`], Pa | u24 |
27//! | 5, `0x04`; 6, `0x05` | `battery 1 voltage`, `battery 2 voltage` | [`Quantity::Voltage`], V | (u16 ÷ 16) × (3.3 × 133 / 33) / 4096, the factor in single precision |
28//! | 7, `0x06` | `temperature` | [`Quantity::Temperature`], °C + 273.15 K | i16 × 0.01 |
29//! | 17–20, `0x07`–`0x0a` | `A Voltage` … `D Voltage` | [`Quantity::Voltage`], V | u16 × 3.3 / 16,384 |
30//! | 16, `0x0d` | `Mag X`, `Mag Y`, `Mag Z` | [`Quantity::Unrecognized`], as written | i16 × 10⁻⁶ each |
31//! | 10, `0x0e` | `lat`, `long`, `GPS MSL`, `SVs`, `GPS flag` | degrees, degrees, [`Quantity::GnssAltitude`] m, [`Quantity::Number`], as written | i32 × f32(10⁻⁷), i32 × f32(10⁻⁷), i32 mm as an f32 × f32(0.001), a byte's bits 0–4, its bit 5 |
32//! | 14, `0x12` | `GPS Time of Week`, `GPS week`, `GPS time flag 1`, `GPS time flag 2`, `GPS leap seconds` | as written (ms, as the CSV's), [`Quantity::Number`], as written, as written, [`Quantity::Number`] | u32, u16, a byte's bit 7, its bit 6, its bits 0–5 |
33//! | 13, `0x10` | `state`, `A on` … `D on`, `A cont` … `D cont` | [`Quantity::Number`] | a u16's top 4 bits (0, 1, 3, 5 read as 0, 1, 2, 3), bits 8–11, bits 7 to 4 |
34//! | 21, `0x15` | `quaternion 1` … `quaternion 4` | [`Quantity::Unrecognized`], as written | i16 / 32,768 each |
35//!
36//! The raw values are big-endian, and are converted as the software converts them, in 32-bit
37//! floats, so that the two readers' channels are equal value for value. The accelerations are
38//! recorded in g, as the CSV's are, and read in m/s²; `lat. X accel.` reads 16 g at −4,096, the
39//! 16 g its sensor measures (p2).
40//! Which battery is the computer's and which the ejection charges' isn't known: both read 6.6 V in
41//! the sample. The magnetometer's unit isn't settled, as for the CSV. The quaternion is kept as
42//! written. The `on` and `cont` names, and the letters, are inferred from the sample: the CSV
43//! names four outputs' firing and continuity flags so, and in the sample C and D fire and lose
44//! continuity when the recovery charges do; A and B never change. What `state`'s numbers mean
45//! isn't stated either: in the sample it turns 1 about a second into the boost, 2 later in the
46//! climb and 3 at apogee.
47//!
48//! **Events.** Each change of `state` is an event named `state N`, and each rise of an `on` flag
49//! from 0 to 1 an event named by its channel, at the record's time.
50//!
51//! **No primary track.** Neither file holds an altitude above the pad, so
52//! [`FlightLog::time_s`] and [`FlightLog::altitude_m`] are empty, as for the Raven's raw `.FIPa`,
53//! and every value is a channel.
54//!
55//! **The `.xtra` file** is a Boost serialization binary archive: a signature,
56//! `serialization::archive`, the archive's version (16), the sizes of an int, a long, a float and
57//! a double (4, 4, 4, 8), the top object's header, and the record count as a u64. Each record is
58//! then its class as an i16 (followed, the first time the class appears, by a byte 1 and the
59//! class's version as a u32), its object id (twice its index), on the first record only a byte 1
60//! and a u32 1, its base object's id (one more), the base's fields (an f32 of 3.3, a byte 0, and
61//! the record's time in seconds as an f32), then the kind's fields, little-endian. A class whose
62//! version isn't the sample's is refused, as its fields may differ. A file that ends inside a
63//! record is read to its last whole one, with a note; bytes after the stated count, 114 in the
64//! sample, aren't read.
65//!
66//! **The raw dump** ([`read_dump`]) is the computer's memory: recordings back to back, then
67//! erased memory, `0xFF`, to its end. A recording opens with type `0x13` and 101 bytes this
68//! reader doesn't read. A byte from `0x80` to `0xFE` adds its low 7 bits to the clock, in the
69//! computer's ticks; any other byte is a record's type, its fields following. The clock is read as
70//! the ticks in milliseconds from the recording's start, as the software reads them. Two things
71//! the dump and the `.xtra` differ in, both the software's: it sums the `.xtra`'s times in 32-bit
72//! floats, a tick at a time, so they fall behind the ticks as the sum grows: in the sample, by
73//! 2.40 s by the end of the recording, and by 0.9 % of the climb's time; and it leaves the dump's
74//! last record out. Neither file's clock is GPS time: over the sample's flight the computer's tick
75//! is 1.009 ms of the GPS's own clock, 1.021 on the pad, so the dump's times run 0.9 % short in
76//! flight and 2.1 % on the pad; the `.xtra`'s, behind the ticks as well, run about 1.8 % short
77//! over the climb (0.9910 ÷ 1.0087 = 0.982).
78//!
79//! **Held to the sample.** Each byte whose meaning the sample doesn't show is held to the one
80//! value it has in every sample record, and a file where it differs is refused, naming the byte:
81//! the `.xtra`'s base fields, 3.3 and 0, the output lines' two flag bytes, 0, and the status's
82//! third set of four bytes, 0; the dump's battery bits 0–3, the output lines' bits 14–15, the
83//! GPS byte's bits 6–7, the status's bits 0–3 and a state number other than the four.
84//! [`recognizes_xtra`] and [`recognizes_dump`] don't read these, so a file that differs there is
85//! refused by name, not left unrecognized. A type or class the sample doesn't hold is refused
86//! too, as its length is unknown; among a dump's first 1,000 records, or as an `.xtra`'s first
87//! record, it leaves the file unrecognized instead.
88//!
89//! ```
90//! use hpr_flightdata::entacore_raw;
91//!
92//! // An invented raw dump: a recording's header, 10 ticks, a pressure of 97,700 Pa (0x017DA4),
93//! // 20 ticks, another, then erased memory.
94//! let mut bytes = vec![0x13];
95//! bytes.extend([0; 101]);
96//! bytes.extend([0x8A, 0x03, 0x01, 0x7D, 0xA4, 0x94, 0x03, 0x01, 0x7D, 0x90]);
97//! bytes.extend([0xFF; 16]);
98//! let recordings = entacore_raw::read_dump(&bytes)?;
99//! let pressure = &recordings[0].channels[0];
100//! assert_eq!(pressure.column, "pressure");
101//! assert_eq!(pressure.time_s, [0.01, 0.03]);
102//! assert_eq!(pressure.values, [97_700.0, 97_680.0]);
103//! # Ok::<(), hpr_flightdata::error::LogError>(())
104//! ```
105
106use std::collections::BTreeMap;
107
108use hpr_core::gravity::STANDARD_GRAVITY_MPS2;
109
110use crate::error::LogError;
111use crate::log::{Channel, Event, FlightLog, LogFormat, Quantity, Stated};
112
113const XTRA_FORMAT: &str = "Entacore AIM .xtra";
114const DUMP_FORMAT: &str = "Entacore AIM raw dump";
115
116/// The `.xtra`'s first 36 bytes: the signature's length and text, the archive's version, the
117/// sizes of an int, a long, a float and a double, and 1, which tells the byte order.
118const XTRA_SIGNATURE: &[u8; 36] =
119    b"\x16\0\0\0serialization::archive\x10\0\x04\x04\x04\x08\x01\0\0\0";
120
121/// The 10 bytes after the signature, the top object's header, as the sample writes them.
122const XTRA_TOP: [u8; 10] = [0, 3, 0, 0, 0, 0, 0, 0, 0, 0];
123
124/// Where the record count and the first record are.
125const XTRA_COUNT_AT: usize = 46;
126const XTRA_RECORDS_AT: usize = 54;
127
128/// The base's first field, as every sample record holds it.
129const XTRA_BASE_FIRST: f32 = 3.3;
130
131/// A recording's opening type in a raw dump, and the bytes that follow it.
132const DUMP_RECORDING: u8 = 0x13;
133const DUMP_RECORDING_BYTES: usize = 101;
134
135/// Erased memory.
136const ERASED: u8 = 0xFF;
137
138/// The records [`recognizes_dump`] reads.
139const RECOGNIZED_RECORDS: usize = 1_000;
140
141/// What a field's raw value is read as.
142#[derive(Clone, Copy)]
143enum Unit {
144    /// Its value times a factor, in the quantity's unit.
145    Scale(f64),
146    /// °C, read in K.
147    Celsius,
148}
149
150/// One field of a kind: its channel's name, quantity and unit.
151struct Field {
152    name: &'static str,
153    quantity: Quantity,
154    unit: Unit,
155}
156
157const fn field(name: &'static str, quantity: Quantity, unit: Unit) -> Field {
158    Field {
159        name,
160        quantity,
161        unit,
162    }
163}
164
165const G: Unit = Unit::Scale(STANDARD_GRAVITY_MPS2);
166const AS_IS: Unit = Unit::Scale(1.0);
167
168/// One kind of record: its class and the class's version in an `.xtra`, its type and length in a
169/// raw dump, its length in an `.xtra`, and its fields.
170struct Kind {
171    class: i16,
172    version: u32,
173    raw_type: u8,
174    raw_bytes: usize,
175    xtra_bytes: usize,
176    fields: &'static [Field],
177}
178
179const KINDS: [Kind; 16] = [
180    Kind {
181        class: 2,
182        version: 3,
183        raw_type: 0x02,
184        raw_bytes: 2,
185        xtra_bytes: 4,
186        fields: &[field("acceleration", Quantity::Acceleration, G)],
187    },
188    Kind {
189        class: 3,
190        version: 3,
191        raw_type: 0x0b,
192        raw_bytes: 4,
193        xtra_bytes: 8,
194        fields: &[
195            field("lat. X accel.", Quantity::Acceleration, G),
196            field("lat. Y accel.", Quantity::Acceleration, G),
197        ],
198    },
199    Kind {
200        class: 15,
201        version: 0,
202        raw_type: 0x0c,
203        raw_bytes: 6,
204        xtra_bytes: 12,
205        fields: &[
206            field("Gyro X", Quantity::AngularRate, AS_IS),
207            field("Gyro Y", Quantity::AngularRate, AS_IS),
208            field("Gyro Z", Quantity::AngularRate, AS_IS),
209        ],
210    },
211    Kind {
212        class: 4,
213        version: 1,
214        raw_type: 0x03,
215        raw_bytes: 3,
216        xtra_bytes: 4,
217        fields: &[field("pressure", Quantity::Pressure, AS_IS)],
218    },
219    Kind {
220        class: 5,
221        version: 1,
222        raw_type: 0x04,
223        raw_bytes: 2,
224        xtra_bytes: 4,
225        fields: &[field("battery 1 voltage", Quantity::Voltage, AS_IS)],
226    },
227    Kind {
228        class: 6,
229        version: 1,
230        raw_type: 0x05,
231        raw_bytes: 2,
232        xtra_bytes: 4,
233        fields: &[field("battery 2 voltage", Quantity::Voltage, AS_IS)],
234    },
235    Kind {
236        class: 7,
237        version: 1,
238        raw_type: 0x06,
239        raw_bytes: 2,
240        xtra_bytes: 4,
241        fields: &[field("temperature", Quantity::Temperature, Unit::Celsius)],
242    },
243    Kind {
244        class: 17,
245        version: 0,
246        raw_type: 0x07,
247        raw_bytes: 2,
248        xtra_bytes: 6,
249        fields: &[field("A Voltage", Quantity::Voltage, AS_IS)],
250    },
251    Kind {
252        class: 18,
253        version: 0,
254        raw_type: 0x08,
255        raw_bytes: 2,
256        xtra_bytes: 6,
257        fields: &[field("B Voltage", Quantity::Voltage, AS_IS)],
258    },
259    Kind {
260        class: 19,
261        version: 0,
262        raw_type: 0x09,
263        raw_bytes: 2,
264        xtra_bytes: 6,
265        fields: &[field("C Voltage", Quantity::Voltage, AS_IS)],
266    },
267    Kind {
268        class: 20,
269        version: 0,
270        raw_type: 0x0a,
271        raw_bytes: 2,
272        xtra_bytes: 6,
273        fields: &[field("D Voltage", Quantity::Voltage, AS_IS)],
274    },
275    Kind {
276        class: 16,
277        version: 0,
278        raw_type: 0x0d,
279        raw_bytes: 6,
280        xtra_bytes: 12,
281        fields: &[
282            field("Mag X", Quantity::Unrecognized, AS_IS),
283            field("Mag Y", Quantity::Unrecognized, AS_IS),
284            field("Mag Z", Quantity::Unrecognized, AS_IS),
285        ],
286    },
287    Kind {
288        class: 10,
289        version: 4,
290        raw_type: 0x0e,
291        raw_bytes: 13,
292        xtra_bytes: 25,
293        fields: &[
294            field("lat", Quantity::Latitude, AS_IS),
295            field("long", Quantity::Longitude, AS_IS),
296            field("GPS MSL", Quantity::GnssAltitude, AS_IS),
297            field("SVs", Quantity::Number, AS_IS),
298            field("GPS flag", Quantity::Unrecognized, AS_IS),
299        ],
300    },
301    Kind {
302        class: 14,
303        version: 2,
304        raw_type: 0x12,
305        raw_bytes: 7,
306        xtra_bytes: 12,
307        fields: &[
308            field("GPS Time of Week", Quantity::Unrecognized, AS_IS),
309            field("GPS week", Quantity::Number, AS_IS),
310            field("GPS time flag 1", Quantity::Unrecognized, AS_IS),
311            field("GPS time flag 2", Quantity::Unrecognized, AS_IS),
312            field("GPS leap seconds", Quantity::Number, AS_IS),
313        ],
314    },
315    Kind {
316        class: 13,
317        version: 1,
318        raw_type: 0x10,
319        raw_bytes: 2,
320        xtra_bytes: 28,
321        fields: &[
322            field("state", Quantity::Number, AS_IS),
323            field("A on", Quantity::Number, AS_IS),
324            field("B on", Quantity::Number, AS_IS),
325            field("C on", Quantity::Number, AS_IS),
326            field("D on", Quantity::Number, AS_IS),
327            field("A cont", Quantity::Number, AS_IS),
328            field("B cont", Quantity::Number, AS_IS),
329            field("C cont", Quantity::Number, AS_IS),
330            field("D cont", Quantity::Number, AS_IS),
331        ],
332    },
333    Kind {
334        class: 21,
335        version: 0,
336        raw_type: 0x15,
337        raw_bytes: 8,
338        xtra_bytes: 16,
339        fields: &[
340            field("quaternion 1", Quantity::Unrecognized, AS_IS),
341            field("quaternion 2", Quantity::Unrecognized, AS_IS),
342            field("quaternion 3", Quantity::Unrecognized, AS_IS),
343            field("quaternion 4", Quantity::Unrecognized, AS_IS),
344        ],
345    },
346];
347
348/// The status's state numbers, by its u16's top 4 bits, as the software converts them.
349const STATES: [(u16, f64); 4] = [(0x0, 0.0), (0x1, 1.0), (0x3, 2.0), (0x5, 3.0)];
350
351/// One sample: its kind, its time and its fields' raw values, as the software converts them.
352struct Sample {
353    kind: &'static Kind,
354    time_s: f64,
355    values: Vec<f64>,
356}
357
358/// Reads a little-endian value of `N` bytes at `at`.
359fn le<const N: usize>(bytes: &[u8], at: usize) -> Option<[u8; N]> {
360    bytes.get(at..at.checked_add(N)?)?.try_into().ok()
361}
362
363/// An `.xtra` kind's fields from its payload, or the byte in it, and why, where one isn't held
364/// to the sample.
365fn xtra_values(kind: &Kind, data: &[u8]) -> Result<Vec<f64>, (usize, String)> {
366    let f64_at = |at: usize| {
367        f64::from_le_bytes([
368            data[at],
369            data[at + 1],
370            data[at + 2],
371            data[at + 3],
372            data[at + 4],
373            data[at + 5],
374            data[at + 6],
375            data[at + 7],
376        ])
377    };
378    let f32_at = |at: usize| {
379        f64::from(f32::from_le_bytes([
380            data[at],
381            data[at + 1],
382            data[at + 2],
383            data[at + 3],
384        ]))
385    };
386    let u32_at =
387        |at: usize| u32::from_le_bytes([data[at], data[at + 1], data[at + 2], data[at + 3]]);
388    // Invariant: `data` is `kind.xtra_bytes` long, which each arm's offsets stay within.
389    Ok(match kind.class {
390        10 => vec![
391            f64_at(0),
392            f64_at(8),
393            f32_at(16),
394            f64::from(u32_at(20)),
395            f64::from(data[24]),
396        ],
397        13 => {
398            for (at, name) in [(4, "first"), (12, "second"), (20, "third")] {
399                if u32_at(at) != 4 {
400                    return Err((
401                        at,
402                        format!(
403                            "the status's {name} set states {} bytes, where every sample \
404                             record's holds 4",
405                            u32_at(at)
406                        ),
407                    ));
408                }
409            }
410            if let Some(at) = data[24..28].iter().position(|&b| b != 0) {
411                return Err((
412                    24 + at,
413                    "the status's third set of four bytes isn't zero, as it is in every sample \
414                     record"
415                        .to_owned(),
416                ));
417            }
418            let mut values = vec![f64::from(u32_at(0))];
419            values.extend(
420                data[8..12]
421                    .iter()
422                    .chain(&data[16..20])
423                    .map(|&b| f64::from(b)),
424            );
425            values
426        }
427        14 => vec![
428            f64::from(u32_at(0)),
429            f64::from(u16::from_le_bytes([data[4], data[5]])),
430            f64::from(data[6]),
431            f64::from(data[7]),
432            f64::from(u32_at(8)),
433        ],
434        17..=20 => {
435            if let Some(at) = data[0..2].iter().position(|&b| b != 0) {
436                return Err((
437                    at,
438                    "an output line's two flag bytes aren't zero, as they are in every sample \
439                     record"
440                        .to_owned(),
441                ));
442            }
443            vec![f32_at(2)]
444        }
445        _ => (0..kind.xtra_bytes / 4).map(|k| f32_at(4 * k)).collect(),
446    })
447}
448
449/// A raw kind's fields from its big-endian payload, converted as the software converts them, or
450/// the byte in it, and why, where a bit isn't held to the sample.
451fn raw_values(kind: &Kind, data: &[u8]) -> Result<Vec<f64>, (usize, String)> {
452    // Invariant: `data` is `kind.raw_bytes` long, which each arm's offsets stay within.
453    let i16_at = |k: usize| i16::from_be_bytes([data[2 * k], data[2 * k + 1]]);
454    let u16_at = |k: usize| u16::from_be_bytes([data[2 * k], data[2 * k + 1]]);
455    let times = |count: usize, factor: f32| -> Vec<f64> {
456        (0..count)
457            .map(|k| f64::from(f32::from(i16_at(k)) * factor))
458            .collect()
459    };
460    Ok(match kind.raw_type {
461        0x02 | 0x0b => (0..kind.raw_bytes / 2)
462            .map(|k| f64::from(f32::from(i16_at(k)) / 256.0))
463            .collect(),
464        0x0c => times(3, 0.07),
465        0x0d => times(3, 1e-6),
466        0x06 => times(1, 0.01),
467        0x15 => (0..4)
468            .map(|k| f64::from(f32::from(i16_at(k)) / 32_768.0))
469            .collect(),
470        0x03 => vec![f64::from(u32::from_be_bytes([
471            0, data[0], data[1], data[2],
472        ]))],
473        0x04 | 0x05 => {
474            let raw = u16_at(0);
475            // Bits 0–3 are in the second byte, big-endian.
476            if raw & 0x000F != 0 {
477                return Err((
478                    1,
479                    format!(
480                        "a battery's bits 0–3 aren't zero ({raw:#06x}), as they are in every sample \
481                     record"
482                    ),
483                ));
484            }
485            // As the software: the 12-bit count times 3.3 V × 133/33 taken in single precision
486            // (f32(3.3) × f32(133/33) = 13.299 999 2, one ulp below f32(13.3)), over 4096,
487            // rounded to an f32. With this factor, every battery value in the sample equals the
488            // `.xtra`'s, as the survey's comparison shows.
489            #[allow(
490                clippy::cast_possible_truncation,
491                reason = "single precision, as the logger's software works it"
492            )]
493            let volts =
494                f64::from(raw >> 4) * f64::from(3.3_f32 * (133.0_f64 / 33.0) as f32) / 4_096.0;
495            #[allow(
496                clippy::cast_possible_truncation,
497                reason = "rounded to an f32, as the logger's software rounds it"
498            )]
499            let volts = volts as f32;
500            vec![f64::from(volts)]
501        }
502        0x07..=0x0a => {
503            let raw = u16_at(0);
504            if raw & 0xC000 != 0 {
505                return Err((
506                    0,
507                    format!(
508                        "an output line's bits 14–15 aren't zero ({raw:#06x}), as they are in every \
509                     sample record"
510                    ),
511                ));
512            }
513            #[allow(
514                clippy::cast_possible_truncation,
515                reason = "an f32 factor, multiplied in f32: the reading is single precision"
516            )]
517            let volts_per_count = (3.3_f64 / 16_384.0) as f32;
518            vec![f64::from(f32::from(raw) * volts_per_count)]
519        }
520        0x0e => {
521            let i32_at = |at: usize| {
522                i32::from_be_bytes([data[at], data[at + 1], data[at + 2], data[at + 3]])
523            };
524            let flags = data[12];
525            if flags & 0xC0 != 0 {
526                return Err((
527                    12,
528                    format!(
529                        "the GPS byte's bits 6–7 aren't zero ({flags:#04x}), as they are in every \
530                     sample record"
531                    ),
532                ));
533            }
534            let degrees = f64::from(1e-7_f32);
535            // The altitude in mm, as an f32 times f32(0.001); `as` rounds to the nearest f32.
536            let altitude_m = i32_at(8) as f32 * 0.001;
537            vec![
538                f64::from(i32_at(0)) * degrees,
539                f64::from(i32_at(4)) * degrees,
540                f64::from(altitude_m),
541                f64::from(flags & 0x1F),
542                f64::from(flags >> 5 & 1),
543            ]
544        }
545        0x10 => {
546            let raw = u16_at(0);
547            let Some((_, state)) = STATES.iter().find(|(top, _)| *top == raw >> 12) else {
548                return Err((
549                    0,
550                    format!("the status's top 4 bits ({raw:#06x}) are no state the sample holds"),
551                ));
552            };
553            if raw & 0x000F != 0 {
554                return Err((
555                    1,
556                    format!(
557                        "the status's bits 0–3 aren't zero ({raw:#06x}), as they are in every sample \
558                     record"
559                    ),
560                ));
561            }
562            let mut values = vec![*state];
563            values.extend((0..4).map(|line| f64::from(raw >> (8 + line) & 1)));
564            values.extend((0..4).map(|line| f64::from(raw >> (7 - line) & 1)));
565            values
566        }
567        0x12 => {
568            let flags = data[6];
569            vec![
570                f64::from(u32::from_be_bytes([data[0], data[1], data[2], data[3]])),
571                f64::from(u16::from_be_bytes([data[4], data[5]])),
572                f64::from(flags >> 7 & 1),
573                f64::from(flags >> 6 & 1),
574                f64::from(flags & 0x3F),
575            ]
576        }
577        // Invariant: `KINDS` holds no other raw type.
578        other => return Err((0, format!("type {other:#04x} has no conversion"))),
579    })
580}
581
582/// The record built from samples: a channel for each field, in the order each kind first
583/// appears, and the status's events.
584fn build(format: LogFormat, samples: &[Sample], notes: Vec<String>) -> FlightLog {
585    let mut order: Vec<i16> = Vec::new();
586    let mut by_kind: BTreeMap<i16, Vec<&Sample>> = BTreeMap::new();
587    for sample in samples {
588        let class = sample.kind.class;
589        if !by_kind.contains_key(&class) {
590            order.push(class);
591        }
592        by_kind.entry(class).or_default().push(sample);
593    }
594    let mut channels = Vec::new();
595    let mut events = Vec::new();
596    for class in order {
597        let kind_samples = &by_kind[&class];
598        // Invariant: each class in `order` has a sample.
599        let kind = kind_samples[0].kind;
600        let time_s: Vec<f64> = kind_samples.iter().map(|s| s.time_s).collect();
601        for (index, field) in kind.fields.iter().enumerate() {
602            let values = kind_samples
603                .iter()
604                .map(|s| match field.unit {
605                    Unit::Scale(factor) => s.values[index] * factor,
606                    Unit::Celsius => s.values[index] + 273.15,
607                })
608                .collect();
609            channels.push(Channel {
610                column: field.name.to_owned(),
611                quantity: field.quantity,
612                time_s: time_s.clone(),
613                values,
614            });
615        }
616        if kind.raw_type == 0x10 {
617            let mut before: Option<&Sample> = None;
618            for sample in kind_samples {
619                if let Some(before) = before {
620                    if sample.values[0] != before.values[0] {
621                        events.push(Event {
622                            time_s: sample.time_s,
623                            name: format!("state {}", sample.values[0]),
624                        });
625                    }
626                    for line in 0..4 {
627                        if sample.values[1 + line] == 1.0 && before.values[1 + line] == 0.0 {
628                            events.push(Event {
629                                time_s: sample.time_s,
630                                name: kind.fields[1 + line].name.to_owned(),
631                            });
632                        }
633                    }
634                }
635                before = Some(sample);
636            }
637        }
638    }
639    FlightLog {
640        format,
641        logger: "Entacore AIM XTRA".to_owned(),
642        serial_number: None,
643        firmware: None,
644        flight_number: None,
645        stated: Stated {
646            samples: Some(samples.len()),
647            ..Stated::default()
648        },
649        time_s: Vec::new(),
650        altitude_m: Vec::new(),
651        vertical_speed_m_s: None,
652        temperature_k: None,
653        battery_v: None,
654        channels,
655        events,
656        notes,
657    }
658}
659
660/// Whether `bytes` are an AIM `.xtra` file: a Boost binary archive's signature, the sample's
661/// top object header, and a first record whose class is an AIM kind's. The rest isn't read here:
662/// [`read_xtra`] refuses a file whose records differ from the sample's, naming the byte.
663pub fn recognizes_xtra(bytes: &[u8]) -> bool {
664    let class = le::<2>(bytes, XTRA_RECORDS_AT).map(i16::from_le_bytes);
665    bytes.starts_with(XTRA_SIGNATURE)
666        && le::<10>(bytes, XTRA_SIGNATURE.len()) == Some(XTRA_TOP)
667        && class.is_some_and(|class| KINDS.iter().any(|kind| kind.class == class))
668}
669
670fn xtra_error(offset: usize, message: String) -> LogError {
671    LogError::Binary {
672        format: XTRA_FORMAT,
673        offset,
674        message,
675    }
676}
677
678/// The `.xtra`'s samples, and its notes.
679fn xtra_samples(bytes: &[u8]) -> Result<(Vec<Sample>, Vec<String>), LogError> {
680    if !bytes.starts_with(XTRA_SIGNATURE) {
681        return Err(LogError::NotThisFormat {
682            format: XTRA_FORMAT,
683            message: "it doesn't open with a Boost binary archive's signature, \
684                      `serialization::archive`, version 16, and its sizes"
685                .to_owned(),
686        });
687    }
688    let top = le::<10>(bytes, XTRA_SIGNATURE.len());
689    if top != Some(XTRA_TOP) {
690        return Err(xtra_error(
691            XTRA_SIGNATURE.len(),
692            "the top object's header isn't the sample's: a version this reader doesn't know"
693                .to_owned(),
694        ));
695    }
696    let count = le::<8>(bytes, XTRA_COUNT_AT)
697        .map(u64::from_le_bytes)
698        .ok_or_else(|| {
699            xtra_error(
700                XTRA_COUNT_AT,
701                "the file ends before its record count".to_owned(),
702            )
703        })?;
704    let mut samples = Vec::new();
705    let mut notes = Vec::new();
706    let mut seen: Vec<i16> = Vec::new();
707    let mut at = XTRA_RECORDS_AT;
708    let mut index: u64 = 0;
709    while index < count {
710        let start = at;
711        let Some(sample) = xtra_record(bytes, &mut at, index, &mut seen)? else {
712            notes.push(if bytes.len() == start {
713                format!("the file ends after record {index} of the {count} it states: read those")
714            } else {
715                format!(
716                    "the file ends inside record {} of the {count} it states, {} bytes into it: \
717                     read the {index} before",
718                    index + 1,
719                    bytes.len() - start
720                )
721            });
722            return Ok((samples, notes));
723        };
724        samples.push(sample);
725        index += 1;
726    }
727    if bytes.len() > at {
728        notes.push(format!(
729            "{} bytes after the {count} records aren't read",
730            bytes.len() - at
731        ));
732    }
733    Ok((samples, notes))
734}
735
736/// The `n` bytes at `*at`, moving `*at` past them; `None` if the file ends first.
737fn take<'a>(bytes: &'a [u8], at: &mut usize, n: usize) -> Option<&'a [u8]> {
738    let data = bytes.get(*at..at.checked_add(n)?)?;
739    *at += n;
740    Some(data)
741}
742
743/// The record at `*at`, the `index`th, moving `*at` past it; `None` if the file ends inside it.
744fn xtra_record(
745    bytes: &[u8],
746    at: &mut usize,
747    index: u64,
748    seen: &mut Vec<i16>,
749) -> Result<Option<Sample>, LogError> {
750    let record = index + 1;
751    let start = *at;
752    let Some(class) = take(bytes, at, 2).map(|b| i16::from_le_bytes([b[0], b[1]])) else {
753        return Ok(None);
754    };
755    let Some(kind) = KINDS.iter().find(|kind| kind.class == class) else {
756        return Err(xtra_error(
757            start,
758            format!("record {record}: class {class}, which no sample holds"),
759        ));
760    };
761    if !seen.contains(&class) {
762        let header = *at;
763        let Some(info) = take(bytes, at, 5) else {
764            return Ok(None);
765        };
766        let version = u32::from_le_bytes([info[1], info[2], info[3], info[4]]);
767        if info[0] != 1 {
768            return Err(xtra_error(
769                header,
770                format!(
771                    "record {record}: class {class}'s header opens with {}, where every sample's \
772                     opens with 1",
773                    info[0]
774                ),
775            ));
776        }
777        if version != kind.version {
778            return Err(xtra_error(
779                header + 1,
780                format!(
781                    "record {record}: class {class}'s version is {version}, where the sample's \
782                     is {}: its fields may differ",
783                    kind.version
784                ),
785            ));
786        }
787        seen.push(class);
788    }
789    let id_at = *at;
790    let Some(id) = take(bytes, at, 4).map(|b| u32::from_le_bytes([b[0], b[1], b[2], b[3]])) else {
791        return Ok(None);
792    };
793    if u64::from(id) != 2 * index {
794        return Err(xtra_error(
795            id_at,
796            format!("record {record}: its object id is {id}, not {}", 2 * index),
797        ));
798    }
799    if index == 0 {
800        let header = *at;
801        let Some(base) = take(bytes, at, 5) else {
802            return Ok(None);
803        };
804        if base != [1, 1, 0, 0, 0] {
805            return Err(xtra_error(
806                header,
807                "the first record's base header isn't the sample's".to_owned(),
808            ));
809        }
810    }
811    let base_at = *at;
812    let Some(base) = take(bytes, at, 13) else {
813        return Ok(None);
814    };
815    let base_id = u32::from_le_bytes([base[0], base[1], base[2], base[3]]);
816    let first = f32::from_le_bytes([base[4], base[5], base[6], base[7]]);
817    let time = f32::from_le_bytes([base[9], base[10], base[11], base[12]]);
818    // The base's fields in turn: its id, the 3.3, the 0.
819    let fault = if u64::from(base_id) != 2 * index + 1 {
820        Some(0)
821    } else if first != XTRA_BASE_FIRST {
822        Some(4)
823    } else if base[8] != 0 {
824        Some(8)
825    } else {
826        None
827    };
828    if let Some(field) = fault {
829        return Err(xtra_error(
830            base_at + field,
831            format!(
832                "record {record}: its base isn't the sample's (an id one past the record's, \
833                 {XTRA_BASE_FIRST} and 0)"
834            ),
835        ));
836    }
837    if !time.is_finite() {
838        return Err(xtra_error(
839            base_at + 9,
840            format!("record {record}: its time isn't a number"),
841        ));
842    }
843    let data_at = *at;
844    let Some(data) = take(bytes, at, kind.xtra_bytes) else {
845        return Ok(None);
846    };
847    let values = xtra_values(kind, data).map_err(|(field, message)| {
848        xtra_error(data_at + field, format!("record {record}: {message}"))
849    })?;
850    Ok(Some(Sample {
851        kind,
852        time_s: f64::from(time),
853        values,
854    }))
855}
856
857/// Reads an AIM `.xtra` file.
858///
859/// # Errors
860///
861/// [`LogError::NotThisFormat`] when it doesn't open with the archive's signature;
862/// [`LogError::Binary`], naming the byte, when the top object's header, a class, a class's
863/// version, an object id or a byte held to the sample isn't as the module describes;
864/// [`LogError::NoData`] when it holds no whole record.
865pub fn read_xtra(bytes: &[u8]) -> Result<FlightLog, LogError> {
866    let (samples, mut notes) = xtra_samples(bytes)?;
867    if samples.is_empty() {
868        return Err(LogError::NoData {
869            format: XTRA_FORMAT,
870        });
871    }
872    held_pad_pressure(&samples, XTRA_FORMAT)?;
873    notes.push(
874        "the times are the software's, summed in 32-bit floats. So they drift from the \
875         computer's own clock as they grow, behind it in the one sample. That clock isn't \
876         the GPS's. On the sample's flight, a tick is 1.009 to 1.021 ms of the GPS's clock. \
877         Over its climb these times run about 1.8 % short of the GPS's"
878            .to_owned(),
879    );
880    Ok(build(LogFormat::EntacoreXtra, &samples, notes))
881}
882
883/// Refuses a recording whose first pressure isn't a pad's, 50,000 to 110,000 Pa: its unit would
884/// be something else.
885fn held_pad_pressure(samples: &[Sample], format: &'static str) -> Result<(), LogError> {
886    match samples.iter().find(|s| s.kind.raw_type == 0x03) {
887        Some(first) if !(50_000.0..=110_000.0).contains(&first.values[0]) => {
888            Err(LogError::NotThisFormat {
889                format,
890                message: format!(
891                    "its first pressure, {}, isn't a pad's in pascals, 50,000 to 110,000",
892                    first.values[0]
893                ),
894            })
895        }
896        _ => Ok(()),
897    }
898}
899
900fn dump_error(offset: usize, message: String) -> LogError {
901    LogError::Binary {
902        format: DUMP_FORMAT,
903        offset,
904        message,
905    }
906}
907
908/// One recording of a raw dump: where it starts, the records walked in it, its samples.
909struct Recording {
910    start: usize,
911    records: usize,
912    samples: Vec<Sample>,
913}
914
915/// The raw dump's recordings, reading at most `limit` records in all. With `held` false, as
916/// [`recognizes_dump`] walks it, a record whose bits aren't held to the sample is passed over,
917/// not refused: only [`read_dump`] refuses it, naming the byte.
918fn dump_recordings(bytes: &[u8], limit: usize, held: bool) -> Result<Vec<Recording>, LogError> {
919    if bytes.first() != Some(&DUMP_RECORDING) {
920        return Err(LogError::NotThisFormat {
921            format: DUMP_FORMAT,
922            message: "it doesn't open with a recording's type, 0x13".to_owned(),
923        });
924    }
925    let mut recordings: Vec<Recording> = Vec::new();
926    let mut ticks: u64 = 0;
927    let mut read = 0;
928    let mut at = 0;
929    while at < bytes.len() && read < limit {
930        let byte = bytes[at];
931        if byte == ERASED {
932            if let Some(stray) = bytes[at..].iter().position(|&b| b != ERASED) {
933                return Err(dump_error(
934                    at + stray,
935                    format!(
936                        "data after the erased memory that starts at byte {at}: the reader \
937                         doesn't know a gap in the memory"
938                    ),
939                ));
940            }
941            break;
942        }
943        if byte & 0x80 != 0 {
944            ticks += u64::from(byte & 0x7F);
945            at += 1;
946            continue;
947        }
948        if byte == DUMP_RECORDING {
949            if bytes.len() - at < 1 + DUMP_RECORDING_BYTES {
950                return Err(dump_error(
951                    at,
952                    "the dump ends inside a recording's opening bytes".to_owned(),
953                ));
954            }
955            recordings.push(Recording {
956                start: at,
957                records: 0,
958                samples: Vec::new(),
959            });
960            ticks = 0;
961            at += 1 + DUMP_RECORDING_BYTES;
962            continue;
963        }
964        let Some(kind) = KINDS.iter().find(|kind| kind.raw_type == byte) else {
965            return Err(dump_error(
966                at,
967                format!("type {byte:#04x}, which no sample holds: its length isn't known"),
968            ));
969        };
970        let Some(data) = bytes.get(at + 1..at + 1 + kind.raw_bytes) else {
971            return Err(dump_error(
972                at,
973                format!("the dump ends inside a record of type {byte:#04x}"),
974            ));
975        };
976        let values = match raw_values(kind, data) {
977            Ok(values) => Some(values),
978            Err((field, message)) if held => return Err(dump_error(at + 1 + field, message)),
979            Err(_) => None,
980        };
981        // Invariant: the dump opens with a recording, so one is open.
982        if let Some(recording) = recordings.last_mut() {
983            recording.records += 1;
984            if let Some(values) = values {
985                recording.samples.push(Sample {
986                    kind,
987                    time_s: ticks as f64 / 1_000.0,
988                    values,
989                });
990            }
991        }
992        read += 1;
993        at += 1 + kind.raw_bytes;
994    }
995    Ok(recordings)
996}
997
998/// Whether `bytes` are an AIM raw dump: a recording's opening, then records of the AIM's types,
999/// up to 1,000, or erased memory. The bits held to the sample aren't read here: [`read_dump`]
1000/// refuses a dump that differs there, naming the byte.
1001pub fn recognizes_dump(bytes: &[u8]) -> bool {
1002    bytes.len() > DUMP_RECORDING_BYTES
1003        && dump_recordings(bytes, RECOGNIZED_RECORDS, false)
1004            .is_ok_and(|recordings| recordings.iter().any(|r| r.records > 0))
1005}
1006
1007/// Reads an AIM raw dump: each recording in it as a record of its own, in the dump's order. A
1008/// recording that holds no record, one opening straight after another, is a record with no
1009/// channel, so that the records keep the dump's count.
1010///
1011/// # Errors
1012///
1013/// [`LogError::NotThisFormat`] when it doesn't open with a recording's type, or a recording's
1014/// first pressure isn't a pad's; [`LogError::Binary`], naming the byte, when a type isn't the
1015/// sample's, a bit held to the sample is set, the dump ends inside a record, or data follows
1016/// erased memory; [`LogError::NoData`] when it holds no record.
1017pub fn read_dump(bytes: &[u8]) -> Result<Vec<FlightLog>, LogError> {
1018    let recordings = dump_recordings(bytes, usize::MAX, true)?;
1019    if recordings.iter().all(|r| r.samples.is_empty()) {
1020        return Err(LogError::NoData {
1021            format: DUMP_FORMAT,
1022        });
1023    }
1024    let count = recordings.len();
1025    recordings
1026        .iter()
1027        .enumerate()
1028        .map(|(index, recording)| {
1029            held_pad_pressure(&recording.samples, DUMP_FORMAT)?;
1030            let notes = vec![
1031                format!(
1032                    "recording {} of the dump's {count}, from byte {}; its {DUMP_RECORDING_BYTES} \
1033                     opening bytes aren't read",
1034                    index + 1,
1035                    recording.start
1036                ),
1037                "the clock is the computer's ticks, read as milliseconds from the recording's \
1038                 start, as the AIM software reads them. On the one sample flight, a tick is \
1039                 1.009 to 1.021 ms of the GPS's clock"
1040                    .to_owned(),
1041            ];
1042            Ok(build(LogFormat::EntacoreDump, &recording.samples, notes))
1043        })
1044        .collect()
1045}
1046
1047/// Reads an AIM raw dump that holds one recording, as [`crate::import::read`] does.
1048///
1049/// # Errors
1050///
1051/// As [`read_dump`], and [`LogError::Binary`] when the dump holds more than one recording,
1052/// naming where the second starts: [`read_dump`] reads each.
1053pub(crate) fn read_dump_one(bytes: &[u8]) -> Result<FlightLog, LogError> {
1054    let mut recordings = read_dump(bytes)?;
1055    if recordings.len() > 1 {
1056        let second = dump_recordings(bytes, usize::MAX, true)?
1057            .get(1)
1058            .map_or(0, |r| r.start);
1059        return Err(dump_error(
1060            second,
1061            format!(
1062                "the dump holds {} recordings, the second from this byte: \
1063                 `entacore_raw::read_dump` reads each as a record of its own",
1064                recordings.len()
1065            ),
1066        ));
1067    }
1068    // Invariant: `read_dump` returns a recording or an error.
1069    recordings.pop().ok_or(LogError::NoData {
1070        format: DUMP_FORMAT,
1071    })
1072}