dandri/MeshCore-OTA
1
0
Fork 0
mirror of https://github.com/vk496/MeshCore.git synced 2026-09-02 13:03:47 +00:00
Code Issues Packages Projects Releases Wiki Activity

Improve OTA cli and protocol

Browse Source
This commit is contained in:
Valentin Kivachuk Burda
2026-06-26 22:11:10 +02:00
parent f149858dc4
commit 659b96ddbb
41 changed files with 3355 additions and 576 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.github/workflows/dev-firmware-rolling.yml
+101
View File
@@ -0,0 +1,101 @@
# Rolling DEV firmware release (fork convenience).
#
# On every push, rebuild ALL device firmwares and replace the assets of ONE rolling prerelease
# (tag `dev-latest`) so other devs can always grab the current binaries from the same place. Also packages
# a demo `.mota` (full + a same-image delta, which is tiny) so the OTA format can be exercised.
#
# These are UNSIGNED development builds for testing only — not official releases.
# To limit which branches trigger this, add a `branches:` filter under `push:` below.
name: Dev Firmware (rolling)
on:
workflow_dispatch:
push:
permissions:
contents: write
# Only the latest push matters — cancel any in-flight run so the release tracks HEAD.
concurrency:
group: dev-firmware-rolling
cancel-in-progress: true
env:
RELEASE_TAG: dev-latest
FIRMWARE_VERSION: dev
MOTA_ENV: Heltec_v3_repeater # representative ESP32 env whose built .bin carries EndF, used for the demo .mota
jobs:
build-and-release:
runs-on: ubuntu-latest
steps:
- name: Clone Repo
uses: actions/checkout@v6
- name: Setup Build Environment
uses: ./.github/actions/setup-build-environment
# build.sh is best-effort: a board that fails to compile is skipped, the rest still land in out/.
# All three roles append to the same out/ directory.
- name: Build all firmwares (companion + repeater + room-server)
run: |
/usr/bin/env bash build.sh build-companion-firmwares
/usr/bin/env bash build.sh build-repeater-firmwares
/usr/bin/env bash build.sh build-room-server-firmwares
echo "Built artifacts:"; ls -la out
- name: Package demo .mota (full + same-image delta)
run: |
set -euo pipefail
pip install --quiet detools # delta codec only (a full .mota needs no extra deps)
BIN="$(ls out/${MOTA_ENV}-*.bin 2>/dev/null | grep -v -- '-merged' | head -1 || true)"
if [ -z "$BIN" ]; then
echo "::warning::no ${MOTA_ENV} .bin built — skipping demo .mota"
else
echo "Using firmware image: $BIN"
# full image .mota (codec 0): payload = the flashable image (BODY||EndF)
python3 tools/mota/mota.py build --fw "$BIN" --target-env "$MOTA_ENV" \
--fw-version 0.0.0 --codec full --out "out/${MOTA_ENV}-demo.full.mota"
# delta .mota (codec 1, sequential+crle): base == target == same image -> near-empty patch
python3 tools/mota/mota.py build --fw "$BIN" --base "$BIN" --target-env "$MOTA_ENV" \
--fw-version 0.0.1 --codec sequential --compression crle --out "out/${MOTA_ENV}-demo.delta.mota"
echo "Demo .mota sizes:"; ls -l out/${MOTA_ENV}-demo.*.mota
fi
- name: Upload workflow artifacts
uses: actions/upload-artifact@v7
with:
name: dev-firmwares
path: out
if-no-files-found: warn
retention-days: 5
- name: Replace the rolling dev release
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
if [ -z "$(ls -A out 2>/dev/null)" ]; then
echo "::error::nothing built — not touching the release"; exit 1
fi
# delete the previous release + its tag so old binaries disappear, then recreate at HEAD
gh release delete "$RELEASE_TAG" --yes --cleanup-tag 2>/dev/null || true
NOTES=$(cat <<EOF
**Automatic development build — latest commit on \`${GITHUB_REF_NAME}\`.**
- Commit: \`${GITHUB_SHA}\`
- Built: $(date -u '+%Y-%m-%d %H:%M:%S UTC')
⚠️ Unsigned dev builds for testing only. The assets here are replaced on every push (this release
always tracks the latest commit). Embedded firmware version string: \`dev-<short-sha>\`.
\`${MOTA_ENV}-demo.full.mota\` / \`*.delta.mota\` are OTA format demos built from this run's
\`${MOTA_ENV}\` image (the delta is a same-image patch, so it is intentionally tiny).
EOF
)
gh release create "$RELEASE_TAG" out/* \
--title "Dev firmware (latest commit)" \
--notes "$NOTES" \
--prerelease \
--target "$GITHUB_SHA"
OTA_STATUS.md
-131
View File
@@ -1,131 +0,0 @@
# OTA-over-LoRa — Implementation Status
Working record of what's built, what's validated (and how), and what remains. Companion to the design
docs: `OTA.md` (spec), `OTA_PLAN.md` (plan), `docs/ota_protocol.md` (wire format).
Everything is gated behind `-D ENABLE_OTA=1` and is byte-for-byte inert when off. Nothing is committed
to git yet — all changes are in the working tree.
## Validation summary
| Phase | Component | Validated how | Status |
|---|---|---|---|
| P0 | `.mota` format + host packager (`tools/mota/`): build, delta (detools), Ed25519 sign, inspect, verify; `EndF` injector | `tools/mota/test_mota.py` (12/12); CLI end-to-end | ✅ |
| P1 | Portable C++ core: `Multihash`, `MerkleTree` (+proofs, gen+verify), `MotaContainer` parse, `BlockBitmap` | `pio test -e native` (21/21), cross-checked vs Python oracle (proofs byte-identical) | ✅ |
| P2 | `EndF` self-scan, `target_id` / `getOtaTargetId()`, `build.sh` injection | **on Heltec v3**: `ota status` reports exact size/hash/target_id matching the build hook | ✅ |
| P3 | `SignerAllowlist`, `OtaStore`, full verify (parse + merkle root + image_hash + **Ed25519** + allowlist) | **on Heltec v3 AND RAK4631**: `verify` → `ok=1 auto=1` with key, `auto=0` without | ✅ cross-platform |
| P4a | `OtaProtocol` message codec (ADV/QUERY/HAVE/GET_MANIFEST/MANIFEST/REQ/DATA) + server proof-gen | native (21/21), proof-gen matches Python | ✅ |
| P4b | `OtaManager` serve+fetch state machine | native: **two-manager full transfer simulation → byte-identical** reassembly | ✅ |
| P4c | Mesh integration: `PAYLOAD_TYPE_OTA` dispatch, lowest-priority hop-capped flood, wired into `simple_repeater` | **full on-air transfer RAK4631→Heltec COMPLETE + VERIFIED** (see below) | ✅ |
| P5 | **Delta apply (ESP32 A/B) via detools 0.53.0**: vendored detools embeddable C decoder (`src/helpers/ota/detools/`, NONE+CRLE) decodes a `--codec sequential --compression crle` patch against the running slot into the inactive slot, hashing→`image_hash` (`ota applydelta`) | **full LoRa run RAK→Heltec**: 129-byte delta (0.01% of a 1.18 MB image) → `detools decoded 1179184 B, hash OK, armed` → reboot → **booted v1.16.9** | ✅ |
| P6 | **Apply (ESP32 A/B)**: verify inactive-slot image vs signed manifest (image_hash + Ed25519 + allowlist) → `esp_ota_set_boot_partition` → reboot | **real role switch on Heltec: repeater → companion, booted correctly** (`ota apply manifest/verify/commit`; after reboot it speaks the companion frame protocol) | ✅ |
| — | Build + flash both platforms | esptool (Heltec/ESP32) + adafruit-nrfutil **DFU** (RAK4631/nRF52) both confirmed | ✅ |
## On-air status (real LoRa, RAK4631 → Heltec v3) — ✅ COMPLETE + VERIFIED
A full signed `.mota` was transferred **over real LoRa from the RAK4631 (nRF52) to the Heltec v3
(ESP32)** — cross-platform — and **completed + verified end-to-end**:
```
fetch=F 1/6 → 3/6 → 4/6 → 5/6 → 6/6 → fetch=C (~26 s)
verify: parsed=1 root=1 img=1 signed=1 sig=1 trust=1 | ok=1 auto=1
```
announce → get-manifest → manifest → windowed request → data; every block **merkle-verified against the
signed root** before storage; the reassembled container then **fully verified** (root + image_hash +
Ed25519 signature + allowlist → auto-appliable). Three real bugs were found and fixed via on-device
testing (none caught by the host sim, which doesn't model the mesh):
- **Windowed requests** (`OTA_REQ_WINDOW`) — was requesting the whole image at once → server TX/pool
congestion. Now paced to the link.
- **Manifest + request retry** in `OtaManager::loop()` — was unrecoverable if a reply dropped.
- **Dedup vs. retries (the key one):** the mesh `hasSeen()` dedup suppressed identical retried requests,
so a single lost reply stalled forever. Fixed: OTA packets are **always processed** (handlers are
idempotent); `hasSeen()` now only gates re-flooding. This makes it genuinely *eventually reliable* —
lossy RF just means more time, exactly as intended.
## Source map (`src/helpers/ota/`)
| File | Role | Portable (native) |
|---|---|---|
| `OtaFormat.h` | wire constants (magics, flags, codecs, msg types) | yes |
| `Multihash.h` | sha2-256 truncations via `Utils::sha256` | yes |
| `MerkleTree.{h,cpp}` | leaf/root (O(log n)), verify, gen proof | yes |
| `MotaContainer.{h,cpp}` | `.mota` parse + root/image-hash checks | yes |
| `BlockBitmap.h` | availability from `leaves[]` (erased = missing) | yes |
| `FirmwareInfo.{h,cpp}` | `EndF` self-scan over a region | yes |
| `OtaStore.h` | staging interface + `OtaStoreRam<N>` | yes |
| `OtaProtocol.{h,cpp}` | message encode/decode | yes |
| `OtaManager.{h,cpp}` | serve+fetch state machine | yes |
| `SignerAllowlist.h` | trusted Ed25519 signer keys | yes |
| `OtaVerify.{h,cpp}` | full verify incl. Ed25519 (uses `Identity`) | device-only |
| `OtaSelf.{h,cpp}` | running-firmware region (ESP32 `esp_partition_read`) | device-only |
| `OtaContext.{h,cpp}` | per-device singleton (manager + stores + allowlist) | device-only |
| `OtaCli.{h,cpp}` | `ota …` CLI commands | device-only |
Core edits (gated): `Packet.h` (`PAYLOAD_TYPE_OTA=0x0C`), `Mesh.{h,cpp}` (dispatch + `createOtaPacket`/
`sendOtaFlood` + hop limit), `MeshCore.h` (`getOtaTargetId`), `CommonCLI.cpp` (`ota` command),
`examples/simple_repeater/MyMesh.{h,cpp}` (`onOtaRecv` + adapter + begin/loop wiring), `build.sh`
(`MOTA_TARGET_ID`), `test/mocks/SHA256.h` (real host SHA-256). Env wiring: `variants/heltec_v3` and
`variants/rak4631` repeater envs (`ENABLE_OTA` + ota sources [+ `EndF` hook on ESP32]).
## `ota` CLI (serial console; also remote-admin over LoRa)
```
ota status target_id, self-fw size/hash, serve/fetch state, key count
ota key add|list|rm <hex> signer allowlist
ota stage <size> prepare serve buffer
ota recv <off> <hex> write a chunk into the serve buffer (host streams the .mota)
ota serve parse+verify the staged .mota and make it servable
ota announce broadcast OTA_ADV for the served .mota
ota verify full verify of the staged/served (or fetched) .mota
ota want <hex>|auto manual cross-target override (deliberate role switch, e.g. companion->repeater)
ota clear reset buffers
```
Host harness: `tools/mota/` packager + the scratch `onair*.py` orchestration scripts.
## Variant coverage (which platforms have OTA, which need special treatment)
OTA is enabled at the platform base so every variant inherits it; only the apply path differs by HW.
| Platform | OTA build | Apply path | Special treatment |
|---|---|---|---|
| **ESP32** (all chips) | ✅ enabled in `[esp32_base]` (`ENABLE_OTA`, `helpers/ota/*.cpp`, `detools.c`, `pio_endf`) | A/B via `esp_ota` + detools-**sequential** decode into the inactive slot | `applydelta` only runs on a **dual-app/OTA partition table** (2 app slots + otadata). `min_spiffs.csv` boards already qualify (1.875 MB slots); `huge_app`/single-app boards (most esp32/S3 defaults, 3.19 MB) build fine but refuse apply (`ERR no A/B slot`) until repartitioned. |
| **nRF52 — RAK4631 hardware** (rak4631 + gat562_30s / evb_pro / tracker_pro / watch13, muziworks_r1_neo, rak_wismesh_tag) | ✅ `[rak4631]` (inline) + `[rak4631_hw]` (shared, the other 6) | single-slot **in-place** detools, applied by the custom OTAFIX bootloader after reboot | Device must run the **OTAFIX bootloader** fork. `detools.c` is NOT built into the app (only the bootloader decodes). |
| **nRF52 — non-RAK** (heltec_t1/t096/t114/mesh_solar/mesh_pocket, lilygo techo*/t_impulse_plus, thinknode_m1/m3/m6, t1000-e, nano_g2_ultra, promicro, xiao_nrf52, ikoka_*, wio*, sensecap_solar, rak3401, keepteen_lt1, meshtiny, minewsemi_me25ls01) | ❌ not enabled | none | **Needs its own bootloader fork** (single-slot, like RAK) before OTA is safe. No A/B slot, and the stock Adafruit/SoftDevice bootloader can't apply in place. Out of scope until per-board bootloaders exist. |
| **RP2040 / STM32** | ❌ not enabled | none | No A/B apply path implemented yet. |
Build-verified this pass (OTA on): ESP32 across all 4 chip families — esp32 `Heltec_v2` (34.6%), S3 `Heltec_v3` (35.3%), C6 `Xiao_C6` (27.9%), and the tight C3 default-partition class up to the fattest config `Heltec_ct62_companion_radio_ble` **96.2%** / `Xiao_C3_companion_radio_ble` 94.6% (the global flash worst case — fits). All 6 RAK4631-hw nRF52 variants build (Flash 55–65%, RAM ≤ 74%). `native` test suite green. (Pre-existing, OTA-unrelated, fail on clean `main` too: `tenstar_c3` stale `helpers/XiaoC3Board.h` include; `generic_espnow` undefined `P_LORA_DIO_1`.)
## Remaining (clearly scoped)
1. **Device-side full-image delivery to the slot** — the apply path is done + validated, but the role-
switch test delivered the 631 KB image to the inactive slot via esptool (simulating the transfer,
which is separately proven on-air). The device writing the slot itself during a *full-image* OTA
needs `esp_ota_write`/`esp_partition_write` streaming + a bulk transfer (the RAM `OtaStore` is for
delta-sized images / bring-up).
2. **Multi-fragment blocks** — v1 uses ≤128-byte single-packet blocks; 1 KB blocks need fragment
reassembly in `OtaManager` (`OTA_DATA` already carries `frag_idx`/`frag_total`).
3. **nRF52 apply** — write the `approval` field + reboot-to-DFU for the bootloader fork (external repo).
4. **nRF52 `EndF` `.hex` build wiring** — currently only the ESP32 `.bin` hook is implemented.
5. **P7 auto-propagation + retention** — 24 h announce, finish-current on supersession, 30-day stale GC.
6. **Companion app frames** (`CMD_OTA_*`) + relay/web-seed ingress for the home-node case.
7. **`hw_id` brick-safety** for cross-target (see plan §6.1) — manifest format change, awaiting confirm.
> Device state: the **Heltec now runs companion_radio_usb** (from the role-switch test); reflash the
> repeater env to continue OTA work. RAK4631 runs the OTA repeater.
8. **`hw_id` brick-safety** for cross-target (`ota want`) — manifest field = `sha2-256:4(manufacturer)`;
allows same-HW role switches but refuses incompatible-HW firmware. Manifest format change → see
`OTA_PLAN.md §6.1` (awaiting confirmation since the format was frozen). The manual override
itself is **done + native-tested**.
## Reproduce
```bash
# host tests
./meshcore/bin/python tools/mota/test_mota.py
./meshcore/bin/pio test -e native -f test_ota
# build + flash (OTA repeater)
./meshcore/bin/pio run -e Heltec_v3_repeater -t upload --upload-port /dev/ttyUSB0
./meshcore/bin/pio run -e RAK_4631_repeater -t upload --upload-port /dev/ttyACM0 # DFU
# on-device verify / on-air transfer: see tools/mota/ + scratchpad onair*.py
```
docs/ota_protocol.md
+359 -137
View File
@@ -1,63 +1,103 @@
# MeshCore OTA — `.mota` container & LoRa protocol (v1 draft)
# MeshCore OTA — `.mota` container & LoRa protocol (v2)
Goals: distribute firmware over LoRa as a self-verifying, resumable, BitTorrent-v2-style block
transfer that survives reboots, never auto-applies without consent, and is portable enough for other
projects (e.g. Meshtastic) to adopt.
This is the **single source of truth** for MeshCore's over-the-air firmware update system ("mOTA"). It is
written for developers who want to implement an interoperable peer (server, fetcher, relay, or host tool)
in another codebase or project. Everything below is implemented and hardware-verified in this repository;
where a section names a source file, that file is the authoritative reference for byte-level details.
**Design goals**
- Distribute firmware over LoRa as a **self-verifying, resumable, BitTorrent-style block transfer** that
survives reboots and never auto-applies without explicit consent.
- **Trustless transport / relay:** any node may carry or relay any block; integrity is content-addressed
against a signed merkle root, so a relay need not be trusted and never needs the signing keys.
- **Lowest priority, always:** OTA traffic is enqueued behind all mesh traffic — "eventually upgradable".
A busy node delays OTA indefinitely rather than competing with real traffic.
- **Portable:** the engine (`src/helpers/ota/OtaManager`) is Arduino/radio/crypto-free and host-testable,
so the same logic drives a device, a simulation, or a third-party implementation.
**Source map** (all under `src/helpers/ota/` unless noted)
| Concern | File |
|---|---|
| Constants, enums, flags | `OtaFormat.h` |
| Container/manifest parse | `MotaContainer.{h,cpp}` |
| Merkle tree + proofs | `MerkleTree.{h,cpp}` |
| EndF self-identity | `FirmwareInfo.{h,cpp}` |
| Wire message codec | `OtaProtocol.{h,cpp}` |
| Session engine (serve+fetch+discovery) | `OtaManager.{h,cpp}` |
| Multi-mota / folder relay | `OtaSource.h`, `MotaSourceSerial.{h,cpp}`, `MotaSeederProto.h` |
| Staging stores | `OtaStore.h`, `OtaStoreFlashNrf52.*`, `OtaStoreFlashEsp32.*` |
| Apply | `OtaApply.*`, bootloader `Adafruit_nRF52_Bootloader_OTAFIX` |
| Device glue (CLI/context) | `OtaCli.cpp`, `OtaContext.h` |
| Host tooling | `tools/mota/` (`mota.py`, `motalib.py`, `mota_seeder.py`) |
---
## 1. Conventions
- **Endianness:** all multi-byte integers are little-endian unless stated.
- **Hashes (multihash):** the hash family is declared once per manifest via `hash_algo`. v1 uses
- **Hashes (multihash):** the hash family is declared once per manifest via `hash_algo`. v2 uses
`0x12` = **SHA-256** (the [multihash](https://github.com/multiformats/multihash) code for sha2-256).
Truncations used:
- `sha2-256:4` — first 4 bytes of the SHA-256 digest. Merkle leaves, internal nodes, root, proofs.
- `sha2-256:8` — first 8 bytes. Base-firmware identity (`base_hash`, `EndF`).
- `sha2-256:4` — first 4 bytes of the SHA-256 digest. Merkle leaves, internal nodes, root, proofs,
`manifest_id`, and the discovery `set_digest`.
- `sha2-256:8` — first 8 bytes. Base-firmware identity (`base_hash`, `EndF.body_hash`).
- `sha2-256:32` — full digest. The image security anchor (`image_hash`).
Digests are stored **bare** (just the truncated bytes); the family is implied by `hash_algo`.
- **Signatures:** Ed25519 (RFC 8032), 64-byte detached signature, 32-byte public key.
Reference constants:
**Reference constants** (`OtaFormat.h`):
| Name | Bytes (hex) | ASCII |
| Name | Value | ASCII / note |
|---|---|---|
| Container `MAGIC` | `6D 4F 54 41` | `mOTA` |
| Container `TRAILER` | `76 6B 34 39 36` | `vk496` |
| `EndF` marker | `45 6E 64 46` | `EndF` |
| `hash_algo` (sha2-256) | `12` | — |
| `approval` = not approved | `FF FF FF FF` | (erased) |
| `hash_algo` (sha2-256) | `0x12` | multihash code |
| `format_ver` | `0x02` | this spec |
| `approval` = not approved | `FF FF FF FF` | erased NOR word |
| `approval` = approved | `41 50 52 56` | `APRV` |
| `format_ver` | `01` | — |
| `MFLAG_FULL` | `0x01` | flags bit0 |
| `MFLAG_SIGNED` | `0x02` | flags bit1 |
| `CODEC_FULL` / `_SEQUENTIAL` / `_INPLACE` | `0` / `1` / `2` | §5 |
| `PAYLOAD_TYPE_OTA` | `0x0C` | MeshCore packet type (`src/Packet.h`) |
| `MAX_PACKET_PAYLOAD` | `184` | usable bytes per packet (`src/MeshCore.h`) |
| Default block size | `1024` | `block_size_log2 = 0x0A` |
| OTA TX priority | `250` | lowest (`OTA_TX_PRIORITY`, `src/Mesh.h`) |
---
## 2. Firmware image & the `EndF` trailer
Every OTA-capable firmware build appends a 16-byte `EndF` trailer to its flashed image so a running
node can discover its own size/identity on any MCU (no linker symbols needed).
Every OTA-capable build appends a 16-byte `EndF` trailer to its flashed image so a running node can
discover its own size/identity on any MCU (no linker symbols needed). Implemented by
`FirmwareInfo.cpp`; appended at build time by `tools/mota/pio_endf.py` (post-build hook).
```
flashed image = BODY (image bytes) || EndF trailer
EndF trailer (16 bytes):
off 0 4 "EndF" (45 6E 64 46)
off 0 4 "EndF" 45 6E 64 46
off 4 4 body_len uint32 LE — length of BODY (excludes this 16-byte trailer)
off 12 8 body_hash sha2-256:8 of BODY
off 8 8 body_hash sha2-256:8 of BODY
```
- **Size discovery:** scan flash from the partition top downward for the `EndF` marker; the byte
before it is the last BODY byte. (Same technique as `NRF52Board::getBootloaderVersion`.)
- **Self-identity / delta base matching:** a node's `body_hash` is read directly from its own `EndF`;
a delta's `base_hash` (§5) must equal it. No self-hashing pass required at match time.
- **Size discovery:** scan flash from the partition top downward for the `EndF` marker; the byte before
it is the last BODY byte. (See `ota_self_firmware()`.)
- **Delta base matching:** a node's `body_hash` is read directly from its own `EndF`; a delta's
`base_hash` (§5) must equal it. No self-hashing pass at match time.
- **No circularity:** `EndF` hashes only the BODY, never itself.
The "reconstructed image" referenced by the manifest is the full `BODY || EndF` (what gets flashed).
> **Implementer note:** the bootloader (and any non-Arduino consumer) MUST locate the body extent by
> scanning for `EndF`, never by trusting a stored size — see the bootloader contract in §12.
---
## 3. The `.mota` container
This is the **distributed** form (host-built, wire-transferred).
The distributed form (host-built, wire-transferred). Parsed by `mota_parse()` in `MotaContainer.cpp`.
```
off size field
@@ -70,62 +110,66 @@ off size field
8 + M + P 5 TRAILER = 76 6B 34 39 36
```
`MOTA_TOTAL_SIZE = 4 + 4 + M + P + 5`.
`MOTA_TOTAL_SIZE = 4 + 4 + M + P + 5`. The manifest `M` **includes** `leaves[]`; the manifest-minus-leaves
prefix (`mfl`, sent over the wire as `OTA_MANIFEST`) is `[8, leaves_off)`.
**Staged (in-flash) form.** Written bottom-aligned so `TRAILER` ends at `staging_region_end`. Identical
bytes, except the device mutates two regions in place (both NOR-safe, no re-erase): the `leaves[]`
slots (filled as blocks arrive) and the 4-byte `approval` field (on user approval). Everything else is
immutable.
bytes, except the device mutates two regions in place (both NOR-safe, no re-erase): the `leaves[]` slots
(filled as blocks arrive — §7) and the 4-byte `approval` field (on owner consent — §4.2). Everything else
is immutable.
---
## 4. The manifest
Fields are serialized in this exact order. Conditional fields are present per `flags`.
Fields serialized in this exact order; conditional fields present per `flags`. Fixed head is **89 bytes**
(through `hw_id`). Parsed by `mota_parse_manifest()`.
```
off size field notes
0 1 format_ver = 0x01
1 1 flags bit0 FULL (0=delta/partial, 1=full image)
bit1 SIGNED
bits2-7 reserved (0)
0 1 format_ver = 0x02
1 1 flags bit0 FULL (0=delta/partial, 1=full image); bit1 SIGNED; bits2-7 reserved 0
2 1 hash_algo 0x12 = sha2-256
3 4 target_id device/arch/role discriminator (§9)
7 4 fw_version MAJOR<<24 | MINOR<<16 | PATCH<<8 | pre (comparable uint32)
11 4 image_size size of the reconstructed image (BODY||EndF)
15 4 payload_size PAYLOAD bytes in this container
19 1 block_size_log2 e.g. 0x0A = 1024
20 4 merkle_root sha2-256:4 over PAYLOAD blocks (§6)
20 4 merkle_root sha2-256:4 over PAYLOAD blocks (§6) — also the manifest_id
24 32 image_hash sha2-256:32 of the reconstructed image — SECURITY anchor
56 1 codec_id 0=full/raw, 1=detools-sequential, 2=detools-in-place
57 8 base_hash [present iff !FULL] sha2-256:8 of the BASE image's BODY (matches EndF.body_hash)
. 32 signer_pubkey [present iff SIGNED] Ed25519 public key
. 64 signature [present iff SIGNED] Ed25519 over all bytes from off 0 up to here (exclusive)
57 32 hw_id NUL-padded ASCII hardware tag (e.g. "RAK4631"); same tag => bootable-compatible.
SIGNED. Applier refuses a mismatch (brick-safety); empty on either side = skip.
--- end of fixed 89-byte head ---
89 8 base_hash [iff !FULL] sha2-256:8 of the BASE image's BODY (== that build's EndF.body_hash)
. 32 signer_pubkey [iff SIGNED] Ed25519 public key
. 64 signature [iff SIGNED] Ed25519 over all bytes from off 0 up to here (exclusive)
. 4 approval ALWAYS present. FF FF FF FF = not approved; 41 50 52 56 ("APRV") = approved
--- end of manifest-minus-leaves (mfl); leaves_off = 8 + mfl in the container ---
. 4*BC leaves[] ALWAYS present. BC = ceil(payload_size / 2^block_size_log2). sha2-256:4 each
```
Self-delimiting: a parser knows every offset from `format_ver`/`flags` + `payload_size` (→ `BC`); no
explicit length field is stored.
Self-delimiting: every offset is known from `flags` + `payload_size` (→ `BC`); no length field is stored.
Manifest size (excluding `leaves[]`): unsigned-full 57+4=61, signed-full 161, unsigned-delta 69,
**signed-delta 165**.
Manifest-minus-leaves size (`mfl`): unsigned-full `89+4 = 93`, signed-full `189`, unsigned-delta `101`,
**signed-delta `197`**. A signed manifest exceeds one packet, so `OTA_MANIFEST` is sent multi-fragment
(§8.4) and reassembled by the fetcher.
### 4.1 Signed region
`signature` covers manifest bytes `[0, signature_offset)` — i.e. everything before it, including
`signer_pubkey` and (for deltas) `base_hash`. It does **not** cover `approval` or `leaves[]`:
`signature` covers manifest bytes `[0, signature_offset)` — everything before it, including `signer_pubkey`
and (for deltas) `base_hash`. It does **not** cover `approval` or `leaves[]`:
- `leaves[]` are verified against the signed `merkle_root` (§6), so they need no separate signature.
- `approval` is device-local consent (§7), deliberately outside the signature.
- `approval` is device-local consent (§4.2), deliberately outside the signature.
### 4.2 The `approval` field
- Distributed and **forced on ingest** to `FF FF FF FF` (a peer can never pre-approve).
- The local user's `ota apply` writes `41 50 52 56` (`"APRV"`) — a single NOR-safe write (only clears
- The local owner's `ota applydelta` writes `41 50 52 56` (`"APRV"`) — a single NOR-safe write (only clears
bits from the erased word). Any partial/other value reads as not-approved (fail-safe).
- Auto-bound to this image: it lives in this `.mota`'s manifest and is re-erased when a new `.mota` is
staged.
- It is a **consent** marker, not a security primitive. Authenticity = `signature` + `image_hash`.
- Bound to this image (lives in this `.mota`'s manifest, re-erased when a new `.mota` is staged).
- A **consent** marker, not a security primitive. Authenticity = `signature` + `image_hash` + `hw_id`.
---
@@ -133,44 +177,45 @@ Manifest size (excluding `leaves[]`): unsigned-full 57+4=61, signed-full 161, un
`PAYLOAD` is either the full reconstructed image (`FULL`) or a delta (`!FULL`).
| `codec_id` | Meaning | Notes |
| `codec_id` | Meaning | Used by |
|---|---|---|
| 0 | full / raw | PAYLOAD = reconstructed image (`BODY||EndF`). Typical for ESP32 (A/B). |
| 1 | detools sequential | needs random read of base + sequential write of result (e.g. ESP32 A→B). |
| 2 | detools in-place | bounded scratch; rewrites the app region in place (nRF52 single-slot). |
| 0 | full / raw | PAYLOAD = reconstructed image (`BODY‖EndF`). ESP32 A/B (and any board for a full image). |
| 1 | detools **sequential** | random read of base + sequential write of result → ESP32 A→B inactive slot. |
| 2 | detools **in-place** | bounded scratch; rewrites the app region in place → nRF52 single-slot. |
For deltas, `base_hash` = the base image's `EndF.body_hash` (sha2-256:8 of its BODY). A node applies a
For deltas, `base_hash` = the base build's `EndF.body_hash` (sha2-256:8 of its BODY). A node applies a
delta only if `base_hash` matches its own `EndF.body_hash`. After applying, the result MUST hash
(sha2-256:32) to `image_hash` before it is flashed — this is the hard security gate.
(sha2-256:32) to `image_hash` before it is booted — the hard security gate.
Compression is internal to the detools patch; the chosen scheme must be supported by the applier
(bootloader contract, §12). Patches are produced by detools 0.53.0 (`tools/mota` → `detools.create_patch`)
and decoded on-device by detools' own embeddable C decoder, vendored verbatim at
`src/helpers/ota/detools/` (see its `README.meshcore.txt`). That build enables only the self-contained
`NONE` + `CRLE` compressions (no malloc / liblzma / heatshrink), so MeshCore deltas use
`--codec sequential --compression crle`. The ESP32 applier (`OtaApply.cpp::ota_apply_detools_mota`)
wires the decoder's callbacks to: read base ← running OTA slot, stream patch ← fetched bytes in RAM,
write output → inactive slot, hashing the output and checking it against `image_hash` before arming.
**A fetcher only requests firmware it can apply.** Each node declares the codec(s) it can apply
(`set_apply_codec`/`set_apply_codec2`): ESP32 accepts `full` + `sequential` (+ `in-place`), nRF52 accepts
`full` + `in-place`. `CODEC_FULL` is always acceptable. A `.mota` with an unsupported codec is rejected at
discovery time, before any blocks are requested.
Compression is internal to the detools patch and must be supported by the applier. Patches are produced by
**detools 0.53.0** (`tools/mota` → `detools.create_patch`) and decoded on-device by detools' embeddable C
decoder, vendored verbatim at `src/helpers/ota/detools/` (see its `README.meshcore.txt`). That build
enables only the self-contained `NONE` + `CRLE` compressions (no malloc/liblzma/heatshrink), so MeshCore
deltas use `--compression crle`. **Do not reimplement the codec** — use the vendored decoder.
---
## 6. Merkle tree (sha2-256:4)
Purpose: verify each PAYLOAD block against the signed `merkle_root` **before** the whole payload
exists, so corruption/forgery is localized to a block.
Verifies each PAYLOAD block against the signed `merkle_root` **before** the whole payload exists, so
corruption/forgery is localized to a block. Implemented in `MerkleTree.cpp`.
- **Blocks:** PAYLOAD is split into `BC = ceil(payload_size / B)` blocks, `B = 2^block_size_log2`
(default 1024). The last block is its real length (**no zero padding**).
- **Blocks:** PAYLOAD splits into `BC = ceil(payload_size / B)` blocks, `B = 2^block_size_log2` (default
1024). The last block is its real length (**no zero padding**).
- **Leaf:** `leaves[i] = sha2-256:4( block_i_bytes )`.
- **Internal node:** `node = sha2-256:4( left || right )` (4+4 = 8 input bytes).
- **Odd level:** if a level has an odd number of nodes, the **last node is promoted unchanged** to the
next level (no duplication).
- **Internal node:** `node = sha2-256:4( left ‖ right )` (4+4 input bytes).
- **Odd level:** an odd count promotes the **last node unchanged** to the next level (no duplication).
- **Root:** reduce until one node remains. `BC == 1` → root = `leaves[0]`. `BC == 0` is invalid.
### 6.1 Proofs
A proof for block `i` is the ordered list of sibling digests from leaf to root, each tagged
left/right. Promoted levels contribute **no** element. Verification (needs `BC` to know the shape):
A proof for block `i` is the ordered list of sibling digests from leaf to root. Promoted levels contribute
**no** element. Verification (needs `BC` to know the tree shape):
```
h = leaf_i ; idx = i ; n = BC ; p = 0
@@ -179,110 +224,287 @@ while n > 1:
pass
else:
sib, side = proof[p] ; p += 1
h = sha2-256:4( sib || h ) if side==left else sha2-256:4( h || sib )
h = sha2-256:4( sib ‖ h ) if side==left else sha2-256:4( h ‖ sib )
idx //= 2 ; n = (n + 1) // 2
accept iff h == merkle_root and p == len(proof)
```
Over LoRa, `leaves[]` are **omitted** from the manifest transfer; a serving node computes a block's
proof on demand from its stored `leaves[]`, and the fetcher fills its own `leaves[i]` as each verified
block lands.
Over LoRa, `leaves[]` are **omitted** from the manifest transfer; a serving node computes a block's proof
on demand from its stored `leaves[]` (`OTA_REQ_PROOF`/`OTA_PROOF`, §8.5), and the fetcher fills its own
`leaves[i]` as each verified block lands.
---
## 7. Block availability (persistent, derived from `leaves[]`)
## 7. Block availability, staging & resume
There is no separate availability structure. **Block `i` is present ⟺ `leaves[i]` is non-erased**
(`!= FF FF FF FF`). Because `leaves[]` live in the staged flash region, availability **survives
reboot**. Commit order per block (crash-safe): (1) verify proof, (2) write block payload to its
offset, (3) write `leaves[i]` **last**. A power loss before step 3 leaves the slot erased → the block
is simply re-fetched (idempotent). On boot a node rebuilds a small in-RAM bitmap (`ceil(BC/8)` bytes)
by scanning `leaves[]`.
(`!= FF FF FF FF`). Because `leaves[]` live in the staged flash region, availability **survives reboot**.
A node holding the complete payload (or relaying/serving its own firmware) advertises `have_all`
instead of a bitmap.
**Commit order per block (crash-safe):** (1) verify proof, (2) write block payload to its offset, (3) write
`leaves[i]` **last**. A power loss before step 3 leaves the slot erased → the block is simply re-fetched
(idempotent). On boot a node rebuilds an in-RAM present-bitmap by scanning `leaves[]`.
**Resume (`OtaManager::resumeStaged` + `OtaStore::checkpoint`/`reopen`):** an interrupted fetch resumes from
the staged container after a reboot — re-parse the stored manifest, recompute geometry, count present
blocks, continue fetching the holes (or jump straight to COMPLETE). The checkpoint cadence (persist progress
every N committed blocks) is runtime-tunable (`ota config checkpoint <N>`, 0 = only finalized containers
resume). Stores keep `leaves[]` in RAM until flush and never auto-GC, preserving resumable progress.
**Flash-store note (RX-safe writes):** a flash page-erase halts the CPU (~85 ms on nRF52) and starves LoRa
RX, so the flash stores (`OtaStoreFlashNrf52`/`OtaStoreFlashEsp32`) **coalesce writes to the erase unit**
(4 KB page / sector) and commit each once off the per-packet path — RAM stays O(one page), not O(image). A
small delta that fits page 0 does zero flash I/O until COMPLETE.
---
## 8. LoRa OTA protocol
Carried in MeshCore packets with **`PAYLOAD_TYPE_OTA = 0x0C`** (subject to change if core devs prefer
reusing `RAW_CUSTOM 0x0F` + subtype). Every OTA packet payload:
Carried in MeshCore packets with **`PAYLOAD_TYPE_OTA = 0x0C`**. Every OTA packet payload is:
```
[0] ota_msg_type
[1..] body
[0] ota_msg_type (OtaMsgType, OtaFormat.h)
[1..] body (fixed per type; encode/decode in OtaProtocol.cpp)
```
- **Routing:** `OTA_ADV`/`OTA_QUERY` flood; `OTA_HAVE`/`OTA_MANIFEST`/`OTA_REQ`/`OTA_DATA` direct.
- **Hop cap:** OTA refuses to retransmit when `getPathHashCount() >= ota_hop_limit` (default **3**,
configurable). No change to core routing.
- **Priority:** enqueued at the lowest TX priority (~250) and only when the duty-cycle/airtime budget
has spare headroom, so OTA never competes with mesh traffic.
- **`manifest_id`** = the manifest's `merkle_root` (4 bytes) — a compact content id.
Message types:
| `ota_msg_type` | val | dir | body |
| `ota_msg_type` | val | routing | purpose |
|---|---|---|---|
| `OTA_ADV` | 0x01 | flood | `target_id(4) fw_version(4) image_size(4) block_size_log2(1) merkle_root(4) image_hash8(8) flags(1) [base_hash(8) if delta] have_all(1)` |
| `OTA_QUERY` | 0x02 | flood | `target_id(4) min_version(4) caps(1)` (caps bit0 want_delta, bit1 want_full) |
| `OTA_HAVE` | 0x03 | direct | `manifest_id(4) bitmap_off(2) bitmap[]` |
| `OTA_GET_MANIFEST` | 0x04 | direct | `manifest_id(4)` |
| `OTA_MANIFEST` | 0x05 | direct | `manifest_id(4) frag_idx(1) frag_total(1) bytes[]` (omits `leaves[]`) |
| `OTA_REQ` | 0x06 | direct | `manifest_id(4) want_off(2) want_bitmap[]` |
| `OTA_DATA` | 0x07 | direct | `manifest_id(4) block_idx(2) frag_idx(1) frag_total(1) [proof in frag0] bytes[]` |
| `OTA_ADV` | 0x01 | flood | tiny per-node beacon (discovery tier 1) |
| `OTA_QUERY` | 0x02 | flood | ask a source for its catalog (discovery tier 2) |
| `OTA_HAVE` | 0x03 | flood | the catalog reply (fragmented, digest-tagged) |
| `OTA_GET_MANIFEST` | 0x04 | direct | request a manifest by `manifest_id` |
| `OTA_MANIFEST` | 0x05 | direct | the manifest-minus-leaves, fragmented |
| `OTA_REQ` | 0x06 | direct | request a window of blocks' DATA |
| `OTA_DATA` | 0x07 | direct | one self-describing fragment of a block's data |
| `OTA_REQ_PROOF` | 0x08 | direct | request the merkle proof for one block |
| `OTA_PROOF` | 0x09 | direct | the merkle proof for one block |
Sizing against the 184-byte `MAX_PACKET_PAYLOAD`: `OTA_DATA` fixed overhead ≈ 9 B → ~175 B/fragment →
**6 fragments per 1 KB block**; a proof for ≤512 blocks ≤ 9×4 = 36 B (carried in `frag0`); an
availability bitmap for 500 blocks ≈ 63 B (one packet).
- **`manifest_id`** = the manifest's `merkle_root` (4 bytes) — a compact content id present in every
transfer message, so a multi-mota server dispatches each request to the right image.
- **Priority:** all OTA packets enqueue at `OTA_TX_PRIORITY = 250` (lowest). OTA never competes with mesh
traffic; on a busy node it is delayed indefinitely.
- **Reliability is *eventual*:** the fetcher re-requests missing fragments/blocks after a timeout, possibly
from a different peer. No hard ACKs, no global ordering.
- **Relay:** replies are flooded, so transparent relay needs no per-requester addressing, and the transfer
is trustless (the fetcher verifies every block against the signed root). Any neighbor may serve any
fragment it has.
Reliability is *eventual*: the fetcher re-requests un-acked blocks after a timeout, possibly from a
different peer. No hard ACKs, no ordering.
### 8.1 Two-tier discovery
### 8.1 Relay seeding (companion frames)
Because a node may serve **many** mOTAs (its own firmware plus an external folder — §10), discovery is split
so the periodic beacon stays tiny regardless of catalog size:
A node need not store a foreign-target `.mota` to serve it: a relay advertises a manifest on behalf of
an external source and **pulls blocks on demand**. Companion-app frames: `CMD_OTA_PROVIDE_MANIFEST`
(app→node, starts advertising), event `PUSH_OTA_BLOCK_REQ(manifest_id, block_idx)` (node→app), reply
`CMD_OTA_PROVIDE_BLOCK(manifest_id, block_idx, bytes)`.
**Tier 1 — `OTA_ADV` beacon** (10 bytes, constant, flooded periodically):
```
seeder_id[4] advertiser node id = pubkey[0:4]; the QUERY address + distinct-source id
n_motas uint8 — count of complete servable mOTAs (saturates at 255)
set_digest[4] sha2-256:4 over the SORTED set of served manifest_ids (see below)
```
`set_digest` is a **content hash of the offering**, not a counter: canonical across nodes, and it changes
iff the set of served mids changes. A peer that has already catalogued this `{seeder, set_digest}` ignores
the beacon (steady state is query-free). For a single served mota, `set_digest = sha2-256:4(mid)`.
**Tier 2 — `OTA_QUERY` → `OTA_HAVE`** (on interest only):
```
OTA_QUERY (flood): seeder_id[4] set_digest[4] filter_target(uint32) # filter_target 0 = everything
OTA_HAVE (flood): seeder_id[4] set_digest[4] frag_idx(1) frag_total(1) n_rows(1) rows[]
HaveRow (14 bytes, OTA_HAVE_ROW_BYTES): mid[4] target_id(4) fw_version(4) codec_id(1) flags(1)
```
A node interested in a source's offering schedules a QUERY; the source replies with its full catalog as
`OTA_HAVE` rows (fragmented if they exceed one packet — up to 12 rows per fragment). The heavy manifest is
fetched per-mid only on commit (§8.3).
### 8.2 Anti-storm (mandatory at mesh scale)
If 50 neighbours all queried a new beacon at once, the mesh would collapse. Mitigations (gossip/mDNS
pattern), all in `OtaManager`:
- **`OTA_HAVE` is flooded and digest-tagged.** EVERY node that overhears it caches the rows **passively**
(keyed by `{seeder, set_digest}`) — no query of its own needed.
- **Jittered query:** a peer needing a catalog schedules its `OTA_QUERY` after a random delay
`OTA_QUERY_MIN_MS (300) + rand(OTA_QUERY_SPREAD_MS (4000))`, derived from `id ⊕ digest ⊕ self`.
- **Overhear suppression:** during the jitter window, overhearing *another* QUERY **or** a HAVE for the same
`{seeder, set_digest}` CANCELS the pending query.
Net effect: a digest change costs ~1 query + ~1 HAVE flood mesh-wide; a stable mesh is query-free.
### 8.3 Fetch handshake
```
fetcher server (any node that has the mid)
OTA_GET_MANIFEST(mid) ───────►
◄─────── OTA_MANIFEST(mid, frag_idx, frag_total, bytes) × frag_total
(reassemble manifest, verify, compute geometry: BC, block_size, payload_size)
for each missing block window:
OTA_REQ(mid, start_block, count) ►
◄─────── OTA_DATA(mid, block_idx, frag_off, data) × (per block)
(reassemble block from frag_off slices)
OTA_REQ_PROOF(mid, block_idx) ────►
◄─────── OTA_PROOF(mid, block_idx, n_proof, proof)
(verify proof vs merkle_root → write block → write leaves[i])
when all blocks present: verify full merkle_root + image_hash → COMPLETE
```
### 8.4 Message bodies (transfer)
All offsets after the 1-byte type. Encoders/decoders in `OtaProtocol.cpp`; constants in `OtaManager.h`.
```
OTA_GET_MANIFEST: manifest_id[4]
OTA_MANIFEST: manifest_id[4] frag_idx(1) frag_total(1) bytes[] # up to OTA_MF_FRAG=176 B/frag
OTA_REQ: manifest_id[4] start_block(uint16) count(1)
OTA_DATA: manifest_id[4] block_idx(uint16) frag_off(uint16) data[] # up to OTA_FRAG_DATA=160 B
OTA_REQ_PROOF: manifest_id[4] block_idx(uint16)
OTA_PROOF: manifest_id[4] block_idx(uint16) n_proof(1) proof[] # n_proof × 4 bytes
```
- **Block ⇆ fragments:** a 1 KB block is split into self-describing `OTA_DATA` fragments. `frag_off` is the
byte offset of `data` within the block, so the global position is `block_idx*block_size + frag_off` —
a fragment is self-placing and may be requested from **any** peer (BitTorrent-style). The fetcher tracks a
per-block slice bitmap and reassembles before requesting the proof.
- **Data and proof are separate phases.** `OTA_DATA` carries no proof; the proof is fetched once per block
via `OTA_REQ_PROOF`/`OTA_PROOF` after the block's data is complete.
### 8.5 Sizing against `MAX_PACKET_PAYLOAD = 184`
| message | fixed overhead | payload/packet |
|---|---|---|
| `OTA_DATA` | 9 B (type+mid4+idx2+off2) | `OTA_FRAG_DATA = 160` → 7 frags per 1 KB block |
| `OTA_MANIFEST` | 7 B | `OTA_MF_FRAG = 176` → signed manifest ≈ 2 frags |
| `OTA_HAVE` | 12 B | 12 rows × 14 B per fragment |
| `OTA_PROOF` | 8 B | up to ~44 sibling digests (≫ any real tree) |
A served mota supports up to `OTA_MAX_BLOCK/4` leaves in the default 4 KB proof scratch (≤1024 blocks ≈ 1 MB
payload); larger self-images pass a bigger scratch buffer.
---
## 9. Identity, trust & versioning
- **`target_id`** (4 B): compile-time `sha2-256:4(pio_env_name + radio_class + ldscript/partition +
platform)`, injected by `build.sh`, read via `MainBoard::getOtaTargetId()`. A node only fetches/serves
matching `target_id`. (The PlatformIO env name uniquely captures hardware AND role/partition.)
- **`fw_version`:** packed comparable uint32 (`MAJOR<<24|MINOR<<16|PATCH<<8|pre`).
- **Signing & allowlist:** a node keeps a runtime-managed allowlist of trusted Ed25519 signer pubkeys
(none embedded in firmware). A `.mota` is eligible for **auto-apply** only if signed by an allowlisted
key, the signature verifies, and `image_hash` matches; otherwise it is manual-apply only with explicit
confirmation. **Transfer needs no trust** — blocks are content-addressed against the signed root, so
any (untrusted) neighbor may relay them.
### 9.1 Supersession & retention
- **Finish-current:** a newer version announced mid-download does not abort the in-progress transfer.
- **Stale GC:** a staged `.mota` carries a persistent `staged_at` epoch; it is discarded after
`ota_stale_ttl` (default **30 days**) unless pinned (`ota keep`) or applying — reclaiming flash from
superseded-complete and stalled-partial images alike. `ota discard` frees the slot immediately.
- **`target_id`** (4 B): compile-time `sha2-256:4(pio_env_name)` (little-endian uint32), injected as
`-D MOTA_TARGET_ID` by `build.sh` and read via `MainBoard::getOtaTargetId()`; `tools/mota` computes the
same from `--target-env`. The PlatformIO env name uniquely captures hardware **and** role/partition, so a
node auto-fetches only matching firmware. A manual `ota pull`/`want` can override target (deliberate role
switch); the `hw_id` brick-safety gate (§4) still applies at apply time.
- **`fw_version`:** packed comparable uint32 (`MAJOR<<24 | MINOR<<16 | PATCH<<8 | pre`).
- **`hw_id`:** 32-byte NUL-padded ASCII hardware tag inside the signed head. The applier refuses a `.mota`
whose `hw_id` differs from the device's own tag (empty on either side = permissive). Brick-safety
independent of signature.
- **Signing & allowlist:** a node keeps a runtime allowlist of trusted Ed25519 signer pubkeys (none embedded
in firmware; `ota key add/list/rm`). A `.mota` is eligible for **auto-install** only if signed by an
allowlisted key, the signature verifies, and `image_hash` matches; otherwise it is manual-apply only with
explicit confirmation. **Transfer needs no trust** — blocks are content-addressed against the signed root.
- **Policies (persisted):** `autofetch` ∈ {off, any, signed} (default off) gates automatic block fetching of
own-target adverts; `autoinstall` ∈ {off, trusted} (default off) gates auto-apply of a COMPLETE signed +
allowlisted fetch. Conservative defaults: a fresh node discovers + announces but never fetches/installs
without operator intent.
- **Supersession:** a newer version announced mid-download does not abort the in-progress transfer
(finish-current).
---
## 10. Apply & bootloader contract (summary)
## 10. Multi-mota serve & the external "folder" relay
- **ESP32:** in-firmware via `Update`/`esp_ota_*` into the inactive A/B slot, then set boot + reboot
(power-safe, rollback-capable). No bootloader changes.
- **nRF52:** running firmware **never** flashes the app. `ota apply` verifies fully, writes the
`approval` field (`"APRV"`), then reboots into DFU. The modified bootloader
(`Adafruit_nRF52_Bootloader_OTAFIX`) locates the staged `.mota` by scanning for `MAGIC`, re-checks
`TRAILER` + signature + `image_hash` + `approval == "APRV"`, applies the codec (delta in-place over
the app region), then clears state and boots. The signature proves author authenticity; `approval`
proves local owner consent — both required.
A node serves a **set** of mOTAs: its own firmware plus, optionally, an external folder of `.mota` files it
relays without holding them in flash. To peers it simply "has N mOTAs"; the relay is trustless (fetchers
verify everything). The serve side (`OtaManager`) keeps a lightweight registry of what it advertises and two
resident "views": `view0` (its own firmware) and one on-demand view loaded from a source when a request
targets an external mota. Every fetch message carries `manifest_id`, so dispatch is a registry lookup.
### 10.1 The `MotaSource` abstraction (`OtaSource.h`)
Transport-agnostic provider of one or more complete `.mota` as random-access bytes. The same serve code
drives USB-serial, BLE, a WiFi URL list, an NFS/samba mount, etc. — only `read()` differs.
```cpp
struct MotaDesc { // catalog metadata + region offsets (no whole image in RAM)
uint8_t mid[4]; uint32_t target_id, fw_version; uint8_t codec_id, flags;
uint32_t total_size, leaves_off, block_count, payload_off, payload_size;
};
class MotaSource {
virtual uint8_t count(); // # mOTAs offered
virtual bool describe(uint8_t idx, MotaDesc& out); // metadata + offsets
virtual bool read(uint8_t idx, uint32_t off, uint8_t* buf, uint32_t len); // random-access bytes
};
```
To serve an external mota the node reads its manifest-minus-leaves + `leaves[]` into RAM (≤4 KB for ≤1024
blocks) and streams payload blocks from the source on demand; proofs are generated from the read leaves.
### 10.2 The `mota-seeder` transport (`MotaSeederProto.h`)
The first concrete `MotaSource` is a host daemon (`tools/mota/mota_seeder.py`) serving a folder over the
device's **USB serial — the same console the CLI uses** (no extra hardware). The device only emits request
frames *while actively serving a fetch*, and reads the reply synchronously, so binary frames coexist with
the text CLI/logs (resync on magic + checksum). Little-endian, XOR-checksummed:
```
request (device → host): 'M' 'S' op(1) args... xsum(1 = XOR of op+args)
response (host → device): 'm' 's' op(1) status(1) payload... xsum(1 = XOR of all prior)
OP_COUNT 0x01 args: - → payload: count(1)
OP_DESCRIBE 0x02 args: idx(1) → payload: MotaDesc wire (38 B)
OP_READ 0x03 args: idx(1) off(4) len(2) → payload: len bytes
MotaDesc wire (38 B): mid[4] target_id(4) fw_version(4) codec(1) flags(1)
total_size(4) leaves_off(4) block_count(4) payload_off(4) payload_size(4)
status: 0 = OK, non-zero = error (out of range / past EOF).
```
Device CLI: `ota folder on` (attach + announce), `ota folder` (list), `ota folder off`. Build flag
`OTA_FOLDER_SERIAL` (default stream = console `Serial`; override `OTA_FOLDER_SERIAL_STREAM` + define
`OTA_FOLDER_SERIAL_BEGIN` for a dedicated UART). Verified on hardware: a RAK4631 relays a host folder to a
Heltec V3 over one USB cable, every block merkle-checked.
---
## 11. Versioning of this spec
## 11. CLI surface (`OtaCli.cpp`)
`format_ver = 1`. Future changes bump `format_ver`; the multihash `hash_algo` allows changing the
digest family without a format bump. Unknown `format_ver`/`codec_id`/`ota_msg_type` values are ignored
(forward-compatible).
User-facing OTA data should travel via `CMD_OTA_*` companion binary frames; the text CLI below is
debug/operator oriented and replies are `snprintf`-bounded into a 160-byte buffer.
```
ota status session + self-fw summary
ota neighbors discovered mOTAs (queries sources; rows arrive async via OTA_HAVE)
ota announce serve self + send a beacon now
ota pull <#|mid8> fetch a chosen mOTA (manual; works regardless of autofetch)
ota drop drop the current fetch session (free the slot)
ota folder on|off attach/detach an external .mota folder (host daemon) ; bare = list
ota self print this firmware's EndF (body/image size, base_hash)
ota applydelta verify + approve + (ESP32) apply / (nRF52) reboot-to-bootloader
ota config [autofetch|autoinstall|checkpoint] ... show/set persisted policy
ota key add|list|rm <hex> trusted signer allowlist
ota dev ... bring-up helpers (stage/recv/serve/verify)
```
---
## 12. Apply & bootloader contract
- **ESP32 (A/B):** applied in-firmware via the detools decoder into the inactive OTA slot
(`OtaApply.cpp::ota_apply_detools_mota` + `OtaStoreFlashEsp32`), then set-boot + reboot (power-safe,
rollback-capable). No bootloader changes. Erase ranges must be sector-aligned (4096).
- **nRF52 (single-slot):** the running firmware **never** flashes the app. `ota applydelta` verifies fully
(`image_hash`, `base_hash`, signature/allowlist, `hw_id`), writes `approval = "APRV"`, then reboots into
the modified bootloader (`Adafruit_nRF52_Bootloader_OTAFIX`). The bootloader:
1. **scans flash for `MAGIC`** to find the staged `.mota` (it must NOT trust any stored size),
2. re-checks `TRAILER`, `image_hash`, `approval == "APRV"`, and that the delta's `base_hash` equals the
running firmware's `EndF.body_hash` (recomputed by scanning for `EndF` — never trust `bank_0_size`),
3. applies the in-place codec over the app region and boots only if the result hashes to `image_hash`.
The signature proves author authenticity; `approval` proves local owner consent — both required to apply.
> **Bootloader testing note:** always test apply with a *real different* image (base ≠ target). A same-image
> (X→X) "delta" trivially reproduces the target and gives a false positive.
---
## 13. Versioning of this spec
`format_ver = 2` (v2 added `hw_id` to the signed head and split discovery/transfer as in §8/§10). Future
changes bump `format_ver`; the multihash `hash_algo` allows changing the digest family without a format
bump. Unknown `format_ver` / `codec_id` / `ota_msg_type` values are ignored (forward-compatible).
platformio.ini
+9 -4
View File
@@ -59,15 +59,19 @@ extends = arduino_base
platform = platformio/espressif32@6.11.0
monitor_filters = esp32_exception_decoder
; OTA is available on every ESP32 variant (A/B via esp_ota; the detools-sequential apply decodes into the
; inactive slot). The build always includes it; for `applydelta` to actually run, the board must use a
; dual-app/OTA partition table (board_build.partitions) with two app slots + otadata. pio_endf appends
; the EndF self-identity trailer. (nRF52 single-slot OTA is enabled per-board, not here — it needs a
; custom bootloader; RP2040/STM32 have no A/B path yet.)
; inactive slot). The received .mota is staged in the INACTIVE OTA slot (OtaStoreFlashEsp32), not RAM, so
; deltas/full images of any size fetch over the air (RX-safe, sector-coalesced). The build always includes
; it; for `applydelta` to actually run, the board must use a dual-app/OTA partition table
; (board_build.partitions) with two app slots + otadata (boards without one fetch-refuse cleanly).
; pio_endf appends the EndF self-identity trailer. (nRF52 single-slot OTA is enabled per-board, not here —
; it needs a custom bootloader; RP2040/STM32 have no A/B path yet.)
extra_scripts = merge-bin.py
post:tools/mota/pio_endf.py
build_flags = ${arduino_base.build_flags}
-D ESP32_PLATFORM
-D ENABLE_OTA=1
-D OTA_FLASH_STORE=1
-D OTA_FOLDER_SERIAL ; `ota folder on` relays a host folder of .mota over the USB console (no extra HW)
; -D ESP32_CPU_FREQ=80 ; change it to your need
build_src_filter = ${arduino_base.build_src_filter}
+<helpers/ota/*.cpp>
@@ -118,6 +122,7 @@ extra_scripts = ${nrf52_base.extra_scripts}
build_flags = ${nrf52_base.build_flags}
-D ENABLE_OTA=1
-D OTA_FLASH_STORE=1
-D OTA_FOLDER_SERIAL ; `ota folder on` relays a host folder of .mota over the USB console (no extra HW)
build_src_filter = ${nrf52_base.build_src_filter}
+<helpers/ota/*.cpp>
lib_deps = ${nrf52_base.lib_deps}
src/Mesh.cpp
+61 -3
View File
@@ -2,6 +2,20 @@
//#include <Arduino.h>
#if defined(ENABLE_OTA)
#include "helpers/ota/OtaContext.h" // OTA mesh-integration is centralized here so every role gets it
#include "helpers/ota/OtaProtocol.h" // decode_adv -> the `ota neighbors` discovery table
#include "helpers/ota/OtaSelf.h" // ota_self_firmware -> auto-advertise our own image
#ifndef OTA_ANNOUNCE_BOOT_MS
#define OTA_ANNOUNCE_BOOT_MS 30000UL // first self-advert ~30 s after boot (let the node settle)
#endif
#ifndef OTA_ANNOUNCE_BURST
#define OTA_ANNOUNCE_BURST 4 // a few closely-spaced boot adverts so co-booting peers catch one
#endif
#ifndef OTA_ANNOUNCE_BURST_MS
#define OTA_ANNOUNCE_BURST_MS 45000UL // spacing during the boot burst (~3 min total), then ...
#endif
#ifndef OTA_ANNOUNCE_INTERVAL_MS
#define OTA_ANNOUNCE_INTERVAL_MS 86400000UL // ... every 24 h — all lowest priority, duty-gated
#endif
#endif
namespace mesh {
@@ -22,7 +36,13 @@ void Mesh::begin() {
#ifdef MOTA_TARGET_ID
my_tid = (uint32_t)(MOTA_TARGET_ID); // sha2-256:4(env name), injected by build.sh
#endif
ota::ota_ctx().begin(my_tid, Mesh::otaSendAdapter, this); // also sets the platform apply codec
const char* my_hw = "";
#ifdef MOTA_HW_ID
my_hw = MOTA_HW_ID; // human-readable hardware tag (per-variant), for the apply hw gate
#endif
ota::ota_ctx().begin(my_tid, Mesh::otaSendAdapter, this, my_hw); // also sets the platform apply codec
ota::ota_ctx().manager.set_seeder_id(self_id.pub_key); // node id (pubkey[0:4]) for advert seeder count
_next_ota_announce = futureMillis(OTA_ANNOUNCE_BOOT_MS); // advertise our own fw shortly after boot
#endif
}
@@ -47,9 +67,44 @@ void Mesh::loop() {
}
}
if (millisHasNowPassed(_next_ota_tick)) {
ota::ota_ctx().manager.loop(); // re-request still-missing OTA blocks (rate-limited)
// one-shot on first tick: resume an interrupted fetch left staged in flash before a reboot. Only adopt
// a PARTIAL container (continue fetching the holes); a COMPLETE one is left for manual/auto-install,
// not re-adopted at boot. requestMissing() (inside resumeStaged) drives the rest via REQ/DATA.
if (!_ota_resumed) {
_ota_resumed = true;
ota::OtaContext& oc = ota::ota_ctx();
if (oc.manager.fetchState() == ota::OtaManager::IDLE && oc.manager.resumeStaged(nullptr)
&& oc.manager.fetchState() == ota::OtaManager::COMPLETE) {
oc.manager.reset_session(); // don't auto-adopt a complete staged container on boot
}
}
ota::ota_ctx().manager.set_clock(_ms->getMillis()); // for discovery jitter/ages + the pending-query timer
ota::ota_ctx().manager.loop(); // re-request still-missing OTA blocks + fire scheduled queries
_next_ota_tick = futureMillis(3000);
}
if (millisHasNowPassed(_next_ota_announce)) { // auto-advertise so peers discover us (tiny beacon)
ota::OtaContext& oc = ota::ota_ctx();
// To be discoverable as a source of our OWN firmware, set up flash-backed self-serve once; then the
// beacon (announce) advertises our served set and peers can QUERY + fetch it.
if (!oc.serving) oc.serving = ota::ota_serve_self(oc, 0);
oc.manager.announce();
// boot burst (a few closely-spaced adverts so a co-booting peer catches one), then settle to daily
_next_ota_announce = futureMillis(_ota_announce_count < OTA_ANNOUNCE_BURST
? OTA_ANNOUNCE_BURST_MS : OTA_ANNOUNCE_INTERVAL_MS);
if (_ota_announce_count < 250) _ota_announce_count++;
}
{ // auto-install (once per COMPLETE fetch): only signed images, and apply_fetched enforces trust
ota::OtaContext& oc = ota::ota_ctx();
if (oc.manager.fetchState() != ota::OtaManager::COMPLETE) {
_ota_autoinstall_tried = false;
} else if (!_ota_autoinstall_tried && !oc.apply_pending
&& oc.autoinstall == ota::OtaContext::AUTOINSTALL_TRUSTED
&& oc.manager.fetched_is_signed()) {
_ota_autoinstall_tried = true;
char msg[100];
oc.apply_fetched(msg); // arms + sets apply_pending only if signed & allowlisted; refused otherwise
}
}
#endif
}
@@ -358,7 +413,10 @@ DispatcherAction Mesh::onRecvPacket(Packet* pkt) {
// dedup would suppress those retries and the transfer could never recover from a lost reply.
// hasSeen() is used ONLY to avoid re-flooding the same packet more than once.
bool seen = _tables->hasSeen(pkt);
ota::ota_ctx().manager.on_message(pkt->payload, pkt->payload_len); // central OTA receive (all roles)
ota::ota_ctx().manager.set_clock(_ms->getMillis()); // discovery jitter/ages
ota::ota_ctx().manager.on_message(pkt->payload, pkt->payload_len); // central OTA receive (beacon/query/
// have/manifest/data/proof; all roles)
ota::ota_ctx().track_session(ota::ota_ctx().manager.fetchState(), _ms->getMillis());
onOtaRecv(pkt); // optional per-example hook
// Re-flood with a hop cap and the LOWEST priority, so OTA never competes with mesh traffic.
uint8_t n = pkt->getPathHashCount();
src/Mesh.h
+4
View File
@@ -168,6 +168,10 @@ protected:
// companion, room, sensor, ...) gets fetch/serve/apply without per-example wiring.
static void otaSendAdapter(void* ctx, const uint8_t* msg, uint16_t len, bool flood);
unsigned long _next_ota_tick = 0;
unsigned long _next_ota_announce = 0; // auto-advertise our own fw: boot burst + every OTA_ANNOUNCE_INTERVAL
uint8_t _ota_announce_count = 0; // adverts sent so far (boot burst before settling to daily)
bool _ota_resumed = false; // one-shot: resumed an interrupted fetch staged in flash on boot
bool _ota_autoinstall_tried = false; // attempted auto-install for the current COMPLETE fetch
#endif
/**
src/MeshCore.h
+10
View File
@@ -74,6 +74,16 @@ public:
return 0;
#endif
}
// Human-readable hardware tag (<=32 ASCII chars, e.g. "RAK4631") naming the hardware this firmware can
// boot on. Same tag == bootable-compatible; the OTA applier refuses a `.mota` whose hw_id differs (brick-
// safety). Defined per-variant via the MOTA_HW_ID build flag; "" when unset (then the check is skipped).
virtual const char* getOtaHwId() const {
#ifdef MOTA_HW_ID
return MOTA_HW_ID;
#else
return "";
#endif
}
#endif
// Power management interface (boards with power management override these)
src/helpers/CommonCLI.cpp
+51 -3
View File
@@ -6,6 +6,7 @@
#include <RTClib.h>
#if defined(ENABLE_OTA)
#include "ota/OtaCli.h"
#include "ota/OtaContext.h" // persist/sync OTA policy + signer allowlist with NodePrefs
#endif
#ifndef BRIDGE_MAX_BAUD
@@ -31,15 +32,33 @@ static bool isValidName(const char *n) {
}
void CommonCLI::loadPrefs(FILESYSTEM* fs) {
bool loaded = false;
if (fs->exists("/com_prefs")) {
loadPrefsInt(fs, "/com_prefs"); // new filename
loadPrefsInt(fs, "/com_prefs"); loaded = true; // new filename
} else if (fs->exists("/node_prefs")) {
loadPrefsInt(fs, "/node_prefs");
savePrefs(fs); // save to new filename
fs->remove("/node_prefs"); // remove old
loaded = true;
}
#if defined(ENABLE_OTA)
if (loaded) syncOtaConfigFromPrefs(); // persisted OTA policy/keys -> OtaContext (else keep safe defaults)
#endif
}
#if defined(ENABLE_OTA)
// Push the persisted OTA policy + signer allowlist into the running OtaContext (called after load).
void CommonCLI::syncOtaConfigFromPrefs() {
mesh::ota::OtaContext& c = mesh::ota::ota_ctx();
c.manager.set_autofetch(_prefs->ota_autofetch);
c.manager.set_checkpoint_blocks(_prefs->ota_checkpoint_blocks);
c.autoinstall = _prefs->ota_autoinstall;
c.allow.clear();
for (uint8_t i = 0; i < _prefs->ota_signer_count && i < MAX_OTA_SIGNERS; i++)
c.allow.add(_prefs->ota_signers[i]);
}
#endif
void CommonCLI::loadPrefsInt(FILESYSTEM* fs, const char* filename) {
#if defined(RP2040_PLATFORM)
File file = fs->open(filename, "r");
@@ -94,7 +113,16 @@ void CommonCLI::loadPrefsInt(FILESYSTEM* fs, const char* filename) {
file.read((uint8_t *)&_prefs->rx_boosted_gain, sizeof(_prefs->rx_boosted_gain)); // 290
file.read((uint8_t *)&_prefs->flood_max_unscoped, sizeof(_prefs->flood_max_unscoped)); // 291
file.read((uint8_t *)&_prefs->flood_max_advert, sizeof(_prefs->flood_max_advert)); // 292
// next: 293
// OTA config (293+). Default first so older prefs files (which lack these) keep conservative
// defaults: a short file makes the reads below no-ops (read returns 0 bytes, values unchanged).
_prefs->ota_autofetch = 0; _prefs->ota_autoinstall = 0; _prefs->ota_signer_count = 0;
_prefs->ota_checkpoint_blocks = 32; // default = OTA_CHECKPOINT_BLOCKS (older prefs lack it -> stays 32)
file.read((uint8_t *)&_prefs->ota_autofetch, sizeof(_prefs->ota_autofetch)); // 293
file.read((uint8_t *)&_prefs->ota_autoinstall, sizeof(_prefs->ota_autoinstall)); // 294
file.read((uint8_t *)&_prefs->ota_signer_count, sizeof(_prefs->ota_signer_count)); // 295
file.read((uint8_t *)_prefs->ota_signers, sizeof(_prefs->ota_signers)); // 296
file.read((uint8_t *)&_prefs->ota_checkpoint_blocks, sizeof(_prefs->ota_checkpoint_blocks)); // 424
// next: 426
// sanitise bad pref values
_prefs->rx_delay_base = constrain(_prefs->rx_delay_base, 0, 20.0f);
@@ -124,6 +152,10 @@ void CommonCLI::loadPrefsInt(FILESYSTEM* fs, const char* filename) {
// sanitise settings
_prefs->rx_boosted_gain = constrain(_prefs->rx_boosted_gain, 0, 1); // boolean
_prefs->ota_autofetch = constrain(_prefs->ota_autofetch, 0, 2);
_prefs->ota_autoinstall = constrain(_prefs->ota_autoinstall, 0, 1);
if (_prefs->ota_checkpoint_blocks > 4096) _prefs->ota_checkpoint_blocks = 32; // 0=never; cap absurd
if (_prefs->ota_signer_count > 4) _prefs->ota_signer_count = 0; // corrupt count -> drop keys
file.close();
}
@@ -187,7 +219,12 @@ void CommonCLI::savePrefs(FILESYSTEM* fs) {
file.write((uint8_t *)&_prefs->rx_boosted_gain, sizeof(_prefs->rx_boosted_gain)); // 290
file.write((uint8_t *)&_prefs->flood_max_unscoped, sizeof(_prefs->flood_max_unscoped)); // 291
file.write((uint8_t *)&_prefs->flood_max_advert, sizeof(_prefs->flood_max_advert)); // 292
// next: 293
file.write((uint8_t *)&_prefs->ota_autofetch, sizeof(_prefs->ota_autofetch)); // 293
file.write((uint8_t *)&_prefs->ota_autoinstall, sizeof(_prefs->ota_autoinstall)); // 294
file.write((uint8_t *)&_prefs->ota_signer_count, sizeof(_prefs->ota_signer_count)); // 295
file.write((uint8_t *)_prefs->ota_signers, sizeof(_prefs->ota_signers)); // 296
file.write((uint8_t *)&_prefs->ota_checkpoint_blocks, sizeof(_prefs->ota_checkpoint_blocks)); // 424
// next: 424
file.close();
}
@@ -312,6 +349,17 @@ void CommonCLI::handleCommand(uint32_t sender_timestamp, char* command, char* re
#if defined(ENABLE_OTA)
} else if (memcmp(command, "ota", 3) == 0 && (command[3] == 0 || command[3] == ' ')) {
mesh::ota::handle_ota_command(command, reply, *_board);
if (mesh::ota::ota_ctx().config_dirty) { // a policy/key changed via the CLI -> persist it
mesh::ota::OtaContext& c = mesh::ota::ota_ctx();
_prefs->ota_autofetch = c.manager.autofetch();
_prefs->ota_checkpoint_blocks = c.manager.checkpoint_blocks();
_prefs->ota_autoinstall = c.autoinstall;
_prefs->ota_signer_count = c.allow.count();
for (uint8_t i = 0; i < c.allow.count() && i < MAX_OTA_SIGNERS; i++)
memcpy(_prefs->ota_signers[i], c.allow.get(i), 32);
_callbacks->savePrefs();
c.config_dirty = false;
}
#endif
} else if (memcmp(command, "sensor get ", 11) == 0) {
const char* key = command + 11;
src/helpers/CommonCLI.h
+9
View File
@@ -63,6 +63,12 @@ struct NodePrefs { // persisted to file
uint8_t rx_boosted_gain; // power settings
uint8_t path_hash_mode; // which path mode to use when sending
uint8_t loop_detect;
// OTA config (persisted; synced to OtaContext on load, written on change). 0 = conservative defaults.
uint8_t ota_autofetch; // OtaManager AUTOFETCH_* (0=off, 1=any-compatible, 2=signed-only)
uint8_t ota_autoinstall; // OtaContext AUTOINSTALL_* (0=off, 1=trusted-only)
uint8_t ota_signer_count; // # of allowlisted signer pubkeys below
uint8_t ota_signers[4][32]; // trusted Ed25519 signer pubkeys (== MAX_OTA_SIGNERS)
uint16_t ota_checkpoint_blocks; // resume checkpoint cadence (blocks); 0=never. Default 32 (runtime-tunable)
};
class CommonCLICallbacks {
@@ -127,6 +133,9 @@ class CommonCLI {
mesh::RTCClock* getRTCClock() { return _rtc; }
void savePrefs();
void loadPrefsInt(FILESYSTEM* _fs, const char* filename);
#if defined(ENABLE_OTA)
void syncOtaConfigFromPrefs(); // persisted OTA policy + signer allowlist -> running OtaContext
#endif
void handleRegionCmd(char* command, char* reply);
void handleGetCmd(uint32_t sender_timestamp, char* command, char* reply);
src/helpers/ota/MotaContainer.cpp
+6 -4
View File
@@ -28,7 +28,7 @@ bool mota_parse(const uint8_t* buf, uint32_t len, MotaManifest& out) {
// helper bounds check
#define NEED(n) do { if ((uint32_t)(end - p) < (uint32_t)(n)) return false; } while (0)
NEED(3 + 16 + 1 + 4 + 32 + 1);
NEED(3 + 16 + 1 + 4 + 32 + 1 + 32); // fixed head incl. hw_id[32]
out.format_ver = p[0];
if (out.format_ver != MOTA_FORMAT_VER) return false;
out.flags = p[1];
@@ -41,7 +41,8 @@ bool mota_parse(const uint8_t* buf, uint32_t len, MotaManifest& out) {
out.merkle_root = p + 20;
out.image_hash = p + 24;
out.codec_id = p[56];
p += 57;
out.hw_id = p + 57; // 32-byte NUL-padded hardware tag (signed)
p += 89;
if (out.block_size_log2 == 0 || out.block_size_log2 > 24) return false;
uint32_t bs = out.block_size();
@@ -77,7 +78,7 @@ bool mota_parse_manifest(const uint8_t* mf, uint32_t len, MotaManifest& out) {
const uint8_t* end = mf + len;
#define NEEDM(n) do { if ((uint32_t)(end - p) < (uint32_t)(n)) return false; } while (0)
NEEDM(57);
NEEDM(89); // fixed head incl. hw_id[32]
out.manifest_start = mf;
out.format_ver = p[0];
if (out.format_ver != MOTA_FORMAT_VER) return false;
@@ -91,7 +92,8 @@ bool mota_parse_manifest(const uint8_t* mf, uint32_t len, MotaManifest& out) {
out.merkle_root = p + 20;
out.image_hash = p + 24;
out.codec_id = p[56];
p += 57;
out.hw_id = p + 57; // 32-byte NUL-padded hardware tag (signed)
p += 89;
if (!out.is_full()) { NEEDM(8); out.base_hash = p; p += 8; }
if (out.is_signed()) {
NEEDM(32); out.signer_pubkey = p; p += 32;
src/helpers/ota/MotaContainer.h
+1
View File
@@ -26,6 +26,7 @@ struct MotaManifest {
const uint8_t* merkle_root = nullptr; // 4
const uint8_t* image_hash = nullptr; // 32
const uint8_t* hw_id = nullptr; // 32 (NUL-padded ASCII hardware tag; signed; v2+)
const uint8_t* base_hash = nullptr; // 8 (delta only)
const uint8_t* signer_pubkey = nullptr; // 32 (signed only)
const uint8_t* signature = nullptr; // 64 (signed only)
src/helpers/ota/MotaSeederProto.h
+44
View File
@@ -0,0 +1,44 @@
#pragma once
#include <stdint.h>
// Wire contract for the "mota-seeder" link: a device (CLIENT) pulls `.mota` bytes on demand from a host
// daemon (SERVER) that owns a folder of `.mota` files. This is the FIRST concrete MotaSource transport
// (docs/ota_protocol.md §9) — the device speaks it over a dedicated Stream (a spare UART / USB-UART), so
// it never contends with the line-based text CLI on the main console.
//
// The device always initiates; every request gets exactly one response. Framing is resync-safe: the
// reader scans for the 2-byte magic, so line noise / a half-read frame just times out and is retried
// (OTA is lowest priority — eventually-upgradable). All multi-byte fields are little-endian.
//
// request (device -> host): 'M' 'S' op(1) args... xsum(1 = XOR of op+args)
// response (host -> device): 'm' 's' op(1) status(1) payload... xsum(1 = XOR of all prior)
//
// OP_COUNT 0x01 args: - resp payload: count(1)
// OP_DESCRIBE 0x02 args: idx(1) resp payload: MotaDesc wire (38 B, see below) [status OK]
// OP_READ 0x03 args: idx(1) off(4) len(2) resp payload: len bytes [status OK]
//
// MotaDesc wire (38 B): mid[4] target_id(4) fw_version(4) codec(1) flags(1) total_size(4)
// leaves_off(4) block_count(4) payload_off(4) payload_size(4)
//
// status: 0 = OK, non-zero = error (idx out of range, read past EOF, ...). On error the response carries
// no payload (just magic+op+status+xsum).
namespace mesh {
namespace ota {
static const uint8_t MOTA_SEEDER_REQ_MAGIC0 = 'M';
static const uint8_t MOTA_SEEDER_REQ_MAGIC1 = 'S';
static const uint8_t MOTA_SEEDER_RSP_MAGIC0 = 'm';
static const uint8_t MOTA_SEEDER_RSP_MAGIC1 = 's';
static const uint8_t MS_OP_COUNT = 0x01;
static const uint8_t MS_OP_DESCRIBE = 0x02;
static const uint8_t MS_OP_READ = 0x03;
static const uint8_t MS_STATUS_OK = 0x00;
static const uint16_t MOTA_DESC_WIRE = 38; // bytes of a MotaDesc on the wire (see layout above)
} // namespace ota
} // namespace mesh
src/helpers/ota/MotaSourceSerial.cpp
+100
View File
@@ -0,0 +1,100 @@
#include "MotaSourceSerial.h"
#include "MotaSeederProto.h"
#include <string.h>
namespace mesh {
namespace ota {
static uint32_t rd_u32le(const uint8_t* p) {
return (uint32_t)p[0] | ((uint32_t)p[1] << 8) | ((uint32_t)p[2] << 16) | ((uint32_t)p[3] << 24);
}
static void wr_u32le(uint8_t* p, uint32_t v) { p[0]=v; p[1]=v>>8; p[2]=v>>16; p[3]=v>>24; }
bool SerialMotaSource::readByteT(uint8_t& b) {
uint32_t t0 = millis();
while ((millis() - t0) < _to) {
int c = _io.read();
if (c >= 0) { b = (uint8_t)c; return true; }
}
return false;
}
bool SerialMotaSource::readExact(uint8_t* b, uint32_t n) {
for (uint32_t i = 0; i < n; i++) if (!readByteT(b[i])) return false;
return true;
}
// One request/response transaction. Resync-safe: drains stale input, frames the request with an XOR
// checksum, then scans for the response magic and validates op+status+checksum before delivering payload.
bool SerialMotaSource::txn(uint8_t op, const uint8_t* args, uint8_t arglen,
uint8_t* payload, uint32_t payload_len) {
while (_io.read() >= 0) {} // drop any stale/partial bytes before a fresh request
uint8_t xs = op;
for (uint8_t i = 0; i < arglen; i++) xs ^= args[i];
_io.write(MOTA_SEEDER_REQ_MAGIC0); _io.write(MOTA_SEEDER_REQ_MAGIC1);
_io.write(op);
if (arglen) _io.write(args, arglen);
_io.write(xs);
_io.flush();
// scan for response magic 'm' 's' (tolerate leading noise)
uint32_t t0 = millis(); bool got = false;
uint8_t prev = 0;
while ((millis() - t0) < _to) {
int c = _io.read();
if (c < 0) continue;
if (prev == MOTA_SEEDER_RSP_MAGIC0 && (uint8_t)c == MOTA_SEEDER_RSP_MAGIC1) { got = true; break; }
prev = (uint8_t)c;
}
if (!got) return false;
uint8_t hdr[2];
if (!readExact(hdr, 2)) return false; // op, status
if (hdr[0] != op) return false;
uint8_t rxs = (uint8_t)(MOTA_SEEDER_RSP_MAGIC0 ^ MOTA_SEEDER_RSP_MAGIC1) ^ hdr[0] ^ hdr[1];
bool ok = (hdr[1] == MS_STATUS_OK);
if (ok && payload_len) {
if (!readExact(payload, payload_len)) return false;
for (uint32_t i = 0; i < payload_len; i++) rxs ^= payload[i];
}
uint8_t xsum;
if (!readByteT(xsum)) return false;
if (xsum != rxs) return false; // corrupt frame -> caller retries
return ok;
}
uint8_t SerialMotaSource::count() {
uint8_t n = 0;
if (!txn(MS_OP_COUNT, nullptr, 0, &n, 1)) return 0;
return n;
}
bool SerialMotaSource::describe(uint8_t idx, MotaDesc& out) {
uint8_t args[1] = { idx };
uint8_t w[MOTA_DESC_WIRE];
if (!txn(MS_OP_DESCRIBE, args, 1, w, MOTA_DESC_WIRE)) return false;
memcpy(out.mid, w, 4);
out.target_id = rd_u32le(w + 4);
out.fw_version = rd_u32le(w + 8);
out.codec_id = w[12];
out.flags = w[13];
out.total_size = rd_u32le(w + 14);
out.leaves_off = rd_u32le(w + 18);
out.block_count = rd_u32le(w + 22);
out.payload_off = rd_u32le(w + 26);
out.payload_size = rd_u32le(w + 30);
// bytes [34,38) reserved (zero) — kept for forward compat without changing MOTA_DESC_WIRE
return true;
}
bool SerialMotaSource::read(uint8_t idx, uint32_t off, uint8_t* buf, uint32_t len) {
if (len > 0xFFFF) return false; // single transaction caps at 64 KB (a block is <=1 KB)
uint8_t args[7];
args[0] = idx;
wr_u32le(args + 1, off);
args[5] = (uint8_t)(len & 0xFF); args[6] = (uint8_t)(len >> 8);
return txn(MS_OP_READ, args, 7, buf, len);
}
} // namespace ota
} // namespace mesh
src/helpers/ota/MotaSourceSerial.h
+35
View File
@@ -0,0 +1,35 @@
#pragma once
#include <Arduino.h>
#include "OtaSource.h"
// A MotaSource backed by a host "mota-seeder" daemon over a dedicated Stream (a spare UART / USB-UART).
// The device pulls catalog + bytes on demand (MotaSeederProto.h); the folder image is never held on the
// device — it streams through. Use a stream that is NOT the text-CLI console so the binary framing never
// collides with command/log text. Reads block on the Stream up to `timeout_ms` (OTA is lowest priority,
// so a serial round-trip's latency is acceptable; keep the daemon on a fast link).
namespace mesh {
namespace ota {
class SerialMotaSource : public MotaSource {
public:
explicit SerialMotaSource(Stream& io, uint32_t timeout_ms = 400) : _io(io), _to(timeout_ms) {}
uint8_t count() override;
bool describe(uint8_t idx, MotaDesc& out) override;
bool read(uint8_t idx, uint32_t off, uint8_t* buf, uint32_t len) override;
private:
// Send a request (op+args) and read its response header; on OK, `payload` (if non-null) receives
// `payload_len` bytes. Returns true iff a well-formed OK response for `op` arrived in time.
bool txn(uint8_t op, const uint8_t* args, uint8_t arglen, uint8_t* payload, uint32_t payload_len);
bool readByteT(uint8_t& b); // one byte within the timeout
bool readExact(uint8_t* b, uint32_t n);
Stream& _io;
uint32_t _to;
};
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaApply.cpp
+233 -10
View File
@@ -12,6 +12,10 @@
extern "C" {
#include "detools/detools.h" // vendored detools 0.53.0 embeddable decoder (CRLE-only build)
}
#if defined(OTA_FLASH_STORE)
#include "OtaStoreFlashEsp32.h" // flash-staged container (delta patch / full image in the slot)
#include "OtaSelf.h" // SelfFwInfo / ota_self_firmware (running-image base_hash gate)
#endif
#elif defined(NRF52_PLATFORM)
#include "OtaVerify.h"
#include "OtaSelf.h"
@@ -87,7 +91,12 @@ bool ota_apply_commit() {
struct DetoolsCtx {
const esp_partition_t* base; // delta base (running image), read at absolute `from_pos`
long from_pos; // absolute byte offset into `base`
const uint8_t* patch; // .mota payload (whole patch held in RAM)
#if defined(OTA_FLASH_STORE)
OtaStoreFlashEsp32* store; // staged container; patch = payload region [patch_base, +patch_len)
uint32_t patch_base; // container offset where the payload (patch) begins
#else
const uint8_t* patch; // .mota payload held wholly in RAM (legacy RAM store)
#endif
uint32_t patch_len;
uint32_t patch_pos;
esp_ota_handle_t out; // inactive slot write handle
@@ -112,7 +121,11 @@ static int dt_from_seek(void* arg, int offset) { // detools uses relati
static int dt_patch_read(void* arg, uint8_t* buf, size_t size) {
DetoolsCtx* c = (DetoolsCtx*)arg;
if (c->patch_pos + size > c->patch_len) return -DETOOLS_IO_FAILED;
#if defined(OTA_FLASH_STORE)
if (!c->store->read(c->patch_base + c->patch_pos, buf, size)) { c->io_ok = false; return -DETOOLS_IO_FAILED; }
#else
memcpy(buf, c->patch + c->patch_pos, size);
#endif
c->patch_pos += (uint32_t)size;
return DETOOLS_OK;
}
@@ -124,6 +137,223 @@ static int dt_to_write(void* arg, const uint8_t* buf, size_t size) {
return DETOOLS_OK;
}
#if defined(OTA_FLASH_STORE)
// --- in-place delta on ESP32 (codec 2) ----------------------------------------------------------
// A single in-place `.mota` can target BOTH nRF52 (bootloader applies it) and ESP32. On ESP32 the
// inactive slot is used as the in-place working memory: we copy the running image (the base) into the
// slot's bottom-staged-container-FREE region [0, write_start), then run detools' in-place decoder over
// that region (it reads the base, erases segments, writes the target back), reading the patch from the
// staged container's payload (which lives at/below write_start, disjoint from the working region). The
// decoded image is hashed against the signed image_hash BEFORE arming, so a bad decode never boots; the
// callbacks are bounded to [0, write_start) so they fail gracefully instead of touching the patch.
// (Sequential is still preferred on ESP32 — it streams straight to the slot with no base-copy; in-place
// exists only for single-artifact distribution. Requires the patch built with --inplace-segment 4096.)
struct InPlaceCtx {
const esp_partition_t* slot; // in-place working memory = slot[0, mem_max)
uint32_t mem_max; // = container write_start; accesses beyond this are refused
OtaStoreFlashEsp32* store; // staged container; patch = payload region [patch_base, +patch_len)
uint32_t patch_base, patch_len, patch_pos;
int step; // detools resume cursor (RAM; no cross-reboot resume of the apply)
bool io_ok;
const char* fail; // first failure point (diagnostic), nullptr until set
uint32_t fa, fn; int frc; // failing addr / len / esp_err
};
static inline int ip_fail(InPlaceCtx* c, const char* w, uint32_t a, size_t n, int rc) {
if (!c->fail) { c->fail = w; c->fa = a; c->fn = (uint32_t)n; c->frc = rc; }
c->io_ok = false; return -DETOOLS_IO_FAILED;
}
static int ip_mem_read(void* a, void* dst, uintptr_t src, size_t n) {
InPlaceCtx* c = (InPlaceCtx*)a;
if ((uint32_t)src + n > c->mem_max) return ip_fail(c, "rd>max", (uint32_t)src, n, 0);
int rc = esp_partition_read(c->slot, (size_t)src, dst, n);
if (rc != ESP_OK) return ip_fail(c, "rd", (uint32_t)src, n, rc);
return DETOOLS_OK;
}
static int ip_mem_write(void* a, uintptr_t dst, void* src, size_t n) {
InPlaceCtx* c = (InPlaceCtx*)a;
if ((uint32_t)dst + n > c->mem_max) return ip_fail(c, "wr>max", (uint32_t)dst, n, 0);
int rc = esp_partition_write(c->slot, (size_t)dst, src, n);
if (rc != ESP_OK) return ip_fail(c, "wr", (uint32_t)dst, n, rc);
return DETOOLS_OK;
}
static int ip_mem_erase(void* a, uintptr_t addr, size_t n) {
InPlaceCtx* c = (InPlaceCtx*)a;
// esp_partition_erase_range requires a SECTOR-aligned size; detools' final in-place segment is partial
// (the image tail past the last full sector). addr is sector-aligned (== --inplace-segment), so round
// the length UP to a full sector. The over-erased bytes are scratch beyond image_size (never hashed),
// and — since detools processes high→low and erases-before-writing — they are never live patch data.
const uint32_t SEC = 4096;
if ((uint32_t)addr % SEC != 0) return ip_fail(c, "er!align", (uint32_t)addr, n, 0);
uint32_t len = ((uint32_t)n + SEC - 1) & ~(SEC - 1);
if ((uint32_t)addr + len > c->mem_max) return ip_fail(c, "er>max", (uint32_t)addr, len, 0);
int rc = esp_partition_erase_range(c->slot, (size_t)addr, len);
if (rc != ESP_OK) return ip_fail(c, "er", (uint32_t)addr, len, rc);
return DETOOLS_OK;
}
static int ip_step_set(void* a, int s) { ((InPlaceCtx*)a)->step = s; return DETOOLS_OK; }
static int ip_step_get(void* a, int* s) { *s = ((InPlaceCtx*)a)->step; return DETOOLS_OK; }
static int ip_patch_read(void* a, uint8_t* b, size_t n) {
InPlaceCtx* c = (InPlaceCtx*)a;
if (c->patch_pos + n > c->patch_len) return ip_fail(c, "patch>len", c->patch_pos, n, 0);
if (!c->store->read(c->patch_base + c->patch_pos, b, n)) return ip_fail(c, "patch_rd", c->patch_pos, n, 0);
c->patch_pos += (uint32_t)n;
return DETOOLS_OK;
}
static bool esp32_inplace_apply(OtaStoreFlashEsp32& store, const MotaManifest& m, ApplyState& st, char* msg) {
const esp_partition_t* slot = store.partition();
const esp_partition_t* base = esp_ota_get_running_partition();
if (!slot || !base) { strcpy(msg, "no slot/base partition"); return false; }
SelfFwInfo fi;
if (!ota_self_firmware(fi) || !fi.valid) { strcpy(msg, "cannot read running firmware (no EndF)"); return false; }
if (!m.base_hash || memcmp(m.base_hash, fi.body_hash, 8) != 0) { strcpy(msg, "not built for the running firmware (base mismatch)"); return false; }
uint32_t mem_max = store.write_start(); // working region [0, mem_max); the patch sits at/above it
if (mem_max == 0) { strcpy(msg, "in-place needs a bottom-staged container"); return false; }
if (fi.image_len > mem_max || m.image_size > mem_max) { strcpy(msg, "in-place region too small for base/image"); return false; }
// load the base (running image) into the working region [0, base_len), sector by sector (erase + copy)
uint8_t buf[512];
for (uint32_t off = 0; off < fi.image_len; ) {
uint32_t sec = off & ~(4096u - 1);
if (esp_partition_erase_range(slot, sec, 4096) != ESP_OK) { strcpy(msg, "base erase failed"); return false; }
uint32_t secend = sec + 4096; if (secend > fi.image_len) secend = fi.image_len;
for (uint32_t p = (off > sec ? off : sec); p < secend; ) {
uint32_t n = secend - p; if (n > sizeof(buf)) n = sizeof(buf);
if (esp_partition_read(base, p, buf, n) != ESP_OK || esp_partition_write(slot, p, buf, n) != ESP_OK) {
strcpy(msg, "base copy failed"); return false; }
p += n;
}
off = secend;
}
// patch in place over the working region; patch streamed from the staged container payload
InPlaceCtx c;
c.slot = slot; c.mem_max = mem_max; c.store = &store;
c.patch_base = store.meta_bytes(); c.patch_len = m.payload_size; c.patch_pos = 0; c.step = 0; c.io_ok = true;
c.fail = nullptr; c.fa = c.fn = 0; c.frc = 0;
int r = detools_apply_patch_in_place_callbacks(ip_mem_read, ip_mem_write, ip_mem_erase,
ip_step_set, ip_step_get, ip_patch_read,
(size_t)m.payload_size, &c);
if (r < 0 || !c.io_ok) {
if (c.fail) sprintf(msg, "in-place decode err %d @%s a=%u n=%u rc=%d max=%u", r, c.fail,
(unsigned)c.fa, (unsigned)c.fn, c.frc, (unsigned)mem_max);
else sprintf(msg, "in-place decode err %d", r);
return false;
}
if ((uint32_t)r != m.image_size) { sprintf(msg, "in-place size %u!=%u", (unsigned)r, (unsigned)m.image_size); return false; }
// verify the decoded slot image against the signed image_hash BEFORE arming (mismatch -> never boots)
SHA256 sha;
for (uint32_t off = 0; off < m.image_size; ) {
uint32_t n = m.image_size - off; if (n > sizeof(buf)) n = sizeof(buf);
if (esp_partition_read(slot, off, buf, n) != ESP_OK) { strcpy(msg, "slot read failed"); return false; }
sha.update(buf, n); off += n;
}
uint8_t hh[32]; sha.finalize(hh, 32);
st.slot_ok = (memcmp(hh, m.image_hash, 32) == 0);
if (!st.slot_ok) { strcpy(msg, "image_hash MISMATCH after in-place decode"); return false; }
if (esp_ota_set_boot_partition(slot) != ESP_OK) { strcpy(msg, "set_boot failed"); return false; }
sprintf(msg, "verified%s; in-place decoded %u B, image hash OK — armed, rebooting to apply",
m.is_signed() ? " (signer trusted)" : " (unsigned)", (unsigned)m.image_size);
return true;
}
// Apply the `.mota` staged in the inactive slot by OtaStoreFlashEsp32 (no contiguous RAM copy).
// FULL: the payload was streamed straight to slot offset 0 during the fetch -> hash the slot image
// and compare to the signed image_hash, then arm. No decode, no copy.
// DELTA (sequential): base = the running slot; the patch is read from the staged payload region (the
// slot's bottom); the reconstructed image is written to the inactive slot via esp_ota_write and
// hashed vs image_hash. esp_ota_begin only erases [0, image_size], which the fetch-time fit
// check kept below the bottom-staged container, so the patch survives while we decode over it.
// DELTA (in-place): copy the running image into the slot's working region then patch in place
// (esp32_inplace_apply); image_hash-gated before arming. Lets one in-place .mota target both
// ESP32 and nRF52. Sequential is still preferred on ESP32 (no base-copy).
// The result is verified (signature/trust up front, image_hash after) and the slot armed; the caller
// reboots once the confirmation reply has gone out.
bool ota_apply_detools_mota(OtaStoreFlashEsp32& store, const SignerAllowlist& allow, ApplyState& st, char* msg) {
st = ApplyState();
const esp_partition_t* slot = store.partition();
if (!slot || store.staged_size() < 16) { strcpy(msg, "no staged update"); return false; }
st.slot_addr = slot->address; st.slot_size = slot->size;
// read + parse the manifest out of the staged container (header = MAGIC(4) + total(4))
uint8_t hdr[8];
if (!store.read(0, hdr, 8) || memcmp(hdr, MOTA_MAGIC, 4) != 0) { strcpy(msg, "bad container"); return false; }
uint8_t mfbuf[256];
uint32_t mflen = store.meta_bytes() > 8 ? store.meta_bytes() - 8 : 0; // manifest+leaves; cap to mfbuf
if (mflen > sizeof(mfbuf)) mflen = sizeof(mfbuf);
MotaManifest m;
if (mflen < 57 || !store.read(8, mfbuf, mflen) || !mota_parse_manifest(mfbuf, mflen, m)) {
strcpy(msg, "manifest parse failed"); return false; }
st.image_size = m.image_size; memcpy(st.image_hash, m.image_hash, 32); st.manifest_ok = true;
if (m.image_size == 0 || m.image_size > slot->size) { strcpy(msg, "image > slot"); return false; }
// signature / trust BEFORE arming an untrusted image (image_hash below is the target-firmware gate)
if (m.is_signed()) {
mesh::Identity signer(m.signer_pubkey);
st.sig_ok = signer.verify(m.signature, m.manifest_start, (int)m.signed_len);
st.trusted = st.sig_ok && allow.contains(m.signer_pubkey);
if (!st.sig_ok) { strcpy(msg, "bad signature"); return false; }
if (!st.trusted) { strcpy(msg, "untrusted signer (pubkey not in allowlist)"); return false; }
}
// ---- FULL: payload already in slot[0]; verify hash + arm ----
if (m.is_full()) {
SHA256 sha; uint8_t buf[512];
for (uint32_t off = 0; off < m.image_size; ) {
uint32_t n = m.image_size - off; if (n > sizeof(buf)) n = sizeof(buf);
if (esp_partition_read(slot, off, buf, n) != ESP_OK) { strcpy(msg, "slot read failed"); return false; }
sha.update(buf, n); off += n;
}
uint8_t hh[32]; sha.finalize(hh, 32);
st.slot_ok = (memcmp(hh, m.image_hash, 32) == 0);
if (!st.slot_ok) { strcpy(msg, "image_hash MISMATCH (slot)"); return false; }
if (esp_ota_set_boot_partition(slot) != ESP_OK) { strcpy(msg, "set_boot failed"); return false; }
sprintf(msg, "verified%s full image %u B in slot — armed, rebooting to apply",
m.is_signed() ? " (trusted)" : " (unsigned)", (unsigned)m.image_size);
return true;
}
// ---- DELTA ----
if (m.codec_id == CODEC_DETOOLS_INPLACE) return esp32_inplace_apply(store, m, st, msg); // single-artifact codec
if (m.codec_id != CODEC_DETOOLS_SEQUENTIAL) { strcpy(msg, "unknown delta codec"); return false; }
// delta must be built for the running firmware (cheap early gate; image_hash is the definitive check)
if (m.base_hash) {
SelfFwInfo fi;
if (!ota_self_firmware(fi) || !fi.valid) { strcpy(msg, "cannot read running firmware (no EndF)"); return false; }
if (memcmp(m.base_hash, fi.body_hash, 8) != 0) { strcpy(msg, "delta not built for the running firmware (base mismatch)"); return false; }
}
const esp_partition_t* base = esp_ota_get_running_partition();
if (!base) { strcpy(msg, "no running partition"); return false; }
esp_ota_handle_t h;
if (esp_ota_begin(slot, m.image_size, &h) != ESP_OK) { strcpy(msg, "ota_begin failed"); return false; }
SHA256 sha;
DetoolsCtx ctx;
ctx.base = base; ctx.from_pos = 0;
ctx.store = &store; ctx.patch_base = store.meta_bytes(); ctx.patch_len = m.payload_size; ctx.patch_pos = 0;
ctx.out = h; ctx.sha = &sha; ctx.out_pos = 0; ctx.io_ok = true;
int r = detools_apply_patch_callbacks(dt_from_read, dt_from_seek, dt_patch_read,
(size_t)m.payload_size, dt_to_write, &ctx);
if (r < 0 || !ctx.io_ok) { esp_ota_abort(h); sprintf(msg, "detools err %d @%u/%u",
ctx.io_ok ? r : -DETOOLS_IO_FAILED, (unsigned)ctx.out_pos, (unsigned)m.image_size); return false; }
if ((uint32_t)r != m.image_size || ctx.out_pos != m.image_size) {
esp_ota_abort(h); sprintf(msg, "size mismatch %u!=%u", (unsigned)ctx.out_pos, (unsigned)m.image_size); return false; }
uint8_t hh[32]; sha.finalize(hh, 32);
st.slot_ok = (memcmp(hh, m.image_hash, 32) == 0);
if (!st.slot_ok) { esp_ota_abort(h); strcpy(msg, "image_hash MISMATCH after decode"); return false; }
if (esp_ota_end(h) != ESP_OK) { strcpy(msg, "ota_end failed"); return false; }
if (esp_ota_set_boot_partition(slot) != ESP_OK) { strcpy(msg, "set_boot failed"); return false; }
sprintf(msg, "verified%s; decoded %u B, image hash OK — armed, rebooting to apply",
m.is_signed() ? " (signer trusted)" : " (unsigned)", (unsigned)m.image_size);
return true;
}
#else // !OTA_FLASH_STORE: legacy RAM-staged apply (whole .mota in a contiguous RAM buffer; bring-up)
bool ota_apply_detools_mota(const uint8_t* buf, uint32_t len, const SignerAllowlist& allow,
ApplyState& st, char* msg) {
st = ApplyState();
@@ -133,9 +363,6 @@ bool ota_apply_detools_mota(const uint8_t* buf, uint32_t len, const SignerAllowl
st.image_size = m.image_size;
memcpy(st.image_hash, m.image_hash, 32);
st.manifest_ok = true;
// signature (if signed): valid Ed25519 AND signer in this device's allowlist — refuse otherwise,
// BEFORE decoding an untrusted image into the slot. (The decoded result is also checked against the
// manifest image_hash below, which is the target-firmware-hash gate.)
if (m.is_signed()) {
mesh::Identity signer(m.signer_pubkey);
st.sig_ok = signer.verify(m.signature, m.manifest_start, (int)m.signed_len);
@@ -143,29 +370,24 @@ bool ota_apply_detools_mota(const uint8_t* buf, uint32_t len, const SignerAllowl
if (!st.sig_ok) { strcpy(msg, "bad signature"); return false; }
if (!st.trusted) { strcpy(msg, "untrusted signer (pubkey not in allowlist)"); return false; }
}
const esp_partition_t* base = esp_ota_get_running_partition(); // delta base = what's running
const esp_partition_t* base = esp_ota_get_running_partition();
const esp_partition_t* out = esp_ota_get_next_update_partition(nullptr);
if (!base || !out) { strcpy(msg, "no A/B slot"); return false; }
st.slot_addr = out->address; st.slot_size = out->size;
if (m.image_size > out->size) { strcpy(msg, "image > slot"); return false; }
esp_ota_handle_t h;
if (esp_ota_begin(out, m.image_size, &h) != ESP_OK) { strcpy(msg, "ota_begin failed"); return false; }
SHA256 sha;
DetoolsCtx ctx;
ctx.base = base; ctx.from_pos = 0;
ctx.patch = m.payload; ctx.patch_len = m.payload_size; ctx.patch_pos = 0;
ctx.out = h; ctx.sha = &sha; ctx.out_pos = 0; ctx.io_ok = true;
int r = detools_apply_patch_callbacks(dt_from_read, dt_from_seek, dt_patch_read,
(size_t)m.payload_size, dt_to_write, &ctx);
if (r < 0 || !ctx.io_ok) { esp_ota_abort(h); sprintf(msg, "detools err %d @%u/%u",
ctx.io_ok ? r : -DETOOLS_IO_FAILED, (unsigned)ctx.out_pos, (unsigned)m.image_size); return false; }
if ((uint32_t)r != m.image_size || ctx.out_pos != m.image_size) {
esp_ota_abort(h); sprintf(msg, "size mismatch %u!=%u", (unsigned)ctx.out_pos, (unsigned)m.image_size); return false; }
uint8_t hh[32]; sha.finalize(hh, 32);
st.slot_ok = (memcmp(hh, m.image_hash, 32) == 0);
if (!st.slot_ok) { esp_ota_abort(h); strcpy(msg, "image_hash MISMATCH after decode"); return false; }
@@ -175,6 +397,7 @@ bool ota_apply_detools_mota(const uint8_t* buf, uint32_t len, const SignerAllowl
m.is_signed() ? " (signer trusted)" : " (unsigned)", (unsigned)m.image_size);
return true;
}
#endif // OTA_FLASH_STORE
bool ota_apply_mota_nrf52(const uint8_t*, uint32_t, const SignerAllowlist&, ApplyState& st, char* msg) {
st = ApplyState(); strcpy(msg, "nRF52-only (ESP32 uses ota_apply_detools_mota)"); return false;
src/helpers/ota/OtaApply.h
+11 -6
View File
@@ -29,14 +29,19 @@ bool ota_apply_set_manifest(const uint8_t* mf, uint32_t len,
bool ota_apply_verify_slot(ApplyState& st); // hash the slot vs image_hash
bool ota_apply_commit(); // set-boot + reboot (no return)
// Apply a detools-sequential delta `.mota` (whole container in `buf`) using detools' own embeddable
// C decoder (CODEC_DETOOLS_SEQUENTIAL, --compression crle). The running slot is the delta base; the
// decoder streams the patch (held in RAM) and writes the reconstructed image into the inactive slot,
// while we hash the output and check it against the signed manifest image_hash. On success the
// inactive slot is set as boot partition; the caller then reboots. `msg` (>=80 bytes) receives a
// human-readable result. Returns true if the slot is verified + armed.
// Apply an ESP32 A/B `.mota` (CODEC_DETOOLS_SEQUENTIAL delta or CODEC_FULL image) and arm the inactive
// slot; the caller reboots after the confirmation reply. `msg` (>=80 bytes) receives a human-readable
// result. With OTA_FLASH_STORE the container is staged in the inactive slot (no contiguous RAM copy): a
// full payload is already in the slot (just verified), a sequential delta is decoded from the running
// slot over the staged patch. Without OTA_FLASH_STORE (bring-up) the whole container is a RAM buffer.
#if defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
class OtaStoreFlashEsp32;
bool ota_apply_detools_mota(OtaStoreFlashEsp32& store,
const SignerAllowlist& allow, ApplyState& st, char* msg);
#else
bool ota_apply_detools_mota(const uint8_t* buf, uint32_t len,
const SignerAllowlist& allow, ApplyState& st, char* msg);
#endif
// nRF52 (RAK4631) single-slot apply. The running app can't rewrite itself, so it does NOT decode: it
// runs the gated verification chain (payload hash -> built-for-this-firmware -> signature/trust) and,
src/helpers/ota/OtaCli.cpp
+219 -78
View File
@@ -5,6 +5,8 @@
#include "Utils.h"
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <Arduino.h> // millis() for session-age display (device-only command surface)
namespace mesh {
namespace ota {
@@ -26,46 +28,216 @@ static char fstate_char(OtaManager::FetchState s) {
}
}
static const char* codec_name(uint8_t c) {
return c == CODEC_FULL ? "full" : (c == CODEC_DETOOLS_SEQUENTIAL ? "seq"
: (c == CODEC_DETOOLS_INPLACE ? "inpl" : "?"));
}
// The everyday OTA surface is BitTorrent-shaped: `ota` shows what you're holding (your running firmware
// as a full mOTA + your one fetch session), `ota neighbors` shows the mOTAs heard around you, `ota pull`
// starts fetching one, `ota drop` frees the session. The raw primitives (manual content load, low-level
// apply steps) live under `ota dev ...` so they don't clutter the everyday surface. Every reply fits one
// packet so it works as remote-admin over LoRa.
static bool handle_dev(const char* d, char* reply, OtaContext& c);
bool handle_ota_command(const char* command, char* reply, mesh::MainBoard& board) {
const char* a = command + 3;
if (*a != 0 && *a != ' ') return false;
while (*a == ' ') a++;
OtaContext& c = ota_ctx();
// ---- raw / internal primitives, tucked under `ota dev ...` ----
if (strncmp(a, "dev", 3) == 0 && (a[3] == 0 || a[3] == ' ')) {
const char* d = a + 3; while (*d == ' ') d++;
return handle_dev(d, reply, c);
}
// ---- inventory dashboard: running fw (self), the one fetch session, serving state ----
if (*a == 0 || strncmp(a, "status", 6) == 0) {
SelfFwInfo fi; bool s = ota_self_firmware(fi);
sprintf(reply, "OTA tid=%08X self=%u serve=%u%s fetch=%c %u/%u keys=%u",
(unsigned)board.getOtaTargetId(), (unsigned)(s ? fi.body_len : 0),
(unsigned)c.serve_expected, c.serving ? "(on)" : "",
fstate_char(c.manager.fetchState()),
char selfhx[9]; if (s && fi.valid) mesh::Utils::toHex(selfhx, fi.body_hash, 4); else strcpy(selfhx, "?");
OtaManager::FetchState fs = c.manager.fetchState();
char midhx[9]; strcpy(midhx, "-");
if (fs != OtaManager::IDLE) mesh::Utils::toHex(midhx, c.manager.fetchManifestId(), 4);
unsigned age = (fs != OtaManager::IDLE && c.session_started_ms) ? (unsigned)((millis() - c.session_started_ms) / 1000) : 0;
sprintf(reply, "OTA tgt=%08X fw=%s | self:%s%uK | sess:%c %u/%u mid=%s age=%us | serv:%s keys=%u",
(unsigned)board.getOtaTargetId(), selfhx, s ? "full " : "?",
(unsigned)((s ? fi.image_len : 0) / 1024), fstate_char(fs),
(unsigned)c.manager.blocksHave(), (unsigned)c.manager.blocksTotal(),
(unsigned)c.allow.count());
midhx, age, c.serving ? "on" : "off", (unsigned)c.allow.count());
// ---- what's available around me (catalogued from beacons + OTA_HAVE), best/most-recent first ----
} else if (strncmp(a, "neighbors", 9) == 0 || strncmp(a, "nbrs", 4) == 0) {
// Kick a fresh round of catalog queries (async — rows arrive over the next seconds); render what we
// have now. The reply buffer is 160 B (serial / one LoRa packet for remote-admin) so writes are bounded.
c.manager.queryAll();
const int CAP = 160;
int n = snprintf(reply, CAP, "nbrs #:mid t=tgt codec seed age *=cur (src=%u)", (unsigned)c.manager.sourceCount());
const uint8_t* cur = (c.manager.fetchState() != OtaManager::IDLE) ? c.manager.fetchManifestId() : nullptr;
uint32_t now = millis(); int shown = 0, more = 0;
for (uint8_t i = 0; i < c.manager.catalogCount(); i++) {
const OtaManager::CatRow* h = c.manager.catalogRow(i);
if (CAP - n < 60) { more++; continue; }
char midhx[9]; mesh::Utils::toHex(midhx, h->mid, 4);
bool on = cur && memcmp(cur, h->mid, 4) == 0;
uint32_t age = (now - h->last_ms) / 1000; if (age > 99999) age = 99999;
n += snprintf(reply + n, CAP - n, "\n %d:%s t=%08X %s seed=%u %us%s", shown + 1, midhx,
(unsigned)h->target_id, codec_name(h->codec), (unsigned)h->n_seeders,
(unsigned)age, on ? "*" : "");
shown++;
}
if (more && n < CAP) snprintf(reply + n, CAP - n, "\n +%d more", more);
if (shown == 0) strcpy(reply, "nbrs: none yet — sources beacon periodically; re-run in a few s (queried now)");
// ---- start fetching a specific catalogued mOTA (by list index or manifest_id) ----
} else if (strncmp(a, "pull", 4) == 0 && (a[4] == 0 || a[4] == ' ')) {
const char* p = a + 4; while (*p == ' ') p++;
if (*p == 0) { strcpy(reply, "usage: ota pull <#|mid8> (see `ota neighbors`)"); return true; }
const OtaManager::CatRow* sel = nullptr; uint8_t mid[4];
if (*p == '#' || (p[0] >= '1' && p[0] <= '9' && (p[1] == 0 || p[1] == ' '))) { // index among catalogue
int idx = atoi(*p == '#' ? p + 1 : p);
if (idx >= 1 && idx <= c.manager.catalogCount()) sel = c.manager.catalogRow((uint8_t)(idx - 1));
} else if (mesh::Utils::fromHex(mid, 4, p)) { // explicit manifest_id
for (uint8_t i = 0; i < c.manager.catalogCount(); i++)
if (memcmp(c.manager.catalogRow(i)->mid, mid, 4) == 0) { sel = c.manager.catalogRow(i); break; }
}
if (!sel) { strcpy(reply, "ERR no such neighbor (see `ota neighbors`)"); return true; }
if (c.apply_pending) { strcpy(reply, "ERR busy applying"); return true; }
uint8_t selmid[4]; uint32_t seltgt = sel->target_id; memcpy(selmid, sel->mid, 4); // sel may move on reset
c.manager.reset_session(); c.fetch_store.clear();
c.manager.pull(selmid, seltgt); // sets want + begins the manifest fetch now
char midhx[9]; mesh::Utils::toHex(midhx, selmid, 4);
sprintf(reply, "OK pulling mid=%s target=%08X (low priority)", midhx, (unsigned)seltgt);
// ---- discard the current session (e.g. a stalled old fetch) to free the slot ----
} else if (strncmp(a, "drop", 4) == 0) {
OtaManager::FetchState fs = c.manager.fetchState();
char midhx[9]; strcpy(midhx, "-");
if (fs != OtaManager::IDLE) mesh::Utils::toHex(midhx, c.manager.fetchManifestId(), 4);
c.manager.reset_session(); c.manager.want(0); c.manager.want_mid(nullptr);
c.fetch_store.clear(); c.serving = false; c.serve_expected = 0; c.session_started_ms = 0;
sprintf(reply, "OK dropped session (was %c mid=%s); slot free for a new pull", fstate_char(fs), midhx);
// ---- broadcast our tiny beacon so peers discover us. If not already serving, set up flash-backed
// self-serve first (so we're a real, fetchable source of our own running firmware). ----
} else if (strncmp(a, "announce", 8) == 0) {
if (!c.serving) c.serving = ota_serve_self(c, 0);
c.manager.announce();
sprintf(reply, "OK beacon sent (serving=%s)", c.serving ? "self fw" : "nothing");
// ---- running firmware identity (compare against a delta's base_hash) ----
} else if (strncmp(a, "self", 4) == 0) {
SelfFwInfo fi;
if (!ota_self_firmware(fi) || !fi.valid) { strcpy(reply, "ERR no EndF (firmware lacks the trailer?)"); return true; }
char hx[17]; mesh::Utils::toHex(hx, fi.body_hash, 8);
sprintf(reply, "self body=%u image=%u base_hash=%s", (unsigned)fi.body_len, (unsigned)fi.image_len, hx);
} else if (strncmp(a, "applydelta", 10) == 0) {
// Apply the fetched update. Destructive (reflashes + reboots) and GATED, not interactive (no "type
// yes" round-trip — unreliable over LoRa): refuse unless the fetch is COMPLETE, then the apply path
// validates in order (payload hash -> built-for-this-firmware -> signature/trust) and returns the
// FIRST failing gate, so the operator knows exactly why it refused; it proceeds only if all pass.
if (c.manager.fetchState() != OtaManager::COMPLETE || c.fetch_store.staged_size() == 0) {
sprintf(reply, "ERR no complete update fetched (fetch=%c %u/%u)",
fstate_char(c.manager.fetchState()), (unsigned)c.manager.blocksHave(),
(unsigned)c.manager.blocksTotal());
return true;
}
// On success the slot is armed but NOT yet rebooted — defer so this reply reaches the operator first;
// the mesh loop reboots once it has been transmitted (same path used by auto-install).
char m2[100];
bool ok = c.apply_fetched(m2);
sprintf(reply, "%s | %s", ok ? "OK" : "ERR", m2);
// ---- external folder relay: advertise + serve `.mota` from a host daemon over the seeder UART, so the
// node hosts MANY images (any architecture) it doesn't hold in flash. Trustless (fetchers verify). --
} else if (strncmp(a, "folder", 6) == 0) {
const char* p = a + 6; while (*p == ' ') p++;
if (strncmp(p, "on", 2) == 0) {
#if defined(OTA_FOLDER_SERIAL)
if (!c.serving) c.serving = ota_serve_self(c, 0); // keep serving our own fw alongside the folder
char m2[120]; c.attach_folder(m2, sizeof(m2)); c.manager.announce();
strncpy(reply, m2, 159); reply[159] = 0;
#else
strcpy(reply, "ERR not built with OTA_FOLDER_SERIAL (set the seeder UART in platformio.ini)");
#endif
} else if (strncmp(p, "off", 3) == 0) {
c.detach_folder(); c.manager.announce();
strcpy(reply, "OK folder detached (still serving own fw)");
} else { // status + list served entries (* = our own fw)
int n = snprintf(reply, 159, "folder=%s serving=%u:", c.folder_active ? "on" : "off",
(unsigned)c.manager.servedCount());
for (uint8_t i = 0; i < c.manager.servedCount() && n < 148; i++) {
const OtaManager::ServeEntry* e = c.manager.servedEntry(i);
if (!e) break;
char midhx[9]; mesh::Utils::toHex(midhx, e->mid, 4);
n += snprintf(reply + n, 159 - n, " %s%s/%08X", e->is_self ? "*" : "", midhx, (unsigned)e->target_id);
}
}
// ---- policy config (persisted via NodePrefs). conservative defaults: autofetch/autoinstall off ----
} else if (strncmp(a, "config", 6) == 0) {
const char* p = a + 6; while (*p == ' ') p++;
if (strncmp(p, "autofetch ", 10) == 0) {
const char* v = p + 10;
uint8_t pol = strncmp(v, "any", 3) == 0 ? OtaManager::AUTOFETCH_ANY
: strncmp(v, "signed", 6) == 0 ? OtaManager::AUTOFETCH_SIGNED
: strncmp(v, "off", 3) == 0 ? OtaManager::AUTOFETCH_OFF : 0xFF;
if (pol == 0xFF) { strcpy(reply, "ERR usage: ota config autofetch <off|any|signed>"); return true; }
c.manager.set_autofetch(pol); c.config_dirty = true; strcpy(reply, "OK autofetch updated (saved)");
} else if (strncmp(p, "autoinstall ", 12) == 0) {
const char* v = p + 12;
uint8_t pol = strncmp(v, "trusted", 7) == 0 ? OtaContext::AUTOINSTALL_TRUSTED
: strncmp(v, "off", 3) == 0 ? OtaContext::AUTOINSTALL_OFF : 0xFF;
if (pol == 0xFF) { strcpy(reply, "ERR usage: ota config autoinstall <off|trusted>"); return true; }
c.autoinstall = pol; c.config_dirty = true; strcpy(reply, "OK autoinstall updated (saved)");
} else if (strncmp(p, "checkpoint ", 11) == 0) { // resume checkpoint cadence (blocks; 0=never)
long n = atol(p + 11);
if (n < 0 || n > 4096) { strcpy(reply, "ERR usage: ota config checkpoint <0..4096> (blocks; 0=never)"); return true; }
c.manager.set_checkpoint_blocks((uint16_t)n); c.config_dirty = true;
sprintf(reply, "OK checkpoint every %ld blocks (saved)%s", n, n == 0 ? " — periodic resume disabled" : "");
} else { // show current policy
uint8_t af = c.manager.autofetch();
sprintf(reply, "ota config: autofetch=%s autoinstall=%s checkpoint=%u keys=%u (persisted)",
af == OtaManager::AUTOFETCH_ANY ? "any" : af == OtaManager::AUTOFETCH_SIGNED ? "signed" : "off",
c.autoinstall == OtaContext::AUTOINSTALL_TRUSTED ? "trusted" : "off",
(unsigned)c.manager.checkpoint_blocks(), (unsigned)c.allow.count());
}
// ---- trusted signer allowlist (security config; persisted) ----
} else if (strncmp(a, "key add ", 8) == 0) {
uint8_t pub[32];
strcpy(reply, (mesh::Utils::fromHex(pub, 32, a + 8) && c.allow.add(pub)) ? "OK key added" : "ERR key");
if (mesh::Utils::fromHex(pub, 32, a + 8) && c.allow.add(pub)) { c.config_dirty = true; strcpy(reply, "OK key added (saved)"); }
else strcpy(reply, "ERR key");
} else if (strncmp(a, "key list", 8) == 0) {
int n = sprintf(reply, "keys=%u:", (unsigned)c.allow.count());
for (uint8_t i = 0; i < c.allow.count() && n < 140; i++) {
char hx[17]; mesh::Utils::toHex(hx, c.allow.get(i), 8);
n += sprintf(reply + n, " %s", hx);
}
} else if (strncmp(a, "key rm ", 7) == 0) {
uint8_t pub[32];
strcpy(reply, (mesh::Utils::fromHex(pub, 32, a + 7) && c.allow.remove(pub)) ? "OK removed" : "ERR");
if (mesh::Utils::fromHex(pub, 32, a + 7) && c.allow.remove(pub)) { c.config_dirty = true; strcpy(reply, "OK removed (saved)"); }
else strcpy(reply, "ERR");
} else if (strncmp(a, "stage ", 6) == 0) {
uint32_t sz = parse_u32(a + 6);
} else {
strcpy(reply, "ota: status|neighbors|announce|pull <#|mid>|drop|folder|config|self|applydelta|key|dev");
}
return true;
}
// Raw / internal primitives (manual content load + low-level apply steps), under `ota dev ...`.
static bool handle_dev(const char* d, char* reply, OtaContext& c) {
if (strncmp(d, "stage ", 6) == 0) {
uint32_t sz = parse_u32(d + 6);
if (sz == 0 || sz > OTA_SERVE_BUF_SIZE) { sprintf(reply, "ERR size 1..%u", OTA_SERVE_BUF_SIZE); }
else { memset(c.serve_buf, 0xFF, sz); c.serve_expected = sz; c.serving = false;
sprintf(reply, "OK stage %u bytes", (unsigned)sz); }
} else if (strncmp(a, "recv ", 5) == 0) {
const char* p = a + 5; uint32_t off = parse_u32(p);
} else if (strncmp(d, "recv ", 5) == 0) {
const char* p = d + 5; uint32_t off = parse_u32(p);
const char* hex = strchr(p, ' ');
if (!hex) { strcpy(reply, "ERR usage: ota recv <off> <hex>"); return true; }
if (!hex) { strcpy(reply, "ERR usage: ota dev recv <off> <hex>"); return true; }
hex++;
int blen = (int)strlen(hex) / 2;
uint8_t tmp[80];
@@ -73,103 +245,72 @@ bool handle_ota_command(const char* command, char* reply, mesh::MainBoard& board
else if (off + blen > c.serve_expected) strcpy(reply, "ERR off>size (stage first)");
else { memcpy(c.serve_buf + off, tmp, blen); sprintf(reply, "OK %d@%u", blen, (unsigned)off); }
} else if (strncmp(a, "serve", 5) == 0) {
} else if (strncmp(d, "serve self", 10) == 0) { // host our own running firmware, served from flash
if (ota_serve_self(c, 0)) {
c.serving = true;
char midhx[9]; mesh::Utils::toHex(midhx, c.serve_self_manifest + 20, 4);
uint32_t img = (uint32_t)c.serve_self_manifest[11] | ((uint32_t)c.serve_self_manifest[12] << 8)
| ((uint32_t)c.serve_self_manifest[13] << 16) | ((uint32_t)c.serve_self_manifest[14] << 24);
sprintf(reply, "OK serving self fw mid=%s (%u B, flash-backed) — peers can pull it", midhx, (unsigned)img);
} else strcpy(reply, "ERR serve self (no EndF / image too big / OOM)");
} else if (strncmp(d, "serve", 5) == 0) {
c.serving = c.manager.serve(c.serve_buf, c.serve_expected);
if (!c.serving) { strcpy(reply, "ERR serve (bad .mota)"); return true; }
VerifyResult r = ota_verify(c.serve_buf, c.serve_expected, c.allow);
sprintf(reply, "OK serving | root=%d img=%d sig=%d trust=%d", r.root_ok, r.image_ok, r.sig_ok, r.trusted);
} else if (strncmp(a, "announce", 8) == 0) {
if (!c.serving) { strcpy(reply, "ERR not serving (ota serve first)"); return true; }
} else if (strncmp(d, "resume", 6) == 0) { // re-adopt a container already staged in flash (test/debug)
bool ok = c.manager.resumeStaged(nullptr);
sprintf(reply, "%s resume: sess=%c %u/%u", ok ? "OK" : "ERR", fstate_char(c.manager.fetchState()),
(unsigned)c.manager.blocksHave(), (unsigned)c.manager.blocksTotal());
} else if (strncmp(d, "announce", 8) == 0) {
if (!c.serving) { strcpy(reply, "ERR not serving (ota dev serve first)"); return true; }
c.manager.announce();
strcpy(reply, "OK announced");
} else if (strncmp(a, "verify", 6) == 0) {
// verify whatever is staged-to-serve, OR the fetched container if a fetch is complete
} else if (strncmp(d, "verify", 6) == 0) {
const uint8_t* buf; uint32_t len;
if (c.manager.fetchState() == OtaManager::COMPLETE) { buf = c.fetch_store.data(); len = c.fetch_store.staged_size(); }
else { buf = c.serve_buf; len = c.serve_expected; }
if (len == 0) { strcpy(reply, "ERR nothing to verify"); return true; }
if (len == 0 || !buf) { strcpy(reply, "ERR nothing to verify (flash-staged: applydelta verifies)"); return true; }
VerifyResult r = ota_verify(buf, len, c.allow);
sprintf(reply, "verify parsed=%d root=%d img=%d signed=%d sig=%d trust=%d | ok=%d auto=%d",
r.parsed, r.root_ok, r.image_ok, r.is_signed, r.sig_ok, r.trusted,
r.integrity_ok(), r.auto_appliable());
r.parsed, r.root_ok, r.image_ok, r.is_signed, r.sig_ok, r.trusted, r.integrity_ok(), r.auto_appliable());
} else if (strncmp(a, "applydelta", 10) == 0) {
// Apply the fetched delta. ESP32: detools-sequential decode into the inactive A/B slot + verify +
// arm (reboot after). nRF52: verify + mark APPROVED in flash + reboot into the bootloader, which
// does the in-place decode + verify before booting it (this call does not return on success).
//
// This is destructive (it reboots and reflashes). It is GATED, not interactive — no "type yes"
// round-trip (unreliable over LoRa). First, refuse unless a full update is present: the fetch must
// be COMPLETE (every block received AND the merkle root re-verified). Then the apply path validates
// in order and returns the FIRST failing gate, so the operator knows exactly why it refused
// (payload hash -> built-for-this-firmware -> signature/trust); it proceeds only if all pass.
if (c.manager.fetchState() != OtaManager::COMPLETE || c.fetch_store.staged_size() == 0) {
sprintf(reply, "ERR no complete update fetched (fetch=%c %u/%u)",
fstate_char(c.manager.fetchState()), (unsigned)c.manager.blocksHave(),
(unsigned)c.manager.blocksTotal());
return true;
}
char m2[100];
#if defined(NRF52_PLATFORM)
bool ok = ota_apply_mota_nrf52(c.fetch_store.data(), c.fetch_store.staged_size(), c.allow, c.apply_st, m2);
#else
bool ok = ota_apply_detools_mota(c.fetch_store.data(), c.fetch_store.staged_size(), c.allow, c.apply_st, m2);
#endif
// On success the update is approved/armed but NOT yet rebooted — arm the deferred handoff so this
// reply reaches the operator first; the mesh loop reboots once it has been transmitted.
if (ok) c.apply_pending = true;
sprintf(reply, "%s | %s", ok ? "OK" : "ERR", m2);
} else if (strncmp(d, "want ", 5) == 0) {
const char* p = d + 5; while (*p == ' ') p++;
if (strncmp(p, "auto", 4) == 0) { c.manager.want(0); c.manager.want_mid(nullptr); strcpy(reply, "OK auto (own target only)"); }
else { uint32_t t = (uint32_t)strtoul(p, nullptr, 16); c.manager.want(t); c.manager.want_mid(nullptr);
sprintf(reply, "OK cross-target: will fetch %08X (you ensure HW compatible)", (unsigned)t); }
} else if (strncmp(a, "self", 4) == 0) {
// running firmware identity (EndF): body_len + body_hash:8 — compare against a delta's base_hash
SelfFwInfo fi;
if (!ota_self_firmware(fi) || !fi.valid) { strcpy(reply, "ERR no EndF (firmware lacks the trailer?)"); return true; }
char hx[17]; mesh::Utils::toHex(hx, fi.body_hash, 8);
sprintf(reply, "self body=%u image=%u base_hash=%s", (unsigned)fi.body_len, (unsigned)fi.image_len, hx);
} else if (strncmp(a, "apply", 5) == 0) {
const char* sub = a + 5;
while (*sub == ' ') sub++;
} else if (strncmp(d, "apply", 5) == 0) {
const char* sub = d + 5; while (*sub == ' ') sub++;
if (strncmp(sub, "slot", 4) == 0) {
uint32_t addr = 0, size = 0;
if (ota_apply_slot_info(&addr, &size)) sprintf(reply, "inactive slot addr=0x%X size=%u", (unsigned)addr, (unsigned)size);
else strcpy(reply, "ERR no A/B slot (apply unsupported on this build)");
} else if (strncmp(sub, "manifest", 8) == 0) {
// the manifest-fixed bytes were loaded into serve_buf via `ota stage`/`ota recv`
if (ota_apply_set_manifest(c.serve_buf, c.serve_expected, c.allow, c.apply_st))
sprintf(reply, "manifest ok img=%u sig=%d trust=%d", (unsigned)c.apply_st.image_size,
c.apply_st.sig_ok, c.apply_st.trusted);
sprintf(reply, "manifest ok img=%u sig=%d trust=%d", (unsigned)c.apply_st.image_size, c.apply_st.sig_ok, c.apply_st.trusted);
else strcpy(reply, "ERR manifest parse / not full-image / unsupported");
} else if (strncmp(sub, "verify", 6) == 0) {
bool ok = ota_apply_verify_slot(c.apply_st);
sprintf(reply, "slot image_hash %s (size=%u)", ok ? "MATCH" : "MISMATCH", (unsigned)c.apply_st.image_size);
} else if (strncmp(sub, "commit", 6) == 0) {
if (!c.apply_st.slot_ok) { strcpy(reply, "ERR run 'ota apply verify' first (slot must match)"); return true; }
// (D2: auto-apply would also require c.apply_st.trusted; a manual commit is allowed here.)
ota_apply_commit(); // sets boot partition + reboots into the new image; no return
if (!c.apply_st.slot_ok) { strcpy(reply, "ERR run 'ota dev apply verify' first (slot must match)"); return true; }
ota_apply_commit(); // set boot partition + reboot; no return
strcpy(reply, "ERR commit failed (no A/B slot?)");
} else {
strcpy(reply, "ERR ota apply (slot|manifest|verify|commit)");
strcpy(reply, "ERR ota dev apply (slot|manifest|verify|commit)");
}
} else if (strncmp(a, "want ", 5) == 0) {
const char* p = a + 5;
while (*p == ' ') p++;
if (strncmp(p, "auto", 4) == 0) { c.manager.want(0); strcpy(reply, "OK auto (own target only)"); }
else {
uint32_t t = (uint32_t)strtoul(p, nullptr, 16); // hex target_id (e.g. from another env)
c.manager.want(t);
sprintf(reply, "OK cross-target: will fetch %08X (you ensure HW compatible)", (unsigned)t);
}
} else if (strncmp(a, "clear", 5) == 0) {
c.serve_expected = 0; c.serving = false; c.fetch_store.clear();
} else if (strncmp(d, "clear", 5) == 0) {
c.serve_expected = 0; c.serving = false; c.fetch_store.clear(); c.manager.reset_session();
strcpy(reply, "OK cleared");
} else {
strcpy(reply, "ERR (status|self|key|stage|recv|serve|announce|verify|want|applydelta|clear)");
strcpy(reply, "ota dev: stage|recv|serve|announce|verify|want|apply slot|manifest|verify|commit|clear");
}
return true;
}
src/helpers/ota/OtaContext.h
+108 -2
View File
@@ -1,5 +1,7 @@
#pragma once
#include <stdio.h> // snprintf (hw_id mismatch message)
#include <string.h> // strncmp/strncpy (hw_id)
#include "OtaManager.h"
#include "OtaStore.h"
#include "SignerAllowlist.h"
@@ -7,6 +9,19 @@
#include "OtaFormat.h"
#if defined(NRF52_PLATFORM) && defined(OTA_FLASH_STORE)
#include "OtaStoreFlashNrf52.h"
#elif defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
#include "OtaStoreFlashEsp32.h"
#endif
#if defined(OTA_FOLDER_SERIAL)
#include "MotaSourceSerial.h" // relay an external folder served by a host daemon over the USB serial
#ifndef OTA_FOLDER_SERIAL_STREAM
#define OTA_FOLDER_SERIAL_STREAM Serial // default: the same USB console the CLI uses (no extra HW)
#endif
#ifndef OTA_FOLDER_SERIAL_BAUD
#define OTA_FOLDER_SERIAL_BAUD 115200
#endif
// The console Serial is already begun by the example; a DEDICATED UART (override the stream) needs init,
// so define OTA_FOLDER_SERIAL_BEGIN to have attach_folder() call .begin(baud) on it.
#endif
// Per-device OTA singleton shared by the CLI (OtaCli) and the mesh adapter (the example's MyMesh).
@@ -31,6 +46,8 @@ struct OtaContext {
OtaManager manager;
#if defined(NRF52_PLATFORM) && defined(OTA_FLASH_STORE)
OtaStoreFlashNrf52 fetch_store; // persistent flash staging (survives reboot; large deltas)
#elif defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
OtaStoreFlashEsp32 fetch_store; // stages in the inactive A/B slot (delta + full, RX-safe)
#else
OtaStoreRam<OTA_FETCH_BUF_SIZE> fetch_store;
#endif
@@ -38,8 +55,62 @@ struct OtaContext {
uint8_t serve_buf[OTA_SERVE_BUF_SIZE];
uint32_t serve_expected = 0; // size declared by `ota stage`
bool serving = false; // manager.serve() succeeded
// flash-backed self-serve: cached merkle leaves (heap, freed on re-serve) + assembled manifest of our
// own running firmware. The payload is read from flash per block; only the metadata is held in RAM.
// serve_self_proof is the proof-gen working buffer (>= block_count*4) — sized to OUR image's block
// count (the manager's fixed 4 KB scratch only covers <=1024 blocks; a >1 MB image needs more).
uint8_t* serve_self_leaves = nullptr;
uint8_t* serve_self_proof = nullptr;
uint8_t serve_self_manifest[96]; // v2 full+unsigned manifest = 89 (fixed incl. hw_id) + 4 approval
ApplyState apply_st; // pending apply (P6)
// OTA policy (persisted via NodePrefs; autofetch lives in the manager). Conservative defaults: a fresh
// node discovers + announces but never fetches/installs without operator intent.
static const uint8_t AUTOINSTALL_OFF = 0, AUTOINSTALL_TRUSTED = 1;
uint8_t autoinstall = AUTOINSTALL_OFF; // 1 = auto-apply a COMPLETE fetch IF signed + allowlisted
bool config_dirty = false; // CLI set a policy/key -> CommonCLI persists + clears
char hw_id[33] = {0}; // this device's hardware tag (from board.getOtaHwId(), set in begin)
// True if the staged .mota's hw_id is compatible with this device: equal tags, or either side empty
// ("unknown" -> can't enforce -> permissive). Brick-safety gate for apply (esp. manual cross-target).
bool hwMatches(const uint8_t* mhw /*32B, may be null*/) const {
if (!hw_id[0] || !mhw) return true;
bool declared = false; for (int i = 0; i < 32; i++) if (mhw[i]) { declared = true; break; }
if (!declared) return true;
return strncmp((const char*)mhw, hw_id, 32) == 0;
}
// Apply the COMPLETE fetched .mota (platform dispatch) and arm the slot; sets apply_pending on success
// so the deferred-reboot path (mesh loop) takes over. Caller ensures the fetch is COMPLETE. Shared by
// manual `ota applydelta` and the auto-install path.
bool apply_fetched(char* msg) {
// hardware-compatibility gate (brick-safety) — refuse a .mota whose hw_id is for different hardware,
// independent of signature; covers a manual cross-target `ota dev want` onto an incompatible board.
{
uint8_t hdr[8], mb[256];
uint32_t total = fetch_store.staged_size();
if (total >= 13 && fetch_store.read(0, hdr, 8) && memcmp(hdr, MOTA_MAGIC, 4) == 0) {
uint32_t mr = total - 8; if (mr > sizeof(mb)) mr = sizeof(mb);
MotaManifest mm;
if (fetch_store.read(8, mb, mr) && mota_parse_manifest(mb, mr, mm) && !hwMatches(mm.hw_id)) {
char want[33] = {0}; memcpy(want, mm.hw_id, 32);
snprintf(msg, 96, "refused: .mota hw_id '%.32s' != this device '%s' (incompatible hardware)", want, hw_id);
return false;
}
}
}
bool ok;
#if defined(NRF52_PLATFORM)
ok = ota_apply_mota_nrf52(fetch_store.data(), fetch_store.staged_size(), allow, apply_st, msg);
#elif defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
ok = ota_apply_detools_mota(fetch_store, allow, apply_st, msg);
#else
ok = ota_apply_detools_mota(fetch_store.data(), fetch_store.staged_size(), allow, apply_st, msg);
#endif
if (ok) apply_pending = true;
return ok;
}
// Deferred apply-reboot: a verified `ota applydelta` approves the update but does NOT reboot inline,
// so the CLI can first deliver the "verified; applying" reply (over LoRa it's the only way the
// operator learns the apply started). The mesh loop then calls ota_reboot_to_apply() once that reply
@@ -48,13 +119,48 @@ struct OtaContext {
uint32_t apply_at = 0; // earliest reboot time (lets the reply get queued + start sending)
uint32_t apply_hard = 0; // hard cap, in case the TX queue never idles on a busy node
void begin(uint32_t target_id, OtaSend send, void* ctx) {
// --- discovery: the "what mOTAs are available around me" view ----------------------------------
// The catalog (heard mOTAs) + the heard-sources table now live in OtaManager (built from beacons +
// OTA_HAVE catalog replies, the two-tier discovery). `ota neighbors` renders manager.catalogRow();
// `ota pull` acts on a mid. Here we only keep the fetch-session age stamp.
uint32_t session_started_ms = 0; // when the fetch session last left IDLE (for the age display)
uint8_t prev_fstate = OtaManager::IDLE;
bool folder_active = false; // an external `.mota` folder is attached + being relayed
// Attach/detach an external folder of `.mota` served by a host daemon over the seeder UART (the node
// then advertises + relays them alongside its own fw). Only built when OTA_FOLDER_SERIAL is configured.
#if defined(OTA_FOLDER_SERIAL)
bool attach_folder(char* msg, size_t cap) {
static SerialMotaSource src(OTA_FOLDER_SERIAL_STREAM, 600);
#ifdef OTA_FOLDER_SERIAL_BEGIN
OTA_FOLDER_SERIAL_STREAM.begin(OTA_FOLDER_SERIAL_BAUD); // dedicated UART; console is already up
#endif
manager.clear_sources(); // idempotent re-attach
if (!manager.add_source(&src)) { strncpy(msg, "ERR no free source slot", cap); return false; }
folder_active = true;
snprintf(msg, cap, "OK folder attached (serial) — serving %u mOTA total (own fw + folder)",
(unsigned)manager.servedCount());
return true;
}
#endif
void detach_folder() { manager.clear_sources(); folder_active = false; }
void track_session(uint8_t fstate, uint32_t now) { // stamp the session start (age display)
if (fstate != prev_fstate) {
if (prev_fstate == OtaManager::IDLE && fstate != OtaManager::IDLE) session_started_ms = now;
prev_fstate = fstate;
}
}
void begin(uint32_t target_id, OtaSend send, void* ctx, const char* hw = nullptr) {
manager.begin(target_id, send, ctx);
if (hw) { strncpy(hw_id, hw, sizeof(hw_id) - 1); hw_id[sizeof(hw_id) - 1] = 0; }
// a node only fetches firmware it can apply: ESP32 A/B -> sequential, nRF52 single-slot -> in-place
#if defined(NRF52_PLATFORM)
manager.set_apply_codec(CODEC_DETOOLS_INPLACE);
#elif defined(ESP32_PLATFORM)
manager.set_apply_codec(CODEC_DETOOLS_SEQUENTIAL);
manager.set_apply_codec(CODEC_DETOOLS_SEQUENTIAL); // preferred (streams straight to the slot)
manager.set_apply_codec2(CODEC_DETOOLS_INPLACE); // also accepted -> a single in-place .mota fits both
#endif
manager.set_fetch_store(&fetch_store);
}
src/helpers/ota/OtaFormat.h
+12 -3
View File
@@ -18,9 +18,16 @@ static const uint8_t ENDF_MAGIC[4] = { 'E', 'n', 'd', 'F' }; // 45 6E 64 4
static const uint32_t ENDF_LEN = 16; // marker(4)+body_len(4)+body_hash8(8)
// ---- manifest -------------------------------------------------------------
static const uint8_t MOTA_FORMAT_VER = 1;
static const uint8_t MOTA_FORMAT_VER = 2; // v2 adds hw_id[32] (a human-readable hardware tag)
static const uint8_t HASH_ALGO_SHA256 = 0x12; // multihash code
// hw_id: a fixed 32-byte, NUL-padded ASCII string naming the hardware a firmware can boot on (e.g.
// "RAK4631", "Heltec_v3"). Same hw_id == bootable-compatible (a role switch on the same board keeps it;
// different MCU/board differs). It sits in the SIGNED region of the manifest, so it can't be tampered.
// The applier refuses a `.mota` whose hw_id differs from the device's own (brick-safety, esp. for a manual
// cross-target `ota dev want`). An empty hw_id on either side = "unknown", and the check is skipped.
static const uint8_t MOTA_HW_ID_LEN = 32;
static const uint8_t MFLAG_FULL = 0x01; // 0 = delta/partial, 1 = full image
static const uint8_t MFLAG_SIGNED = 0x02;
@@ -46,8 +53,10 @@ enum OtaMsgType : uint8_t {
OTA_HAVE = 0x03,
OTA_GET_MANIFEST = 0x04,
OTA_MANIFEST = 0x05,
OTA_REQ = 0x06,
OTA_DATA = 0x07,
OTA_REQ = 0x06, // request a window of blocks' DATA fragments
OTA_DATA = 0x07, // one fragment of a block's data (self-describing by frag_off; no proof)
OTA_REQ_PROOF = 0x08, // request the merkle proof for one block (data + proof are fetched separately)
OTA_PROOF = 0x09, // the merkle proof for one block
};
static const uint16_t OTA_DEFAULT_BLOCK_SIZE = 1024;
src/helpers/ota/OtaManager.cpp
+482 -97
View File
@@ -15,80 +15,369 @@ static void wr_u32(uint8_t* p, uint32_t v) { p[0]=v; p[1]=v>>8; p[2]=v>>16; p[3]
void OtaManager::begin(uint32_t my_target_id, OtaSend send, void* ctx) {
_target = my_target_id; _send = send; _ctx = ctx;
_fstate = IDLE; _has_serve = false; _have = 0; _fbc = 0;
_fstate = IDLE; _have = 0; _fbc = 0;
_n_serve = 0; _n_src_obj = 0; _view0.valid = false; _srcv.valid = false;
}
// ---------------- serve ----------------
// ---------------- serve (multi-mota registry) ----------------
//
// A node offers a SET of mOTAs: its own firmware (view0) plus any external "folder" sources (OtaSource).
// Every fetch message carries the manifest_id, so a request dispatches to the matching ServeView via
// resolve() — view0 is resident; an external mota is (re)loaded on demand into _srcv. The catalog (what
// we advertise / answer OTA_QUERY with) is the lightweight _serve[] registry.
bool OtaManager::serve(const uint8_t* mota, uint32_t len) {
if (!mota_parse(mota, len, _sm)) return false;
_serve_buf = mota; _serve_len = len; _has_serve = true;
if (!mota_parse(mota, len, _view0.m)) return false;
_view0.mfl = (uint16_t)(_view0.m.leaves - _view0.m.manifest_start); // contiguous container
_view0.read = nullptr; _view0.read_ctx = nullptr; // payload is contiguous _view0.m.payload
_view0.scratch = _scratch; _view0.scratch_sz = sizeof(_scratch); // <=1024 blocks (RAM .mota is small)
_view0.valid = true;
registerSelfEntry();
return true;
}
void OtaManager::announce() {
if (!_has_serve) return;
bool OtaManager::serve_self(const uint8_t* manifest, uint16_t mfl, const uint8_t* leaves,
uint32_t block_count, uint8_t* proof_scratch, uint32_t proof_scratch_sz,
ServeReadFn read, void* ctx) {
if (proof_scratch_sz < (uint64_t)block_count * 4) return false; // proof-gen needs count*4 working bytes
if (!mota_parse_manifest(manifest, mfl, _view0.m)) return false; // fixed fields: root, image_hash, sizes
_view0.m.manifest_start = manifest;
_view0.m.leaves = leaves; // pre-computed, caller-owned (heap)
_view0.m.payload = nullptr; // read on demand via `read`
_view0.m.block_count = block_count;
_view0.mfl = mfl; _view0.read = read; _view0.read_ctx = ctx;
_view0.scratch = proof_scratch; _view0.scratch_sz = proof_scratch_sz; // sized for our (large) image
_view0.valid = true;
registerSelfEntry();
return true;
}
// (Re)build registry slot 0 from view0 (our own fw / RAM mota). Keeps any source entries in [1..].
void OtaManager::registerSelfEntry() {
if (!_view0.valid) return;
ServeEntry& e = _serve[0];
memcpy(e.mid, _view0.m.merkle_root, 4);
e.target_id = _view0.m.target_id; e.fw_version = _view0.m.fw_version;
e.codec_id = _view0.m.codec_id; e.flags = _view0.m.flags;
e.is_self = true; e.src = nullptr; e.src_idx = 0;
if (_n_serve == 0) _n_serve = 1;
}
bool OtaManager::add_source(MotaSource* src) {
if (!src || _n_src_obj >= OTA_MAX_SOURCE_OBJ) return false;
_src_list[_n_src_obj++] = src;
refresh_sources();
return true;
}
void OtaManager::refresh_sources() {
uint8_t base = _view0.valid ? 1 : 0; // entry 0 stays our own fw
if (_view0.valid) registerSelfEntry();
_n_serve = base;
for (uint8_t s = 0; s < _n_src_obj; s++) {
MotaSource* src = _src_list[s];
if (!src) continue;
uint8_t cnt = src->count();
for (uint8_t i = 0; i < cnt && _n_serve < OTA_MAX_SERVE; i++) {
MotaDesc d;
if (!src->describe(i, d)) continue;
if (serveEntryIndex(d.mid) >= 0) continue; // already offered (e.g. our own fw in the folder)
ServeEntry& e = _serve[_n_serve++];
memcpy(e.mid, d.mid, 4);
e.target_id = d.target_id; e.fw_version = d.fw_version;
e.codec_id = d.codec_id; e.flags = d.flags;
e.is_self = false; e.src = src; e.src_idx = i; e.desc = d;
}
}
_srcv.valid = false; // a loaded source view may now be stale; reloads on demand
}
void OtaManager::clear_sources() {
_n_src_obj = 0; _srcv.valid = false;
_n_serve = _view0.valid ? 1 : 0;
if (_view0.valid) registerSelfEntry();
}
int OtaManager::serveEntryIndex(const uint8_t* mid) const {
for (uint8_t i = 0; i < _n_serve; i++)
if (memcmp(_serve[i].mid, mid, 4) == 0) return i;
return -1;
}
OtaManager::ServeView* OtaManager::resolve(const uint8_t* mid) {
if (_view0.valid && memcmp(mid, _view0.m.merkle_root, 4) == 0) return &_view0;
if (_srcv.valid && memcmp(mid, _srcv_mid, 4) == 0) return &_srcv;
int i = serveEntryIndex(mid);
if (i < 0) return nullptr;
if (_serve[i].is_self) return _view0.valid ? &_view0 : nullptr;
return loadSource(_serve[i]) ? &_srcv : nullptr;
}
// Load an external mota into the on-demand _srcv: read its manifest-minus-leaves + leaves[] from the
// source into RAM, parse, and wire a payload reader that streams blocks from the source on REQ. (The
// payload itself is NOT held in RAM — only the small head + the leaves, <=4 KB for <=1024 blocks.)
bool OtaManager::loadSource(const ServeEntry& e) {
const MotaDesc& d = e.desc;
if (!e.src || d.leaves_off < 8) return false;
uint16_t mfl = (uint16_t)(d.leaves_off - 8);
if (mfl == 0 || mfl > sizeof(_src_manifest)) return false;
if (d.block_count == 0 || (uint64_t)d.block_count * 4 > sizeof(_src_leaves)) return false;
if (!e.src->read(e.src_idx, 8, _src_manifest, mfl)) return false;
if (!mota_parse_manifest(_src_manifest, mfl, _srcv.m)) return false;
if (memcmp(_srcv.m.merkle_root, d.mid, 4) != 0) return false; // descriptor/bytes disagree
if (_srcv.m.block_count != d.block_count) return false;
if (!e.src->read(e.src_idx, d.leaves_off, _src_leaves, d.block_count * 4)) return false;
_srcv.m.manifest_start = _src_manifest;
_srcv.m.leaves = _src_leaves;
_srcv.m.payload = nullptr;
_srcv.mfl = mfl;
_srcv_rdctx.src = e.src; _srcv_rdctx.idx = e.src_idx; _srcv_rdctx.payload_off = d.payload_off;
_srcv.read = srcReadTramp; _srcv.read_ctx = &_srcv_rdctx;
_srcv.scratch = _scratch; _srcv.scratch_sz = sizeof(_scratch);
memcpy(_srcv_mid, d.mid, 4);
_srcv.valid = true;
return true;
}
// ServeReadFn trampoline: payload-relative offset -> absolute source read.
bool OtaManager::srcReadTramp(void* c, uint32_t off, uint8_t* buf, uint32_t len) {
SrcReadCtx* x = (SrcReadCtx*)c;
return x->src->read(x->idx, x->payload_off + off, buf, len);
}
// sha2-256:4 over the SORTED set of mids we serve — peers use it to tell if our offering changed. Sorting
// makes it canonical across nodes regardless of insert order; for a single mota it is mh4(mid) (unchanged).
void OtaManager::setDigest(uint8_t out[4]) const {
if (_n_serve == 0) { memset(out, 0, 4); return; }
uint8_t order[OTA_MAX_SERVE];
for (uint8_t i = 0; i < _n_serve; i++) order[i] = i;
for (uint8_t i = 1; i < _n_serve; i++) { // insertion sort by mid (n <= 12)
uint8_t v = order[i]; int j = (int)i - 1;
while (j >= 0 && memcmp(_serve[order[j]].mid, _serve[v].mid, 4) > 0) { order[j+1] = order[j]; j--; }
order[j+1] = v;
}
uint8_t cat[OTA_MAX_SERVE * 4];
for (uint8_t i = 0; i < _n_serve; i++) memcpy(cat + (uint32_t)i * 4, _serve[order[i]].mid, 4);
mh4(out, cat, (size_t)_n_serve * 4);
}
void OtaManager::announce() { // tiny per-node beacon (constant size, independent of how many mOTAs)
AdvMsg a;
a.target_id = _sm.target_id;
a.fw_version = _sm.fw_version;
memcpy(a.manifest_id, _sm.merkle_root, 4);
a.flags = _sm.flags;
a.have_all = 1;
a.codec_id = _sm.codec_id;
uint8_t b[32];
memcpy(a.seeder_id, _seeder_id, 4);
a.n_motas = _n_serve;
setDigest(a.set_digest);
uint8_t b[16];
emit(b, encode_adv(b, sizeof(b), a), true);
}
// OTA_QUERY: two roles. (1) OVERHEAR-SUPPRESSION — any node that has a pending query for the same
// {source,digest} cancels it (someone else already asked; the broadcast HAVE is coming). (2) If the query
// is addressed to US, reply with our catalog (broadcast, tagged with our digest so every overhearer caches
// it). All served mOTAs matching filter_target are returned, fragmented if they exceed one packet.
void OtaManager::handleQuery(const uint8_t* m, uint16_t n) {
QueryMsg q;
if (!decode_query(m, n, q)) return;
if (_pq_active && memcmp(_pq_seeder, q.seeder_id, 4) == 0 && memcmp(_pq_digest, q.set_digest, 4) == 0)
_pq_active = false; // (1) suppress our own pending query
if (_n_serve == 0 || memcmp(q.seeder_id, _seeder_id, 4) != 0) return; // (2) only WE answer queries to us
uint8_t dg[4]; setDigest(dg);
uint8_t rowbuf[OTA_MAX_SERVE * OTA_HAVE_ROW_BYTES];
uint8_t nm = 0;
for (uint8_t i = 0; i < _n_serve; i++) {
const ServeEntry& e = _serve[i];
if (q.filter_target != 0 && q.filter_target != e.target_id) continue;
uint8_t* row = rowbuf + (uint32_t)nm * OTA_HAVE_ROW_BYTES;
memcpy(row, e.mid, 4);
wr_u32(row + 4, e.target_id); wr_u32(row + 8, e.fw_version);
row[12] = e.codec_id; row[13] = e.flags;
nm++;
}
const uint8_t per = (uint8_t)((MAX_PACKET_PAYLOAD - 12) / OTA_HAVE_ROW_BYTES); // rows per HAVE fragment
uint8_t ftotal = (uint8_t)((nm + per - 1) / per); if (ftotal == 0) ftotal = 1;
for (uint8_t fi = 0; fi < ftotal; fi++) {
uint8_t bse = (uint8_t)(fi * per);
uint8_t cnt = (uint8_t)((nm - bse > per) ? per : (nm - bse));
HaveMsg hv; memcpy(hv.seeder_id, _seeder_id, 4); memcpy(hv.set_digest, dg, 4);
hv.frag_idx = fi; hv.frag_total = ftotal; hv.n_rows = cnt;
hv.rows = cnt ? (rowbuf + (uint32_t)bse * OTA_HAVE_ROW_BYTES) : nullptr;
uint8_t b[MAX_PACKET_PAYLOAD];
emit(b, encode_have(b, sizeof(b), hv), true); // broadcast: all neighbours cache it
}
}
void OtaManager::handleGetManifest(const uint8_t* m, uint16_t n) {
GetManifestMsg gm;
if (!decode_get_manifest(m, n, gm) || !_has_serve) return;
if (memcmp(gm.manifest_id, _sm.merkle_root, 4) != 0) return;
// manifest-minus-leaves == bytes [manifest_start, leaves)
uint32_t mfl = (uint32_t)(_sm.leaves - _sm.manifest_start);
uint8_t b[MAX_PACKET_PAYLOAD];
ManifestMsg mm;
memcpy(mm.manifest_id, _sm.merkle_root, 4);
mm.frag_idx = 0; mm.frag_total = 1; // fits one fragment (signed manifest <= ~165 B)
mm.bytes = _sm.manifest_start; mm.len = (uint16_t)mfl;
emit(b, encode_manifest(b, sizeof(b), mm), false);
if (!decode_get_manifest(m, n, gm)) return;
ServeView* v = resolve(gm.manifest_id);
if (!v) return;
// A signed v2 manifest (with hw_id[32]) exceeds one LoRa packet, so send it as fragments. Each carries
// up to OTA_MF_FRAG manifest bytes; the client reassembles by frag_idx. (Re-sent in full on a retry.)
uint32_t mfl = v->mfl;
const uint8_t* src = v->m.manifest_start;
uint8_t ftotal = (uint8_t)((mfl + OTA_MF_FRAG - 1) / OTA_MF_FRAG); if (ftotal == 0) ftotal = 1;
for (uint8_t fi = 0; fi < ftotal; fi++) {
uint32_t off = (uint32_t)fi * OTA_MF_FRAG;
uint32_t fl = mfl - off; if (fl > OTA_MF_FRAG) fl = OTA_MF_FRAG;
ManifestMsg mm;
memcpy(mm.manifest_id, v->m.merkle_root, 4);
mm.frag_idx = fi; mm.frag_total = ftotal;
mm.bytes = src + off; mm.len = (uint16_t)fl;
uint8_t b[MAX_PACKET_PAYLOAD];
emit(b, encode_manifest(b, sizeof(b), mm), false);
}
}
void OtaManager::handleReq(const uint8_t* m, uint16_t n) {
ReqMsg rq;
if (!decode_req(m, n, rq) || !_has_serve) return;
if (memcmp(rq.manifest_id, _sm.merkle_root, 4) != 0) return;
uint32_t bs = _sm.block_size();
if (!decode_req(m, n, rq)) return;
ServeView* v = resolve(rq.manifest_id);
if (!v) return;
uint32_t bs = v->m.block_size();
for (uint32_t k = 0; k < rq.count; k++) {
uint32_t idx = rq.start_block + k;
if (idx >= _sm.block_count) break;
if (idx >= v->m.block_count) break;
uint32_t off = idx * bs;
uint32_t blen = (off + bs <= _sm.payload_size) ? bs : (_sm.payload_size - off);
uint8_t proof[32 * 4];
uint8_t np = merkle_gen_proof(_sm.leaves, _sm.block_count, idx, _scratch, proof);
DataMsg dm;
memcpy(dm.manifest_id, _sm.merkle_root, 4);
dm.block_idx = (uint16_t)idx; dm.frag_idx = 0; dm.frag_total = 1;
dm.n_proof = np; dm.proof = proof;
dm.data = _sm.payload + off; dm.data_len = (uint16_t)blen;
uint8_t b[MAX_PACKET_PAYLOAD];
emit(b, encode_data(b, sizeof(b), dm), false);
uint32_t blen = (off + bs <= v->m.payload_size) ? bs : (v->m.payload_size - off);
uint8_t blk[OTA_MAX_BLOCK];
const uint8_t* data;
if (v->read) { if (!v->read(v->read_ctx, off, blk, blen)) break; data = blk; }
else { data = v->m.payload + off; }
// send the block's data as self-describing fragments (frag_off); the proof is fetched separately
for (uint32_t fo = 0; fo < blen; fo += OTA_FRAG_DATA) {
uint32_t fl = (fo + OTA_FRAG_DATA <= blen) ? OTA_FRAG_DATA : (blen - fo);
DataMsg dm;
memcpy(dm.manifest_id, v->m.merkle_root, 4);
dm.block_idx = (uint16_t)idx; dm.frag_off = (uint16_t)fo;
dm.data = data + fo; dm.data_len = (uint16_t)fl;
uint8_t b[MAX_PACKET_PAYLOAD];
emit(b, encode_data(b, sizeof(b), dm), false);
}
}
}
void OtaManager::handleReqProof(const uint8_t* m, uint16_t n) {
ReqProofMsg rp;
if (!decode_req_proof(m, n, rp)) return;
ServeView* v = resolve(rp.manifest_id);
if (!v) return;
if (rp.block_idx >= v->m.block_count) return;
if ((uint64_t)v->m.block_count * 4 > v->scratch_sz) return; // proof-gen needs block_count*4 scratch
uint8_t proof[32 * 4];
uint8_t np = merkle_gen_proof(v->m.leaves, v->m.block_count, rp.block_idx, v->scratch, proof);
ProofMsg pm;
memcpy(pm.manifest_id, v->m.merkle_root, 4);
pm.block_idx = rp.block_idx; pm.n_proof = np; pm.proof = proof;
uint8_t b[MAX_PACKET_PAYLOAD];
emit(b, encode_proof(b, sizeof(b), pm), false);
}
// ---------------- fetch ----------------
// A tiny per-node BEACON: record the source; ask it for its catalog (OTA_QUERY) only when we're
// interested AND its set-digest is one we haven't catalogued yet (so a stable mesh is query-free).
void OtaManager::handleAdv(const uint8_t* m, uint16_t n) {
AdvMsg a;
if (!decode_adv(m, n, a)) return;
// auto-fetch matches our own target; a manual `want(T)` override accepts target T instead.
uint32_t accept = _desired_target ? _desired_target : _target;
if (a.target_id != accept) return; // not the firmware we're (auto/manually) after
if (!codecOk(a.codec_id)) return; // fw we can't apply on this platform — don't fetch
bool have_sid = (_seeder_id[0] | _seeder_id[1] | _seeder_id[2] | _seeder_id[3]) != 0;
if (have_sid && memcmp(a.seeder_id, _seeder_id, 4) == 0) return; // our own beacon, re-flooded
if (a.n_motas == 0) return; // source offers nothing
int slot = -1, lru = 0; // find/insert the source (LRU evict)
for (int i = 0; i < _n_src; i++) {
if (memcmp(_sources[i].seeder, a.seeder_id, 4) == 0) { slot = i; break; }
if (_sources[i].last_ms < _sources[lru].last_ms) lru = i;
}
bool fresh = (slot < 0);
if (fresh) { slot = (_n_src < OTA_MAX_SOURCES) ? _n_src++ : lru; _sources[slot] = Source{}; }
Source& s = _sources[slot];
bool changed = fresh || memcmp(s.digest, a.set_digest, 4) != 0;
memcpy(s.seeder, a.seeder_id, 4); memcpy(s.digest, a.set_digest, 4);
s.n_motas = a.n_motas; s.last_ms = _now_ms;
if (changed) s.have_catalog = false;
// interested = auto-fetch enabled, or a manual pull/want is pending. (Browsing queries via queryAll().)
bool interested = (_autofetch != AUTOFETCH_OFF) || _have_desired_mid || _desired_target;
if (interested && !s.have_catalog) scheduleQuery(a.seeder_id, a.set_digest); // jittered + suppressible
}
// Schedule a catalog query after a random jitter (id ⊕ digest, so neighbours pick different delays). The
// node with the shortest jitter sends; the rest overhear that QUERY (or the broadcast HAVE) and suppress.
void OtaManager::scheduleQuery(const uint8_t* seeder, const uint8_t* digest) {
if (_pq_active && memcmp(_pq_seeder, seeder, 4) == 0 && memcmp(_pq_digest, digest, 4) == 0) return; // already pending
memcpy(_pq_seeder, seeder, 4); memcpy(_pq_digest, digest, 4);
uint32_t j = (rd_u32(seeder) ^ rd_u32(digest) ^ rd_u32(_seeder_id)) % OTA_QUERY_SPREAD_MS;
_pq_at = _now_ms + OTA_QUERY_MIN_MS + j;
_pq_active = true;
}
void OtaManager::sendQuery(const uint8_t* seeder, const uint8_t* digest, uint32_t filter_target) {
QueryMsg q; memcpy(q.seeder_id, seeder, 4); memcpy(q.set_digest, digest, 4); q.filter_target = filter_target;
uint8_t b[16];
emit(b, encode_query(b, sizeof(b), q), true); // FLOODED so neighbours overhear it and suppress
}
// User-initiated browse (`ota neighbors`): ask every known source now (no jitter — infrequent + explicit).
void OtaManager::queryAll() { for (uint8_t i = 0; i < _n_src; i++) sendQuery(_sources[i].seeder, _sources[i].digest, 0); }
// A catalog reply: record each mOTA (deduped by mid; distinct-source count for the UI), and if a row
// matches our fetch interest (auto-fetch own-target, or a pending pull/want), begin fetching it.
void OtaManager::handleHave(const uint8_t* m, uint16_t n) {
HaveMsg hv;
if (!decode_have(m, n, hv)) return;
bool have_sid = (_seeder_id[0] | _seeder_id[1] | _seeder_id[2] | _seeder_id[3]) != 0;
if (have_sid && memcmp(hv.seeder_id, _seeder_id, 4) == 0) return; // our own catalog
// PASSIVE: any overheard HAVE marks its source catalogued + cancels a pending query for it (storm
// suppression) — every node caches the rows below, even one that never queried.
for (uint8_t i = 0; i < _n_src; i++)
if (memcmp(_sources[i].seeder, hv.seeder_id, 4) == 0 && memcmp(_sources[i].digest, hv.set_digest, 4) == 0)
_sources[i].have_catalog = true;
if (_pq_active && memcmp(_pq_seeder, hv.seeder_id, 4) == 0 && memcmp(_pq_digest, hv.set_digest, 4) == 0)
_pq_active = false;
for (uint8_t r = 0; r < hv.n_rows && hv.rows; r++) {
const uint8_t* row = hv.rows + (uint32_t)r * OTA_HAVE_ROW_BYTES;
const uint8_t* mid = row;
uint32_t target = rd_u32(row + 4), fwver = rd_u32(row + 8);
uint8_t codec = row[12], flags = row[13];
int slot = -1, lru = 0; // upsert into the catalog (dedup by mid)
for (int i = 0; i < _n_cat; i++) {
if (memcmp(_catalog[i].mid, mid, 4) == 0) { slot = i; break; }
if (_catalog[i].last_ms < _catalog[lru].last_ms) lru = i;
}
if (slot < 0) {
slot = (_n_cat < OTA_MAX_CATALOG) ? _n_cat++ : lru;
_catalog[slot] = CatRow{};
memcpy(_catalog[slot].mid, mid, 4); memcpy(_catalog[slot].seeder0, hv.seeder_id, 4);
_catalog[slot].n_seeders = 1;
} else if (memcmp(_catalog[slot].seeder0, hv.seeder_id, 4) != 0 && _catalog[slot].n_seeders < 255) {
_catalog[slot].n_seeders++; // another distinct source has it
}
CatRow& c = _catalog[slot];
c.target_id = target; c.fw_version = fwver; c.codec = codec; c.flags = flags; c.last_ms = _now_ms;
if (wantRow(mid, target, codec, flags)) startFetch(mid, target);
}
}
bool OtaManager::wantRow(const uint8_t* mid, uint32_t target, uint8_t codec, uint8_t flags) const {
if (!_fetch || _fstate == FETCHING || _fstate == WANT_MANIFEST) return false; // busy with a session
if (_fstate == COMPLETE && memcmp(mid, _fid, 4) == 0) return false; // already have it
if (!codecOk(codec)) return false; // can't apply this codec
if (_have_desired_mid) // manual pull of a specific mid
return memcmp(mid, _desired_mid, 4) == 0 && (_desired_target == 0 || target == _desired_target);
if (_desired_target) return target == _desired_target; // cross-target want (role switch)
if (_autofetch == AUTOFETCH_OFF) return false; // discover only
if (target != _target) return false; // auto-fetch = our own target
if (_autofetch == AUTOFETCH_SIGNED && !(flags & MFLAG_SIGNED)) return false; // signed-only policy
return true;
}
// Begin (or resume) fetching a chosen mid: try a staged-partial resume first, else request the manifest.
void OtaManager::startFetch(const uint8_t* mid, uint32_t target) {
(void)target;
if (!_fetch || _fstate == FETCHING || _fstate == WANT_MANIFEST) return;
if (_fstate == COMPLETE && memcmp(a.manifest_id, _fid, 4) == 0) return; // already have it
// interested: ask for the manifest
memcpy(_fid, a.manifest_id, 4);
if (resumeStaged(mid)) return; // resume a partial container left in flash
memcpy(_fid, mid, 4);
_fstate = WANT_MANIFEST;
_mf_total = 0; _mf_mask = 0; _mf_len = 0; // fresh manifest reassembly
GetManifestMsg gm; memcpy(gm.manifest_id, _fid, 4);
uint8_t b[16];
emit(b, encode_get_manifest(b, sizeof(b), gm), false);
@@ -98,16 +387,27 @@ void OtaManager::handleManifest(const uint8_t* m, uint16_t n) {
ManifestMsg mm;
if (!decode_manifest(m, n, mm) || !_fetch) return;
if (_fstate != WANT_MANIFEST || memcmp(mm.manifest_id, _fid, 4) != 0) return;
if (mm.frag_total != 1) return; // multi-fragment manifest not supported yet
if (mm.frag_total == 0 || mm.frag_total > OTA_MF_MAXFRAG || mm.frag_idx >= mm.frag_total) return;
const uint8_t* mf = mm.bytes; // manifest-minus-leaves
uint32_t mfl = mm.len;
if (mfl < 57) { _fstate = FAILED; return; }
// reassemble the (possibly multi-fragment) manifest into _mf_buf; place fragment frag_idx at its offset
uint32_t foff = (uint32_t)mm.frag_idx * OTA_MF_FRAG;
if (foff + mm.len > sizeof(_mf_buf)) return;
if (mm.frag_total != _mf_total) { _mf_total = mm.frag_total; _mf_mask = 0; _mf_len = 0; } // (re)start
memcpy(_mf_buf + foff, mm.bytes, mm.len);
_mf_mask |= (uint16_t)(1u << mm.frag_idx);
if (mm.frag_idx == mm.frag_total - 1) _mf_len = foff + mm.len; // last fragment fixes the length
uint16_t full = (mm.frag_total >= 16) ? 0xFFFF : (uint16_t)((1u << mm.frag_total) - 1);
if (_mf_mask != full || _mf_len == 0) return; // wait until every fragment is in
const uint8_t* mf = _mf_buf; // fully reassembled manifest-minus-leaves
uint32_t mfl = _mf_len;
if (mfl < 89) { _fstate = FAILED; return; } // fixed head incl. hw_id[32]
if (!codecOk(mf[56])) { _fstate = IDLE; return; } // codec we can't apply (lying/stale ADV) — abort
uint32_t payload_size = rd_u32(mf + 15);
uint8_t bsl = mf[19];
uint32_t bs = 1u << bsl;
if (bs == 0 || payload_size == 0) { _fstate = FAILED; return; }
// a block must fit our reassembly buffer (and be non-empty) — reject an oversized block_size up front
if (bs == 0 || bs > OTA_MAX_BLOCK || payload_size == 0) { _fstate = FAILED; return; }
uint32_t bc = (payload_size + bs - 1) / bs;
memcpy(_froot, mf + 20, 4);
@@ -115,6 +415,11 @@ void OtaManager::handleManifest(const uint8_t* m, uint16_t n) {
uint32_t payload_off = leaves_off + bc * 4;
uint32_t total = payload_off + payload_size + 5;
// Hand the store the parsed layout BEFORE begin(), so a partition-backed store (ESP32) can choose
// placement and refuse an unfittable fetch up front: a FULL payload streams to the inactive slot,
// a delta's whole container is staged together. (image_size at mf+11, is_full from flags at mf+1.)
bool is_full = (mf[1] & MFLAG_FULL) != 0;
if (!_fetch->plan_layout(is_full, rd_u32(mf + 11), payload_off, payload_size)) { _fstate = FAILED; return; }
if (!_fetch->begin(total)) { _fstate = FAILED; return; }
// declare the metadata extent so a flash store can pin it (leaves are written all transfer long)
if (!_fetch->set_meta_size(payload_off)) { _fstate = FAILED; return; }
@@ -125,12 +430,63 @@ void OtaManager::handleManifest(const uint8_t* m, uint16_t n) {
!_fetch->write(8, mf, mfl) ||
!_fetch->write(total - 5, MOTA_TRAILER, 5)) { _fstate = FAILED; return; }
_fflags = mf[1]; // manifest flags (FULL/SIGNED) of the fetch in progress (auto-install gate)
_fpoff = payload_off; _floff = leaves_off; _fpsize = payload_size; _fbc = bc; _fbs = bs;
_ftotal = total; _have = 0; _fstate = FETCHING;
// fresh transfer: clear any per-block reassembly state from a prior session
_reasm_block = 0xFFFFFFFFu; _reasm_mask = 0; _reasm_need = 0; _awaiting_proof = false;
_loop_last_have = 0; _loop_last_mask = 0;
OTA_DBG("OTA: FETCHING bc=%u bs=%u total=%u\n", (unsigned)bc, (unsigned)bs, (unsigned)total);
requestMissing();
}
bool OtaManager::resumeStaged(const uint8_t* want_mid) {
if (!_fetch || _fstate == FETCHING || _fstate == WANT_MANIFEST) return false;
if (!_fetch->reopen()) return false; // nothing persisted in the store
uint32_t total = _fetch->staged_size();
uint8_t hdr[8];
if (total < 13 || !_fetch->read(0, hdr, 8) || memcmp(hdr, MOTA_MAGIC, 4) != 0) return false;
// read + parse the stored manifest (everything before leaves[]) to recompute the geometry
uint8_t mbuf[256];
uint32_t mread = total - 8; if (mread > sizeof(mbuf)) mread = sizeof(mbuf);
MotaManifest m;
if (!_fetch->read(8, mbuf, mread) || !mota_parse_manifest(mbuf, mread, m)) return false;
if (want_mid && memcmp(m.merkle_root, want_mid, 4) != 0) return false; // a different fw is staged
if (!codecOk(m.codec_id)) return false;
uint32_t mfl = (uint32_t)(m.approval - m.manifest_start) + 4; // manifest-minus-leaves length
uint32_t bs = m.block_size();
if (bs == 0 || bs > OTA_MAX_BLOCK) return false;
uint32_t bc = m.block_count;
uint32_t leaves_off = 8 + mfl;
uint32_t payload_off = leaves_off + bc * 4;
if ((uint64_t)payload_off + m.payload_size + 5 != total) return false; // geometry must match the header
memcpy(_fid, m.merkle_root, 4);
memcpy(_froot, m.merkle_root, 4);
_fflags = m.flags;
_fpoff = payload_off; _floff = leaves_off; _fpsize = m.payload_size; _fbc = bc; _fbs = bs;
_ftotal = total;
_have = 0;
for (uint32_t i = 0; i < bc; i++) if (blockPresent(i)) _have++; // count blocks whose leaf survived
_reasm_block = 0xFFFFFFFFu; _reasm_mask = 0; _reasm_need = 0; _awaiting_proof = false;
_loop_last_have = 0; _loop_last_mask = 0;
OTA_DBG("OTA: RESUME have=%u/%u total=%u\n", (unsigned)_have, (unsigned)bc, (unsigned)total);
if (_have >= bc) { // already complete -> verify root + finalize
if (bc * 4 <= sizeof(_scratch) && _fetch->read(_floff, _scratch, bc * 4)) {
uint8_t root[4]; merkle_root(root, _scratch, bc);
_fstate = (memcmp(root, _froot, 4) == 0) ? COMPLETE : FAILED;
} else {
_fstate = COMPLETE;
}
if (_fstate == COMPLETE) _fetch->finalize();
return true;
}
_fstate = FETCHING; // resume fetching the holes
requestMissing();
return true;
}
uint32_t OtaManager::blockLen(uint32_t i) const {
uint32_t off = i * _fbs;
return (off + _fbs <= _fpsize) ? _fbs : (_fpsize - off);
@@ -146,69 +502,92 @@ void OtaManager::handleData(const uint8_t* m, uint16_t n) {
DataMsg dm;
if (!decode_data(m, n, dm) || !_fetch) return;
if (_fstate != FETCHING || memcmp(dm.manifest_id, _fid, 4) != 0) return;
if (dm.frag_total != 1) return; // single-fragment blocks only (v1)
if (dm.block_idx >= _fbc) return;
if (blockPresent(dm.block_idx)) return; // already have it
uint32_t want = blockLen(dm.block_idx);
if (dm.data_len != want) return;
// verify the block against the (signed) root via its proof — reject forged/corrupt data
if (!merkle_verify(dm.data, dm.data_len, dm.block_idx, dm.proof, dm.n_proof, _froot, _fbc)) return;
// commit: payload block first, then the leaf (the commit marker)
if (!_fetch->write(_fpoff + dm.block_idx * _fbs, dm.data, dm.data_len)) return;
uint8_t leaf[4];
merkle_leaf(leaf, dm.data, dm.data_len);
if (!_fetch->write(_floff + dm.block_idx * 4, leaf, 4)) return;
if (blockPresent(dm.block_idx)) return; // already stored + verified
uint32_t blen = blockLen(dm.block_idx);
if (dm.frag_off % OTA_FRAG_DATA != 0) return; // canonical FRAG_DATA-aligned slices only
if ((uint32_t)dm.frag_off + dm.data_len > blen) return; // slice out of the block
if (dm.block_idx != _reasm_block) { // (re)start reassembly for this block
_reasm_block = dm.block_idx; _reasm_mask = 0; _awaiting_proof = false;
uint32_t nf = (blen + OTA_FRAG_DATA - 1) / OTA_FRAG_DATA;
_reasm_need = (nf >= 16) ? 0xFFFF : (uint16_t)((1u << nf) - 1);
}
uint32_t kf = dm.frag_off / OTA_FRAG_DATA;
if (kf >= 16) return;
memcpy(_reasm_buf + dm.frag_off, dm.data, dm.data_len);
_reasm_mask |= (uint16_t)(1u << kf);
if (_reasm_mask != _reasm_need || _awaiting_proof) return; // wait for all slices (or proof already asked)
// block fully reassembled -> request its proof (data + proof are fetched separately)
_awaiting_proof = true;
ReqProofMsg rp; memcpy(rp.manifest_id, _fid, 4); rp.block_idx = (uint16_t)_reasm_block;
uint8_t b[16]; emit(b, encode_req_proof(b, sizeof(b), rp), false);
}
void OtaManager::handleProof(const uint8_t* m, uint16_t n) {
ProofMsg pm;
if (!decode_proof(m, n, pm) || !_fetch) return;
if (_fstate != FETCHING || memcmp(pm.manifest_id, _fid, 4) != 0) return;
if (!_awaiting_proof || pm.block_idx != _reasm_block) return; // not the block we're verifying
uint32_t blen = blockLen(_reasm_block);
if (!merkle_verify(_reasm_buf, blen, _reasm_block, pm.proof, pm.n_proof, _froot, _fbc)) {
_reasm_block = 0xFFFFFFFFu; _reasm_mask = 0; _awaiting_proof = false; // bad -> drop, re-fetch the block
return;
}
// verified -> commit the payload block, then its leaf (the present marker)
if (!_fetch->write(_fpoff + (uint32_t)_reasm_block * _fbs, _reasm_buf, blen)) return;
uint8_t leaf[4]; merkle_leaf(leaf, _reasm_buf, blen);
if (!_fetch->write(_floff + (uint32_t)_reasm_block * 4, leaf, 4)) return;
_have++;
OTA_DBG("OTA: block %u OK have=%u/%u\n", (unsigned)dm.block_idx, (unsigned)_have, (unsigned)_fbc);
// if the current request window is fully received, immediately ask for the next one
// (paces the transfer to the link rate instead of flooding the whole image at once)
if (_have < _fbc) {
bool window_done = true;
for (uint32_t i = _req_start; i < _req_start + _req_count && i < _fbc; i++) {
if (!blockPresent(i)) { window_done = false; break; }
}
if (window_done) requestMissing();
}
if (_have >= _fbc) {
// recompute the root over all stored leaves as a final cross-check
// (read leaves into the scratch buffer; bounded by OTA_PROOFGEN_SCRATCH)
if (_fbc * 4 <= sizeof(_scratch) && _fetch->read(_floff, _scratch, _fbc * 4)) {
uint8_t root[4];
merkle_root(root, _scratch, _fbc);
_fstate = (memcmp(root, _froot, 4) == 0) ? COMPLETE : FAILED;
} else {
_fstate = COMPLETE; // per-block proofs already guaranteed integrity vs the root
}
if (_fstate == COMPLETE) _fetch->finalize(); // commit the staged container to persistent storage
OTA_DBG("OTA: transfer %s\n", _fstate == COMPLETE ? "COMPLETE" : "FAILED(root)");
OTA_DBG("OTA: block %u OK have=%u/%u\n", (unsigned)_reasm_block, (unsigned)_have, (unsigned)_fbc);
_reasm_block = 0xFFFFFFFFu; _reasm_mask = 0; _awaiting_proof = false;
// periodically persist progress (meta/leaf page + open payload) so a reboot can resume (no-op for RAM);
// cadence is runtime-tunable via `ota config checkpoint <N>` (0 = never)
if (_checkpoint_blocks && _have % _checkpoint_blocks == 0) _fetch->checkpoint();
if (_have < _fbc) { requestMissing(); return; } // next block
// all blocks present -> final root cross-check + finalize
if (_fbc * 4 <= sizeof(_scratch) && _fetch->read(_floff, _scratch, _fbc * 4)) {
uint8_t root[4]; merkle_root(root, _scratch, _fbc);
_fstate = (memcmp(root, _froot, 4) == 0) ? COMPLETE : FAILED;
} else {
_fstate = COMPLETE; // per-block proofs already guaranteed integrity vs the root
}
if (_fstate == COMPLETE) _fetch->finalize(); // commit the staged container to persistent storage
OTA_DBG("OTA: transfer %s\n", _fstate == COMPLETE ? "COMPLETE" : "FAILED(root)");
}
void OtaManager::requestMissing() {
if (_fstate != FETCHING) return;
// request a small WINDOW of the next missing blocks (keeps the server's TX queue small,
// so OTA never floods/saturates the mesh — docs/ota_protocol.md §8)
// Per-block serial flow (split data/proof). If the current block's data is fully reassembled and we
// are waiting on its proof, (re)send the proof request rather than re-fetching the data — this also
// recovers from a lost PROOF reply.
if (_awaiting_proof && _reasm_block != 0xFFFFFFFFu) {
ReqProofMsg rp; memcpy(rp.manifest_id, _fid, 4); rp.block_idx = (uint16_t)_reasm_block;
uint8_t b[16]; emit(b, encode_req_proof(b, sizeof(b), rp), false);
OTA_DBG("OTA: REQ_PROOF block=%u (have=%u/%u)\n",
(unsigned)_reasm_block, (unsigned)_have, (unsigned)_fbc);
return;
}
// Otherwise request the DATA fragments of the next missing block. One block at a time keeps the
// server's TX queue tiny so OTA never floods the mesh (docs/ota_protocol.md §8); a block's fragments
// are self-describing (frag_off) so they may be served by ANY peer, BitTorrent-style.
uint32_t start = 0;
while (start < _fbc && blockPresent(start)) start++;
if (start >= _fbc) return;
uint32_t count = _fbc - start;
if (count > OTA_REQ_WINDOW) count = OTA_REQ_WINDOW;
_req_start = start; _req_count = count;
_req_start = start; _req_count = 1;
ReqMsg rq; memcpy(rq.manifest_id, _fid, 4);
rq.start_block = (uint16_t)start; rq.count = (uint8_t)count;
rq.start_block = (uint16_t)start; rq.count = 1;
uint8_t b[16];
OTA_DBG("OTA: REQ start=%u count=%u (have=%u/%u)\n",
(unsigned)start, (unsigned)count, (unsigned)_have, (unsigned)_fbc);
OTA_DBG("OTA: REQ block=%u (have=%u/%u mask=%04x)\n",
(unsigned)start, (unsigned)_have, (unsigned)_fbc, (unsigned)_reasm_mask);
emit(b, encode_req(b, sizeof(b), rq), false);
}
void OtaManager::loop() {
// fire a scheduled catalog query once its jitter has elapsed (unless overhearing already suppressed it)
if (_pq_active && (int32_t)(_now_ms - _pq_at) >= 0) {
_pq_active = false;
sendQuery(_pq_seeder, _pq_digest, 0); // unfiltered: one broadcast HAVE serves everyone
}
if (_fstate == WANT_MANIFEST) {
// the MANIFEST reply may have been lost on a marginal link — retry GET_MANIFEST
GetManifestMsg gm; memcpy(gm.manifest_id, _fid, 4);
@@ -217,9 +596,11 @@ void OtaManager::loop() {
return;
}
if (_fstate != FETCHING) return;
// retry only when a tick passed with no progress (avoids re-request spam during active flow)
if (_have == _loop_last_have) requestMissing();
// retry only when a whole tick passed with NO progress — neither a committed block nor a new fragment
// of the in-flight block. This avoids re-request spam while a block's fragments are still streaming in.
if (_have == _loop_last_have && _reasm_mask == _loop_last_mask) requestMissing();
_loop_last_have = _have;
_loop_last_mask = _reasm_mask;
}
// ---------------- dispatch ----------------
@@ -227,10 +608,14 @@ void OtaManager::loop() {
void OtaManager::on_message(const uint8_t* msg, uint16_t len) {
switch (ota_msg_type(msg, len)) {
case OTA_ADV: handleAdv(msg, len); break;
case OTA_QUERY: handleQuery(msg, len); break;
case OTA_HAVE: handleHave(msg, len); break;
case OTA_GET_MANIFEST: handleGetManifest(msg, len); break;
case OTA_MANIFEST: handleManifest(msg, len); break;
case OTA_REQ: handleReq(msg, len); break;
case OTA_DATA: handleData(msg, len); break;
case OTA_REQ_PROOF: handleReqProof(msg, len); break;
case OTA_PROOF: handleProof(msg, len); break;
default: break;
}
}
src/helpers/ota/OtaManager.h
+216 -15
View File
@@ -5,14 +5,16 @@
#include "OtaFormat.h"
#include "OtaStore.h"
#include "MotaContainer.h"
#include "OtaSource.h"
// Transport-agnostic OTA session engine (docs/ota_protocol.md §5/§8). It SERVES a complete `.mota`
// (answering GET_MANIFEST / REQ) and/or FETCHES one into an OtaStore (verifying every block against
// the signed merkle root via proofs). It is portable (no Arduino / radio / Ed25519) so it can be
// driven by a host simulation; a thin Mesh adapter wires it to PAYLOAD_TYPE_OTA on device.
//
// v1 assumes single-fragment blocks (block_size small enough to fit one packet). Multi-fragment
// reassembly (for 1 KB blocks) is a later optimization; the wire format already carries frag fields.
// Transfer is per-block and 2-phase: a 1 KB logical block is fetched as self-describing DATA fragments
// (frag_off, so any peer can serve any fragment — BitTorrent-style), reassembled, then its merkle PROOF
// is requested separately and verified against the signed root before the block is committed.
namespace mesh {
namespace ota {
@@ -20,12 +22,49 @@ namespace ota {
// Emit an OTA message (one packet payload). `flood`=true for announce/query, false for direct replies.
typedef void (*OtaSend)(void* ctx, const uint8_t* msg, uint16_t len, bool flood);
// Read `len` payload bytes at offset `off` from the serve source (flash-backed self-serve); false on
// error. nullptr means the payload is a contiguous RAM buffer (the staged `.mota`).
typedef bool (*ServeReadFn)(void* ctx, uint32_t off, uint8_t* buf, uint32_t len);
#ifndef OTA_PROOFGEN_SCRATCH
#define OTA_PROOFGEN_SCRATCH 4096 // server proof-gen working buffer (supports up to 1024 blocks)
#endif
#ifndef OTA_REQ_WINDOW
#define OTA_REQ_WINDOW 6 // blocks requested per REQ (keeps the server's TX queue small)
#ifndef OTA_MAX_BLOCK
#define OTA_MAX_BLOCK 1024 // largest logical block (merkle leaf unit) = reassembly buffer size
#endif
#ifndef OTA_CHECKPOINT_BLOCKS
#define OTA_CHECKPOINT_BLOCKS 32 // persist progress (store.checkpoint) every N committed blocks (resume)
#endif
#ifndef OTA_MF_FRAG
#define OTA_MF_FRAG 176 // manifest bytes per OTA_MANIFEST fragment (<= MAX_PACKET_PAYLOAD - header)
#endif
#ifndef OTA_MF_MAXFRAG
#define OTA_MF_MAXFRAG 4 // max manifest fragments (a signed v2 manifest is ~2)
#endif
#ifndef OTA_MAX_SOURCES
#define OTA_MAX_SOURCES 12 // heard OTA sources (beacon senders) tracked (LRU); ~12 B each
#endif
#ifndef OTA_MAX_SERVE
#define OTA_MAX_SERVE 12 // mOTAs THIS node offers (own fw + external folder); == one HAVE fragment
#endif
#ifndef OTA_MAX_SOURCE_OBJ
#define OTA_MAX_SOURCE_OBJ 4 // external MotaSource objects (folders/transports) attached at once
#endif
#ifndef OTA_SRC_MANIFEST_MAX
#define OTA_SRC_MANIFEST_MAX 256 // manifest-minus-leaves buffer for the loaded source mota (head+sig+approval)
#endif
#ifndef OTA_MAX_CATALOG
#define OTA_MAX_CATALOG 12 // distinct mOTAs catalogued from OTA_HAVE replies (LRU)
#endif
#ifndef OTA_QUERY_MIN_MS
#define OTA_QUERY_MIN_MS 300 // min delay before sending a catalog query (overhear-suppression window)
#endif
#ifndef OTA_QUERY_SPREAD_MS
#define OTA_QUERY_SPREAD_MS 4000 // random jitter span so 50 neighbours don't all query at once (storm)
#endif
#ifndef OTA_FRAG_DATA
#define OTA_FRAG_DATA 160 // data bytes per DATA fragment (<= MAX_PACKET_PAYLOAD - 9-byte header)
#endif
// nRF52 note: a flash page-erase halts the CPU (~85 ms, code runs from flash) and starves the LoRa RX,
// so writing to flash on every received packet drops in-flight DATA and the transfer stalls. The SD-safe
@@ -40,43 +79,167 @@ class OtaManager {
public:
enum FetchState : uint8_t { IDLE, WANT_MANIFEST, FETCHING, COMPLETE, FAILED };
// --- multi-mota serve --- A ServeView is everything a serve handler needs for ONE mota. Two can be
// resident: view0 = our own fw / a RAM `.mota` (always loaded), plus one on-demand source view that is
// (re)loaded from a MotaSource when a request targets a different external mota. Requests dispatch by
// manifest_id (carried in every fetch message) -> the matching ServeView via resolve().
struct ServeView {
bool valid = false;
MotaManifest m; // parsed manifest (fields/pointers into the backing buffers)
uint16_t mfl = 0; // manifest-minus-leaves length (the OTA_MANIFEST payload)
ServeReadFn read = nullptr; // payload reader (nullptr => m.payload is contiguous in RAM)
void* read_ctx = nullptr;
uint8_t* scratch = nullptr; // proof-gen working buffer (>= block_count*4)
uint32_t scratch_sz = 0;
};
// A lightweight catalog entry: what we advertise per mota + how to load its ServeView on demand.
struct ServeEntry {
uint8_t mid[4];
uint32_t target_id, fw_version;
uint8_t codec_id, flags;
bool is_self; // true => entry is view0 (our own fw / RAM mota)
MotaSource* src; // else: load from this external source ...
uint8_t src_idx; // ... at this index
MotaDesc desc; // cached region offsets (source entries)
};
// Context for the source-payload reader trampoline (maps a payload-relative offset to a source read).
struct SrcReadCtx { MotaSource* src; uint8_t idx; uint32_t payload_off; };
void begin(uint32_t my_target_id, OtaSend send, void* ctx);
// --- serve --- Provide a complete, contiguous `.mota` to serve (caller keeps it alive).
bool serve(const uint8_t* mota, uint32_t len);
void announce(); // broadcast OTA_ADV for the served .mota
// Serve from a non-contiguous source (e.g. our own firmware in flash): a pre-assembled manifest
// (manifest-minus-leaves, `mfl` bytes), the pre-computed merkle `leaves` (kept alive by caller), and a
// `read` callback for payload blocks. Lets a node host its own image without holding it in RAM.
bool serve_self(const uint8_t* manifest, uint16_t mfl, const uint8_t* leaves, uint32_t block_count,
uint8_t* proof_scratch, uint32_t proof_scratch_sz, ServeReadFn read, void* ctx);
// Attach an external "folder" of `.mota` images (USB-serial daemon, BLE, WiFi URLs, NFS/samba, ...).
// The node then advertises + RELAYS them transparently alongside its own fw — peers just see more mOTAs.
// Re-enumerates the source into the serve registry. Returns false if no slots remain. (Trustless: the
// fetcher verifies merkle+signature, so the source is never trusted — see OtaSource.h.)
bool add_source(MotaSource* src);
// Re-read every attached source's catalog (call when the folder's contents change). Rebuilds entries
// [1..] from the sources; entry 0 (our own fw) is preserved.
void refresh_sources();
// Drop all external sources (keep serving our own fw).
void clear_sources();
uint8_t servedCount() const { return _n_serve; } // total mOTAs we offer (own fw + folder)
// Read-only view of one served entry (for `ota serve` listing): mid/target/fwver/codec/flags + is_self.
const ServeEntry* servedEntry(uint8_t i) const { return i < _n_serve ? &_serve[i] : nullptr; }
// Broadcast the tiny per-node BEACON (OTA_ADV): seeder_id + count + set-digest of everything we serve.
// Constant size regardless of how many mOTAs — peers ask for the catalog via OTA_QUERY only on interest.
void announce();
// --- fetch --- Provide the staging store; fetching starts on a matching OTA_ADV.
void set_fetch_store(OtaStore* s) { _fetch = s; }
// Resume a fetch from a container already persisted in the store (after a reboot). want_mid=nullptr
// accepts whatever is staged; otherwise only resumes if the staged manifest_id matches. Re-parses the
// stored manifest, recomputes geometry, counts present blocks, and continues FETCHING the holes (or goes
// straight to COMPLETE if all blocks are present). Returns true if it adopted a staged container.
bool resumeStaged(const uint8_t* want_mid);
// Manual cross-target override (decision: deliberate role switch, e.g. companion -> repeater on the
// same hardware). Normally a node only auto-fetches its OWN target_id; `want(T)` makes it accept an
// ADV for target T instead (T=0 restores auto). The user takes responsibility for HW compatibility;
// a hw_id brick-safety check is the planned safety layer (see docs/ota_protocol.md / plan).
void want(uint32_t target_id) { _desired_target = target_id; }
void want(uint32_t target_id) { _desired_target = target_id; reDiscover(); }
uint32_t wanted() const { return _desired_target; }
uint32_t target() const { return _target; } // this node's own OTA target_id (set in begin)
// Pull a SPECIFIC advertised mOTA by manifest_id (e.g. `ota pull <#>` picks the one more peers have),
// not just any firmware for the target. mid=nullptr clears the filter (accept any mid for the target).
void want_mid(const uint8_t* mid) {
if (mid) { for (int i = 0; i < 4; i++) _desired_mid[i] = mid[i]; _have_desired_mid = true; }
else _have_desired_mid = false;
reDiscover();
}
// Begin fetching a chosen mid now (sets want + starts the manifest fetch / resume). Used by `ota pull`
// once the user picks a catalogued mOTA (the source is reached via the flooded GET_MANIFEST).
void pull(const uint8_t* mid, uint32_t target) { want(target); want_mid(mid); startFetch(mid, target); }
// Ask every known source for its catalog (populates `ota neighbors`). Async — rows arrive via OTA_HAVE.
void queryAll();
// Coarse clock for source/catalog ages + LRU (the Mesh adapter feeds millis; 0 in host tests is fine).
void set_clock(uint32_t ms) { _now_ms = ms; }
// Codec compatibility: a node only fetches/accepts fw it can actually apply. CODEC_FULL is always
// acceptable; the platform's single delta codec is set here (ESP32 A/B -> sequential, nRF52 single-
// slot -> in-place). A mismatching `.mota` is rejected at OTA_ADV time, before fetching anything.
void set_apply_codec(uint8_t c) { _apply_codec = c; }
bool codecOk(uint8_t c) const { return c == CODEC_FULL || c == _apply_codec; }
// A platform may apply MORE than one delta codec (ESP32 does both sequential AND in-place, so a single
// in-place `.mota` can target both ESP32 and nRF52). 0xFF = unset.
void set_apply_codec2(uint8_t c) { _apply_codec2 = c; }
bool codecOk(uint8_t c) const { return c == CODEC_FULL || c == _apply_codec || c == _apply_codec2; }
// Auto-fetch policy (manual `ota pull` always works regardless): 0=off (discover only), 1=any
// compatible own-target advert, 2=only signed adverts. Conservative default = off.
static const uint8_t AUTOFETCH_OFF = 0, AUTOFETCH_ANY = 1, AUTOFETCH_SIGNED = 2;
void set_autofetch(uint8_t p) { _autofetch = p; reDiscover(); }
uint8_t autofetch() const { return _autofetch; }
// Resume checkpoint cadence (runtime-tunable, persisted in NodePrefs): persist progress every N
// committed blocks. 0 = never (resume only from a finalized container). Default OTA_CHECKPOINT_BLOCKS.
void set_checkpoint_blocks(uint16_t n) { _checkpoint_blocks = n; }
uint16_t checkpoint_blocks() const { return _checkpoint_blocks; }
bool fetched_is_signed() const { return (_fflags & MFLAG_SIGNED) != 0; } // flags of the fetched manifest
// This node's id (pubkey[0:4]), stamped into adverts we send so receivers can count distinct seeders.
void set_seeder_id(const uint8_t* id4) { if (id4) for (int i = 0; i < 4; i++) _seeder_id[i] = id4[i]; }
void on_message(const uint8_t* msg, uint16_t len); // feed one received OTA message
void loop(); // drive fetch (re-request missing blocks)
// Drop the current fetch session back to IDLE (so a fresh `ota pull` / advert starts a new one).
void reset_session() {
_fstate = IDLE; _have = 0; _req_count = 0;
_reasm_block = 0xFFFFFFFFu; _reasm_mask = 0; _reasm_need = 0; _awaiting_proof = false;
_loop_last_have = 0; _loop_last_mask = 0;
_mf_total = 0; _mf_mask = 0; _mf_len = 0;
}
FetchState fetchState() const { return _fstate; }
uint32_t blocksHave() const { return _have; }
uint32_t blocksTotal() const { return _fbc; }
const uint8_t* fetchManifestId() const { return _fid; }
// --- discovery catalog (for `ota neighbors`): mOTAs heard around us via OTA_HAVE, deduped by mid ---
struct CatRow {
uint8_t mid[4];
uint32_t target_id, fw_version;
uint8_t codec, flags;
uint8_t seeder0[4]; // first source that advertised this mid
uint8_t n_seeders; // distinct sources advertising it (saturates) — "N nodes have it"
uint32_t last_ms;
};
uint8_t catalogCount() const { return _n_cat; }
const CatRow* catalogRow(uint8_t i) const { return i < _n_cat ? &_catalog[i] : nullptr; }
uint8_t sourceCount() const { return _n_src; } // distinct OTA sources (beacon senders) heard
private:
void emit(const uint8_t* b, uint16_t n, bool flood) { if (_send && n) _send(_ctx, b, n, flood); }
void handleAdv(const uint8_t* m, uint16_t n);
void handleAdv(const uint8_t* m, uint16_t n); // beacon -> sources table (+ query if interested)
void handleQuery(const uint8_t* m, uint16_t n); // serve: reply OTA_HAVE catalog
void handleHave(const uint8_t* m, uint16_t n); // peer: catalog rows (+ startFetch if a row matches)
void handleGetManifest(const uint8_t* m, uint16_t n);
void handleManifest(const uint8_t* m, uint16_t n);
void handleReq(const uint8_t* m, uint16_t n);
void handleData(const uint8_t* m, uint16_t n);
void handleReqProof(const uint8_t* m, uint16_t n);
void handleProof(const uint8_t* m, uint16_t n);
void startFetch(const uint8_t* mid, uint32_t target); // begin/resume a fetch of a chosen mid
bool wantRow(const uint8_t* mid, uint32_t target, uint8_t codec, uint8_t flags) const; // fetch this row?
int serveEntryIndex(const uint8_t* mid) const; // registry slot serving this mid (-1 if none)
ServeView* resolve(const uint8_t* mid); // pick/load the ServeView for this mid (nullptr)
bool loadSource(const ServeEntry& e); // load an external mota into _srcv (head+leaves)
void registerSelfEntry(); // (re)build entry[0] from view0
static bool srcReadTramp(void* c, uint32_t off, uint8_t* buf, uint32_t len); // source payload reader
void sendQuery(const uint8_t* seeder, const uint8_t* digest, uint32_t filter_target); // ask a source for its catalog
void scheduleQuery(const uint8_t* seeder, const uint8_t* digest); // jittered + suppressible
void reDiscover() { for (uint8_t i = 0; i < _n_src; i++) _sources[i].have_catalog = false; _pq_active = false; }
void setDigest(uint8_t out[4]) const; // sha2-256:4 over our served mids
bool blockPresent(uint32_t i) const;
void requestMissing();
uint32_t blockLen(uint32_t i) const;
@@ -85,12 +248,18 @@ private:
OtaSend _send = nullptr;
void* _ctx = nullptr;
// serve
bool _has_serve = false;
const uint8_t* _serve_buf = nullptr;
uint32_t _serve_len = 0;
MotaManifest _sm;
uint8_t _scratch[OTA_PROOFGEN_SCRATCH];
// serve (multi-mota): view0 = our own fw / a RAM `.mota`; _srcv = the currently-loaded external mota.
ServeView _view0;
ServeView _srcv;
uint8_t _srcv_mid[4] = {0};
SrcReadCtx _srcv_rdctx = {nullptr, 0, 0};
ServeEntry _serve[OTA_MAX_SERVE]; // catalog (what we advertise) — entry 0 is view0
uint8_t _n_serve = 0;
MotaSource* _src_list[OTA_MAX_SOURCE_OBJ] = {nullptr};
uint8_t _n_src_obj = 0;
uint8_t _src_manifest[OTA_SRC_MANIFEST_MAX]; // manifest-minus-leaves of the loaded source mota
uint8_t _src_leaves[OTA_PROOFGEN_SCRATCH]; // leaves[] of the loaded source mota (<=1024 blocks)
uint8_t _scratch[OTA_PROOFGEN_SCRATCH]; // proof-gen / fetch root-check working buffer
// fetch
OtaStore* _fetch = nullptr;
@@ -99,10 +268,42 @@ private:
uint8_t _froot[4] = {0};
uint32_t _ftotal = 0, _fpoff = 0, _floff = 0, _fpsize = 0, _fbc = 0, _fbs = 0;
uint32_t _have = 0;
uint32_t _req_start = 0, _req_count = 0; // current outstanding request window
uint32_t _req_start = 0, _req_count = 0; // last block requested (per-block serial flow; telemetry)
uint32_t _loop_last_have = 0; // for stall detection in loop()
uint32_t _desired_target = 0; // manual cross-target override (0 = auto / own target)
uint8_t _desired_mid[4] = {0,0,0,0}; // pull a specific manifest_id (see want_mid)
bool _have_desired_mid = false;
uint8_t _apply_codec = CODEC_DETOOLS_SEQUENTIAL; // platform's delta codec (OtaContext sets it)
uint8_t _apply_codec2 = 0xFF; // optional 2nd accepted delta codec (ESP32: in-place)
uint8_t _seeder_id[4] = {0,0,0,0}; // our node id (pubkey[0:4]) for advert seeder counting
uint8_t _autofetch = AUTOFETCH_OFF; // auto-fetch policy (persisted in NodePrefs)
uint16_t _checkpoint_blocks = OTA_CHECKPOINT_BLOCKS; // resume checkpoint cadence (persisted)
uint8_t _fflags = 0; // flags of the manifest currently being fetched
// multi-fragment reassembly of the current block (per-block 2-phase: fetch data, then its proof)
uint32_t _reasm_block = 0xFFFFFFFFu; // block being reassembled / awaiting proof (none)
uint16_t _reasm_mask = 0; // received FRAG_DATA-slice bitmap (bit k = slice @ k*FRAG_DATA)
uint16_t _reasm_need = 0; // full mask once all slices of the current block are in
bool _awaiting_proof = false; // data complete; REQ_PROOF sent, verify on PROOF
uint16_t _loop_last_mask = 0; // fragment-level stall detection in loop()
uint8_t _reasm_buf[OTA_MAX_BLOCK];
// multi-fragment manifest reassembly (a signed v2 manifest exceeds one packet)
uint8_t _mf_buf[256];
uint8_t _mf_total = 0; // frag_total of the manifest being reassembled (0 = none)
uint16_t _mf_mask = 0; // received manifest-fragment bitmap
uint32_t _mf_len = 0; // assembled manifest length (set by the last fragment)
// discovery: heard sources (beacon senders) + the catalog assembled from their OTA_HAVE replies
struct Source { uint8_t seeder[4]; uint8_t digest[4]; uint8_t n_motas; uint32_t last_ms; bool have_catalog; };
Source _sources[OTA_MAX_SOURCES];
uint8_t _n_src = 0;
CatRow _catalog[OTA_MAX_CATALOG];
uint8_t _n_cat = 0;
uint32_t _now_ms = 0; // coarse clock (fed by set_clock; for ages/LRU/jitter)
// pending catalog query (jittered + suppressed on overhearing a matching QUERY/HAVE — anti-storm)
bool _pq_active = false;
uint8_t _pq_seeder[4] = {0};
uint8_t _pq_digest[4] = {0};
uint32_t _pq_at = 0; // _now_ms deadline to actually send the query
};
} // namespace ota
src/helpers/ota/OtaProtocol.cpp
+56 -12
View File
@@ -25,16 +25,41 @@ struct R {
};
} // namespace
uint16_t encode_adv(uint8_t* buf, uint16_t cap, const AdvMsg& m) {
W w(buf, cap); w.u8(OTA_ADV); w.u32(m.target_id); w.u32(m.fw_version);
w.raw(m.manifest_id, 4); w.u8(m.flags); w.u8(m.have_all); w.u8(m.codec_id);
uint16_t encode_adv(uint8_t* buf, uint16_t cap, const AdvMsg& m) { // tiny per-node beacon
W w(buf, cap); w.u8(OTA_ADV); w.raw(m.seeder_id, 4); w.u8(m.n_motas); w.raw(m.set_digest, 4);
return w.ok ? w.n : 0;
}
bool decode_adv(const uint8_t* buf, uint16_t len, AdvMsg& m) {
R r(buf, len); if (r.u8() != OTA_ADV) return false;
m.target_id = r.u32(); m.fw_version = r.u32();
const uint8_t* id = r.raw(4); if (id) memcpy(m.manifest_id, id, 4);
m.flags = r.u8(); m.have_all = r.u8(); m.codec_id = r.u8();
const uint8_t* sid = r.raw(4); if (sid) memcpy(m.seeder_id, sid, 4);
m.n_motas = r.u8();
const uint8_t* d = r.raw(4); if (d) memcpy(m.set_digest, d, 4);
return r.ok;
}
uint16_t encode_query(uint8_t* buf, uint16_t cap, const QueryMsg& m) {
W w(buf, cap); w.u8(OTA_QUERY); w.raw(m.seeder_id, 4); w.raw(m.set_digest, 4); w.u32(m.filter_target);
return w.ok ? w.n : 0;
}
bool decode_query(const uint8_t* buf, uint16_t len, QueryMsg& m) {
R r(buf, len); if (r.u8() != OTA_QUERY) return false;
const uint8_t* sid = r.raw(4); if (sid) memcpy(m.seeder_id, sid, 4);
const uint8_t* dg = r.raw(4); if (dg) memcpy(m.set_digest, dg, 4);
m.filter_target = r.u32();
return r.ok;
}
uint16_t encode_have(uint8_t* buf, uint16_t cap, const HaveMsg& m) {
W w(buf, cap); w.u8(OTA_HAVE); w.raw(m.seeder_id, 4); w.raw(m.set_digest, 4);
w.u8(m.frag_idx); w.u8(m.frag_total); w.u8(m.n_rows); w.raw(m.rows, (uint16_t)m.n_rows * OTA_HAVE_ROW_BYTES);
return w.ok ? w.n : 0;
}
bool decode_have(const uint8_t* buf, uint16_t len, HaveMsg& m) {
R r(buf, len); if (r.u8() != OTA_HAVE) return false;
const uint8_t* sid = r.raw(4); if (sid) memcpy(m.seeder_id, sid, 4);
const uint8_t* dg = r.raw(4); if (dg) memcpy(m.set_digest, dg, 4);
m.frag_idx = r.u8(); m.frag_total = r.u8(); m.n_rows = r.u8();
m.rows = r.raw((uint16_t)m.n_rows * OTA_HAVE_ROW_BYTES);
return r.ok;
}
@@ -73,21 +98,40 @@ bool decode_req(const uint8_t* buf, uint16_t len, ReqMsg& m) {
}
uint16_t encode_data(uint8_t* buf, uint16_t cap, const DataMsg& m) {
W w(buf, cap); w.u8(OTA_DATA); w.raw(m.manifest_id, 4); w.u16(m.block_idx);
w.u8(m.frag_idx); w.u8(m.frag_total);
if (m.frag_idx == 0) { w.u8(m.n_proof); w.raw(m.proof, (uint16_t)m.n_proof * 4); }
W w(buf, cap); w.u8(OTA_DATA); w.raw(m.manifest_id, 4); w.u16(m.block_idx); w.u16(m.frag_off);
w.raw(m.data, m.data_len);
return w.ok ? w.n : 0;
}
bool decode_data(const uint8_t* buf, uint16_t len, DataMsg& m) {
R r(buf, len); if (r.u8() != OTA_DATA) return false;
const uint8_t* id = r.raw(4); if (id) memcpy(m.manifest_id, id, 4);
m.block_idx = r.u16(); m.frag_idx = r.u8(); m.frag_total = r.u8();
m.n_proof = 0; m.proof = nullptr;
if (m.frag_idx == 0) { m.n_proof = r.u8(); m.proof = r.raw((uint16_t)m.n_proof * 4); }
m.block_idx = r.u16(); m.frag_off = r.u16();
m.data_len = r.remaining(); m.data = r.raw(m.data_len);
return r.ok;
}
uint16_t encode_req_proof(uint8_t* buf, uint16_t cap, const ReqProofMsg& m) {
W w(buf, cap); w.u8(OTA_REQ_PROOF); w.raw(m.manifest_id, 4); w.u16(m.block_idx);
return w.ok ? w.n : 0;
}
bool decode_req_proof(const uint8_t* buf, uint16_t len, ReqProofMsg& m) {
R r(buf, len); if (r.u8() != OTA_REQ_PROOF) return false;
const uint8_t* id = r.raw(4); if (id) memcpy(m.manifest_id, id, 4);
m.block_idx = r.u16();
return r.ok;
}
uint16_t encode_proof(uint8_t* buf, uint16_t cap, const ProofMsg& m) {
W w(buf, cap); w.u8(OTA_PROOF); w.raw(m.manifest_id, 4); w.u16(m.block_idx);
w.u8(m.n_proof); w.raw(m.proof, (uint16_t)m.n_proof * 4);
return w.ok ? w.n : 0;
}
bool decode_proof(const uint8_t* buf, uint16_t len, ProofMsg& m) {
R r(buf, len); if (r.u8() != OTA_PROOF) return false;
const uint8_t* id = r.raw(4); if (id) memcpy(m.manifest_id, id, 4);
m.block_idx = r.u16(); m.n_proof = r.u8(); m.proof = r.raw((uint16_t)m.n_proof * 4);
return r.ok;
}
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaProtocol.h
+52 -13
View File
@@ -12,17 +12,38 @@
namespace mesh {
namespace ota {
// ---- OTA_ADV: "I have (part of) fw X for target T" (flood, periodic + on demand) ----
// ---- OTA_ADV: tiny per-NODE beacon (flood, periodic). CONSTANT size regardless of how many mOTAs a
// node serves — it just says "I'm a source, here's how many + a digest of my set". A peer that's
// interested asks for the catalog via OTA_QUERY. (Replaces the old per-mOTA advert so a folder node with
// N images costs one 10-byte beacon, not N adverts.) ----
struct AdvMsg {
uint32_t target_id;
uint32_t fw_version;
uint8_t manifest_id[4]; // = merkle_root
uint8_t flags; // manifest flags (FULL/SIGNED)
uint8_t have_all; // 1 = holder has the complete payload
uint8_t codec_id; // manifest codec (0=full,1=detools-seq,2=detools-inplace) — lets a
// receiver reject fw it can't apply before fetching anything
uint8_t seeder_id[4]; // advertiser's node id = pubkey[0:4]; the QUERY address + distinct-source id
uint8_t n_motas; // # of complete, servable mOTAs (saturates at 255)
uint8_t set_digest[4]; // sha2-256:4 over the sorted set of served mids; "did my offering change?"
};
// ---- OTA_QUERY: "list what you serve" — addressed to a source by seeder_id, FLOODED so neighbours
// overhear it (storm suppression). set_digest identifies the offering being asked about (so an overhearer
// can suppress its own pending query for the same {source,digest}). filter_target=0 = everything. ----
struct QueryMsg {
uint8_t seeder_id[4]; // which source this query is for (the source matches its own id)
uint8_t set_digest[4]; // the offering digest we're asking about (for overhear-suppression)
uint32_t filter_target; // 0 = all (the scalable default); else only mOTAs for this target_id
};
// ---- OTA_HAVE: the compact catalog (source -> mesh), FLOODED + tagged with set_digest so EVERY node
// that overhears it caches the rows (passive, no query needed). Fragmented. ----
// body: seeder_id(4) set_digest(4) frag_idx(1) frag_total(1) n_rows(1) rows[ mid(4) target(4) fwver(4) codec(1) flags(1) ]
struct HaveRow { uint8_t mid[4]; uint32_t target_id; uint32_t fw_version; uint8_t codec_id; uint8_t flags; };
struct HaveMsg {
uint8_t seeder_id[4];
uint8_t set_digest[4]; // the offering this catalog describes (overhearers cache by it)
uint8_t frag_idx, frag_total;
uint8_t n_rows; // rows in THIS fragment
const uint8_t* rows; // points into buf: n_rows * OTA_HAVE_ROW_BYTES
};
static const uint8_t OTA_HAVE_ROW_BYTES = 14; // mid4 + target4 + fwver4 + codec1 + flags1
// ---- OTA_GET_MANIFEST: request the manifest for a content id (direct) ----
struct GetManifestMsg { uint8_t manifest_id[4]; };
@@ -37,23 +58,35 @@ struct ManifestMsg {
// ---- OTA_REQ: request a window of blocks (direct) ----
struct ReqMsg { uint8_t manifest_id[4]; uint16_t start_block; uint8_t count; };
// ---- OTA_DATA: one (fragment of a) block (direct) ----
// body: manifest_id(4) block_idx(2) frag_idx(1) frag_total(1) [frag0: n_proof(1) proof(n_proof*4)] data[]
// ---- OTA_DATA: one self-describing fragment of a block's data (proof is fetched separately) ----
// body: manifest_id(4) block_idx(2) frag_off(2) data[]
// `frag_off` is the byte offset of `data` within block `block_idx` (global position = block_idx*block_size
// + frag_off), so a fragment is self-placing and can be requested from ANY peer (BitTorrent-style).
struct DataMsg {
uint8_t manifest_id[4];
uint16_t block_idx;
uint8_t frag_idx, frag_total;
uint8_t n_proof; // only meaningful on frag_idx==0
const uint8_t* proof; // n_proof*4 bytes (frag0 only)
uint16_t frag_off;
const uint8_t* data; uint16_t data_len;
};
// ---- OTA_REQ_PROOF: request the merkle proof for one (reassembled) block (direct) ----
struct ReqProofMsg { uint8_t manifest_id[4]; uint16_t block_idx; };
// ---- OTA_PROOF: the merkle proof (ordered sibling digests) for one block (direct) ----
struct ProofMsg { uint8_t manifest_id[4]; uint16_t block_idx; uint8_t n_proof; const uint8_t* proof; };
// Each encode_* returns the total payload length (incl. the leading msg-type byte), 0 on overflow.
// Each decode_* returns true on success (and points struct fields into `buf`).
uint16_t encode_adv(uint8_t* buf, uint16_t cap, const AdvMsg& m);
bool decode_adv(const uint8_t* buf, uint16_t len, AdvMsg& m);
uint16_t encode_query(uint8_t* buf, uint16_t cap, const QueryMsg& m);
bool decode_query(const uint8_t* buf, uint16_t len, QueryMsg& m);
uint16_t encode_have(uint8_t* buf, uint16_t cap, const HaveMsg& m);
bool decode_have(const uint8_t* buf, uint16_t len, HaveMsg& m);
uint16_t encode_get_manifest(uint8_t* buf, uint16_t cap, const GetManifestMsg& m);
bool decode_get_manifest(const uint8_t* buf, uint16_t len, GetManifestMsg& m);
@@ -66,6 +99,12 @@ bool decode_req(const uint8_t* buf, uint16_t len, ReqMsg& m);
uint16_t encode_data(uint8_t* buf, uint16_t cap, const DataMsg& m);
bool decode_data(const uint8_t* buf, uint16_t len, DataMsg& m);
uint16_t encode_req_proof(uint8_t* buf, uint16_t cap, const ReqProofMsg& m);
bool decode_req_proof(const uint8_t* buf, uint16_t len, ReqProofMsg& m);
uint16_t encode_proof(uint8_t* buf, uint16_t cap, const ProofMsg& m);
bool decode_proof(const uint8_t* buf, uint16_t len, ProofMsg& m);
inline uint8_t ota_msg_type(const uint8_t* buf, uint16_t len) { return len ? buf[0] : 0xFF; }
} // namespace ota
src/helpers/ota/OtaSelf.cpp
+84
View File
@@ -9,6 +9,16 @@
#include "OtaFlashLayout_nrf52.h"
#endif
#if defined(ESP32_PLATFORM) || defined(NRF52_PLATFORM)
#include "OtaContext.h" // serve our own fw from flash (cache leaves, read payload on demand)
#include "MerkleTree.h"
#include <SHA256.h>
#include <stdlib.h>
#ifndef OTA_SELF_LEAVES_MAX
#define OTA_SELF_LEAVES_MAX 65536u // cap heap for cached leaves (~16k blocks @1 KB = up to ~16 MB image)
#endif
#endif
namespace mesh {
namespace ota {
@@ -60,5 +70,79 @@ bool ota_self_firmware(SelfFwInfo& out) {
}
#endif
#if defined(ESP32_PLATFORM)
bool ota_self_read(uint32_t off, uint8_t* buf, uint32_t len) {
const esp_partition_t* p = esp_ota_get_running_partition();
return p && esp_partition_read(p, off, buf, len) == ESP_OK;
}
#elif defined(NRF52_PLATFORM)
bool ota_self_read(uint32_t off, uint8_t* buf, uint32_t len) {
if ((uint64_t)MOTA_NRF52_APP_BASE + off + len > MOTA_NRF52_FS_START) return false;
memcpy(buf, (const uint8_t*)(uintptr_t)(MOTA_NRF52_APP_BASE + off), len);
return true;
}
#else
bool ota_self_read(uint32_t, uint8_t*, uint32_t) { return false; }
#endif
#if defined(ESP32_PLATFORM) || defined(NRF52_PLATFORM)
static bool self_read_cb(void* ctx, uint32_t off, uint8_t* buf, uint32_t len) {
(void)ctx; return ota_self_read(off, buf, len);
}
static void wr_u32le(uint8_t* p, uint32_t v) {
p[0] = (uint8_t)v; p[1] = (uint8_t)(v >> 8); p[2] = (uint8_t)(v >> 16); p[3] = (uint8_t)(v >> 24);
}
// Build (once) the full-image manifest + merkle leaves for the running firmware, cache them in `c`, and
// hand the manager a flash-read callback for the payload. The image is read ONCE here to compute the
// leaves + image_hash; thereafter a block REQ reads only that block (proof comes from the cached leaves).
bool ota_serve_self(OtaContext& c, uint32_t fw_version) {
SelfFwInfo fi;
if (!ota_self_firmware(fi) || !fi.valid) return false;
// 1 KB logical blocks (delivered as multiple LoRa fragments): 8x fewer merkle leaves than 128 B, so a
// ~530 KB image is ~518 blocks (proof-gen scratch ~2 KB) instead of ~4150 (which overflowed the scratch).
const uint32_t image_size = fi.image_len, BS = OTA_DEFAULT_BLOCK_SIZE;
const uint32_t bc = (image_size + BS - 1) / BS;
if ((uint64_t)bc * 4 > OTA_SELF_LEAVES_MAX) return false;
free(c.serve_self_leaves); free(c.serve_self_proof);
c.serve_self_leaves = (uint8_t*)malloc((size_t)bc * 4);
c.serve_self_proof = (uint8_t*)malloc((size_t)bc * 4); // proof-gen working buffer (sized to OUR image)
if (!c.serve_self_leaves || !c.serve_self_proof) {
free(c.serve_self_leaves); free(c.serve_self_proof);
c.serve_self_leaves = c.serve_self_proof = nullptr;
return false;
}
SHA256 sha; uint8_t blk[BS];
for (uint32_t i = 0, off = 0; i < bc; i++, off += BS) {
uint32_t blen = (off + BS <= image_size) ? BS : (image_size - off);
if (!ota_self_read(off, blk, blen)) {
free(c.serve_self_leaves); free(c.serve_self_proof);
c.serve_self_leaves = c.serve_self_proof = nullptr;
return false;
}
merkle_leaf(c.serve_self_leaves + (size_t)i * 4, blk, blen);
sha.update(blk, blen);
}
uint8_t image_hash[32]; sha.finalize(image_hash, 32);
uint8_t root[4]; merkle_root(root, c.serve_self_leaves, bc);
uint8_t* m = c.serve_self_manifest; // assemble v2 manifest-minus-leaves (full, unsigned) = 93 bytes
memset(m, 0, 96);
m[0] = MOTA_FORMAT_VER; m[1] = MFLAG_FULL; m[2] = HASH_ALGO_SHA256;
wr_u32le(m + 3, c.manager.target()); wr_u32le(m + 7, fw_version);
wr_u32le(m + 11, image_size); wr_u32le(m + 15, image_size); // full: payload == image
m[19] = 10; // block_size_log2 = 10 (1024 B logical block)
memcpy(m + 20, root, 4);
memcpy(m + 24, image_hash, 32);
m[56] = CODEC_FULL;
memcpy(m + 57, c.hw_id, strlen(c.hw_id) < 32 ? strlen(c.hw_id) : 32); // hw_id[32] (NUL-padded by memset)
memcpy(m + 89, APPROVAL_NOT, 4); // approval marker (fetching device's apply-gate handles it)
return c.manager.serve_self(m, 93, c.serve_self_leaves, bc,
c.serve_self_proof, (size_t)bc * 4, self_read_cb, nullptr);
}
#endif
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaSelf.h
+10
View File
@@ -14,5 +14,15 @@ namespace ota {
// platform or no valid EndF is present (e.g. firmware built without the EndF build hook).
bool ota_self_firmware(SelfFwInfo& out);
// Read `len` bytes of the running firmware image at offset `off` (ESP32: running partition via
// esp_partition_read; nRF52: memory-mapped app region). false on unsupported platforms.
bool ota_self_read(uint32_t off, uint8_t* buf, uint32_t len);
// Compute (once) + cache our running firmware's manifest + merkle leaves in `c`, then serve it from
// flash as a full `.mota` (payload read on demand per block; only metadata held in RAM). Returns false
// if no EndF / image too big / OOM. Device platforms only.
struct OtaContext;
bool ota_serve_self(OtaContext& c, uint32_t fw_version); // target = this node's own (c.manager.target())
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaSource.h
+55
View File
@@ -0,0 +1,55 @@
#pragma once
#include <stdint.h>
#include <stddef.h>
#include "OtaFormat.h"
// Transport-agnostic "folder of firmware" abstraction (docs/ota_protocol.md §9). A node can RELAY
// `.mota` images it does not hold in flash — a user drops several `.mota` (different architectures) into
// some external store and the node advertises + serves them as if it held them. Peers just see "this node
// has N mOTAs"; the node knows they are external. The store is reached through a MotaSource, so the SAME
// serve code drives a USB-serial host daemon, BLE, a WiFi URL list, an NFS/samba mount, ... — only the
// `read()` plumbing differs per transport.
//
// The relay is TRUSTLESS: the fetcher verifies every block against the signed merkle root, so a source is
// never trusted. A wrong descriptor or wrong bytes simply makes the fetch fail its merkle/signature check
// — a malicious or buggy source cannot forge firmware, only deny it.
namespace mesh {
namespace ota {
// A parsed top-level descriptor of one `.mota` a source provides: enough to advertise it in the catalog
// AND to locate every region for serving, WITHOUT holding the whole image in RAM. Offsets are absolute
// byte positions within the `.mota` container (which always begins MAGIC(4) total(4) manifest...).
struct MotaDesc {
uint8_t mid[4] = {0}; // merkle_root (the content id peers fetch by)
uint32_t target_id = 0;
uint32_t fw_version = 0;
uint8_t codec_id = 0;
uint8_t flags = 0;
uint32_t total_size = 0; // full `.mota` length (bytes)
uint32_t leaves_off = 0; // byte offset of the merkle leaves[] (manifest-minus-leaves = [8, leaves_off))
uint32_t block_count = 0; // == number of leaves (== number of payload blocks)
uint32_t payload_off = 0; // byte offset of the payload
uint32_t payload_size = 0;
};
// One or more complete `.mota` images, reachable as random-access bytes. Implementations are device-side
// and transport-specific (the engine in OtaManager is portable and never includes this directly for I/O;
// it only calls through the interface).
class MotaSource {
public:
virtual ~MotaSource() {}
// Number of complete, servable mOTAs this source currently offers (may change as the folder changes;
// the manager re-enumerates on add_source / refresh).
virtual uint8_t count() = 0;
// Cheap metadata + region offsets for mota `idx`. False if idx is out of range or unparsable.
virtual bool describe(uint8_t idx, MotaDesc& out) = 0;
// Random-access read of `len` bytes at absolute offset `off` of mota `idx` into `buf`. Returns true iff
// exactly `len` bytes were produced. May block on the transport (serial round-trip); OTA is lowest
// priority so latency here is acceptable.
virtual bool read(uint8_t idx, uint32_t off, uint8_t* buf, uint32_t len) = 0;
};
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaStore.h
+35 -1
View File
@@ -3,6 +3,7 @@
#include <stdint.h>
#include <stddef.h>
#include <string.h>
#include "OtaFormat.h" // MOTA_MAGIC (resume: detect a persisted partial container)
// Staging backend for an in-transit `.mota` (docs/ota_protocol.md §7). Blocks may arrive out of order
// and progress must survive reboots, so the store is random-access. The transfer/verify logic is
@@ -34,13 +35,37 @@ public:
// reaches COMPLETE (radio idle), so a flash store does its page writes off the RX critical path.
// After this returns, a flash store's data() view is coherent. No-op for purely in-RAM stores.
virtual void finalize() {}
// Optional: persist in-progress state (the metadata/leaf-progress page + any open payload buffer) so a
// reboot mid-transfer can resume. Called by OtaManager every OTA_CHECKPOINT_BLOCKS committed blocks.
// A flash store flushes its pinned meta page + the open payload page (consistency: payload-before-leaves)
// so every block whose leaf is persisted also has its payload in flash. No-op for RAM stores. Infrequent
// (at LoRa block rates, ~once per many minutes) so the extra erases don't matter.
virtual void checkpoint() {}
// Optional: re-attach to a container ALREADY persisted in the backing store (after a reboot), WITHOUT
// erasing. Returns true if a syntactically valid container (MOTA_MAGIC header + plausible total) is
// present and the store is now set up to read/continue-writing it; false if none (caller starts fresh).
// OtaManager then reads + parses the stored manifest to recompute geometry and resume the fetch.
virtual bool reopen() { return false; }
// Optional: declare the container's logical layout once the manifest is parsed, BEFORE begin(), so a
// store backed by a single spare A/B partition (ESP32) can choose placement and reject an unfittable
// fetch up front. A FULL image's payload IS the final firmware (no decode), so it can stream straight
// to the inactive slot's offset 0 while the small meta/leaves/trailer persist elsewhere; a delta's
// whole container is staged together (the decoder reads the patch from it at apply). image_size is the
// reconstructed image; [payload_off, payload_off+payload_size) is the payload region in the container.
// Return false if it cannot fit the backing store (the transfer is then refused before any block).
virtual bool plan_layout(bool is_full, uint32_t image_size, uint32_t payload_off, uint32_t payload_size) {
(void)is_full; (void)image_size; (void)payload_off; (void)payload_size; return true;
}
};
// Fixed-capacity RAM store — for native tests and device bring-up of the transfer/verify path.
// (Does NOT survive reboot; a persistent flash store replaces it for production — see D1.)
template <uint32_t CAP>
class OtaStoreRam : public OtaStore {
uint8_t _buf[CAP];
uint8_t _buf[CAP] = {}; // zero-init so a never-written store's reopen() finds no MOTA_MAGIC
uint32_t _total = 0;
public:
bool begin(uint32_t total_size) override {
@@ -62,6 +87,15 @@ public:
uint32_t capacity() const override { return CAP; }
uint32_t staged_size() const override { return _total; }
void clear() override { _total = 0; }
// RAM doesn't survive a real reboot, but the buffer persists within a process — enough to exercise the
// manager's resume path in native tests. Recover `total` from the stored header so read() bounds work.
bool reopen() override {
if (memcmp(_buf, MOTA_MAGIC, 4) != 0) return false;
uint32_t t = (uint32_t)_buf[4] | ((uint32_t)_buf[5] << 8) | ((uint32_t)_buf[6] << 16) | ((uint32_t)_buf[7] << 24);
if (t < 13 || t > CAP) return false;
_total = t;
return true;
}
const uint8_t* data() const { return _buf; } // contiguous view (RAM store only)
};
src/helpers/ota/OtaStoreFlashEsp32.cpp
+247
View File
@@ -0,0 +1,247 @@
#include "OtaStoreFlashEsp32.h"
#if defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
#include "OtaDebug.h"
#include "MotaContainer.h" // mota_parse_manifest (reopen: rebuild geometry from the staged manifest)
#include <string.h>
#include <stdlib.h> // malloc/free (the meta buffer is sized per fetch)
#include "esp_ota_ops.h" // esp_ota_get_next_update_partition (the inactive A/B slot)
namespace mesh {
namespace ota {
static inline uint32_t round_up_sec(uint32_t x) {
const uint32_t S = 4096;
return (x + S - 1) & ~(S - 1);
}
OtaStoreFlashEsp32::~OtaStoreFlashEsp32() { free(_meta); }
bool OtaStoreFlashEsp32::acquire() {
if (!_part) {
_part = esp_ota_get_next_update_partition(nullptr);
_psize = _part ? _part->size : 0;
}
return _part != nullptr;
}
// Compute the slot placement from _full/_image_size/_meta_bytes/_pay_size (already set). Returns false if
// it won't fit. Shared by plan_layout (fresh fetch) and reopen (resume) so both derive identical geometry.
bool OtaStoreFlashEsp32::layout() {
uint32_t total = _meta_bytes + _pay_size + 5;
if (_full) {
// payload streams to slot offset 0 (it IS the image); header+manifest+leaves+trailer persist at the
// bottom so the container survives a reboot (resume / re-serve).
_meta_span = _meta_bytes; // routing boundary: [0,meta) -> RAM meta buffer
_meta_flush = round_up_sec(_meta_bytes + 5); // meta + 5-byte trailer, whole sectors
if (_meta_flush > OTA_ESP32_META_CAP) return false;
uint32_t bottom = (_psize - _meta_flush) & ~(SEC - 1);
_meta_part = bottom;
_pay_log0 = _meta_bytes; _pay_part0 = 0;
_write_start = 0;
if (_image_size > bottom) return false; // image would overrun the bottom meta region
} else {
// whole container staged bottom-aligned; the decoded image fills the slot from offset 0.
_meta_span = round_up_sec(_meta_bytes); // pin whole sectors covering meta (+ spillover payload)
_meta_flush = _meta_span;
if (_meta_flush > OTA_ESP32_META_CAP) return false;
if (total > _psize) return false;
_write_start = (_psize - total) & ~(SEC - 1);
_meta_part = _write_start;
_pay_log0 = _meta_span; _pay_part0 = _write_start + _meta_span;
if (_image_size > _write_start) return false; // decoded output would overlap the staged container
}
_total = total;
return true;
}
// Choose placement from the parsed manifest and refuse anything that won't fit, BEFORE any block is
// staged. (See the header for the delta-vs-full layout rationale.)
bool OtaStoreFlashEsp32::plan_layout(bool is_full, uint32_t image_size, uint32_t payload_off, uint32_t payload_size) {
if (!acquire()) return false;
_full = is_full; _image_size = image_size; _meta_bytes = payload_off; _pay_size = payload_size;
bool ok = layout();
OTA_DBG("OTA esp32: plan %s total=%u image=%u meta=%u write_start=%u meta_part=%u ok=%d\n",
is_full ? "FULL" : "DELTA", (unsigned)_total, (unsigned)image_size, (unsigned)_meta_bytes,
(unsigned)_write_start, (unsigned)_meta_part, (int)ok);
return ok;
}
bool OtaStoreFlashEsp32::begin(uint32_t total_size) {
if (!_part || _total == 0 || total_size != _total) return false; // plan_layout must have run + agreed
free(_meta);
_meta = (uint8_t*)malloc(_meta_flush); // header+manifest+leaves(+full trailer)
if (!_meta) { _total = 0; return false; }
memset(_meta, 0xFF, _meta_flush);
memset(_trailer, 0xFF, sizeof(_trailer));
_pay_open = false; _pay_sec = 0; _pay_max_sec = 0; _flushed = false;
_io_ok = true;
return true;
}
void OtaStoreFlashEsp32::clear() {
free(_meta); _meta = nullptr;
_total = 0; _pay_open = false; _flushed = false; // _part kept (re-acquire is fine)
}
uint8_t* OtaStoreFlashEsp32::meta_slot(uint32_t L) {
if (in_trailer(L)) return _full ? (_meta + _meta_bytes + (L - (_total - 5)))
: (_trailer + (L - (_total - 5)));
if (L < _meta_span) return _meta + L;
return nullptr; // payload -> sliding sector / flash
}
const uint8_t* OtaStoreFlashEsp32::meta_slot_c(uint32_t L) const {
if (in_trailer(L)) return _full ? (_meta + _meta_bytes + (L - (_total - 5)))
: (_trailer + (L - (_total - 5)));
if (L < _meta_span) return _meta + L;
return nullptr;
}
// Bytes from `pos` that stay in one region, and (for payload) one flash sector.
uint32_t OtaStoreFlashEsp32::run(uint32_t pos, uint32_t remain) const {
if (pos >= _total - 5) return remain; // trailer (<=5, one buffer)
if (pos < _meta_span) { uint32_t c = _meta_span - pos; return remain < c ? remain : c; }
uint32_t poff = pay_part(pos);
uint32_t to_sec = SEC - (poff % SEC);
uint32_t to_end = (_total - 5) - pos; // don't cross into the trailer
uint32_t c = remain;
if (to_sec < c) c = to_sec;
if (to_end < c) c = to_end;
return c;
}
void OtaStoreFlashEsp32::flush_sector(uint32_t slot_off, const uint8_t* buf, uint32_t n) {
if (!_io_ok) return;
if (esp_partition_erase_range(_part, slot_off, n) != ESP_OK) { _io_ok = false; return; }
if (esp_partition_write(_part, slot_off, buf, n) != ESP_OK) { _io_ok = false; return; }
OTA_DBG("OTA esp32: flushed %u B @ slot+%u\n", (unsigned)n, (unsigned)slot_off);
}
void OtaStoreFlashEsp32::flush_pay() {
if (_pay_open) { flush_sector((uint32_t)_pay_sec * SEC, _pay, SEC); _pay_open = false; }
}
void OtaStoreFlashEsp32::open_pay(uint32_t sec) {
if (_pay_open) flush_pay();
if (sec < _pay_max_sec) {
// revisiting an already-flushed sector (out-of-order block) -> read it back so the gaps we don't
// touch are preserved (they were programmed as 0xFF or earlier block data); we erase+reprogram on flush.
if (esp_partition_read(_part, (size_t)sec * SEC, _pay, SEC) != ESP_OK) _io_ok = false;
} else {
memset(_pay, 0xFF, SEC); // fresh sector
_pay_max_sec = sec;
}
_pay_sec = sec; _pay_open = true;
}
bool OtaStoreFlashEsp32::write(uint32_t offset, const uint8_t* d, uint32_t len) {
if ((uint64_t)offset + len > _total || !_io_ok) return false;
for (uint32_t pos = offset, end = offset + len; pos < end; ) {
uint32_t n = run(pos, end - pos);
if (uint8_t* dst = meta_slot(pos)) { // meta / leaves / trailer -> pinned RAM
memcpy(dst, d, n);
} else { // payload -> sliding sector
uint32_t poff = pay_part(pos);
uint32_t sec = poff / SEC;
if (!_pay_open || sec != _pay_sec) open_pay(sec);
memcpy(_pay + (poff % SEC), d, n);
}
pos += n; d += n;
if (!_io_ok) return false;
}
return true;
}
bool OtaStoreFlashEsp32::read(uint32_t offset, uint8_t* buf, uint32_t len) const {
if ((uint64_t)offset + len > _total) return false;
for (uint32_t pos = offset, end = offset + len; pos < end; ) {
uint32_t n = run(pos, end - pos);
if (const uint8_t* src = meta_slot_c(pos)) {
memcpy(buf, src, n);
} else {
uint32_t poff = pay_part(pos);
if (_pay_open && poff / SEC == _pay_sec) memcpy(buf, _pay + (poff % SEC), n); // still in RAM
else if (esp_partition_read(_part, poff, buf, n) != ESP_OK) return false; // flushed -> flash
}
pos += n; buf += n;
}
return true;
}
void OtaStoreFlashEsp32::finalize() {
if (_flushed || _total == 0) return;
if (!_full) {
// delta: drop the 5-byte trailer into the sliding payload sector(s) so it flushes with them (the
// trailer sits right after the payload; this also covers the rare case it spills into a fresh sector).
uint32_t tpoff = _write_start + (_total - 5);
for (uint32_t off = 0; off < 5; ) {
uint32_t sec = (tpoff + off) / SEC;
if (!_pay_open || sec != _pay_sec) open_pay(sec);
uint32_t in = SEC - ((tpoff + off) % SEC); if (in > 5 - off) in = 5 - off;
memcpy(_pay + ((tpoff + off) % SEC), _trailer + off, in);
off += in;
}
}
flush_pay(); // last payload sector(s) (+ delta trailer)
flush_sector(_meta_part, _meta, _meta_flush); // meta (+ trailer for full)
_flushed = true;
OTA_DBG("OTA esp32: finalize %s io_ok=%d\n", _full ? "FULL" : "DELTA", (int)_io_ok);
}
// Persist mid-transfer progress so a reboot can resume. Flush the open payload sector (KEEP it buffered so
// continued in-order writes aren't lost), then the meta region (leaves). Payload-before-leaves keeps it
// consistent: every block whose leaf is now in flash also has its payload in flash. Infrequent.
void OtaStoreFlashEsp32::checkpoint() {
if (_total == 0 || _flushed || !_io_ok) return;
if (_pay_open) flush_sector((uint32_t)_pay_sec * SEC, _pay, SEC); // flush but leave _pay/_pay_open intact
flush_sector(_meta_part, _meta, _meta_flush); // header + manifest + leaves (+full trailer)
}
// Re-attach to a container already staged in the slot (after a reboot), WITHOUT erasing. The header
// (MOTA_MAGIC + total) sits at a sector boundary (meta_part for full, write_start for delta — both
// sector-aligned), so scan sector starts from the bottom up. A candidate is accepted only if: magic +
// plausible total, the manifest parses, the container geometry is self-consistent with the total, AND the
// recomputed placement lands the meta exactly where we found the magic. Any miss -> false (fetch fresh),
// so a stray/stale match can only cost a restart, never a corrupt adopt.
bool OtaStoreFlashEsp32::reopen() {
if (!acquire() || _psize < SEC) return false;
uint8_t hb[8];
// the meta/container is staged at the BOTTOM of the slot, so scan upward from there; cap the scan (a
// miss just means "fetch fresh"). 128 sectors (512 KB) covers any full image's meta + typical deltas.
uint32_t scanned = 0;
for (uint32_t o = (_psize - SEC) & ~(SEC - 1); ; o -= SEC) {
if (esp_partition_read(_part, o, hb, 8) == ESP_OK && memcmp(hb, MOTA_MAGIC, 4) == 0) {
uint32_t total = (uint32_t)hb[4] | ((uint32_t)hb[5] << 8) | ((uint32_t)hb[6] << 16) | ((uint32_t)hb[7] << 24);
if (total >= 13 && total <= _psize) {
uint8_t mbuf[256]; uint32_t mread = total - 8; if (mread > sizeof(mbuf)) mread = sizeof(mbuf);
MotaManifest m;
if (esp_partition_read(_part, o + 8, mbuf, mread) == ESP_OK && mota_parse_manifest(mbuf, mread, m)) {
uint32_t mfl = (uint32_t)(m.approval - m.manifest_start) + 4;
uint32_t payload_off = 8 + mfl + m.block_count * 4;
if ((uint64_t)payload_off + m.payload_size + 5 == total) {
_full = m.is_full(); _image_size = m.image_size; _meta_bytes = payload_off; _pay_size = m.payload_size;
if (layout() && _meta_part == o) { // geometry agrees AND magic is where we'd place meta
free(_meta); _meta = (uint8_t*)malloc(_meta_flush);
if (!_meta) { _total = 0; return false; }
if (esp_partition_read(_part, _meta_part, _meta, _meta_flush) != ESP_OK) {
free(_meta); _meta = nullptr; _total = 0; return false; }
memset(_trailer, 0xFF, sizeof(_trailer)); // delta trailer (re-written at finalize); full reads it from _meta
_pay_open = false; _pay_sec = 0; _flushed = false; _io_ok = true;
_pay_max_sec = (_pay_part0 + _pay_size + SEC) / SEC; // treat all payload sectors as seen -> RMW preserves committed blocks
OTA_DBG("OTA esp32: reopen %s total=%u meta_part=%u\n", _full ? "FULL" : "DELTA", (unsigned)total, (unsigned)o);
return true;
}
}
}
}
}
if (o == 0 || ++scanned >= 128) break;
}
return false;
}
} // namespace ota
} // namespace mesh
#endif
src/helpers/ota/OtaStoreFlashEsp32.h
+116
View File
@@ -0,0 +1,116 @@
#pragma once
#if defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
#include "OtaStore.h"
#include "esp_partition.h"
// Persistent flash-backed OtaStore for ESP32 (A/B). Stages the received `.mota` in the INACTIVE OTA
// slot, so a delta/full of any size is bounded to O(one sector) of RAM, never O(mota) -- lifting the old
// RAM store's ~16 KB ceiling (a full 1 MB+ image now fetches over the air). Placement is chosen from the
// parsed manifest (plan_layout), keyed only on full-vs-delta:
//
// - DELTA (codec sequential/in-place): the whole container is staged bottom-aligned in the slot. At
// apply the decoder reads the patch from it -- sequential: base from the running slot -> output to
// the inactive slot from offset 0; in-place: copy running->slot then patch in place. The decoded
// image fills the slot from offset 0, so plan_layout refuses unless `image_size + container` fits
// (the output must never reach the bottom-staged container; the fit makes them disjoint).
//
// - FULL (codec full): the payload IS the final image (no decode), so it streams straight to slot
// offset 0 while the small header+manifest+merkle-leaves and the trailer persist at the bottom of the
// slot (so the whole container survives a reboot -- for fetch-resume and for re-serving to other
// nodes). One default for every A/B full payload; plan_layout refuses unless `image_size + meta`
// fits, and if that doesn't fit nothing would.
//
// RX-safety: an ESP32 flash erase+write disables the XIP instruction cache and stalls code running from
// flash (the dispatcher loop), exactly like the nRF52 page erase that starved the LoRa RX. So writes are
// COALESCED to the 4 KB sector and each sector is programmed once. The meta region (header+manifest+
// leaves -- written all transfer long as blocks/leaves arrive, often out of order) is PINNED in RAM and
// flushed at finalize() with the radio idle; the bulk payload streams through ONE sliding sector buffer,
// flushing the sector it leaves behind (~1 flush per 4 KB, off the per-packet path). OTA is also the
// lowest-priority TX, so a brief stall yields to real traffic. A small delta whose container fits the
// pinned meta region does ZERO flash I/O until COMPLETE.
namespace mesh {
namespace ota {
#ifndef OTA_ESP32_META_CAP
#define OTA_ESP32_META_CAP 65536 // max heap for header+manifest+merkle leaves (4 B/block) -> ~16k blocks
#endif // (a 1.27 MB full image at 128 B LoRa blocks ~= 40 KB of leaves)
class OtaStoreFlashEsp32 : public OtaStore {
static const uint32_t SEC = 4096; // ESP32 NOR flash erase unit
const esp_partition_t* _part = nullptr; // inactive OTA slot (acquired in plan_layout/begin)
uint32_t _psize = 0; // slot size
// container geometry (from plan_layout / handleManifest)
uint32_t _total = 0; // container size (0 = none staged)
uint32_t _meta_bytes = 0; // header+manifest+leaves (== payload offset in the container)
uint32_t _pay_size = 0; // payload bytes
uint32_t _image_size = 0; // reconstructed image (delta) / == _pay_size (full)
bool _full = false;
// logical->partition placement (see header doc). For delta everything is contiguous at _write_start;
// for full the payload lands at slot 0 and meta+trailer at the bottom.
uint32_t _write_start = 0; // delta: container offset 0 in the slot (sector-aligned)
uint32_t _meta_span = 0; // container bytes held in the RAM meta buffer (whole sectors)
uint32_t _meta_part = 0; // slot offset the meta buffer flushes to
uint32_t _pay_log0 = 0; // first container offset that streams to the payload region
uint32_t _pay_part0 = 0; // slot offset of that first payload byte
uint32_t _trailer_part = 0; // slot offset of the 5-byte trailer
// RX-safe staging buffers
uint8_t* _meta = nullptr; // heap, sized per fetch: header+manifest+leaves(+full trailer)
uint8_t _pay[SEC]; // one sliding payload sector (slot-sector aligned)
uint32_t _pay_sec = 0; // slot sector index currently in _pay (0 = none open)
uint8_t _trailer[5];
uint32_t _meta_flush = 0; // whole-sector byte count to program for the meta buffer
uint32_t _pay_max_sec = 0; // highest payload slot-sector opened (out-of-order detection)
bool _pay_open = false; // a sliding sector is currently buffered in _pay
bool _flushed = false;
bool _io_ok = true; // cleared if any flash erase/write/read fails
bool acquire(); // resolve the inactive slot (idempotent)
bool layout(); // compute placement from _full/_image_size/_meta_bytes/_pay_size
uint32_t pay_part(uint32_t L) const { return _pay_part0 + (L - _pay_log0); } // payload slot offset
bool in_trailer(uint32_t L) const { return L >= _total - 5; }
uint32_t run(uint32_t pos, uint32_t remain) const; // bytes from `pos` that stay in one region+sector
uint8_t* meta_slot(uint32_t L); // RAM home of a meta/trailer byte (nullptr if it's payload)
const uint8_t* meta_slot_c(uint32_t L) const;
void open_pay(uint32_t sec); // make `sec` the buffered sliding sector (RMW if revisited)
void flush_pay(); // erase+write the open sliding sector
void flush_sector(uint32_t slot_off, const uint8_t* buf, uint32_t n); // erase+program sector(s)
public:
~OtaStoreFlashEsp32() override;
bool plan_layout(bool is_full, uint32_t image_size, uint32_t payload_off, uint32_t payload_size) override;
bool begin(uint32_t total_size) override;
bool write(uint32_t offset, const uint8_t* data, uint32_t len) override;
bool read(uint32_t offset, uint8_t* buf, uint32_t len) const override;
uint32_t capacity() const override { return _psize; } // loose bound; plan_layout does the real check
uint32_t staged_size() const override { return _total; }
void clear() override;
bool set_meta_size(uint32_t meta_bytes) override { return meta_bytes < OTA_ESP32_META_CAP; }
void finalize() override;
void checkpoint() override; // persist meta(leaves) + open payload sector so a reboot can resume
bool reopen() override; // re-attach to a container already staged in the slot (scan + rebuild geometry)
// No contiguous RAM/mmap view: the staged container lives in the slot (split for FULL). The ESP32
// apply path reads what it needs via read()/esp_partition_read instead of a data() pointer, so this
// returns nullptr (kept for interface parity with the nRF52 store, whose flash is memory-mapped).
const uint8_t* data() const { return nullptr; }
// Apply-path accessors: where the staged container physically lives in the inactive slot.
const esp_partition_t* partition() const { return _part; }
uint32_t write_start() const { return _write_start; } // delta: container offset 0 in the slot
bool is_full() const { return _full; }
uint32_t image_size() const { return _image_size; }
uint32_t meta_bytes() const { return _meta_bytes; }
uint32_t payload_slot_off(uint32_t k) const { return _pay_part0 + k; } // slot off of payload byte k
};
} // namespace ota
} // namespace mesh
#endif
src/helpers/ota/OtaStoreFlashNrf52.cpp
+37
View File
@@ -115,6 +115,43 @@ void OtaStoreFlashNrf52::finalize() {
_flushed = true;
}
// Persist mid-transfer progress so a reboot can resume. Order matters for consistency: flush the open
// payload page FIRST, then page 0 (the leaf-progress markers) -- so every block whose leaf is now in flash
// also has its payload in flash. Infrequent (every OTA_CHECKPOINT_BLOCKS blocks), so the 2 extra page
// erases don't matter; at LoRa block rates it's roughly once per many minutes.
void OtaStoreFlashNrf52::checkpoint() {
if (_total == 0 || _flushed) return;
flush_pay(); // keep _pay_idx open (it may still receive writes); just re-flush its bytes
flush_page(0, _meta_page); // header + manifest + leaves accumulated so far
}
// Re-attach to a container already staged in flash (after a reboot), without erasing. The container is
// bottom-aligned (begin: start = (FS_START - total) & ~(PG-1)) and flash is memory-mapped, so scan page
// starts from just below FS_START down to the app end for MOTA_MAGIC with a self-consistent total; adopt
// the first match (highest address = most recent for the common single-container case). The manager then
// parses the loaded manifest and validates geometry/root, so a stale leftover is rejected there.
bool OtaStoreFlashNrf52::reopen() {
uint32_t app_end = MOTA_NRF52_APP_BASE;
SelfFwInfo fi;
if (ota_self_firmware(fi) && fi.valid) app_end = MOTA_NRF52_APP_BASE + fi.image_len;
for (uint32_t start = (MOTA_NRF52_FS_START - PG) & ~(PG - 1); start >= app_end; start -= PG) {
const uint8_t* p = (const uint8_t*)(uintptr_t)start;
if (memcmp(p, MOTA_MAGIC, 4) != 0) continue;
uint32_t total = (uint32_t)p[4] | ((uint32_t)p[5] << 8) | ((uint32_t)p[6] << 16) | ((uint32_t)p[7] << 24);
if (total < 13 || total > capacity()) continue;
if (((MOTA_NRF52_FS_START - total) & ~(PG - 1)) != start) continue; // must match begin()'s placement
_write_start = start;
_total = total;
memcpy(_meta_page, p, PG); // load page 0 (header+manifest+leaves) into RAM to continue
memcpy(_trailer, p + (total - 5), 5); // recover the trailer tail (flushed at last finalize, if any)
_pay_idx = 0;
_flushed = false;
OTA_DBG("OTA flash: reopen total=%u start=%08x\n", (unsigned)total, (unsigned)start);
return true;
}
return false;
}
} // namespace ota
} // namespace mesh
src/helpers/ota/OtaStoreFlashNrf52.h
+2
View File
@@ -62,6 +62,8 @@ public:
void clear() override { _total = 0; _pay_idx = 0; _flushed = false; }
bool set_meta_size(uint32_t meta_bytes) override { return meta_bytes <= PG; } // leaves must fit page 0
void finalize() override;
void checkpoint() override; // persist page 0 (leaves) + the open payload page so a reboot can resume
bool reopen() override; // re-attach to a container already staged in flash (scan for it)
// Contiguous view (flash is memory-mapped). VALID ONLY AFTER finalize() — before that, page 0 and the
// tail are still in RAM. OtaManager/OtaCli/verify use this only once the transfer is COMPLETE.
test/test_ota/mota_vectors.h
+11 -4
View File
File diff suppressed because one or more lines are too long
test/test_ota/test_ota_core.cpp
+265 -46
View File
@@ -48,6 +48,8 @@ TEST(OtaParse, ParsesReferenceContainer) {
EXPECT_EQ(m.codec_id, EXP_CODEC_ID);
EXPECT_EQ(0, memcmp(m.merkle_root, EXP_MERKLE_ROOT, 4));
EXPECT_EQ(0, memcmp(m.image_hash, EXP_IMAGE_HASH, 32));
ASSERT_NE(m.hw_id, nullptr);
EXPECT_EQ(0, memcmp(m.hw_id, EXP_HW_ID, 32)); // v2 hardware tag ("TESTHW" NUL-padded)
EXPECT_EQ(0, memcmp(m.approval, APPROVAL_NOT, 4)); // distributed = not approved
EXPECT_FALSE(m.is_approved());
}
@@ -304,14 +306,35 @@ TEST(OtaMerkle, GenProofMatchesPythonAndVerifies) {
TEST(OtaProtocol, CodecRoundTrips) {
uint8_t buf[200];
AdvMsg adv{0x11223344, 0x02000000, {0x29,0x17,0xe4,0xf7}, MFLAG_FULL | MFLAG_SIGNED, 1, CODEC_DETOOLS_INPLACE};
// OTA_ADV is now a tiny per-node beacon: seeder_id + n_motas + set_digest
AdvMsg adv{{0x29,0x17,0xe4,0xf7}, 7, {0xde,0xad,0xbe,0xef}};
uint16_t n = encode_adv(buf, sizeof(buf), adv);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_ADV);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_ADV); EXPECT_EQ(n, 10);
AdvMsg a2; ASSERT_TRUE(decode_adv(buf, n, a2));
EXPECT_EQ(a2.target_id, adv.target_id); EXPECT_EQ(a2.fw_version, adv.fw_version);
EXPECT_EQ(0, memcmp(a2.manifest_id, adv.manifest_id, 4));
EXPECT_EQ(a2.flags, adv.flags); EXPECT_EQ(a2.have_all, 1);
EXPECT_EQ(a2.codec_id, CODEC_DETOOLS_INPLACE);
EXPECT_EQ(0, memcmp(a2.seeder_id, adv.seeder_id, 4));
EXPECT_EQ(a2.n_motas, 7);
EXPECT_EQ(0, memcmp(a2.set_digest, adv.set_digest, 4));
// OTA_QUERY: ask a source (by seeder_id) for the offering set_digest, optionally filtered to a target
QueryMsg qy{{0x29,0x17,0xe4,0xf7}, {0xd1,0xd2,0xd3,0xd4}, 0x11223344};
n = encode_query(buf, sizeof(buf), qy);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_QUERY);
QueryMsg q2; ASSERT_TRUE(decode_query(buf, n, q2));
EXPECT_EQ(0, memcmp(q2.seeder_id, qy.seeder_id, 4));
EXPECT_EQ(0, memcmp(q2.set_digest, qy.set_digest, 4));
EXPECT_EQ(q2.filter_target, 0x11223344u);
// OTA_HAVE: a 2-row catalog (mid, target, fwver, codec, flags per row) tagged with the offering digest
uint8_t rows[2 * 14];
for (int i = 0; i < 2 * 14; i++) rows[i] = (uint8_t)(i + 1);
HaveMsg hv{{0x29,0x17,0xe4,0xf7}, {0xd1,0xd2,0xd3,0xd4}, 0, 1, 2, rows};
n = encode_have(buf, sizeof(buf), hv);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_HAVE);
HaveMsg h2; ASSERT_TRUE(decode_have(buf, n, h2));
EXPECT_EQ(0, memcmp(h2.seeder_id, hv.seeder_id, 4));
EXPECT_EQ(0, memcmp(h2.set_digest, hv.set_digest, 4));
EXPECT_EQ(h2.frag_total, 1); EXPECT_EQ(h2.n_rows, 2);
EXPECT_EQ(0, memcmp(h2.rows, rows, 2 * 14));
GetManifestMsg gm{{1,2,3,4}};
n = encode_get_manifest(buf, sizeof(buf), gm);
@@ -330,20 +353,36 @@ TEST(OtaProtocol, CodecRoundTrips) {
ReqMsg r2; ASSERT_TRUE(decode_req(buf, n, r2));
EXPECT_EQ(r2.start_block, 7); EXPECT_EQ(r2.count, 5);
uint8_t proof[12]; for (int i = 0; i < 12; i++) proof[i] = (uint8_t)(0xA0 + i);
// DATA is one self-describing fragment of a block (frag_off places it; proof is fetched separately)
uint8_t data[100]; for (int i = 0; i < 100; i++) data[i] = (uint8_t)(i * 3);
DataMsg dm{{0,1,2,3}, 42, 0, 1, 3, proof, data, 100};
DataMsg dm{{0,1,2,3}, 42, 0, data, 100}; // block 42, fragment at offset 0
n = encode_data(buf, sizeof(buf), dm);
DataMsg d2; ASSERT_TRUE(decode_data(buf, n, d2));
EXPECT_EQ(d2.block_idx, 42); EXPECT_EQ(d2.frag_idx, 0); EXPECT_EQ(d2.n_proof, 3);
EXPECT_EQ(0, memcmp(d2.proof, proof, 12));
EXPECT_EQ(d2.block_idx, 42); EXPECT_EQ(d2.frag_off, 0);
EXPECT_EQ(d2.data_len, 100); EXPECT_EQ(0, memcmp(d2.data, data, 100));
// a non-frag0 DATA carries no proof
DataMsg dm2{{0,1,2,3}, 42, 2, 6, 0, nullptr, data, 50};
// a later slice of the same block (non-zero frag_off)
DataMsg dm2{{0,1,2,3}, 42, 160, data, 50};
n = encode_data(buf, sizeof(buf), dm2);
DataMsg d3; ASSERT_TRUE(decode_data(buf, n, d3));
EXPECT_EQ(d3.frag_idx, 2); EXPECT_EQ(d3.n_proof, 0); EXPECT_EQ(d3.data_len, 50);
EXPECT_EQ(d3.block_idx, 42); EXPECT_EQ(d3.frag_off, 160); EXPECT_EQ(d3.data_len, 50);
// REQ_PROOF: request the merkle proof for one (reassembled) block
ReqProofMsg rp{{7,7,8,8}, 13};
n = encode_req_proof(buf, sizeof(buf), rp);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_REQ_PROOF);
ReqProofMsg rp2; ASSERT_TRUE(decode_req_proof(buf, n, rp2));
EXPECT_EQ(0, memcmp(rp2.manifest_id, rp.manifest_id, 4)); EXPECT_EQ(rp2.block_idx, 13);
// PROOF: ordered sibling digests for one block
uint8_t proof[12]; for (int i = 0; i < 12; i++) proof[i] = (uint8_t)(0xA0 + i);
ProofMsg pm{{7,7,8,8}, 13, 3, proof};
n = encode_proof(buf, sizeof(buf), pm);
ASSERT_GT(n, 0); EXPECT_EQ(ota_msg_type(buf, n), OTA_PROOF);
ProofMsg pm2; ASSERT_TRUE(decode_proof(buf, n, pm2));
EXPECT_EQ(0, memcmp(pm2.manifest_id, pm.manifest_id, 4));
EXPECT_EQ(pm2.block_idx, 13); EXPECT_EQ(pm2.n_proof, 3);
EXPECT_EQ(0, memcmp(pm2.proof, proof, 12));
}
// --- full transfer simulation between two OtaManagers (P4b) ------------------------------------
@@ -355,6 +394,54 @@ struct SendTo { OtaManager* dest; };
static void sim_send(void* ctx, const uint8_t* msg, uint16_t len, bool /*flood*/) {
g_q.push_back({((SendTo*)ctx)->dest, std::vector<uint8_t>(msg, msg + len)});
}
// Drive the bus to quiescence: deliver queued messages; when idle, advance the client's clock (monotonic
// across calls, so a jittered query scheduled in a prior pump still comes due) and call loop() (fires the
// scheduled catalog query / block re-requests). Two idle ticks in a row = quiescent.
static uint32_t g_clk = 0;
static void pump(OtaManager& client, int guard_max = 200000) {
int idle = 0, guard = 0;
while (guard++ < guard_max) {
if (!g_q.empty()) {
SimMsg m = std::move(g_q.front()); g_q.erase(g_q.begin());
m.dest->on_message(m.bytes.data(), (uint16_t)m.bytes.size());
idle = 0;
} else {
g_clk += 5000; client.set_clock(g_clk); client.loop();
if (!g_q.empty()) { idle = 0; continue; }
if (++idle >= 2) break;
}
}
}
// A test MotaSource backing an external "folder" with one or more complete `.mota` images held in RAM —
// the simplest concrete transport (a real device uses serial/BLE/WiFi/FS, same interface). describe()
// parses each container for the catalog + region offsets; read() is a bounds-checked memcpy.
class RamMotaSource : public mesh::ota::MotaSource {
public:
void add(const uint8_t* buf, uint32_t len) { if (_n < 8) { _buf[_n] = buf; _len[_n] = len; _n++; } }
uint8_t count() override { return _n; }
bool describe(uint8_t idx, mesh::ota::MotaDesc& d) override {
if (idx >= _n) return false;
MotaManifest m;
if (!mota_parse(_buf[idx], _len[idx], m)) return false;
std::memcpy(d.mid, m.merkle_root, 4);
d.target_id = m.target_id; d.fw_version = m.fw_version;
d.codec_id = m.codec_id; d.flags = m.flags;
d.total_size = _len[idx];
d.leaves_off = (uint32_t)(m.leaves - _buf[idx]);
d.block_count = m.block_count;
d.payload_off = (uint32_t)(m.payload - _buf[idx]);
d.payload_size = m.payload_size;
return true;
}
bool read(uint8_t idx, uint32_t off, uint8_t* out, uint32_t len) override {
if (idx >= _n || (uint64_t)off + len > _len[idx]) return false;
std::memcpy(out, _buf[idx] + off, len);
return true;
}
private:
const uint8_t* _buf[8] = {nullptr}; uint32_t _len[8] = {0}; uint8_t _n = 0;
};
}
TEST(OtaTransfer, TwoManagersFullTransfer) {
@@ -366,18 +453,12 @@ TEST(OtaTransfer, TwoManagersFullTransfer) {
server.begin(/*server's own target irrelevant for serving*/ 0, sim_send, &to_client);
client.begin(SIM_TARGET_ID, sim_send, &to_server);
client.set_fetch_store(&store);
client.set_autofetch(OtaManager::AUTOFETCH_ANY); // tests exercise fetch-on-advert; policy default is OFF
ASSERT_TRUE(server.serve(SIM_MOTA, SIM_MOTA_LEN));
server.announce(); // -> client hears the ADV and starts fetching
server.announce(); // -> client hears the beacon, queries, catalogs, then fetches
// drain the message bus until the client completes (event cascade does the whole transfer)
int guard = 0;
while (!g_q.empty() && guard++ < 100000) {
SimMsg m = std::move(g_q.front());
g_q.erase(g_q.begin());
m.dest->on_message(m.bytes.data(), (uint16_t)m.bytes.size());
if (g_q.empty() && client.fetchState() == OtaManager::FETCHING) client.loop();
}
pump(client); // beacon -> query -> have -> startFetch -> full transfer
EXPECT_EQ(client.fetchState(), OtaManager::COMPLETE);
EXPECT_EQ(client.blocksHave(), client.blocksTotal());
@@ -394,6 +475,138 @@ TEST(OtaTransfer, TwoManagersFullTransfer) {
EXPECT_TRUE(mota_check_image_hash_full(m));
}
// Same end-to-end transfer, but with 1 KB logical blocks: each block is delivered as several
// self-describing DATA fragments (frag_off), reassembled by the client, then its merkle PROOF is
// requested + verified separately before the block is committed. Exercises the multi-fragment path.
TEST(OtaTransfer, MultiFragmentBlocks) {
g_q.clear();
OtaManager server, client;
OtaStoreRam<4096> store;
SendTo to_client{&client}, to_server{&server};
server.begin(0, sim_send, &to_client);
client.begin(SIM_TARGET_ID, sim_send, &to_server);
client.set_fetch_store(&store);
client.set_autofetch(OtaManager::AUTOFETCH_ANY);
ASSERT_TRUE(server.serve(SIM_MOTA_1K, SIM_MOTA_1K_LEN));
server.announce();
pump(client);
EXPECT_EQ(client.fetchState(), OtaManager::COMPLETE);
EXPECT_EQ(client.blocksTotal(), SIM_MOTA_1K_BLOCKS); // 1 KB blocks => fewer, larger blocks
EXPECT_EQ(client.blocksHave(), client.blocksTotal());
ASSERT_EQ(store.staged_size(), SIM_MOTA_1K_LEN);
EXPECT_EQ(0, std::memcmp(store.data(), SIM_MOTA_1K, SIM_MOTA_1K_LEN));
MotaManifest m;
ASSERT_TRUE(mota_parse(store.data(), store.staged_size(), m));
EXPECT_TRUE(mota_check_root(m));
EXPECT_TRUE(mota_check_image_hash_full(m));
}
// Multi-mota folder serve: a node serves its OWN fw (view0) PLUS an external folder (RamMotaSource) of
// other `.mota`. Peers discover BOTH via the tiny beacon -> query -> broadcast HAVE catalog, then fetch an
// external mota end-to-end. The relaying node never holds the folder image in RAM — it streams the
// manifest/leaves/blocks from the source on demand (loadSource + srcReadTramp + proof-gen from read
// leaves). The fetched bytes must equal the original `.mota` (proves the trustless relay is byte-exact).
TEST(OtaFolder, ServesSelfPlusFolderAndFetchesExternal) {
g_q.clear();
OtaManager server, client;
OtaStoreRam<4096> store;
SendTo to_client{&client}, to_server{&server};
server.begin(/*own target irrelevant for serving*/ 0, sim_send, &to_client);
uint8_t srv_id[4] = {0xAB, 0xCD, 0xEF, 0x01}; server.set_seeder_id(srv_id);
client.begin(SIM_TARGET_ID, sim_send, &to_server);
client.set_fetch_store(&store);
MotaManifest mSelf, mExt;
ASSERT_TRUE(mota_parse(SIM_MOTA, SIM_MOTA_LEN, mSelf)); // served as our own fw (view0)
ASSERT_TRUE(mota_parse(SIM_MOTA_1K, SIM_MOTA_1K_LEN, mExt)); // served from the external folder
ASSERT_TRUE(server.serve(SIM_MOTA, SIM_MOTA_LEN)); // entry 0 = self
static RamMotaSource folder;
folder.add(SIM_MOTA_1K, SIM_MOTA_1K_LEN); // an external image (different mid)
folder.add(SIM_MOTA, SIM_MOTA_LEN); // same as self -> must be DEDUPED
ASSERT_TRUE(server.add_source(&folder));
EXPECT_EQ(server.servedCount(), 2); // self + 1 distinct folder mota (dedup)
// discovery: beacon -> the client catalogs the source, queries it, and the broadcast HAVE fills the
// catalog with BOTH served mids.
server.announce();
pump(client);
client.queryAll();
pump(client);
EXPECT_EQ(client.catalogCount(), 2);
// fetch the EXTERNAL (folder) mota by mid -> served via the source, relayed block-by-block.
client.pull(mExt.merkle_root, mExt.target_id);
pump(client);
EXPECT_EQ(client.fetchState(), OtaManager::COMPLETE);
EXPECT_EQ(client.blocksHave(), client.blocksTotal());
ASSERT_EQ(store.staged_size(), SIM_MOTA_1K_LEN);
EXPECT_EQ(0, std::memcmp(store.data(), SIM_MOTA_1K, SIM_MOTA_1K_LEN)); // byte-exact relay
MotaManifest got;
ASSERT_TRUE(mota_parse(store.data(), store.staged_size(), got));
EXPECT_TRUE(mota_check_root(got));
EXPECT_TRUE(mota_check_image_hash_full(got));
}
// Fetch-resume across a reboot: a client commits some blocks, "reboots" (a fresh OtaManager on the SAME
// persisted store), and resumeStaged() re-adopts the partial container and finishes the remaining blocks —
// without re-fetching the manifest or the blocks already present.
TEST(OtaTransfer, ResumeAfterReboot) {
g_q.clear();
OtaManager server, client;
OtaStoreRam<4096> store;
SendTo to_client{&client}, to_server{&server};
server.begin(0, sim_send, &to_client);
client.begin(SIM_TARGET_ID, sim_send, &to_server);
client.set_fetch_store(&store);
client.set_autofetch(OtaManager::AUTOFETCH_ANY);
ASSERT_TRUE(server.serve(SIM_MOTA_1K, SIM_MOTA_1K_LEN));
server.announce();
// drive only until the first block commits, then "crash"
int idle = 0, guard = 0;
while (guard++ < 100000) {
if (!g_q.empty()) {
SimMsg msg = std::move(g_q.front()); g_q.erase(g_q.begin());
msg.dest->on_message(msg.bytes.data(), (uint16_t)msg.bytes.size());
idle = 0;
} else {
g_clk += 5000; client.set_clock(g_clk); client.loop();
if (!g_q.empty()) { idle = 0; } else if (++idle >= 2) break;
}
if (client.blocksHave() >= 1) break;
}
ASSERT_GE(client.blocksHave(), 1u);
ASSERT_LT(client.blocksHave(), client.blocksTotal()); // genuinely partial
uint32_t had = client.blocksHave();
g_q.clear(); // in-flight packets are lost in the "reboot"
// "reboot": a brand-new manager on the SAME store (its bytes survived) resumes the partial
OtaManager client2;
to_client.dest = &client2; // server now replies to the rebooted client
SendTo to_server2{&server};
client2.begin(SIM_TARGET_ID, sim_send, &to_server2);
client2.set_fetch_store(&store);
ASSERT_TRUE(client2.resumeStaged(nullptr)); // adopt whatever is staged
EXPECT_EQ(client2.blocksHave(), had); // resumed exactly where we left off
EXPECT_EQ(client2.fetchState(), OtaManager::FETCHING);
EXPECT_EQ(client2.blocksTotal(), SIM_MOTA_1K_BLOCKS);
pump(client2);
EXPECT_EQ(client2.fetchState(), OtaManager::COMPLETE);
ASSERT_EQ(store.staged_size(), SIM_MOTA_1K_LEN);
EXPECT_EQ(0, std::memcmp(store.data(), SIM_MOTA_1K, SIM_MOTA_1K_LEN)); // byte-identical to the original
}
TEST(OtaTransfer, ClientRejectsWrongTarget) {
g_q.clear();
OtaManager server, client;
@@ -402,13 +615,10 @@ TEST(OtaTransfer, ClientRejectsWrongTarget) {
server.begin(0, sim_send, &to_client);
client.begin(SIM_TARGET_ID ^ 0x1u, sim_send, &to_server); // different target -> not interested
client.set_fetch_store(&store);
client.set_autofetch(OtaManager::AUTOFETCH_ANY); // tests exercise fetch-on-advert; policy default is OFF
ASSERT_TRUE(server.serve(SIM_MOTA, SIM_MOTA_LEN));
server.announce();
int guard = 0;
while (!g_q.empty() && guard++ < 1000) {
SimMsg m = std::move(g_q.front()); g_q.erase(g_q.begin());
m.dest->on_message(m.bytes.data(), (uint16_t)m.bytes.size());
}
pump(client); // catalogs the row but wantRow rejects it (wrong target) -> never fetches
EXPECT_EQ(client.fetchState(), OtaManager::IDLE); // never started
}
@@ -422,46 +632,55 @@ TEST(OtaTransfer, ManualCrossTargetFetch) {
server.begin(0, sim_send, &to_client);
client.begin(SIM_TARGET_ID ^ 0xABCDu, sim_send, &to_server); // DIFFERENT own target
client.set_fetch_store(&store);
client.set_autofetch(OtaManager::AUTOFETCH_ANY); // tests exercise fetch-on-advert; policy default is OFF
ASSERT_TRUE(server.serve(SIM_MOTA, SIM_MOTA_LEN));
// without the override: ignores the ADV (wrong target)
// without the override: catalogs the row but won't fetch (wrong target)
server.announce();
for (int g = 0; !g_q.empty() && g < 1000; g++) { SimMsg m = std::move(g_q.front()); g_q.erase(g_q.begin()); m.dest->on_message(m.bytes.data(), (uint16_t)m.bytes.size()); }
pump(client);
EXPECT_EQ(client.fetchState(), OtaManager::IDLE);
// with want(): deliberately fetch the different-target firmware to completion
client.want(SIM_TARGET_ID);
server.announce();
int guard = 0;
while (!g_q.empty() && guard++ < 100000) {
SimMsg m = std::move(g_q.front()); g_q.erase(g_q.begin());
m.dest->on_message(m.bytes.data(), (uint16_t)m.bytes.size());
if (g_q.empty() && client.fetchState() == OtaManager::FETCHING) client.loop();
}
pump(client);
EXPECT_EQ(client.fetchState(), OtaManager::COMPLETE);
ASSERT_EQ(store.staged_size(), SIM_MOTA_LEN);
EXPECT_EQ(0, std::memcmp(store.data(), SIM_MOTA, SIM_MOTA_LEN));
}
// A node must not fetch firmware it can't apply: an ADV whose codec the platform can't decode is
// rejected at ADV time (never requests the manifest). FULL + the platform's delta codec are accepted.
// Encode a 1-row OTA_HAVE catalog (the discovery reply a peer acts on).
static uint16_t make_have1(uint8_t* buf, uint16_t cap, const uint8_t mid[4],
uint32_t target, uint32_t fwver, uint8_t codec, uint8_t flags) {
uint8_t row[OTA_HAVE_ROW_BYTES];
memcpy(row, mid, 4);
row[4]=target; row[5]=target>>8; row[6]=target>>16; row[7]=target>>24;
row[8]=fwver; row[9]=fwver>>8; row[10]=fwver>>16; row[11]=fwver>>24;
row[12]=codec; row[13]=flags;
HaveMsg hv{{0xAA,0xBB,0xCC,0xDD}, {0,0,0,0}, 0, 1, 1, row};
return encode_have(buf, cap, hv);
}
// A node must not fetch firmware it can't apply: a catalog row whose codec the platform can't decode is
// not fetched. FULL + the platform's delta codec(s) are accepted.
TEST(OtaTransfer, RejectsIncompatibleCodec) {
g_q.clear();
OtaManager client; OtaStoreRam<4096> store;
SendTo to_server{&client}; // dest unused (we only check client state)
client.begin(SIM_TARGET_ID, sim_send, &to_server);
client.set_fetch_store(&store);
client.set_apply_codec(CODEC_DETOOLS_INPLACE); // nRF52-style: accepts only full + in-place
uint8_t b[32];
client.set_autofetch(OtaManager::AUTOFETCH_ANY);
client.set_apply_codec(CODEC_DETOOLS_INPLACE); // nRF52-style: accepts only full + in-place
uint8_t b[64];
// our target, but a SEQUENTIAL delta -> incompatible -> ignored (stays IDLE, no GET_MANIFEST)
AdvMsg seq{SIM_TARGET_ID, 0x01000000, {1,2,3,4}, 0, 1, CODEC_DETOOLS_SEQUENTIAL};
client.on_message(b, encode_adv(b, sizeof(b), seq));
// a SEQUENTIAL delta for our target -> incompatible -> not fetched (stays IDLE)
uint8_t midA[4] = {1,2,3,4};
client.on_message(b, make_have1(b, sizeof(b), midA, SIM_TARGET_ID, 0x01000000, CODEC_DETOOLS_SEQUENTIAL, 0));
EXPECT_EQ(client.fetchState(), OtaManager::IDLE);
EXPECT_TRUE(g_q.empty());
// our target, IN-PLACE delta -> compatible -> proceeds to request the manifest
AdvMsg ip{SIM_TARGET_ID, 0x01000000, {5,6,7,8}, 0, 1, CODEC_DETOOLS_INPLACE};
client.on_message(b, encode_adv(b, sizeof(b), ip));
// an IN-PLACE delta for our target -> compatible -> begins fetching (requests the manifest)
uint8_t midB[4] = {5,6,7,8};
client.on_message(b, make_have1(b, sizeof(b), midB, SIM_TARGET_ID, 0x01000000, CODEC_DETOOLS_INPLACE, 0));
EXPECT_EQ(client.fetchState(), OtaManager::WANT_MANIFEST);
g_q.clear();
}
tools/mota/README.md
+38 -3
View File
@@ -1,9 +1,8 @@
# `mota` — MeshCore OTA packaging tool
Host-side tooling for building and validating `.mota` firmware-update containers.
Implements the wire spec in [`docs/ota_protocol.md`](../../docs/ota_protocol.md) (v1).
Part of the OTA-over-LoRa work — see `OTA_PLAN.md` (milestone **P0**).
Implements the wire spec in [`docs/ota_protocol.md`](../../docs/ota_protocol.md) — the single source of
truth for the `.mota` format and the OTA-over-LoRa protocol.
## Setup
@@ -64,6 +63,42 @@ $PY tools/mota/mota.py verify fw_v1.16.0_delta.mota --pub signer.priv.pub --bas
# signing, tamper detection, approval enforcement
```
## Folder serve (`mota_seeder.py`) — relay many `.mota` from a host
A node can advertise + serve `.mota` it does **not** hold in flash, by relaying them from a folder on a
host computer. Drop several `.mota` (any architecture) into a folder; the node then shows up to peers as
having all of them. The relay is **trustless** — fetchers verify the merkle root + signature, so the
host/daemon never needs the signing keys and a bad file simply fails the fetch.
```bash
# just connect the MeshCore node to the PC over its normal USB — no extra hardware:
pip install pyserial
./tools/mota/mota_seeder.py --port /dev/ttyACM0 --baud 115200 --dir ./my_firmware/ -v
```
The daemon owns the port, auto-sends `ota folder on` to the node, then answers the node's byte requests.
It speaks a tiny binary request/response protocol (`src/helpers/ota/MotaSeederProto.h`) over the **same
USB serial the CLI uses** — the node only emits request frames *while actively serving a fetch* and reads
the reply synchronously, so the binary frames coexist with the text CLI / logs (the daemon resyncs on a
magic + checksum and surfaces device text as `[dev]` lines). Peers discover the folder mOTAs via the
normal beacon → query → HAVE catalog and `ota pull <mid>` fetches them block-by-block straight from the
host folder. Verified on HW: a RAK4631 relayed a host folder to a Heltec V3, which fetched a folder mota
to COMPLETE (every block merkle-checked) over the single USB.
`OTA_FOLDER_SERIAL` is already enabled on the stock RAK4631 / ESP32 OTA repeater builds (inert until you
run `ota folder on` — which the daemon does for you). To point the relay at a *dedicated* UART instead of
the console, override in `platformio.ini`:
```ini
build_flags =
-D OTA_FOLDER_SERIAL_STREAM=Serial1 ; default is the USB console `Serial`
-D OTA_FOLDER_SERIAL_BEGIN ; call .begin() on it (console is already initialized)
-D OTA_FOLDER_SERIAL_BAUD=115200
```
CLI on the node: `ota folder on` (attach + announce), `ota folder` (list served mOTAs, `*`=own fw),
`ota folder off` (detach).
## `EndF` build integration
`EndF` must live in the **flashed** firmware (not just inside the `.mota`), because a node serves its
tools/mota/gen_vectors.py
+14 -1
View File
@@ -30,7 +30,7 @@ def build_full():
m = ml.build_manifest(
target_id=0x11223344, fw_version=ml.pack_version("1.16.0"),
image_size=len(image), payload=image, block_size=1024,
image_hash=ml.mh32(image), codec_id=ml.CODEC_FULL, is_full=True)
image_hash=ml.mh32(image), codec_id=ml.CODEC_FULL, is_full=True, hw_id="TESTHW")
return ml.build_container(m, image), m, image
@@ -86,6 +86,7 @@ def main():
f"static const uint8_t EXP_CODEC_ID = {m.codec_id};",
_carr("EXP_MERKLE_ROOT", m.merkle_root),
_carr("EXP_IMAGE_HASH", m.image_hash),
_carr("EXP_HW_ID", m.hw_id),
f"static const uint32_t PROOF_INDEX = {proof_idx}u;",
f"static const uint8_t PROOF_NSIB = {len(siblings)//4};",
_carr("PROOF_SIBLINGS", siblings),
@@ -119,6 +120,18 @@ def main():
lines.append(f"static const uint32_t SIM_MOTA_LEN = {len(sim_blob)};")
lines.append(f"static const uint32_t SIM_TARGET_ID = 0x{sm.target_id:08x}u;")
# 1 KB-block signed .mota: each logical block spans MULTIPLE LoRa DATA fragments (1024 / 160 = 7),
# so this exercises the multi-fragment reassembly + split proof path the device actually uses.
sm1k = ml.build_manifest(target_id=0xCAFEBABE, fw_version=ml.pack_version("3.0.0"),
image_size=len(sim_image), payload=sim_image, block_size=1024,
image_hash=ml.mh32(sim_image), codec_id=ml.CODEC_FULL,
is_full=True, sign_priv=sim_priv)
sim1k_blob = ml.build_container(sm1k, sim_image)
lines.append("// 1 KB-block signed .mota (multi-fragment per block) for the reassembly transfer test")
lines.append(_carr("SIM_MOTA_1K", sim1k_blob))
lines.append(f"static const uint32_t SIM_MOTA_1K_LEN = {len(sim1k_blob)};")
lines.append(f"static const uint32_t SIM_MOTA_1K_BLOCKS = {sm1k.block_count};")
# detools sequential+crle delta vector: base image, the real detools 0.53.0 patch, and the
# expected target image. The native test applies DT_PATCH to DT_BASE with the *vendored detools
# C decoder* (src/helpers/ota/detools) and must reproduce DT_TARGET byte-for-byte -- proving the
tools/mota/mota.py
+4
View File
@@ -100,12 +100,14 @@ def cmd_build(args):
is_full=is_full,
base_hash=base_hash,
sign_priv=sign_priv,
hw_id=args.hw_id,
)
blob = ml.build_container(manifest, payload)
Path(args.out).write_bytes(blob)
print(f"wrote {args.out} ({len(blob)} bytes)")
print(f" codec : {ml.CODEC_NAMES[codec_id]}")
print(f" hw_id : {manifest.hw_id.rstrip(bytes([0])).decode('ascii', 'replace') or '(none)'}")
print(f" payload : {len(payload)} bytes ({manifest.block_count} blocks of {args.block_size})")
print(f" image_size : {image_size} bytes (BODY+EndF)")
print(f" merkle_root : {manifest.merkle_root.hex()}")
@@ -201,6 +203,8 @@ def main(argv=None):
help="delta patch compression (decode-cheap 'crle' default; must be supported by "
"the applier. Ignored for --codec full, whose payload is the raw flashable image)")
b.add_argument("--block-size", type=int, default=ml.DEFAULT_BLOCK_SIZE)
b.add_argument("--hw-id", default="", help="hardware tag (<=32 ASCII chars, e.g. RAK4631) the firmware "
"can boot on; the device refuses a .mota whose hw_id differs from its own. Empty = unset.")
b.add_argument("--sign", help="Ed25519 private key file (hex) to sign the manifest")
b.add_argument("--inplace-memory", type=int, default=4096, help="detools in-place memory_size")
b.add_argument("--inplace-segment", type=int, default=4096, help="detools in-place segment_size")
tools/mota/mota_seeder.py
Executable
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env python3
"""mota-seeder: serve a FOLDER of .mota firmware to a MeshCore node over a serial link.
The node (a SerialMotaSource) pulls the catalog + bytes on demand and RELAYS them into the mesh as if it
held them — peers just see "this node has N mOTAs". Drop several .mota (any architecture) into a folder
and point this daemon at the node's seeder UART; the node advertises + serves them all. The relay is
trustless (fetchers verify merkle+signature), so this daemon never needs the signing keys.
Protocol: src/helpers/ota/MotaSeederProto.h (little-endian, XOR-checksummed, resync on magic).
request (node -> here): 'M''S' op(1) args... xsum
response (here -> node): 'm''s' op(1) status(1) payload... xsum
OP_COUNT 0x01 -> count(1) ; OP_DESCRIBE 0x02 idx -> MotaDesc(38) ; OP_READ 0x03 idx off(4) len(2) -> bytes
Usage: mota_seeder.py --port /dev/ttyUSB0 --baud 115200 --dir ./firmware_folder [--watch]
Requires: pyserial, and motalib.py (this same tools/mota dir) for parsing.
"""
import argparse, glob, os, struct, sys, time
import serial # pyserial
import motalib
REQ_MAGIC = b"MS"
RSP_MAGIC = b"ms"
OP_COUNT, OP_DESCRIBE, OP_READ = 0x01, 0x02, 0x03
ST_OK, ST_ERR = 0x00, 0x01
DESC_WIRE = 38
HEAD = 89 # fixed manifest head incl hw_id[32]
def mota_offsets(blob: bytes):
"""Parse a .mota and return its catalog descriptor + region offsets (mirrors MotaContainer.cpp)."""
p = motalib.parse_container(blob)
m = p.manifest
base = 0 if m.is_full else 8
sig = 96 if m.is_signed else 0
mfl = HEAD + base + sig + 4 # manifest-minus-leaves: head + base_hash? + sig? + approval
leaves_off = 8 + mfl # container = MAGIC(4) total(4) manifest...
bc = m.block_count
payload_off = leaves_off + bc * 4
return {
"mid": bytes(m.merkle_root),
"target_id": m.target_id,
"fw_version": m.fw_version,
"codec_id": m.codec_id,
"flags": m.flags,
"total_size": len(blob),
"leaves_off": leaves_off,
"block_count": bc,
"payload_off": payload_off,
"payload_size": m.payload_size,
}
def desc_wire(d) -> bytes:
w = bytearray(DESC_WIRE)
w[0:4] = d["mid"]
struct.pack_into("<IIB", w, 4, d["target_id"], d["fw_version"], d["codec_id"])
w[13] = d["flags"]
struct.pack_into("<IIII", w, 14, d["total_size"], d["leaves_off"], d["block_count"], d["payload_off"])
struct.pack_into("<I", w, 30, d["payload_size"])
# [34:38) reserved 0
return bytes(w)
def load_folder(path):
"""Return a sorted list of {path, blob, desc} for every parseable .mota in the folder."""
items = []
for f in sorted(glob.glob(os.path.join(path, "*.mota"))):
try:
blob = open(f, "rb").read()
d = mota_offsets(blob)
items.append({"path": f, "blob": blob, "desc": d})
except Exception as e:
print(f" ! skip {os.path.basename(f)}: {e}", file=sys.stderr)
return items
def xor(data: bytes, seed: int = 0) -> int:
x = seed
for b in data:
x ^= b
return x & 0xFF
def send_rsp(ser, op, status, payload=b""):
body = bytes([op, status]) + payload
frame = RSP_MAGIC + body + bytes([xor(RSP_MAGIC + body)])
ser.write(frame)
ser.flush()
def read_exact(ser, n):
buf = bytearray()
while len(buf) < n:
chunk = ser.read(n - len(buf))
if not chunk:
return None
buf += chunk
return bytes(buf)
def handle_one(ser, items, verbose):
"""Read one request body (magic already consumed) and send its response."""
op = read_exact(ser, 1)
if op is None:
return
op = op[0]
if op == OP_COUNT:
args = b""
elif op == OP_DESCRIBE:
args = read_exact(ser, 1)
elif op == OP_READ:
args = read_exact(ser, 7)
else:
return
if args is None:
return
xs = read_exact(ser, 1)
if xs is None or xs[0] != xor(args, op):
return # bad checksum -> ignore; node retries
if op == OP_COUNT:
send_rsp(ser, op, ST_OK, bytes([min(len(items), 255)]))
if verbose:
print(f" COUNT -> {len(items)}")
elif op == OP_DESCRIBE:
idx = args[0]
if idx < len(items):
send_rsp(ser, op, ST_OK, desc_wire(items[idx]["desc"]))
if verbose:
print(f" DESCRIBE {idx} -> {os.path.basename(items[idx]['path'])}")
else:
send_rsp(ser, op, ST_ERR)
elif op == OP_READ:
idx = args[0]
off, lo, hi = struct.unpack("<IBB", args[1:7])
length = lo | (hi << 8)
if idx < len(items) and off + length <= len(items[idx]["blob"]):
send_rsp(ser, op, ST_OK, items[idx]["blob"][off:off + length])
if verbose:
print(f" READ {idx} @{off} +{length}")
else:
send_rsp(ser, op, ST_ERR)
def serve(ser, items, verbose):
"""Scan the shared USB stream for request frames ('M''S'), answering each. Device CLI replies / logs
interleave on the same wire — they're surfaced as [dev] lines and skipped (resync on the magic)."""
skipped = bytearray()
def emit(byte):
skipped.append(byte)
if byte == 0x0A: # newline: flush one device text line
line = bytes(skipped).decode("utf-8", "replace").strip()
skipped.clear()
if line:
print(f" [dev] {line}")
prev = None
while True:
b = ser.read(1)
if not b:
continue
c = b[0]
if prev == ord('M') and c == ord('S'):
handle_one(ser, items, verbose) # 'M''S' consumed; read the rest of the request
prev = None
continue
if prev is not None: # confirmed device text (not a frame start)
emit(prev)
prev = c
def main():
ap = argparse.ArgumentParser(description="Serve a folder of .mota to a MeshCore node over serial.")
ap.add_argument("--port", required=True, help="serial device of the node's seeder UART (e.g. /dev/ttyUSB0)")
ap.add_argument("--baud", type=int, default=115200)
ap.add_argument("--dir", required=True, help="folder containing .mota files to serve")
ap.add_argument("--no-enable", action="store_true",
help="don't auto-send `ota folder on/off` (run those on the node CLI yourself)")
ap.add_argument("-v", "--verbose", action="store_true")
a = ap.parse_args()
items = load_folder(a.dir)
print(f"mota-seeder: {len(items)} mOTA in {a.dir}")
for it in items:
d = it["desc"]
print(f" - {os.path.basename(it['path'])}: mid={d['mid'].hex().upper()} "
f"target={d['target_id']:08X} fw={d['fw_version']} codec={d['codec_id']} "
f"blocks={d['block_count']} size={d['total_size']}")
if not items:
print(" (no .mota found — nothing to serve)", file=sys.stderr)
ser = serial.Serial(a.port, a.baud, timeout=0.2)
if not a.no_enable:
time.sleep(0.5) # let the node settle, then turn folder relay on via its CLI
ser.write(b"ota folder on\r\n"); ser.flush()
print("sent `ota folder on` to the node")
print(f"serving on {a.port} @ {a.baud} — Ctrl-C to stop")
try:
serve(ser, items, a.verbose)
except KeyboardInterrupt:
if not a.no_enable:
try:
ser.write(b"ota folder off\r\n"); ser.flush(); time.sleep(0.2)
except Exception:
pass
print("\nbye")
if __name__ == "__main__":
main()
tools/mota/motalib.py
+18 -3
View File
@@ -9,7 +9,7 @@ The wire format (all integers little-endian):
manifest = format_ver(1) flags(1) hash_algo(1) target_id(4) fw_version(4)
image_size(4) payload_size(4) block_size_log2(1) merkle_root(4)
image_hash(32) codec_id(1)
image_hash(32) codec_id(1) hw_id(32)
[base_hash(8) if delta] [signer_pubkey(32) signature(64) if signed]
approval(4) leaves[](4*BC)
@@ -33,7 +33,7 @@ TRAILER = b"vk496" # 76 6B 34 39 36
ENDF_MAGIC = b"EndF" # 45 6E 64 46
ENDF_LEN = 16 # marker(4) + body_len(4) + body_hash8(8)
FORMAT_VER = 1
FORMAT_VER = 2 # v2 adds hw_id[32] (a NUL-padded ASCII hardware tag) in the signed head
HASH_ALGO_SHA256 = 0x12 # multihash code for sha2-256
FLAG_FULL = 0x01
@@ -237,6 +237,7 @@ class Manifest:
merkle_root: bytes = b"\0\0\0\0"
image_hash: bytes = b"\0" * 32
codec_id: int = CODEC_FULL
hw_id: bytes = b"\0" * 32 # 32-byte NUL-padded ASCII hardware tag (signed)
base_hash: Optional[bytes] = None # 8 bytes, delta only
signer_pubkey: Optional[bytes] = None # 32 bytes, signed only
signature: Optional[bytes] = None # 64 bytes, signed only
@@ -269,6 +270,7 @@ class Manifest:
out += self.merkle_root
out += self.image_hash
out += bytes([self.codec_id])
out += self.hw_id # 32-byte hardware tag (part of the signed head)
if not self.is_full:
out += self.base_hash
if self.is_signed:
@@ -285,9 +287,20 @@ class Manifest:
return bytes(out)
def hw_id_bytes(s) -> bytes:
"""Pack a hardware tag (str or bytes) into the fixed 32-byte NUL-padded field."""
if s is None:
return b"\0" * 32
raw = s.encode("ascii") if isinstance(s, str) else bytes(s)
if len(raw) > 32:
raise ValueError("hw_id must be <= 32 bytes")
return raw + b"\0" * (32 - len(raw))
def _validate_lengths(m: Manifest):
assert len(m.merkle_root) == 4
assert len(m.image_hash) == 32
assert len(m.hw_id) == 32
assert len(m.approval) == 4
if not m.is_full:
assert m.base_hash is not None and len(m.base_hash) == 8, "delta requires 8-byte base_hash"
@@ -298,7 +311,7 @@ def _validate_lengths(m: Manifest):
def build_manifest(*, target_id: int, fw_version: int, image_size: int, payload: bytes,
block_size: int, image_hash: bytes, codec_id: int, is_full: bool,
base_hash: Optional[bytes] = None, sign_priv=None) -> Manifest:
base_hash: Optional[bytes] = None, sign_priv=None, hw_id=None) -> Manifest:
assert (block_size & (block_size - 1)) == 0, "block_size must be a power of two"
leaves = leaf_hashes(payload, block_size)
m = Manifest(
@@ -311,6 +324,7 @@ def build_manifest(*, target_id: int, fw_version: int, image_size: int, payload:
merkle_root=merkle_root(leaves),
image_hash=image_hash,
codec_id=codec_id,
hw_id=hw_id_bytes(hw_id),
base_hash=None if is_full else base_hash,
leaves=leaves,
)
@@ -367,6 +381,7 @@ def parse_container(blob: bytes) -> Parsed:
m.merkle_root = take(4)
m.image_hash = take(32)
m.codec_id = take(1)[0]
m.hw_id = take(32)
if not m.is_full:
m.base_hash = take(8)
if m.is_signed:
tools/mota/test_mota.py
+25
View File
@@ -101,6 +101,31 @@ def test_full_build_parse_verify():
assert ml.verify(parsed) == []
def test_hw_id_roundtrip_and_signed():
# the v2 hw_id is a 32-byte NUL-padded ASCII tag in the SIGNED head; it must round-trip + be covered
# by the signature (tampering it breaks verification).
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey
fw = _fw(77, 4 * 1024)
image, _ = ml.ensure_endf(fw)
priv = Ed25519PrivateKey.from_private_bytes(bytes(range(32)))
m = ml.build_manifest(
target_id=0xABCD, fw_version=ml.pack_version("2.0.0"),
image_size=len(image), payload=image, block_size=1024,
image_hash=ml.mh32(image), codec_id=ml.CODEC_FULL, is_full=True,
sign_priv=priv, hw_id="RAK4631")
blob = ml.build_container(m, image)
parsed = ml.parse_container(blob)
assert parsed.manifest.format_ver == 2
assert parsed.manifest.hw_id == b"RAK4631" + b"\0" * (32 - 7)
assert parsed.manifest.hw_id.rstrip(b"\0").decode() == "RAK4631"
assert ml.verify(parsed) == []
# flip a byte of the on-wire hw_id -> signature must fail (it's in the signed region)
bad = bytearray(blob)
hw_off = 8 + 57 # MAGIC(4)+total(4) + fixed head up to codec(57) = start of hw_id
bad[hw_off] ^= 0xFF
assert ml.verify(ml.parse_container(bytes(bad))) != []
def test_tampered_payload_detected():
fw = _fw(11, 10 * 1024)
image, _ = ml.ensure_endf(fw)
variants/heltec_v3/platformio.ini
+1
View File
@@ -6,6 +6,7 @@ build_flags =
${sensor_base.build_flags}
-I variants/heltec_v3
-D HELTEC_LORA_V3
-D MOTA_HW_ID='"Heltec_v3"' ; OTA hardware tag (apply refuses a .mota for different hw)
-D ESP32_CPU_FREQ=80
-D P_LORA_DIO_1=14
-D P_LORA_NSS=8
variants/rak4631/platformio.ini
+2
View File
@@ -10,6 +10,7 @@ build_flags = ${nrf52_base.build_flags}
-I variants/rak4631
-D RAK_4631
-D RAK_BOARD
-D MOTA_HW_ID='"RAK4631"' ; OTA hardware tag (apply refuses a .mota for different hw)
-D NRF52_POWER_MANAGEMENT
-D PIN_BOARD_SCL=14
-D PIN_BOARD_SDA=13
@@ -28,6 +29,7 @@ build_flags = ${nrf52_base.build_flags}
-D ENV_INCLUDE_BME680_BSEC=1
-D ENABLE_OTA=1 ; OTA delta updates on every RAK4631 role (single-slot, bootloader-applied)
-D OTA_FLASH_STORE=1 ; stage the received .mota in flash (survives reboot into the bootloader)
-D OTA_FOLDER_SERIAL ; `ota folder on` relays a host folder of .mota over the USB console (no extra HW)
build_src_filter = ${nrf52_base.build_src_filter}
+<../variants/rak4631>
+<helpers/sensors>