mirror of
https://github.com/mikecarper/MeshCore.git
synced 2026-09-25 17:23:35 +00:00
fix: harden LoRa OTA chain and storage
This commit is contained in:
mikecarper
@@ -11,6 +11,12 @@ site/
|
||||
out.*/
|
||||
out-*/
|
||||
build-logs/
|
||||
# Local resume state, extracted firmware, and physical-test logs from the
|
||||
# pinned RAK3401 chain runner. These may contain device-specific settings.
|
||||
rak3401-mota-chain-work/
|
||||
# Generic runner work directories can also contain endpoint-bound recovery
|
||||
# settings and diagnostic logs. Never stage them with a broad `git add`.
|
||||
meshcore-lora-ota-*/
|
||||
.direnv/
|
||||
.DS_Store
|
||||
.vscode/settings.json
|
||||
|
||||
@@ -2714,6 +2714,13 @@ declare_build_capability_contract() {
|
||||
fi
|
||||
|
||||
if is_rak_i2c_voltage_monitor_ota_target "$env_name"; then
|
||||
# These two reduced RAK OTA profiles deliberately retain their board GPS
|
||||
# provider as well as the compact INA monitor set. Keep the release
|
||||
# manifest honest so a space-oriented target name is not mistaken for a
|
||||
# GPS-less image. This evidence is emitted only by the linked WisBlock GPS
|
||||
# provider; generic CLI command text is not sufficient proof.
|
||||
record_build_expectation \
|
||||
"sensor.gps" "meshcore.capability.rak_wisblock_gps.v1"
|
||||
record_build_expectation "sensor.ina219" "INA219"
|
||||
record_build_expectation "sensor.ina226" "INA226"
|
||||
record_build_expectation "sensor.ina260" "INA260"
|
||||
|
||||
+89
-26
@@ -262,6 +262,22 @@ runner can restore those periods but cannot reconstruct an unreported saved
|
||||
level. A radio change later recalculates from a preserved saved minimum, so
|
||||
moving back to a slower tuple returns to the operator's saved level.
|
||||
|
||||
The OTA source has a stricter policy than the destination. For every source
|
||||
with a managed serial or TCP CLI, the runner reads and retains its exact RXPS
|
||||
preference, using the legacy fixed-period query only when the detailed query is
|
||||
unavailable. It then disables RXPS and verifies the readback before any target
|
||||
discovery or TempRadio change. Source RXPS stays off through catalog serving,
|
||||
download, installation, and post-install identity verification. Cleanup first
|
||||
proves that the source has returned to its normal radio, then restores and
|
||||
verifies the exact saved level/preamble or fixed-period state once. A source
|
||||
whose RXPS state cannot be read, disabled, or restored safely fails closed.
|
||||
If current firmware explicitly rejects an RXPS disable or restore with `radio
|
||||
busy; retry`, the runner retries that idempotent mutation at staggered
|
||||
210–378 ms intervals. All 32 delays are distinct and contribute about 9.4
|
||||
seconds of waits; source-CLI command round-trip time is additional. This avoids
|
||||
repeatedly sampling one radio phase while retaining a strict attempt cap. Other
|
||||
rejections are not replayed.
|
||||
|
||||
## 4. Run an ESP32 update
|
||||
|
||||
The ZIP can contain a compatible ready `.mota` or the exact board-and-role
|
||||
@@ -427,7 +443,9 @@ later as a catalog timeout.
|
||||
If the source is already on the exact TempRadio tuple through a scheduled or
|
||||
manual operation, `--source-already-temp` lets a TCP source run without a raw
|
||||
CLI link. The script cannot verify, extend, or shorten that source window, so
|
||||
leave a comfortable time margin.
|
||||
leave a comfortable time margin. It also cannot inspect or change that
|
||||
unmanaged source's RXPS state; disable source RXPS yourself before starting
|
||||
the run and restore it only after returning the source to its normal radio.
|
||||
|
||||
Use `--controller-baud` or `--source-baud` only for a build whose corresponding
|
||||
interface is genuinely configured to another speed.
|
||||
@@ -458,7 +476,9 @@ Useful controls:
|
||||
- `--no-install` downloads and verifies the image but leaves it staged. By
|
||||
default the runner then schedules the target, relays, and a script-configured
|
||||
source back to their normal radios. Combining it with
|
||||
`--leave-controller-radio` deliberately preserves the TempRadio topology.
|
||||
`--leave-controller-radio` deliberately preserves the destination, relays,
|
||||
and separate controller on the TempRadio topology; a managed source is still
|
||||
returned to normal so its exact RXPS setting can be restored.
|
||||
If the version gate required RXPS off, it stays off while that topology is
|
||||
preserved; use `target-rxps-settings.json` to restore it only after sending
|
||||
`normalradio`.
|
||||
@@ -467,10 +487,14 @@ Useful controls:
|
||||
downloading or staged on the target. Without it, that update is preserved.
|
||||
- `--source-shares-controller` is for a Full Companion whose USB Binary API is
|
||||
the controller while its TCP port `5001` is the source. It verifies that the
|
||||
source's port-`5000` public key equals the controller key. The Binary API
|
||||
first moves the shared physical radio, port `5002` then enables the local OTA
|
||||
egress gate, and cleanup sends `normalradio` before restoring the saved
|
||||
Binary radio tuple.
|
||||
source's port-`5000` public key equals the controller key. Port `5002` uses a
|
||||
bounded local `tempradio` override to move the shared physical radio without
|
||||
overwriting its saved normal tuple; the Binary API remains the authenticated
|
||||
transport. Cleanup sends local `normalradio`, proves that override inactive,
|
||||
and then reasserts the saved Binary tuple. It cannot be combined with
|
||||
`--leave-controller-radio`,
|
||||
because exact source RXPS restoration requires that shared physical radio to
|
||||
be back on its verified normal tuple.
|
||||
- `--require-system-watchdog-off` checks `get system.watchdog` immediately
|
||||
before every `ota install` transmission and refuses installation unless the
|
||||
destination reports `> off`. Use it for nRF52 chains whose bootloader cannot
|
||||
@@ -511,12 +535,14 @@ the destination.
|
||||
hashes, Merkle root, full-image hash where applicable, identity fields,
|
||||
signature, codec, base, and the firmware's 1024-byte maximum block size.
|
||||
Version-3 bootloader packages are refused before any target state changes.
|
||||
4. Save the controller's normal radio tuple, read every participant's version,
|
||||
save the destination's RXPS state, select the qualified RXPS policy, and
|
||||
show the confirmation prompt.
|
||||
5. Apply and verify that RXPS policy, then start TempRadio on the target,
|
||||
far-to-near relays, and the source;
|
||||
finally switch the controller to the same tuple and read it back. The runner
|
||||
4. Read and save a managed source's exact RXPS preference, disable and verify
|
||||
source RXPS, save the controller's normal radio tuple, read every
|
||||
participant's version, save the destination's RXPS state, select the
|
||||
qualified destination policy, and show the confirmation prompt.
|
||||
5. Apply and verify the destination RXPS policy, then start TempRadio on the
|
||||
target, far-to-near relays, and source. A separate controller is moved and
|
||||
read back through Binary; a shared Full Companion instead schedules its
|
||||
bounded local override while Binary remains the transport. The runner
|
||||
rejects a TempRadio window that cannot cover setup, seeder startup,
|
||||
discovery, the transfer timeout, final polling, and install checks.
|
||||
6. Start `motatool serve`, discover the exact eight-hex manifest ID, request
|
||||
@@ -527,14 +553,17 @@ the destination.
|
||||
7. Recheck that exact ID, give the target a short final TempRadio safety window,
|
||||
and request `ota install`. Then shorten each relay's TempRadio window so the
|
||||
normal multi-hop route returns, stop the seeder, shorten the source window,
|
||||
restore the controller, and probe `ota self` at 10 and 20 seconds instead of
|
||||
sleeping for 90 seconds. The exact new body hash is the readiness signal;
|
||||
restore the controller, and probe `ota self` every 10 seconds through the
|
||||
configured readiness window (five minutes by default). The exact new body
|
||||
hash is the readiness signal;
|
||||
only then does the runner require the exact package version. A relayed run
|
||||
continues the 10-second probes through the mandatory relay-return window. A
|
||||
source supplied with `--source-already-temp` is never modified.
|
||||
`--leave-controller-radio` moves the controller back to TempRadio only after
|
||||
this normal-channel verification. Restore the destination's exact original
|
||||
RXPS setting after normal-channel identity is proven.
|
||||
RXPS setting after normal-channel identity is proven. A managed source stays
|
||||
RXPS-off through that verification and its exact setting is restored only
|
||||
after its own TempRadio state is proven inactive.
|
||||
|
||||
Remote replies are matched only after queued messages have been drained and
|
||||
only when they come from the intended contact and fit the command. A ready
|
||||
@@ -565,26 +594,54 @@ budget. If a bounded window expires, rerun the same package after the nodes
|
||||
return to their normal channel; the manifest-ID check resumes its partial
|
||||
download without replacing it.
|
||||
|
||||
The working directory is retained and printed at exit. It contains the exact
|
||||
The working directory is created before a managed source can be changed, then
|
||||
retained and printed at exit. It contains the exact
|
||||
served mOTA, `motatool-serve.log`, extracted build inputs when needed, and
|
||||
`controller-radio.txt`. When the destination started with RXPS enabled it also
|
||||
contains protected `target-rxps-settings.json` for manual recovery. It contains
|
||||
no saved admin password.
|
||||
`controller-radio.txt`. A managed source also gets a protected
|
||||
`source-rxps-settings.json` containing its exact original preference and
|
||||
idempotent restore command. Its contents and directory entry are flushed before
|
||||
RXPS is disabled. When the destination
|
||||
started with RXPS enabled, protected `target-rxps-settings.json` records its
|
||||
manual recovery state. The RAK3401 chain points every nested step at one
|
||||
chain-root source record, so a rerun after host power loss does not adopt the
|
||||
temporary RXPS-off state as the original. A retained record is accepted only
|
||||
for the same managed CLI endpoint. These files contain no saved admin password.
|
||||
After exact source restoration, a standalone run atomically retires its record;
|
||||
the chain retains its shared record between steps and retires it only after the
|
||||
verified endpoint restoration completes.
|
||||
|
||||
## Interruption and recovery
|
||||
|
||||
Ctrl-C stops the seeder, detaches its serial folder, makes one best-effort
|
||||
request to shorten a source TempRadio window started by the script, and attempts
|
||||
to restore the controller. The target and relays remain on TempRadio only until
|
||||
their bounded windows end; rebooting also restores their saved radio settings.
|
||||
A normal cleanup restores the destination's exact RXPS periods. If that remote
|
||||
restore cannot be confirmed, use `target-rxps-settings.json` after the target
|
||||
returns to its normal channel.
|
||||
Ctrl-C stops the seeder, detaches its serial folder, makes a bounded attempt to
|
||||
shorten a source TempRadio window started by the script, and attempts to
|
||||
restore the controller. For a managed source it leaves RXPS off until the
|
||||
source is proven back on its normal radio, then restores and verifies the
|
||||
saved source preference. A transient success-path restore failure remains
|
||||
armed for one more idempotent cleanup attempt. The target and relays remain on
|
||||
TempRadio only until their bounded windows end; rebooting also restores their
|
||||
saved radio settings. A normal cleanup restores the destination's exact RXPS
|
||||
periods. If that remote restore cannot be confirmed, use
|
||||
`target-rxps-settings.json` after the target returns to its normal channel.
|
||||
A partial download remains safe. Once the target is reachable again (after its
|
||||
TempRadio window ends, or after putting the controller back on that tuple),
|
||||
rerunning the same package recognizes its manifest ID and resumes the existing
|
||||
session instead of clearing it.
|
||||
|
||||
When a chained run has already proved the newly running body hash, its retained
|
||||
previous package can briefly report `verifying staged blocks` after TempRadio
|
||||
reactivates the OTA manager. The runner waits only through the configured
|
||||
discovery timeout, keeps checking source liveness, and accepts only the same
|
||||
manifest becoming `ready to install` or the manager becoming idle. A changed
|
||||
ID, failed or incomplete state, or timeout stops the chain. It then proves the
|
||||
exact installed body again. If that same session is still attached and ready,
|
||||
the runner detaches it with `ota cancel`; if the manager has become idle, it
|
||||
sends no cancel. An ordinary-channel `no download` status proves only that the
|
||||
manager is idle, not that persistent staging was erased. The runner therefore
|
||||
does not issue or describe an IDLE cancel as durable cleanup. The next chain
|
||||
transition explicitly re-adopts and proves the expected previous MID before detaching it;
|
||||
after the final install OTAFIX has consumed the approval word, so any retained
|
||||
container is inert and is replaced by the next valid pull.
|
||||
|
||||
A hard process kill or host power loss cannot run cleanup. Recover a serial
|
||||
controller using the tuple saved in the printed work directory:
|
||||
|
||||
@@ -598,6 +655,12 @@ $radio = (Get-Content '.\meshcore-lora-ota-...\controller-radio.txt' -Raw).Trim(
|
||||
meshcli -s COM7 set radio $radio
|
||||
```
|
||||
|
||||
For a managed source, first return it to its ordinary radio, then inspect
|
||||
`source-rxps-settings.json` and issue its exact `restore_command` through the
|
||||
same serial or TCP-console endpoint recorded in that file. The command is
|
||||
idempotent; confirm the full setting with `get radio.rxps.config` before
|
||||
resuming an update.
|
||||
|
||||
If you stop during final confirmation, reconnect on the node's normal channel
|
||||
and run `ota self` and `ver`. A completed run returns success only when
|
||||
`ota self` reports a valid new body hash and `ver` exactly matches the package;
|
||||
|
||||
+10
-1
@@ -824,7 +824,7 @@ ota rescue install <base_hash16> internal-flash nRF52 only: recover from failed
|
||||
ota bootloader [status] capable allowlisted nRF52 repeater: installed BL identity/caps + staged confirmation
|
||||
ota bootloader install <MID8> <HASH16>
|
||||
explicitly verify/arm one complete trusted v3 package; never automatic
|
||||
ota cancel | drop | stop drop the current fetch session (frees the slot)
|
||||
ota cancel | drop | stop drop the fetch; durably invalidate device staging, or retain a folder partial for resume
|
||||
ota announce | adv serve self + send a beacon now
|
||||
ota self | id print this firmware's EndF (body/image size, base_hash)
|
||||
ota qspi | storage QSPI nRF52 only: JEDEC/SR1/stage/latched storage error (read-only)
|
||||
@@ -834,6 +834,15 @@ ota key | keys [add|rm <hex>] trusted signer allowlist ; bare = list
|
||||
ota dev ... bring-up helpers (stage/recv/serve/resume <MID8>/verify)
|
||||
```
|
||||
|
||||
For a device-backed pull, current firmware returns success only after the
|
||||
persistent store can no longer be reopened; flash/SD/QSPI I/O or readback
|
||||
failure is reported as an error even though the in-memory manager session was
|
||||
dropped. For a `folder` pull, cancellation detaches the live transfer but
|
||||
deliberately leaves the host `.part` file available for a later resume.
|
||||
If the shared receive engine is currently performing the MeshTower SD
|
||||
auto-archive capture, cancellation likewise detaches that archive transfer and
|
||||
retains its `.part` file; it does not erase the unrelated manual-install store.
|
||||
|
||||
---
|
||||
|
||||
<a id="12-apply-bootloader-contract"></a>
|
||||
|
||||
+12
-1
@@ -212,6 +212,15 @@ falls back to flash. To **stop** a download you no longer want:
|
||||
ota cancel
|
||||
```
|
||||
|
||||
For a download staged on the node, success means current firmware also
|
||||
invalidated the persistent flash/SD/QSPI copy; it reports an error if that
|
||||
media operation cannot be verified. For a `folder` capture, cancel only
|
||||
detaches the live session and retains the host partial so it can be resumed or
|
||||
removed on the host deliberately. On a MeshTower SD auto-archive capture,
|
||||
cancel also detaches the live archive session and retains its card partial for
|
||||
the archive service to resume later; use `ota cache off` to stop new archive
|
||||
captures.
|
||||
|
||||
### 4. Install a downloaded update
|
||||
|
||||
Once `ota status` shows the download is **ready to install**:
|
||||
@@ -292,7 +301,9 @@ complete target inventory, storage layouts, and safety contract.
|
||||
|
||||
### 5. If something goes wrong
|
||||
|
||||
- A download that stalls or gets interrupted just **resumes** later, or you can `ota cancel` and try again.
|
||||
- A download that stalls or gets interrupted just **resumes** later. Use
|
||||
`ota cancel` to durably discard device staging; a folder capture keeps its
|
||||
host partial for a later resume.
|
||||
- A legacy app-only internal-flash **nRF52** that still runs but reports `no EndF` can use the pre-provisioned rescue path
|
||||
if its physical EndF is intact and only app-side validation is failing. Fetch the exact `[rescue]`
|
||||
in-place delta with an explicit acknowledgement, obtain its 16-hex-digit `base_hash` from the package
|
||||
|
||||
+235
-97
@@ -1,24 +1,32 @@
|
||||
# RAK3401 1W repeater compact LoRa update chain
|
||||
|
||||
> Status: physically qualified on 19-Aug-2026. All nine exact transitions
|
||||
> completed on a deployed RAK3401 1W with its existing bootloader, using a
|
||||
> Heltec V4 source at 909.950 MHz / 500 kHz / SF5 / CR5. Every post-boot EndF
|
||||
> hash matched. The same files also pass independent reconstruction, container
|
||||
> verification, and both deployed Preview 6 and current OTAFIX simulators.
|
||||
> Status: **unreleased candidate with physically passed package transitions.**
|
||||
> On 29-Aug-2026 all ten exact package transitions completed on the target
|
||||
> RAK3401 with its deployed `0.9.2-OTAFIX2.4` bootloader and a Heltec V4 source at
|
||||
> 909.950 MHz / 500 kHz / SF5 / CR5. Every intermediate EndF hash matched.
|
||||
> Independent SWD readback then matched the endpoint application, original
|
||||
> bootloader, SoftDevice/MBR, and UICR byte-for-byte. The host runner received
|
||||
> cleanup/recovery fixes after that test and has not had a new clean end-to-end
|
||||
> physical rerun. Multi-hop and alternate-bandwidth estimates remain planning
|
||||
> data, not physical qualification.
|
||||
|
||||
Use this asset:
|
||||
The candidate is a local qualification artifact, not a GitHub release:
|
||||
|
||||
```text
|
||||
RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip
|
||||
RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip
|
||||
```
|
||||
|
||||
- ZIP SHA-256: `9f80eef191b88833bf4d2e4fea559cf5233ca53f9266ba310d447f37fa445f3a`
|
||||
- inner `SHA256SUMS.txt` SHA-256: `73d96e23237896a3e342fe736be12d94087a813bf09ad609fb55330bbe586055`
|
||||
- endpoint image SHA-256: `2784e4b645bc3dc198de0b8b18d3d7369cd02eca61cd71c46a51b61854da5345`
|
||||
- endpoint EndF body hash: `4BB1526BF647547D`
|
||||
- candidate ZIP SHA-256: `c0b33f4568985e8b2b8dc99411295907212cf2bad21764b6333d5e0ba298fd61`
|
||||
- inner `SHA256SUMS.txt` SHA-256: `3f8c4af8096b96a4aa6506825c387cc8a06f74d5213a29c9387bd11689546881`
|
||||
- endpoint image SHA-256: `31c182c888ceb1135e5afb2376610d93cee2e807b556c838e07fd4486c79d095`
|
||||
- endpoint EndF body hash: `9179B98432895924`
|
||||
|
||||
Until that asset is uploaded to the release tag pinned by the runner, pass its
|
||||
local path with `--bundle`.
|
||||
Always pass the exact local path with `--bundle`. The runner has no download
|
||||
URL for this candidate. `--verify-only` remains the safe default. The hidden
|
||||
`--accept-test-candidate` override is intentionally absent from `--help` and
|
||||
remains required until this exact artifact is published with its qualification
|
||||
record; direct physical success does not silently turn a local file into a
|
||||
released deployment asset.
|
||||
|
||||
## Exact destination
|
||||
|
||||
@@ -29,29 +37,32 @@ target:
|
||||
- hardware: `RAK_3401`
|
||||
- role: `RAK_3401_repeater_lora_ota_no_external_sensors`
|
||||
- start: `v1.16.7.0-c1caa5ad`, EndF `71F4026CBE4B8B74`
|
||||
- endpoint source: MeshCore `e742333a0ee393b0d55d2414f25b28f2f67e1ea1`
|
||||
- endpoint label: `v1.17.1.02-halo-keymind-cascade-dev-e742333a`
|
||||
- endpoint packed version: `0x01110102` (EndF tools render this as `1.17.1.2`)
|
||||
- endpoint source: MeshCore `fd98bc90f7fb25d5956facd550e227b217abbe95`
|
||||
- endpoint label: `v1.17.1.5-halo-keymind-cascade-dev-fd98bc90`
|
||||
- endpoint packed version: `0x01110105`
|
||||
- deployment target key: `63d8df6387eaffd2e25db7d2a8ad967a65202182a48d681d7e7a9260f917280d`
|
||||
|
||||
### Historical endpoint GPS limitation
|
||||
### Retained and omitted hardware support
|
||||
|
||||
This chain's exact `RAK_3401_repeater_lora_ota_no_external_sensors` endpoint
|
||||
compiles with `ENV_INCLUDE_GPS` removed. It will not detect, configure, or read
|
||||
a RAK12501 GPS module. GPS requires the ordinary full-sensor
|
||||
`RAK_3401_repeater` firmware and the RAK12501 must be installed in sensor slot
|
||||
A. Do not use slot D with the RAK13302 1 W radio: the GPS reset/PPS signals
|
||||
would overlap the radio's BUSY/DIO1 signals. The full-sensor build is a
|
||||
different target and is not an endpoint of this compact OTA chain.
|
||||
Despite the legacy `no_external_sensors` target name, the endpoint retains the
|
||||
RAK12501 GPS provider and the common INA3221, INA219, INA226, and INA260 I2C
|
||||
voltage/current monitors. Install RAK12501 in sensor slot A. Do not use slot D
|
||||
with the RAK13302 1 W radio because the GPS reset/PPS signals overlap the
|
||||
radio's BUSY/DIO1 wiring.
|
||||
|
||||
This limitation describes the immutable released chain endpoint identified
|
||||
above. Newly built RAK3401 OTA repeater firmware retains RAK12501 support in
|
||||
slot A, but that newer image is not retroactively part of this exact chain.
|
||||
The reduced profile omits optional external environmental and ranging devices:
|
||||
AHTX0, BME280, BMP280, SHTC3, SHT4x, LPS22HB, MLX90614, VL53L0X, BME680,
|
||||
BMP085, RAK12035, and BME680 BSEC. Board radio, display, buttons, battery
|
||||
telemetry, GPS, and the INA monitor family remain. No additional “remove I2C
|
||||
temperature” image is needed: the optional I2C temperature/environment drivers
|
||||
are already in the omitted set, while removing the remaining I2C/INA support
|
||||
would discard desired voltage/current monitoring and create another firmware
|
||||
identity without helping this route.
|
||||
|
||||
Do not use the chain on another target ID, hardware family, starting image, or
|
||||
firmware body hash. The runner checks all four.
|
||||
|
||||
## Why the old bootloader works
|
||||
## Why Preview 5 and the old staging ceiling work
|
||||
|
||||
The deployed bootloader is not changed and no package uses the newer expanded
|
||||
`0xED000` ceiling. Each package remains bottom-aligned below the old
|
||||
@@ -66,31 +77,43 @@ The old bootloader already reads and validates that encoded detools geometry
|
||||
before its first destructive write. The previous application receiver was the
|
||||
limiting part: it reserved a fixed `0x98000` workspace before accepting a
|
||||
download. Step 1 therefore remains byte-for-byte identical to the physically
|
||||
passed 89,844-byte package. Step 2 is a 46,363-byte fixed-workspace package
|
||||
passed 89,844-byte package. Step 2 is a 46,364-byte fixed-workspace package
|
||||
that installs the compatibility receiver. Later receivers stage above the
|
||||
real EndF image extent and let the bootloader enforce each package's selected
|
||||
workspace.
|
||||
|
||||
An exhaustive search tested every page-aligned workspace relevant to a route
|
||||
shorter than nine packages: 9,430 candidate geometries across 272 possible
|
||||
layer-skipping edges. None fit. The 61 edges on the shortest-path graph were
|
||||
then swept across another 2,296 workspace choices; 840 nine-package routes
|
||||
were compared by total transfer size. The selected route is the byte-minimum
|
||||
shortest route:
|
||||
Every package in this candidate is an ordinary application container using
|
||||
format ABI 2 and in-place codec 2. The historical bridge applications scan the
|
||||
bootloader capability marker byte-by-byte, so they recognize the released
|
||||
Preview 5 RAK4631 marker at absolute address `0xFCDEA` even though it is two
|
||||
bytes off a word boundary. The `fd98bc90` endpoint deliberately retains that
|
||||
exact Preview 5 application-update fallback. Its separately compiled internal
|
||||
bootloader-update feature remains unavailable on Preview 5 because privileged
|
||||
self-update requires a current ABI 3 marker with the exact storage profile.
|
||||
Installing this chain therefore does not replace the bootloader, and the final
|
||||
application can still accept a future valid ABI-2/codec-2 application update.
|
||||
|
||||
The schema-2 exhaustive search checked 17,402 candidate geometries and found
|
||||
121 feasible forward edges in the declared 33-image inventory. Fourteen
|
||||
ten-package routes tied for the minimum package count; the selected route has
|
||||
the smallest total container size among them. That minimum applies to the
|
||||
declared inventory, not every conceivable firmware image. The selected route
|
||||
is:
|
||||
|
||||
| Step | From | To | Workspace | Package | Margin |
|
||||
|---:|---|---|---:|---:|---:|
|
||||
| 1 | 1.16.7.0 | 1.16.7.9 | `0x98000` | 89,844 | 0 |
|
||||
| 2 | 1.16.7.9 | 1.16.7.10 | `0x98000` | 46,363 | 40,960 |
|
||||
| 3 | 1.16.7.10 | 1.16.8.7 | `0x91000` | 106,030 | 12,288 |
|
||||
| 2 | 1.16.7.9 | 1.16.7.10 | `0x98000` | 46,364 | 40,960 |
|
||||
| 3 | 1.16.7.10 | 1.16.8.7 | `0x91000` | 106,029 | 12,288 |
|
||||
| 4 | 1.16.8.7 | 1.16.9.105 | `0x89000` | 149,927 | 0 |
|
||||
| 5 | 1.16.9.105 | 1.16.9.110 | `0x8E000` | 111,307 | 16,384 |
|
||||
| 6 | 1.16.9.110 | 1.16.9.111 | `0x8D000` | 70,679 | 61,440 |
|
||||
| 7 | 1.16.9.111 | 1.16.9.116 | `0x89000` | 143,441 | 4,096 |
|
||||
| 8 | 1.16.9.116 | 1.16.9.118 | `0x81000` | 98,188 | 86,016 |
|
||||
| 9 | 1.16.9.118 | 1.17.1.02 | `0x7D000` | 186,385 | 12,288 |
|
||||
| 5 | 1.16.9.105 | 1.16.9.110 | `0x8E000` | 111,312 | 16,384 |
|
||||
| 6 | 1.16.9.110 | 1.16.9.113 | `0x8F000` | 120,624 | 4,096 |
|
||||
| 7 | 1.16.9.113 | 1.16.9.117 | `0x8B000` | 139,637 | 0 |
|
||||
| 8 | 1.16.9.117 | 1.16.10.0 | `0x7E000` | 174,202 | 20,480 |
|
||||
| 9 | 1.16.10.0 | 1.17.1.3 | `0x6B000` | 271,569 | 0 |
|
||||
| 10 | 1.17.1.3 | 1.17.1.5 | `0x74000` | 173,334 | 61,440 |
|
||||
|
||||
Total mOTA transfer data is 1,002,164 bytes. `ROUTE.json`, `CHAIN.csv`, and
|
||||
Total mOTA transfer data is 1,382,842 bytes. `ROUTE.json`, `CHAIN.csv`, and
|
||||
`validation-results.json` in the bundle pin the exact geometry and image hash
|
||||
for every transition.
|
||||
|
||||
@@ -105,17 +128,27 @@ No password or device is needed:
|
||||
|
||||
```bash
|
||||
python3 tools/lora_ota/rak3401_mota_chain.py \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip \
|
||||
--work-dir ./rak3401-mota-chain-work \
|
||||
--motatool /path/to/motatool \
|
||||
--verify-only
|
||||
```
|
||||
|
||||
The runner checks the outer ZIP, complete inner checksum coverage, all nine
|
||||
manifests, continuity, image anchors, final recovery image, and every container
|
||||
with `motatool`.
|
||||
When given the ZIP, the runner checks its pinned outer hash and binds the
|
||||
extraction cache to that exact archive. An explicitly supplied extracted root
|
||||
starts at the pinned inner checksum list. In either case it checks complete
|
||||
inner coverage, all ten manifests, continuity, image anchors, both recovery
|
||||
images, and every container with `motatool`. Passing this command means offline
|
||||
and simulator qualification only; it does not unlock live use.
|
||||
|
||||
## Direct recoverable bench run
|
||||
## Current guarded reproduction command
|
||||
|
||||
These are the same direct-link arguments used for the physical transition run,
|
||||
but the current host runner also contains later cleanup and recovery fixes.
|
||||
That revised runner has not been cleanly rerun through all ten transitions.
|
||||
Keep local USB/SWD recovery available while the artifact remains unreleased.
|
||||
The hidden override bypasses only the candidate-status block; all identity,
|
||||
checksum, bootloader, route, watchdog, and post-boot gates still run.
|
||||
|
||||
Restore the test start locally with
|
||||
`recovery/test-start/RAK3401-test-start-v1.16.7-c1caa5ad.uf2`, then run:
|
||||
@@ -124,7 +157,7 @@ Restore the test start locally with
|
||||
export MESHCORE_ADMIN_PASSWORD='password'
|
||||
|
||||
python3 tools/lora_ota/rak3401_mota_chain.py \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip \
|
||||
--work-dir ./rak3401-mota-chain-work \
|
||||
--controller-serial /dev/ttyACM0 \
|
||||
--source-tcp 192.168.1.51:5001 \
|
||||
@@ -135,6 +168,7 @@ python3 tools/lora_ota/rak3401_mota_chain.py \
|
||||
--ota-hops 0 \
|
||||
--legacy-full-airtime \
|
||||
--motatool /path/to/motatool \
|
||||
--accept-test-candidate \
|
||||
--yes
|
||||
```
|
||||
|
||||
@@ -145,23 +179,76 @@ zero and restores it at the endpoint. Use that option only where the selected
|
||||
frequency and local duty-cycle rules permit a full transmit budget; omit it
|
||||
otherwise.
|
||||
|
||||
### Measured direct-link timing
|
||||
|
||||
The first `ready to install` transfer ages were:
|
||||
|
||||
| Step | Blocks | Transfer |
|
||||
|---:|---:|---:|
|
||||
| 1 | 88 | 598 s |
|
||||
| 2 | 45 | 42 s |
|
||||
| 3 | 103 | 81 s |
|
||||
| 4 | 146 | 105 s |
|
||||
| 5 | 109 | 76 s |
|
||||
| 6 | 118 | 94 s |
|
||||
| 7 | 136 | 102 s |
|
||||
| 8 | 170 | 127 s |
|
||||
| 9 | 264 | 187 s |
|
||||
| 10 | 169 | 193 s |
|
||||
|
||||
The physically observed bulk-transfer total for those exact packages is 1,605
|
||||
seconds (26m45s). The log came from the pre-cleanup-fix runner revision. Step 1
|
||||
was measured before the host began forcing source RXPS off and is therefore a
|
||||
conservative outlier. A new complete run with that fix is projected at roughly
|
||||
18–19 minutes of bulk transfer, but that projection is not a substitute for
|
||||
the measured total. Allow about 70–75 minutes direct at BW500 for validation,
|
||||
ten transfers, installs/reboots, retained-store checks, and final restoration.
|
||||
|
||||
Step 10's longer per-block time was not a V4 TempRadio delay: the source RXPS
|
||||
readback was off. Its v1.17.1.3 requester uses non-overlapping adaptive flights
|
||||
and a five-second recovery floor and logged more block reloads than step 8.
|
||||
The V4 itself intentionally schedules its local radio handoff after 1.5 seconds
|
||||
so the normal-channel command reply can drain; the host waits three seconds and
|
||||
then checks the live TempRadio state.
|
||||
|
||||
For planning only, scale the conservative measured 26m45s bulk baseline by the
|
||||
qualified adaptive-preamble airtime and by each additional relay transmission:
|
||||
|
||||
| Bandwidth, SF5/CR5 | Direct / 0 relays | 1 relay | 2 relays |
|
||||
|---:|---:|---:|---:|
|
||||
| 500 kHz | 26m45s | 53m30s | 1h20m15s |
|
||||
| 250 kHz | 46m51s | 1h33m42s | 2h20m33s |
|
||||
| 125 kHz | 1h27m03s | 2h54m07s | 4h21m11s |
|
||||
| 62.5 kHz | 2h54m07s | 5h48m15s | 8h42m23s |
|
||||
|
||||
These are transfer-only estimates, not measured alternate-bandwidth or relay
|
||||
results. Packet loss, relay contention, discovery, administrative commands,
|
||||
staged verification, and ten reboot cycles add time. The bandwidth factors are
|
||||
not simple powers of two because the adaptive wire preamble is 128 symbols at
|
||||
500 kHz, 64 at 250 kHz, and 32 at 125/62.5 kHz.
|
||||
|
||||
## Manual operation
|
||||
|
||||
The pinned runner is the safer method. If it cannot be used, follow every
|
||||
check below and keep a written copy of each original setting. Do not skip a
|
||||
step even when a later package appears in `ota ls`.
|
||||
The automated runner is preferred because it binds every transition to the
|
||||
expected MID and body hash. The manual details below document the checks used
|
||||
during qualification and recovery; they are not permission to skip the
|
||||
runner's candidate gate. Keep a written copy of each original setting and
|
||||
never skip a step even when a later package appears in `ota ls`.
|
||||
|
||||
### 1. Verify and extract the asset
|
||||
|
||||
```bash
|
||||
sha256sum RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip
|
||||
unzip RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip
|
||||
cd RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a
|
||||
sha256sum RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip
|
||||
unzip RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip
|
||||
cd RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90
|
||||
sha256sum -c SHA256SUMS.txt
|
||||
for package in motas/*.mota; do motatool verify "$package" || exit 1; done
|
||||
```
|
||||
|
||||
The outer hash must be `9f80eef191b88833bf4d2e4fea559cf5233ca53f9266ba310d447f37fa445f3a`.
|
||||
The outer hash must be
|
||||
`c0b33f4568985e8b2b8dc99411295907212cf2bad21764b6333d5e0ba298fd61`,
|
||||
and the SHA-256 of the extracted `SHA256SUMS.txt` must be
|
||||
`3f8c4af8096b96a4aa6506825c387cc8a06f74d5213a29c9387bd11689546881`.
|
||||
Do not continue after any checksum or verification error.
|
||||
|
||||
### 2. Record and prepare the destination
|
||||
@@ -218,13 +305,12 @@ tests use `hops 0`: `hops 1` makes the target echo each source response despite
|
||||
there being no intermediate relay, increasing half-duplex loss and legacy
|
||||
three-second retries.
|
||||
|
||||
The exact compact chain never qualifies for fast RXPS: its bridge and endpoint
|
||||
versions predate the v1.17.1.5 adaptive-preamble contract. Keep RXPS off for
|
||||
this historical chain whether using SF5/BW250 or the faster direct-bench
|
||||
SF5/BW500 tuple. For a later update whose target, controller, source, and
|
||||
relays are all v1.17.1.5 or newer, use the generic LoRa OTA runner; it can
|
||||
retain RXPS at the qualified level-8/preamble-64 boundary for SF5/BW250 or the
|
||||
level-8/preamble-128 boundary for SF5/BW500.
|
||||
The exact candidate cannot use fast RXPS throughout because its historical
|
||||
bridge applications predate the v1.17.1.5 adaptive-preamble contract. Keep
|
||||
RXPS off for all ten steps whether using SF5/BW250 or the faster direct-bench
|
||||
SF5/BW500 tuple. After the final endpoint is proven, a later update whose
|
||||
target, controller, source, and relays are all v1.17.1.5 or newer can use the
|
||||
generic LoRa OTA runner and its qualified adaptive-preamble boundaries.
|
||||
|
||||
Put the source on the identical TempRadio tuple. A current ASCII-first Full
|
||||
Companion recognizes `motatool`'s initial `ota folder on` line directly:
|
||||
@@ -243,30 +329,48 @@ every step so the source emits a fresh catalog advert. Leave it running during
|
||||
the download and stop it with Ctrl-C only after the destination reports `ready
|
||||
to install`.
|
||||
|
||||
### 3. Install all nine packages in order
|
||||
### 3. Install all ten packages in order
|
||||
|
||||
| Step | Manifest ID | Before hash | Expected version | After hash |
|
||||
|---:|---|---|---|---|
|
||||
| 1 | `C147BCEF` | `71F4026CBE4B8B74` | `1.16.7.9` | `42BC53A64288E845` |
|
||||
| 2 | `C510B628` | `42BC53A64288E845` | `1.16.7.10` | `6F6E51D63BF3E93F` |
|
||||
| 3 | `8CAD7045` | `6F6E51D63BF3E93F` | `1.16.8.7` | `F04DA8DB515E5C6D` |
|
||||
| 4 | `8B1EEFF6` | `F04DA8DB515E5C6D` | `1.16.9.105` | `3894A4D7B11ED282` |
|
||||
| 5 | `687E3BEA` | `3894A4D7B11ED282` | `1.16.9.110` | `CDC5CA630881DA15` |
|
||||
| 6 | `0D59A34B` | `CDC5CA630881DA15` | `1.16.9.111` | `D0CC4A9E135B9A4D` |
|
||||
| 7 | `2C5C2082` | `D0CC4A9E135B9A4D` | `1.16.9.116` | `66272A0E7FEF2773` |
|
||||
| 8 | `994AB743` | `66272A0E7FEF2773` | `1.16.9.118` | `8E00019AA08E00BF` |
|
||||
| 9 | `FE986948` | `8E00019AA08E00BF` | `1.17.1.02` | `4BB1526BF647547D` |
|
||||
| 2 | `BEB578FC` | `42BC53A64288E845` | `1.16.7.10` | `1AD2AA8B6C478DA6` |
|
||||
| 3 | `6D8EF814` | `1AD2AA8B6C478DA6` | `1.16.8.7` | `D709A25308BC1831` |
|
||||
| 4 | `40AC4CEB` | `D709A25308BC1831` | `1.16.9.105` | `457BEDA5E6406C81` |
|
||||
| 5 | `F6C8890B` | `457BEDA5E6406C81` | `1.16.9.110` | `E76BFA205634CAB8` |
|
||||
| 6 | `82405E89` | `E76BFA205634CAB8` | `1.16.9.113` | `65328FC8A1FBED2D` |
|
||||
| 7 | `80F40DB6` | `65328FC8A1FBED2D` | `1.16.9.117` | `AF7532E13337FADD` |
|
||||
| 8 | `DBF6310D` | `AF7532E13337FADD` | `1.16.10.0` | `22002359BBDFA76E` |
|
||||
| 9 | `C2979E08` | `22002359BBDFA76E` | `1.17.1.3` | `FE65A6135A1E7B3F` |
|
||||
| 10 | `319DEC92` | `FE65A6135A1E7B3F` | `1.17.1.5` | `9179B98432895924` |
|
||||
|
||||
For each row, first prove that `ota self` exactly matches its **Before hash**.
|
||||
Then run:
|
||||
For each row, first prove that `ota self` exactly matches its **Before hash**,
|
||||
then inspect the manager before changing it:
|
||||
|
||||
```text
|
||||
ota status
|
||||
```
|
||||
|
||||
Proceed to discovery only after that status is reconciled. When the manager is
|
||||
idle, run:
|
||||
|
||||
```text
|
||||
ota cancel
|
||||
ota ls
|
||||
ota pull MANIFEST_ID flash
|
||||
ota status
|
||||
```
|
||||
|
||||
If the first status reports this row's MID as partial, use `ota pull` with that
|
||||
same MID to resume it. If this row's MID is already ready, do not pull or
|
||||
cancel it; continue with the install checks below. If status reports the
|
||||
immediately previous row's MID as `verifying staged blocks`, wait through the
|
||||
bounded discovery window; require either that exact MID to become `ready to
|
||||
install` or the manager to become `no download`, and re-prove the current
|
||||
**Before hash**. Cancel only the same previous MID while it is visibly
|
||||
attached and ready. If it has become idle, send no cancel. Stop on any other
|
||||
MID, failed or incomplete state, ambiguous reply, or timeout. Never issue a
|
||||
blind `ota cancel`: it can discard a resumable or unrelated session.
|
||||
|
||||
Repeat `ota status` at a restrained interval until it says `ready to install`.
|
||||
If the manifest is initially absent, wait for a fresh source advert, run
|
||||
`ota ls` again, or restart `motatool`; do not pull a different ID. Stop the
|
||||
@@ -277,19 +381,35 @@ get system.watchdog
|
||||
ota install
|
||||
```
|
||||
|
||||
The watchdog reply must still be `> off`. Allow up to 90 seconds for the USB
|
||||
port to disappear and return. Then run `ver`, `ota self`, and `ota help`.
|
||||
The watchdog reply must still be `> off`. Allow the current default five-minute
|
||||
readiness window for the USB/LoRa identity to return; automation probes every
|
||||
10 seconds and returns early as soon as the exact body is visible. Then run
|
||||
`ver`, `ota self`, and `ota help`.
|
||||
Require the row's exact **After hash**, require `bootloader: apply OK`, and
|
||||
require `rescue install <hash16>` in the help before exposing that bridge to
|
||||
the next package. Clear only the now-proven retained download with `ota cancel`,
|
||||
reapply the transfer guardrails and TempRadio tuple, restart `motatool`, and
|
||||
continue with the next row.
|
||||
the next package. This rescue-help gate applies to steps 1-9; step 10 is the
|
||||
endpoint and is not exposed to another package in this chain. A normal-radio
|
||||
`ota status` response of `no download` proves only that the manager is idle;
|
||||
legacy internal-flash firmware does not durably erase a retained container by
|
||||
accepting an IDLE `ota cancel`. Do not claim otherwise. When the same expected
|
||||
manifest reports `verifying staged blocks` after the next TempRadio starts,
|
||||
wait only for the bounded discovery window and require that it become `ready
|
||||
to install` or `no download`; stop on a changed ID, failed/incomplete state, or
|
||||
timeout. Prove the exact running body before cancelling an attached completed
|
||||
session. OTAFIX consumes the approval word during a successful install, so a
|
||||
retained endpoint container is inert and a later valid pull replaces it.
|
||||
Reapply the transfer
|
||||
guardrails and TempRadio tuple, restart `motatool`, and continue with the next
|
||||
row.
|
||||
|
||||
If the board boots but `ota self` says the EndF is invalid, stop. Only when
|
||||
`ota status` still identifies that row's complete staged package may the
|
||||
guarded `ota rescue install BEFORE_HASH` command be used. It is not a force
|
||||
option and must use that row's exact Before hash. If the board does not boot,
|
||||
recover the documented start/recovery UF2 locally over USB.
|
||||
recover the documented start/recovery UF2 locally over USB. The fd98 endpoint
|
||||
deliberately disables rescue on its shared internal bootloader-update build;
|
||||
if its EndF is invalid, use local DFU/SWD rather than attempting another LoRa
|
||||
package.
|
||||
|
||||
### 4. Intermediate relays and restoration
|
||||
|
||||
@@ -307,7 +427,7 @@ window. The source's manifest-fragment gap independently follows active packet
|
||||
airtime and duty spacing, clamped to 100-1000 ms. These two delays solve
|
||||
different problems.
|
||||
|
||||
After step 9 is proven, restore each saved destination and relay value exactly,
|
||||
After step 10 is proven, restore each saved destination and relay value exactly,
|
||||
including `rxdelay`, RXPS, CPU power saving, `af`, `ota config hops`, and relay
|
||||
timing. Current full-parser repeater firmware accepts `normalradio` and restores
|
||||
the saved tuple after replying on the temporary channel:
|
||||
@@ -316,7 +436,7 @@ the saved tuple after replying on the temporary channel:
|
||||
normalradio
|
||||
```
|
||||
|
||||
The historical endpoint in this pinned bundle predates that command. If it
|
||||
The fd98 endpoint supports that command. If a recovery build unexpectedly
|
||||
replies `Unknown command`, shorten its lease to one minute with the same tuple:
|
||||
|
||||
```text
|
||||
@@ -335,20 +455,26 @@ A Full Companion source is the exception: its local TCP console supports
|
||||
`normalradio` and the automated runner uses it before restoring the shared
|
||||
Binary API radio tuple.
|
||||
|
||||
Final success requires version `v1.17.1.02-halo-keymind-cascade-dev-e742333a`,
|
||||
body hash `4BB1526BF647547D`, target `2FA509C1`, hardware `RAK_3401`, and
|
||||
Final lab success requires version
|
||||
`v1.17.1.5-halo-keymind-cascade-dev-fd98bc90`, body hash
|
||||
`9179B98432895924`, target `2FA509C1`, hardware `RAK_3401`, and
|
||||
`get system.watchdog` reporting `> on`. If a relay cannot be restored before
|
||||
its TempRadio lease ends, wait for it to return to the normal channel and
|
||||
restore its saved `rxdelay` and `txdelay` there.
|
||||
|
||||
## Two-relay deployment
|
||||
|
||||
For two intermediate relays, list them farthest-to-nearest and use three OTA
|
||||
hops. Run a non-mutating preflight first:
|
||||
The exact direct package transitions above were physically observed; this
|
||||
post-run host-runner revision and multi-hop operation are not cleanly
|
||||
end-to-end qualified. A controlled, recoverable multi-hop lab can exercise two
|
||||
intermediate relays by
|
||||
listing them farthest-to-nearest and using three OTA hops. Run a non-mutating
|
||||
preflight first; even preflight requires the hidden candidate override because
|
||||
the artifact remains unreleased and the command connects to live devices:
|
||||
|
||||
```bash
|
||||
python3 tools/lora_ota/rak3401_mota_chain.py \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.02-e742333a.zip \
|
||||
--bundle /path/to/RAK3401-update-chain-v1.16.7-c1caa5ad-to-v1.17.1.5-fd98bc90.zip \
|
||||
--work-dir ./rak3401-mota-chain-work \
|
||||
--controller-serial /dev/ttyACM0 \
|
||||
--source-tcp SOURCE_IP:5001 \
|
||||
@@ -360,6 +486,7 @@ python3 tools/lora_ota/rak3401_mota_chain.py \
|
||||
--temp-radio 909.950,250,5,5,120 \
|
||||
--ota-hops 3 \
|
||||
--motatool /path/to/motatool \
|
||||
--accept-test-candidate \
|
||||
--preflight-only
|
||||
```
|
||||
|
||||
@@ -370,16 +497,27 @@ legal at the deployment location.
|
||||
## Rescue and completion
|
||||
|
||||
The guarded `ota rescue install <base_body_hash>` command is present after
|
||||
step 1. It is not a force command: it refuses a valid normal EndF, a base
|
||||
mismatch, wrong target/hardware, or invalid payload. A failure before step 1
|
||||
completes still requires local USB recovery because the deployed start image
|
||||
predates the rescue command.
|
||||
step 1 and remains available through the step-9 bridge. It is not a force
|
||||
command: it refuses a valid normal EndF, a base mismatch, wrong
|
||||
target/hardware, or invalid payload. A failure before step 1 completes still
|
||||
requires local USB recovery because the deployed start image predates the
|
||||
rescue command. The final fd98 endpoint uses the stricter shared-slot profile;
|
||||
its invalid-EndF recovery is local DFU/SWD, not the rescue command.
|
||||
|
||||
Before the first mutation, the runner saves the destination's RXPS periods,
|
||||
CPU power-saving state, RX flood delay, airtime factor, and OTA hop reach in
|
||||
the persistent work directory. It reads the destination, controller, source,
|
||||
and relay versions before applying the same RXPS policy as the generic runner.
|
||||
The pinned historical chain therefore verifies `radio.rxps off` after every
|
||||
the persistent work directory, flushing both the file and its directory entry
|
||||
before changing a guardrail. It also saves the managed source's exact RXPS
|
||||
preference in the chain-root `source-rxps-settings.json` before disabling it.
|
||||
Every step reuses that protected record, so a rerun after a killed host process
|
||||
restores the chain-start preference instead of treating the temporary off state
|
||||
as original. The record is accepted only for the same CLI endpoint. The runner
|
||||
retires that chain-root record only after it has proved the source is back on
|
||||
its normal radio and restored the exact preference at the verified endpoint.
|
||||
The runner
|
||||
reads the destination, controller, source, and relay versions before applying
|
||||
the same RXPS policy as the generic runner.
|
||||
The mixed historical chain therefore verifies `radio.rxps off` after every
|
||||
bridge reboot; a future all-v1.17.1.5-or-newer SF5/BW250 chain would instead
|
||||
keep RXPS on under the qualified level-8/preamble-64 boundary. It also verifies
|
||||
`powersaving off` and `rxdelay 0`, plus `af 0` when
|
||||
@@ -398,8 +536,8 @@ the relay leaves TempRadio.
|
||||
|
||||
The runner also keeps the watchdog off across the chain, rechecks identity and
|
||||
OTA reach before every package, requires the exact post-boot EndF hash, and
|
||||
only re-enables the watchdog after step 9. Success requires endpoint body hash
|
||||
`4BB1526BF647547D`, target `2FA509C1`, hardware `RAK_3401`, normal radio
|
||||
only re-enables the watchdog after step 10. Lab success requires endpoint body
|
||||
hash `9179B98432895924`, target `2FA509C1`, hardware `RAK_3401`, normal radio
|
||||
`910.525 MHz / 62.5 kHz / SF7 / CR5`, and the watchdog verified on.
|
||||
|
||||
## External-radio startup and manual USB recovery
|
||||
|
||||
@@ -647,19 +647,45 @@ bool handle_ota_command(const char* command, char* reply, mesh::MainBoard& board
|
||||
} else if (is_cmd(a, "drop|cancel|stop", &rest)) {
|
||||
if (c.apply_pending) { strcpy(reply, "ERR update is armed; reboot is pending"); return true; }
|
||||
OtaManager::FetchState fs = c.manager.fetchState();
|
||||
const bool was_folder = c.fetch_to_folder;
|
||||
bool was_sd_archive = false;
|
||||
char midhx[9]; strcpy(midhx, "-");
|
||||
if (fs != OtaManager::IDLE) mesh::Utils::toHex(midhx, c.manager.fetchManifestId(), 4);
|
||||
#if defined(NRF52_PLATFORM) && defined(OTA_SD_STORE)
|
||||
was_sd_archive = c.sdCacheFetching();
|
||||
c.stopSdCacheFetch();
|
||||
#endif
|
||||
c.manager.reset_session(); c.manager.want(0); c.manager.want_mid(nullptr);
|
||||
if (was_folder && c.folder_dest) c.folder_dest->clear();
|
||||
c.fetch_to_folder = false;
|
||||
c.manager.set_fetch_store(&c.fetch_store); // revert to the default flash store (a folder pull switched it)
|
||||
#if defined(NRF52_PLATFORM) && !defined(OTA_SD_STORE) && !defined(OTA_QSPI_STORE)
|
||||
c.manager.set_accept_full(false);
|
||||
#endif
|
||||
c.fetch_store.clear(); c.serving = false; c.serve_expected = 0; c.session_started_ms = 0;
|
||||
snprintf(reply, 160, "OK dropped session (was %c mid=%s); slot free for a new pull", fstate_char(fs), midhx);
|
||||
const bool discarded = !was_folder && !was_sd_archive &&
|
||||
c.fetch_store.discard();
|
||||
// Fetch cancellation and serving are independent. In particular, a
|
||||
// manual ESP32 serve view can point into serve_buf, so leave the manager
|
||||
// view and its caller-owned buffer intact until `ota dev clear` (which
|
||||
// detaches the view before releasing the buffer).
|
||||
c.session_started_ms = 0;
|
||||
if (was_folder) {
|
||||
snprintf(reply, 160,
|
||||
"OK dropped folder session (was %c mid=%s); host file left untouched",
|
||||
fstate_char(fs), midhx);
|
||||
} else if (was_sd_archive) {
|
||||
snprintf(reply, 160,
|
||||
"OK dropped SD archive session (was %c mid=%s); partial retained for resume",
|
||||
fstate_char(fs), midhx);
|
||||
} else if (!discarded) {
|
||||
snprintf(reply, 160,
|
||||
"ERR dropped live session (was %c mid=%s), but persistent OTA slot invalidation failed",
|
||||
fstate_char(fs), midhx);
|
||||
} else {
|
||||
snprintf(reply, 160,
|
||||
"OK dropped session (was %c mid=%s); OTA receive slot confirmed clear",
|
||||
fstate_char(fs), midhx);
|
||||
}
|
||||
|
||||
// ---- broadcast our tiny beacon so peers discover us. If not already serving, set up flash-backed
|
||||
// self-serve first (so we're a real, fetchable source of our own running firmware). ----
|
||||
@@ -1141,13 +1167,27 @@ static bool handle_dev(const char* d, char* reply, OtaContext& c) {
|
||||
}
|
||||
|
||||
} else if (strncmp(d, "clear", 5) == 0) {
|
||||
const bool was_folder = c.fetch_to_folder;
|
||||
bool was_sd_archive = false;
|
||||
#if defined(NRF52_PLATFORM) && defined(OTA_SD_STORE)
|
||||
was_sd_archive = c.sdCacheFetching();
|
||||
c.stopSdCacheFetch();
|
||||
#endif
|
||||
c.manager.clear_primary();
|
||||
c.serve_expected = 0; c.serving = false; c.releaseServeBuffer();
|
||||
c.fetch_store.clear(); c.manager.reset_session(); c.fetch_to_folder = false;
|
||||
strcpy(reply, "OK cleared");
|
||||
c.manager.reset_session();
|
||||
if (was_folder && c.folder_dest) c.folder_dest->clear();
|
||||
c.fetch_to_folder = false;
|
||||
c.manager.set_fetch_store(&c.fetch_store);
|
||||
if (was_folder) {
|
||||
strcpy(reply, "OK cleared folder session; host file left untouched");
|
||||
} else if (was_sd_archive) {
|
||||
strcpy(reply, "OK cleared SD archive session; partial retained for resume");
|
||||
} else {
|
||||
const bool discarded = c.fetch_store.discard();
|
||||
strcpy(reply, discarded ? "OK OTA receive slot confirmed clear"
|
||||
: "ERR RAM state cleared; persistent OTA slot invalidation failed");
|
||||
}
|
||||
|
||||
} else {
|
||||
strcpy(reply, "ota dev: stage|recv|serve|announce|resume [mid8]|verify|want|apply|clear");
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
#pragma once
|
||||
|
||||
#include <stdint.h>
|
||||
|
||||
namespace mesh {
|
||||
namespace ota {
|
||||
|
||||
// Physical placement of one ESP32 flash-store container. Keep this independent
|
||||
// of ESP-IDF so reopen/discard geometry can be tested on the native host.
|
||||
struct MotaEsp32StageLayout {
|
||||
uint32_t total = 0;
|
||||
uint32_t meta_span = 0;
|
||||
uint32_t meta_flush = 0;
|
||||
uint32_t write_start = 0;
|
||||
uint32_t meta_part = 0;
|
||||
uint32_t pay_log0 = 0;
|
||||
uint32_t pay_part0 = 0;
|
||||
};
|
||||
|
||||
inline bool mota_esp32_align_up(uint32_t value, uint32_t alignment,
|
||||
uint32_t& result) {
|
||||
if (alignment == 0 || value > UINT32_MAX - (alignment - 1u)) return false;
|
||||
result = ((value + alignment - 1u) / alignment) * alignment;
|
||||
return true;
|
||||
}
|
||||
|
||||
inline uint32_t mota_esp32_align_down(uint32_t value,
|
||||
uint32_t alignment) {
|
||||
return alignment ? (value / alignment) * alignment : 0u;
|
||||
}
|
||||
|
||||
inline bool mota_esp32_stage_layout(
|
||||
uint32_t partition_size, uint32_t sector_size, uint32_t meta_capacity,
|
||||
bool is_full, uint32_t image_size, uint32_t meta_bytes,
|
||||
uint32_t payload_size, MotaEsp32StageLayout& out) {
|
||||
if (sector_size == 0 || partition_size < sector_size ||
|
||||
partition_size % sector_size != 0) {
|
||||
return false;
|
||||
}
|
||||
const uint64_t total64 =
|
||||
(uint64_t)meta_bytes + (uint64_t)payload_size + 5u;
|
||||
if (total64 > UINT32_MAX) return false;
|
||||
|
||||
MotaEsp32StageLayout layout;
|
||||
layout.total = (uint32_t)total64;
|
||||
if (is_full) {
|
||||
// A FULL payload is the final application image, byte for byte. Reject a
|
||||
// malformed manifest before begin() can erase or stream anything, and
|
||||
// prove that the complete logical container fits the inactive partition.
|
||||
if (payload_size != image_size || layout.total > partition_size) {
|
||||
return false;
|
||||
}
|
||||
layout.meta_span = meta_bytes;
|
||||
if (meta_bytes > UINT32_MAX - 5u ||
|
||||
!mota_esp32_align_up(meta_bytes + 5u, sector_size,
|
||||
layout.meta_flush) ||
|
||||
layout.meta_flush > meta_capacity ||
|
||||
layout.meta_flush > partition_size) {
|
||||
return false;
|
||||
}
|
||||
layout.meta_part = mota_esp32_align_down(
|
||||
partition_size - layout.meta_flush, sector_size);
|
||||
layout.pay_log0 = meta_bytes;
|
||||
layout.pay_part0 = 0;
|
||||
layout.write_start = 0;
|
||||
if (image_size > layout.meta_part) return false;
|
||||
} else {
|
||||
if (!mota_esp32_align_up(meta_bytes, sector_size,
|
||||
layout.meta_span) ||
|
||||
layout.meta_span > meta_capacity || layout.total > partition_size) {
|
||||
return false;
|
||||
}
|
||||
layout.meta_flush = layout.meta_span;
|
||||
layout.write_start = mota_esp32_align_down(
|
||||
partition_size - layout.total, sector_size);
|
||||
layout.meta_part = layout.write_start;
|
||||
layout.pay_log0 = layout.meta_span;
|
||||
if (layout.write_start > UINT32_MAX - layout.meta_span) return false;
|
||||
layout.pay_part0 = layout.write_start + layout.meta_span;
|
||||
if (image_size > layout.write_start) return false;
|
||||
}
|
||||
out = layout;
|
||||
return true;
|
||||
}
|
||||
|
||||
// The probe separates ordinary non-candidates from partition I/O failure:
|
||||
// true/false is I/O success, while `reopenable` says whether this sector holds
|
||||
// a header that the store's reopen path would adopt.
|
||||
typedef bool (*MotaEsp32ProbeStagedHeader)(void* context, uint32_t offset,
|
||||
bool& reopenable);
|
||||
typedef bool (*MotaEsp32InvalidateStagedHeader)(void* context,
|
||||
uint32_t offset);
|
||||
|
||||
inline bool mota_esp32_discard_staged_headers(
|
||||
uint32_t partition_size, uint32_t sector_size, void* context,
|
||||
MotaEsp32ProbeStagedHeader probe_header,
|
||||
MotaEsp32InvalidateStagedHeader invalidate_header,
|
||||
uint32_t* invalidated_count = nullptr) {
|
||||
if (invalidated_count) *invalidated_count = 0;
|
||||
if (sector_size == 0 || partition_size < sector_size ||
|
||||
partition_size % sector_size != 0 || !probe_header ||
|
||||
!invalidate_header) {
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ok = true;
|
||||
uint32_t count = 0;
|
||||
uint32_t offset = mota_esp32_align_down(
|
||||
partition_size - sector_size, sector_size);
|
||||
for (;;) {
|
||||
bool reopenable = false;
|
||||
if (!probe_header(context, offset, reopenable)) {
|
||||
ok = false;
|
||||
} else if (reopenable) {
|
||||
if (invalidate_header(context, offset)) {
|
||||
++count;
|
||||
} else {
|
||||
ok = false;
|
||||
}
|
||||
}
|
||||
if (offset < sector_size) break;
|
||||
offset -= sector_size;
|
||||
}
|
||||
if (invalidated_count) *invalidated_count = count;
|
||||
return ok;
|
||||
}
|
||||
|
||||
} // namespace ota
|
||||
} // namespace mesh
|
||||
@@ -47,6 +47,7 @@ static const uint32_t MOTA_NRF52_STAGE_CEILING_EXPANDED = MOTA_NRF52_APP_END;
|
||||
// explicit ceiling so it cannot accidentally cross an Internal ExtraFS.
|
||||
static const uint32_t MOTA_NRF52_FS_START = MOTA_NRF52_EXTRAFS_START;
|
||||
static const uint32_t MOTA_NRF52_FLASH_PAGE = 4096u;
|
||||
static const uint32_t MOTA_NRF52_CONTAINER_MIN_SIZE = 8u + 197u + 5u;
|
||||
static const uint8_t GPREGRET_OTA_APPLY = 0x6Au; // distinct from DFU magics 0x57/0x4E/0xA8
|
||||
static const uint8_t GPREGRET_OTA_BOOTLOADER_UPDATE = 0x6Bu;
|
||||
static const uint8_t GPREGRET2_OTA_STAGE_LEGACY = 0xD4u;
|
||||
@@ -272,5 +273,71 @@ inline bool mota_nrf52_stage_plan(uint32_t total_size, uint32_t app_base, uint32
|
||||
mota_nrf52_layout_stage_ceiling(), out_start);
|
||||
}
|
||||
|
||||
// Durable internal-store cancellation has to work even when the live
|
||||
// OtaStore object is empty (for example, autofetch is off after a reboot). It
|
||||
// also has to invalidate every older bottom-aligned header: invalidating only
|
||||
// the highest container could expose a larger stale container below it on the
|
||||
// next reopen scan. Keep the page walk and layout recognition pure so native
|
||||
// tests can exercise fresh-object and multiple-header cases without nRF flash.
|
||||
typedef bool (*MotaNrf52ReadStagedHeader)(void* context, uint32_t address,
|
||||
uint32_t& total_size);
|
||||
typedef bool (*MotaNrf52InvalidateStagedHeader)(void* context,
|
||||
uint32_t address);
|
||||
|
||||
inline bool mota_nrf52_staged_header_matches_layout(
|
||||
uint32_t address, uint32_t total_size, uint32_t app_base,
|
||||
uint32_t app_end, uint32_t effective_stage_ceiling) {
|
||||
uint32_t planned = 0;
|
||||
if (mota_nrf52_stage_plan(total_size, app_base, app_end,
|
||||
effective_stage_ceiling, planned) &&
|
||||
planned == address) {
|
||||
return true;
|
||||
}
|
||||
|
||||
// A device that moved from the legacy D4000 ceiling to the expanded ED000
|
||||
// ceiling can still contain an older, otherwise reopenable legacy header.
|
||||
// The expanded layout has no filesystem in that range, so it is safe to
|
||||
// invalidate both geometries. A legacy-only build never scans above D4000.
|
||||
return effective_stage_ceiling == MOTA_NRF52_STAGE_CEILING_EXPANDED &&
|
||||
mota_nrf52_stage_plan(total_size, app_base, app_end,
|
||||
MOTA_NRF52_STAGE_CEILING_LEGACY, planned) &&
|
||||
planned == address;
|
||||
}
|
||||
|
||||
inline bool mota_nrf52_discard_staged_headers(
|
||||
uint32_t app_base, uint32_t app_end, uint32_t effective_stage_ceiling,
|
||||
void* context, MotaNrf52ReadStagedHeader read_header,
|
||||
MotaNrf52InvalidateStagedHeader invalidate_header,
|
||||
uint32_t* invalidated_count = nullptr) {
|
||||
if (invalidated_count) *invalidated_count = 0;
|
||||
if (!read_header || !invalidate_header ||
|
||||
!mota_nrf52_layout_valid(app_base, effective_stage_ceiling) ||
|
||||
app_end < app_base || app_end > effective_stage_ceiling) {
|
||||
return false;
|
||||
}
|
||||
|
||||
bool ok = true;
|
||||
uint32_t count = 0;
|
||||
uint32_t address = effective_stage_ceiling - MOTA_NRF52_FLASH_PAGE;
|
||||
while (address >= app_end) {
|
||||
uint32_t total_size = 0;
|
||||
if (read_header(context, address, total_size) &&
|
||||
total_size >= MOTA_NRF52_CONTAINER_MIN_SIZE &&
|
||||
mota_nrf52_staged_header_matches_layout(
|
||||
address, total_size, app_base, app_end,
|
||||
effective_stage_ceiling)) {
|
||||
if (invalidate_header(context, address)) {
|
||||
++count;
|
||||
} else {
|
||||
ok = false;
|
||||
}
|
||||
}
|
||||
if (address < MOTA_NRF52_FLASH_PAGE) break;
|
||||
address -= MOTA_NRF52_FLASH_PAGE;
|
||||
}
|
||||
if (invalidated_count) *invalidated_count = count;
|
||||
return ok;
|
||||
}
|
||||
|
||||
} // namespace ota
|
||||
} // namespace mesh
|
||||
|
||||
@@ -24,6 +24,17 @@ public:
|
||||
virtual uint32_t staged_size() const = 0; // total_size from begin(), 0 if none
|
||||
virtual void clear() = 0;
|
||||
|
||||
// Durably discard a staged container. `clear()` resets the store object's
|
||||
// live session state and is intentionally cheap enough for begin()/retry
|
||||
// paths; a persistent backend may therefore leave its resumable bytes in
|
||||
// place. User-facing cancellation needs the stronger operation: after a
|
||||
// successful discard, a fresh store object must not be able to reopen the
|
||||
// old container. The conservative default clears only the live object and
|
||||
// returns false: a backend must opt in only when it can prove that its
|
||||
// durable copy is gone. A false result means cancellation must not be
|
||||
// reported as having cleared the persistent slot.
|
||||
virtual bool discard() { clear(); return false; }
|
||||
|
||||
// Optional: declare the size of the leading metadata (header + manifest + merkle leaves, i.e.
|
||||
// everything before the payload). A flash-backed store keeps that region - which is updated
|
||||
// throughout the transfer (a leaf is committed per block) - pinned in one RAM page, so it can
|
||||
@@ -90,9 +101,17 @@ public:
|
||||
uint32_t capacity() const override { return CAP; }
|
||||
uint32_t staged_size() const override { return _total; }
|
||||
void clear() override { _total = 0; }
|
||||
bool discard() override {
|
||||
_total = 0;
|
||||
const size_t marker_bytes =
|
||||
CAP < sizeof(MOTA_MAGIC) ? CAP : sizeof(MOTA_MAGIC);
|
||||
memset(_buf, 0, marker_bytes);
|
||||
return true;
|
||||
}
|
||||
// RAM doesn't survive a real reboot, but the buffer persists within a process - enough to exercise the
|
||||
// manager's resume path in native tests. Recover `total` from the stored header so read() bounds work.
|
||||
bool reopen() override {
|
||||
if (CAP < 8u) return false;
|
||||
if (memcmp(_buf, MOTA_MAGIC, 4) != 0) return false;
|
||||
uint32_t t = (uint32_t)_buf[4] | ((uint32_t)_buf[5] << 8) | ((uint32_t)_buf[6] << 16) | ((uint32_t)_buf[7] << 24);
|
||||
if (t < 13 || t > CAP) return false;
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
#if defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
|
||||
|
||||
#include "OtaDebug.h"
|
||||
#include "OtaByteIO.h" // align_up / align_down (flash-sector geometry)
|
||||
#include "OtaByteIO.h" // rd_u32le (persisted header total)
|
||||
#include "MotaContainer.h" // mota_parse_manifest (reopen: rebuild geometry from the staged manifest)
|
||||
#include <string.h>
|
||||
#include <stdlib.h> // malloc/free (the meta buffer is sized per fetch)
|
||||
@@ -22,33 +22,29 @@ bool OtaStoreFlashEsp32::acquire() {
|
||||
return _part != nullptr;
|
||||
}
|
||||
|
||||
void OtaStoreFlashEsp32::set_layout(const MotaEsp32StageLayout& layout) {
|
||||
_total = layout.total;
|
||||
_meta_span = layout.meta_span;
|
||||
_meta_flush = layout.meta_flush;
|
||||
_write_start = layout.write_start;
|
||||
_meta_part = layout.meta_part;
|
||||
_pay_log0 = layout.pay_log0;
|
||||
_pay_part0 = layout.pay_part0;
|
||||
_trailer_part = _full ? _meta_part + _meta_bytes
|
||||
: _write_start + _total - 5u;
|
||||
}
|
||||
|
||||
// Compute the slot placement from _full/_image_size/_meta_bytes/_pay_size (already set). Returns false if
|
||||
// it won't fit. Shared by plan_layout (fresh fetch) and reopen (resume) so both derive identical geometry.
|
||||
// it won't fit. Shared by plan_layout (fresh fetch), reopen (resume), and discard validation so all three
|
||||
// derive identical geometry.
|
||||
bool OtaStoreFlashEsp32::layout() {
|
||||
uint32_t total = _meta_bytes + _pay_size + 5;
|
||||
if (_full) {
|
||||
// payload streams to slot offset 0 (it IS the image); header+manifest+leaves+trailer persist at the
|
||||
// bottom so the container survives a reboot (resume / re-serve).
|
||||
_meta_span = _meta_bytes; // routing boundary: [0,meta) -> RAM meta buffer
|
||||
_meta_flush = align_up(_meta_bytes + 5, SEC); // meta + 5-byte trailer, whole sectors
|
||||
if (_meta_flush > OTA_ESP32_META_CAP) return false;
|
||||
uint32_t bottom = align_down(_psize - _meta_flush, SEC);
|
||||
_meta_part = bottom;
|
||||
_pay_log0 = _meta_bytes; _pay_part0 = 0;
|
||||
_write_start = 0;
|
||||
if (_image_size > bottom) return false; // image would overrun the bottom meta region
|
||||
} else {
|
||||
// whole container staged bottom-aligned; the decoded image fills the slot from offset 0.
|
||||
_meta_span = align_up(_meta_bytes, SEC); // pin whole sectors covering meta (+ spillover payload)
|
||||
_meta_flush = _meta_span;
|
||||
if (_meta_flush > OTA_ESP32_META_CAP) return false;
|
||||
if (total > _psize) return false;
|
||||
_write_start = align_down(_psize - total, SEC);
|
||||
_meta_part = _write_start;
|
||||
_pay_log0 = _meta_span; _pay_part0 = _write_start + _meta_span;
|
||||
if (_image_size > _write_start) return false; // decoded output would overlap the staged container
|
||||
MotaEsp32StageLayout planned;
|
||||
if (!mota_esp32_stage_layout(
|
||||
_psize, SEC, OTA_ESP32_META_CAP, _full, _image_size,
|
||||
_meta_bytes, _pay_size, planned)) {
|
||||
return false;
|
||||
}
|
||||
_total = total;
|
||||
set_layout(planned);
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -84,6 +80,120 @@ void OtaStoreFlashEsp32::clear() {
|
||||
_total = 0; _pay_open = false; _flushed = false; // _part kept (re-acquire is fine)
|
||||
}
|
||||
|
||||
OtaStoreFlashEsp32::CandidateProbe OtaStoreFlashEsp32::probe_candidate(
|
||||
uint32_t offset, StagedCandidate& candidate) const {
|
||||
if (!_part || _psize < SEC || offset > _psize - SEC) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
|
||||
uint8_t header[8];
|
||||
if (esp_partition_read(_part, offset, header, sizeof(header)) != ESP_OK) {
|
||||
return CandidateProbe::IO_ERROR;
|
||||
}
|
||||
if (memcmp(header, MOTA_MAGIC, sizeof(MOTA_MAGIC)) != 0) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
|
||||
const uint32_t total = rd_u32le(header + sizeof(MOTA_MAGIC));
|
||||
if (total < 8u + MOTA_MFL + 5u || total > _psize) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
|
||||
uint8_t manifest_bytes[256];
|
||||
uint32_t manifest_read = total - 8u;
|
||||
if (manifest_read > sizeof(manifest_bytes)) {
|
||||
manifest_read = sizeof(manifest_bytes);
|
||||
}
|
||||
if (esp_partition_read(_part, offset + 8u, manifest_bytes,
|
||||
manifest_read) != ESP_OK) {
|
||||
return CandidateProbe::IO_ERROR;
|
||||
}
|
||||
|
||||
MotaManifest manifest;
|
||||
if (!mota_parse_manifest(manifest_bytes, manifest_read, manifest)) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
const uint32_t manifest_len =
|
||||
(uint32_t)(manifest.approval - manifest.manifest_start) +
|
||||
sizeof(APPROVAL_YES);
|
||||
const uint64_t payload_off64 =
|
||||
8u + (uint64_t)manifest_len + (uint64_t)manifest.block_count * 4u;
|
||||
if (payload_off64 > UINT32_MAX ||
|
||||
payload_off64 + (uint64_t)manifest.payload_size + 5u != total) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
|
||||
MotaEsp32StageLayout planned;
|
||||
if (!mota_esp32_stage_layout(
|
||||
_psize, SEC, OTA_ESP32_META_CAP, manifest.is_full(),
|
||||
manifest.image_size, (uint32_t)payload_off64,
|
||||
manifest.payload_size, planned) ||
|
||||
planned.meta_part != offset || planned.total != total) {
|
||||
return CandidateProbe::INVALID;
|
||||
}
|
||||
|
||||
candidate.layout = planned;
|
||||
candidate.full = manifest.is_full();
|
||||
candidate.image_size = manifest.image_size;
|
||||
candidate.meta_bytes = (uint32_t)payload_off64;
|
||||
candidate.pay_size = manifest.payload_size;
|
||||
return CandidateProbe::VALID;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashEsp32::probe_staged_header(
|
||||
void* context, uint32_t offset, bool& reopenable) {
|
||||
OtaStoreFlashEsp32* store = static_cast<OtaStoreFlashEsp32*>(context);
|
||||
StagedCandidate candidate;
|
||||
const CandidateProbe result = store->probe_candidate(offset, candidate);
|
||||
reopenable = result == CandidateProbe::VALID;
|
||||
return result != CandidateProbe::IO_ERROR;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashEsp32::invalidate_staged_header(
|
||||
void* context, uint32_t offset) {
|
||||
OtaStoreFlashEsp32* store = static_cast<OtaStoreFlashEsp32*>(context);
|
||||
if (!store->_part || store->_psize < SEC || offset > store->_psize - SEC ||
|
||||
esp_partition_read(store->_part, offset, store->_pay, SEC) != ESP_OK) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// Both markers are inside the already-validated first metadata sector.
|
||||
// One sector RMW avoids erasing a delta's adjacent metadata/payload twice.
|
||||
memset(store->_pay, 0, sizeof(MOTA_MAGIC));
|
||||
memset(store->_pay + 8u + MOTA_OFF_APPROVAL, 0,
|
||||
sizeof(APPROVAL_YES));
|
||||
if (esp_partition_erase_range(store->_part, offset, SEC) != ESP_OK ||
|
||||
esp_partition_write(store->_part, offset, store->_pay, SEC) != ESP_OK) {
|
||||
return false;
|
||||
}
|
||||
|
||||
uint8_t check[sizeof(MOTA_MAGIC)];
|
||||
if (esp_partition_read(store->_part, offset, check, sizeof(check)) != ESP_OK ||
|
||||
memcmp(check, store->_pay, sizeof(check)) != 0 ||
|
||||
esp_partition_read(store->_part, offset + 8u + MOTA_OFF_APPROVAL,
|
||||
check, sizeof(check)) != ESP_OK ||
|
||||
memcmp(check, store->_pay + 8u + MOTA_OFF_APPROVAL,
|
||||
sizeof(check)) != 0) {
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashEsp32::discard() {
|
||||
bool ok = acquire();
|
||||
uint32_t invalidated = 0;
|
||||
if (ok) {
|
||||
ok = mota_esp32_discard_staged_headers(
|
||||
_psize, SEC, this, probe_staged_header,
|
||||
invalidate_staged_header, &invalidated);
|
||||
}
|
||||
OTA_DBG("OTA esp32: discard invalidated=%u ok=%d\n",
|
||||
(unsigned)invalidated, (int)ok);
|
||||
clear();
|
||||
_io_ok = ok;
|
||||
return ok;
|
||||
}
|
||||
|
||||
uint8_t* OtaStoreFlashEsp32::meta_slot(uint32_t L) {
|
||||
if (in_trailer(L)) return _full ? (_meta + _meta_bytes + (L - (_total - 5)))
|
||||
: (_trailer + (L - (_total - 5)));
|
||||
@@ -207,34 +317,26 @@ void OtaStoreFlashEsp32::checkpoint() {
|
||||
// so a stray/stale match can only cost a restart, never a corrupt adopt.
|
||||
bool OtaStoreFlashEsp32::reopen() {
|
||||
if (!acquire() || _psize < SEC) return false;
|
||||
uint8_t hb[8];
|
||||
// The meta/container is staged at the bottom of the slot. Large delta containers can begin well above
|
||||
// the final 512 KB, so scan the complete partition rather than silently losing resumability after reboot.
|
||||
for (uint32_t o = align_down(_psize - SEC, SEC); ; o -= SEC) {
|
||||
if (esp_partition_read(_part, o, hb, 8) == ESP_OK && memcmp(hb, MOTA_MAGIC, 4) == 0) {
|
||||
uint32_t total = rd_u32le(hb + 4);
|
||||
if (total >= 13 && total <= _psize) {
|
||||
uint8_t mbuf[256]; uint32_t mread = total - 8; if (mread > sizeof(mbuf)) mread = sizeof(mbuf);
|
||||
MotaManifest m;
|
||||
if (esp_partition_read(_part, o + 8, mbuf, mread) == ESP_OK && mota_parse_manifest(mbuf, mread, m)) {
|
||||
uint32_t mfl = (uint32_t)(m.approval - m.manifest_start) + 4;
|
||||
uint32_t payload_off = 8 + mfl + m.block_count * 4;
|
||||
if ((uint64_t)payload_off + m.payload_size + 5 == total) {
|
||||
_full = m.is_full(); _image_size = m.image_size; _meta_bytes = payload_off; _pay_size = m.payload_size;
|
||||
if (layout() && _meta_part == o) { // geometry agrees AND magic is where we'd place meta
|
||||
free(_meta); _meta = (uint8_t*)malloc(_meta_flush);
|
||||
if (!_meta) { _total = 0; return false; }
|
||||
if (esp_partition_read(_part, _meta_part, _meta, _meta_flush) != ESP_OK) {
|
||||
free(_meta); _meta = nullptr; _total = 0; return false; }
|
||||
memset(_trailer, 0xFF, sizeof(_trailer)); // delta trailer (re-written at finalize); full reads it from _meta
|
||||
_pay_open = false; _pay_sec = 0; _flushed = false; _io_ok = true;
|
||||
_pay_max_sec = (_pay_part0 + _pay_size + SEC) / SEC; // treat all payload sectors as seen -> RMW preserves committed blocks
|
||||
OTA_DBG("OTA esp32: reopen %s total=%u meta_part=%u\n", _full ? "FULL" : "DELTA", (unsigned)total, (unsigned)o);
|
||||
return true;
|
||||
}
|
||||
}
|
||||
}
|
||||
for (uint32_t o = mota_esp32_align_down(_psize - SEC, SEC); ; o -= SEC) {
|
||||
StagedCandidate candidate;
|
||||
if (probe_candidate(o, candidate) == CandidateProbe::VALID) {
|
||||
_full = candidate.full;
|
||||
_image_size = candidate.image_size;
|
||||
_meta_bytes = candidate.meta_bytes;
|
||||
_pay_size = candidate.pay_size;
|
||||
set_layout(candidate.layout);
|
||||
free(_meta); _meta = (uint8_t*)malloc(_meta_flush);
|
||||
if (!_meta) { _total = 0; return false; }
|
||||
if (esp_partition_read(_part, _meta_part, _meta, _meta_flush) != ESP_OK) {
|
||||
free(_meta); _meta = nullptr; _total = 0; return false;
|
||||
}
|
||||
memset(_trailer, 0xFF, sizeof(_trailer)); // delta trailer (re-written at finalize); full reads it from _meta
|
||||
_pay_open = false; _pay_sec = 0; _flushed = false; _io_ok = true;
|
||||
_pay_max_sec = (_pay_part0 + _pay_size + SEC) / SEC; // treat all payload sectors as seen -> RMW preserves committed blocks
|
||||
OTA_DBG("OTA esp32: reopen %s total=%u meta_part=%u\n", _full ? "FULL" : "DELTA", (unsigned)_total, (unsigned)o);
|
||||
return true;
|
||||
}
|
||||
if (o == 0) break;
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
#if defined(ESP32_PLATFORM) && defined(OTA_FLASH_STORE)
|
||||
|
||||
#include "OtaStore.h"
|
||||
#include "OtaFlashLayout_esp32.h"
|
||||
#include "esp_partition.h"
|
||||
|
||||
// Persistent flash-backed OtaStore for ESP32 (A/B). Stages the received `.mota` in the INACTIVE OTA
|
||||
@@ -41,6 +42,15 @@ namespace ota {
|
||||
class OtaStoreFlashEsp32 : public OtaStore {
|
||||
static const uint32_t SEC = 4096; // ESP32 NOR flash erase unit
|
||||
|
||||
enum class CandidateProbe : uint8_t { INVALID, VALID, IO_ERROR };
|
||||
struct StagedCandidate {
|
||||
MotaEsp32StageLayout layout;
|
||||
bool full = false;
|
||||
uint32_t image_size = 0;
|
||||
uint32_t meta_bytes = 0;
|
||||
uint32_t pay_size = 0;
|
||||
};
|
||||
|
||||
const esp_partition_t* _part = nullptr; // inactive OTA slot (acquired in plan_layout/begin)
|
||||
uint32_t _psize = 0; // slot size
|
||||
|
||||
@@ -62,7 +72,7 @@ class OtaStoreFlashEsp32 : public OtaStore {
|
||||
|
||||
// RX-safe staging buffers
|
||||
uint8_t* _meta = nullptr; // heap, sized per fetch: header+manifest+leaves(+full trailer)
|
||||
uint8_t _pay[SEC]; // one sliding payload sector (slot-sector aligned)
|
||||
alignas(4) uint8_t _pay[SEC]; // one sliding payload sector (slot-sector aligned)
|
||||
uint32_t _pay_sec = 0; // slot sector index currently in _pay (0 = none open)
|
||||
uint8_t _trailer[5];
|
||||
uint32_t _meta_flush = 0; // whole-sector byte count to program for the meta buffer
|
||||
@@ -73,6 +83,12 @@ class OtaStoreFlashEsp32 : public OtaStore {
|
||||
|
||||
bool acquire(); // resolve the inactive slot (idempotent)
|
||||
bool layout(); // compute placement from _full/_image_size/_meta_bytes/_pay_size
|
||||
void set_layout(const MotaEsp32StageLayout& layout);
|
||||
CandidateProbe probe_candidate(uint32_t offset,
|
||||
StagedCandidate& candidate) const;
|
||||
static bool probe_staged_header(void* context, uint32_t offset,
|
||||
bool& reopenable);
|
||||
static bool invalidate_staged_header(void* context, uint32_t offset);
|
||||
uint32_t pay_part(uint32_t L) const { return _pay_part0 + (L - _pay_log0); } // payload slot offset
|
||||
bool in_trailer(uint32_t L) const { return L >= _total - 5; }
|
||||
uint32_t run(uint32_t pos, uint32_t remain) const; // bytes from `pos` that stay in one region+sector
|
||||
@@ -92,6 +108,7 @@ public:
|
||||
uint32_t capacity() const override { return _psize; } // loose bound; plan_layout does the real check
|
||||
uint32_t staged_size() const override { return _total; }
|
||||
void clear() override;
|
||||
bool discard() override;
|
||||
bool set_meta_size(uint32_t meta_bytes) override { return meta_bytes < OTA_ESP32_META_CAP; }
|
||||
bool finalize() override;
|
||||
void checkpoint() override; // persist meta(leaves) + open payload sector so a reboot can resume
|
||||
|
||||
@@ -13,6 +13,18 @@
|
||||
namespace mesh {
|
||||
namespace ota {
|
||||
|
||||
void OtaStoreFlashNrf52::reset_session() {
|
||||
_write_start = 0;
|
||||
_stage_ceiling = MOTA_NRF52_STAGE_CEILING_LEGACY;
|
||||
_total = 0;
|
||||
_pay_idx = 0;
|
||||
_flushed = false;
|
||||
_io_ok = true;
|
||||
_planned_bootloader = false;
|
||||
_planned_total = 0;
|
||||
_planned_start = 0;
|
||||
}
|
||||
|
||||
// A valid EndF gives the exact live-image extent. Legacy app-only builds keep
|
||||
// the conservative rescue fallback. Shared internal bootloader-update builds
|
||||
// reject every package kind without EndF because their normal linker may place
|
||||
@@ -73,6 +85,59 @@ static bool flash_matches(uint32_t addr, const uint8_t* expected, uint32_t n) {
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashNrf52::read_staged_header(
|
||||
void*, uint32_t address, uint32_t& total_size) {
|
||||
const uint8_t* header = (const uint8_t*)(uintptr_t)address;
|
||||
if (memcmp(header, MOTA_MAGIC, sizeof(MOTA_MAGIC)) != 0) return false;
|
||||
total_size = rd_u32le(header + sizeof(MOTA_MAGIC));
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashNrf52::invalidate_staged_header(
|
||||
void* context, uint32_t address) {
|
||||
OtaStoreFlashNrf52* store = static_cast<OtaStoreFlashNrf52*>(context);
|
||||
const uint8_t* page = (const uint8_t*)(uintptr_t)address;
|
||||
memcpy(store->_meta_page, page, PG);
|
||||
|
||||
// Magic prevents application reopen; approval prevents an explicitly
|
||||
// triggered legacy bootloader from accepting a half-cleared container.
|
||||
// Rewrite the page once so both durable markers are consumed together.
|
||||
memset(store->_meta_page, 0, sizeof(MOTA_MAGIC));
|
||||
memset(store->_meta_page + 8u + MOTA_OFF_APPROVAL, 0,
|
||||
sizeof(APPROVAL_YES));
|
||||
if (flash_nrf5x_write(address, store->_meta_page, PG) < 0) {
|
||||
store->_io_ok = false;
|
||||
return false;
|
||||
}
|
||||
flash_nrf5x_flush();
|
||||
if (!flash_matches(address, store->_meta_page, PG)) {
|
||||
store->_io_ok = false;
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashNrf52::discard() {
|
||||
const uint32_t app_base = mota_nrf52_app_base();
|
||||
const uint32_t stage_ceiling = ota_nrf52_effective_stage_ceiling();
|
||||
uint32_t app_end = 0;
|
||||
if (!protected_app_end(app_base, stage_ceiling, app_end)) {
|
||||
reset_session();
|
||||
_io_ok = false;
|
||||
return false;
|
||||
}
|
||||
|
||||
uint32_t invalidated = 0;
|
||||
const bool ok = mota_nrf52_discard_staged_headers(
|
||||
app_base, app_end, stage_ceiling, this, read_staged_header,
|
||||
invalidate_staged_header, &invalidated);
|
||||
OTA_DBG("OTA flash: discard invalidated=%u ok=%d\n",
|
||||
(unsigned)invalidated, (int)ok);
|
||||
reset_session();
|
||||
_io_ok = ok;
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool OtaStoreFlashNrf52::flush_page(uint32_t page_idx, const uint8_t* buf) {
|
||||
if (!_io_ok || _write_start > _stage_ceiling - PG ||
|
||||
page_idx > (_stage_ceiling - PG - _write_start) / PG) {
|
||||
@@ -133,7 +198,7 @@ bool OtaStoreFlashNrf52::begin(uint32_t total_size) {
|
||||
const bool planned_bootloader = _planned_bootloader;
|
||||
const uint32_t planned_total = _planned_total;
|
||||
const uint32_t planned_start = _planned_start;
|
||||
clear();
|
||||
reset_session();
|
||||
|
||||
// never collide with the running application image (its extent comes from its EndF trailer)
|
||||
const uint32_t app_base = mota_nrf52_app_base();
|
||||
|
||||
@@ -57,6 +57,10 @@ class OtaStoreFlashNrf52 : public OtaStore {
|
||||
uint8_t* write_slot(uint32_t pos);
|
||||
bool flush_pay(); // commit _pay_page to flash (erase + program, one page)
|
||||
bool flush_page(uint32_t page_idx, const uint8_t* buf); // write + verify a full page
|
||||
void reset_session(); // RAM bookkeeping only; never touches staged flash
|
||||
static bool read_staged_header(void* context, uint32_t address,
|
||||
uint32_t& total_size);
|
||||
static bool invalidate_staged_header(void* context, uint32_t address);
|
||||
|
||||
public:
|
||||
bool plan_layout(bool is_full, uint32_t image_size, uint32_t payload_off,
|
||||
@@ -66,11 +70,8 @@ public:
|
||||
bool read(uint32_t offset, uint8_t* buf, uint32_t len) const override;
|
||||
uint32_t capacity() const override;
|
||||
uint32_t staged_size() const override { return _total; }
|
||||
void clear() override {
|
||||
_stage_ceiling = MOTA_NRF52_STAGE_CEILING_LEGACY;
|
||||
_total = 0; _pay_idx = 0; _flushed = false; _io_ok = true;
|
||||
_planned_bootloader = false; _planned_total = 0; _planned_start = 0;
|
||||
}
|
||||
void clear() override { reset_session(); }
|
||||
bool discard() override;
|
||||
bool set_meta_size(uint32_t meta_bytes) override { return meta_bytes <= PG; } // leaves must fit page 0
|
||||
bool finalize() override;
|
||||
void checkpoint() override; // persist page 0 (leaves) + the open payload page so a reboot can resume
|
||||
|
||||
@@ -864,7 +864,7 @@ bool OtaStoreQspiNrf52::reopen() {
|
||||
return true;
|
||||
}
|
||||
|
||||
void OtaStoreQspiNrf52::clear() {
|
||||
bool OtaStoreQspiNrf52::discard() {
|
||||
bool ok = ensureFlash();
|
||||
uint8_t zero[4] = { 0, 0, 0, 0 };
|
||||
uint8_t check[sizeof(zero)];
|
||||
@@ -872,7 +872,11 @@ void OtaStoreQspiNrf52::clear() {
|
||||
ok = rawWrite(0, zero, sizeof(zero)) && rawRead(0, check, sizeof(check)) &&
|
||||
memcmp(check, zero, sizeof(zero)) == 0;
|
||||
}
|
||||
if (ok && _total >= 8 + MOTA_OFF_APPROVAL + sizeof(zero)) {
|
||||
// The raw QSPI store is dedicated to OTA, and APRV has a fixed manifest
|
||||
// offset. A fresh store object has _total == 0 even when an older approved
|
||||
// container remains in flash, so invalidate both gates from flash capacity
|
||||
// rather than trusting this session's RAM state.
|
||||
if (ok && _flash_size >= 8 + MOTA_OFF_APPROVAL + sizeof(zero)) {
|
||||
const uint32_t approval = 8 + MOTA_OFF_APPROVAL;
|
||||
ok = rawWrite(approval, zero, sizeof(zero)) && rawRead(approval, check, sizeof(check)) &&
|
||||
memcmp(check, zero, sizeof(zero)) == 0;
|
||||
@@ -883,8 +887,11 @@ void OtaStoreQspiNrf52::clear() {
|
||||
resetSession();
|
||||
if (!ok) fail(saved_error[0] ? saved_error : "QSPI container invalidation failed");
|
||||
releaseFlash();
|
||||
return ok;
|
||||
}
|
||||
|
||||
void OtaStoreQspiNrf52::clear() { (void)discard(); }
|
||||
|
||||
bool OtaStoreQspiNrf52::approve_for_bootloader() {
|
||||
if (!finalize()) return false;
|
||||
const uint32_t approval = 8 + MOTA_OFF_APPROVAL;
|
||||
|
||||
@@ -155,6 +155,7 @@ public:
|
||||
uint32_t capacity() const override;
|
||||
uint32_t staged_size() const override { return _total; }
|
||||
void clear() override;
|
||||
bool discard() override;
|
||||
bool set_meta_size(uint32_t meta_bytes) override;
|
||||
bool finalize() override;
|
||||
void checkpoint() override;
|
||||
|
||||
@@ -509,17 +509,32 @@ bool OtaStoreSdNrf52::reopen() {
|
||||
return true;
|
||||
}
|
||||
|
||||
void OtaStoreSdNrf52::clear() {
|
||||
bool OtaStoreSdNrf52::discard() {
|
||||
_total = 0;
|
||||
_planned_bootloader = false;
|
||||
_first_sector = 0;
|
||||
_allocated_sectors = 0;
|
||||
clear_sd_auth_record();
|
||||
if (!mount()) return;
|
||||
if (!mount()) return false;
|
||||
if (_file && *_file) _file->close();
|
||||
_sd->remove(PATH);
|
||||
if (_sd->exists(PATH) && !_sd->remove(PATH)) {
|
||||
fail("SD OTA file removal failed");
|
||||
return false;
|
||||
}
|
||||
if (_sd->exists(PATH)) {
|
||||
fail("SD OTA file still exists after removal");
|
||||
return false;
|
||||
}
|
||||
SdCard* const card = _sd->card();
|
||||
if (card && !card->syncDevice()) {
|
||||
fail("SD OTA removal sync failed");
|
||||
return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
void OtaStoreSdNrf52::clear() { (void)discard(); }
|
||||
|
||||
bool OtaStoreSdNrf52::approve_for_bootloader(
|
||||
const uint8_t expected_boot_image_hash[32],
|
||||
const uint8_t authorized_container_hash[32]) {
|
||||
|
||||
@@ -30,6 +30,7 @@ public:
|
||||
uint32_t capacity() const override;
|
||||
uint32_t staged_size() const override { return _total; }
|
||||
void clear() override;
|
||||
bool discard() override;
|
||||
bool set_meta_size(uint32_t meta_bytes) override;
|
||||
bool finalize() override;
|
||||
void checkpoint() override;
|
||||
|
||||
@@ -172,6 +172,17 @@ static RAK12035_SoilMoisture RAK12035;
|
||||
#endif
|
||||
|
||||
#ifdef RAK_WISBLOCK_GPS
|
||||
// Release capability manifests scan the linked firmware image rather than
|
||||
// trusting build flags. Give the actual WisBlock provider a unique marker and
|
||||
// keep an observable reference from begin() so LTO/section GC cannot discard
|
||||
// it. The marker lives inside this provider guard, so generic GPS CLI text
|
||||
// cannot produce a false positive.
|
||||
extern "C" {
|
||||
extern const char meshcore_capability_rak_wisblock_gps[];
|
||||
const char meshcore_capability_rak_wisblock_gps[] __attribute__((used)) =
|
||||
"meshcore.capability.rak_wisblock_gps.v1";
|
||||
}
|
||||
|
||||
// -1 = no enable pin; out-of-range values are no-ops in pinMode/digitalWrite,
|
||||
// while 0 would be a real GPIO (P0.00 = LFXO crystal on nRF52)
|
||||
static uint32_t gpsResetPin = -1;
|
||||
@@ -702,6 +713,10 @@ static const size_t SENSOR_TABLE_SIZE = (sizeof(SENSOR_TABLE) / sizeof(SENSOR_TA
|
||||
bool EnvironmentSensorManager::begin() {
|
||||
#if ENV_INCLUDE_GPS
|
||||
#ifdef RAK_WISBLOCK_GPS
|
||||
// A volatile read anchors the externally linked marker even under LTO.
|
||||
const volatile char* capability_marker =
|
||||
meshcore_capability_rak_wisblock_gps;
|
||||
(void)*capability_marker;
|
||||
rakGPSInit();
|
||||
#else
|
||||
initBasicGPS();
|
||||
|
||||
@@ -783,4 +783,23 @@ expectations=" ${BUILD_EXPECTATIONS[*]} "
|
||||
[[ "$expectations" != *"companion.network_terminal="* ]] \
|
||||
|| fail "nRF52 Full contract promised an ESP32-only network terminal"
|
||||
|
||||
BUILD_PROFILE_FOR_TARGET=standard
|
||||
for rak_target in \
|
||||
RAK_3401_repeater_lora_ota_no_external_sensors \
|
||||
RAK_4631_repeater_lora_ota_no_external_sensors; do
|
||||
BUILD_CAPABILITIES=()
|
||||
BUILD_REDUCTIONS=()
|
||||
BUILD_EXPECTATIONS=()
|
||||
declare_build_capability_contract "$rak_target" NRF52_PLATFORM
|
||||
expectations=" ${BUILD_EXPECTATIONS[*]} "
|
||||
[[ "$expectations" == *"sensor.gps=meshcore.capability.rak_wisblock_gps.v1"* ]] \
|
||||
|| fail "$rak_target reduced OTA contract omitted retained GPS"
|
||||
[[ "$expectations" != *"sensor.gps=gps setloc"* ]] \
|
||||
|| fail "$rak_target reduced OTA contract still accepts generic GPS CLI text"
|
||||
for ina in 219 226 260 3221; do
|
||||
[[ "$expectations" == *"sensor.ina${ina}=INA${ina}"* ]] \
|
||||
|| fail "$rak_target reduced OTA contract omitted retained INA${ina}"
|
||||
done
|
||||
done
|
||||
|
||||
echo "test_build_profiles: OK"
|
||||
|
||||
@@ -1274,6 +1274,60 @@ TEST(OtaStoreRamTest, RandomAccessAndErasedSentinel) {
|
||||
for (int i = 0; i < 8; i++) EXPECT_EQ(rd[i], 0xFF);
|
||||
}
|
||||
|
||||
TEST(OtaStoreRamTest, ClearKeepsResumeFixtureButDiscardConsumesHeader) {
|
||||
OtaStoreRam<4096> s;
|
||||
constexpr uint32_t total = 8u + MOTA_MFL + 5u;
|
||||
uint8_t header[8] = {'m', 'O', 'T', 'A',
|
||||
static_cast<uint8_t>(total),
|
||||
static_cast<uint8_t>(total >> 8),
|
||||
static_cast<uint8_t>(total >> 16),
|
||||
static_cast<uint8_t>(total >> 24)};
|
||||
ASSERT_TRUE(s.begin(total));
|
||||
ASSERT_TRUE(s.write(0, header, sizeof(header)));
|
||||
|
||||
// clear() remains the cheap RAM-session reset used by reboot/resume tests.
|
||||
s.clear();
|
||||
EXPECT_TRUE(s.reopen());
|
||||
EXPECT_EQ(s.staged_size(), total);
|
||||
|
||||
// User-facing cancellation uses discard(), which must prevent a fresh
|
||||
// reopen even for the native in-process persistence model.
|
||||
EXPECT_TRUE(s.discard());
|
||||
EXPECT_EQ(s.staged_size(), 0u);
|
||||
EXPECT_FALSE(s.reopen());
|
||||
}
|
||||
|
||||
TEST(OtaStoreRamTest, SeederSizedStoreDiscardsWithoutAssumingAHeaderFits) {
|
||||
// OTA_SEEDER_ONLY intentionally uses OtaStoreRam<1>: it needs a valid
|
||||
// manager destination object but never stages firmware locally.
|
||||
OtaStoreRam<1> s;
|
||||
EXPECT_TRUE(s.discard());
|
||||
EXPECT_EQ(s.staged_size(), 0u);
|
||||
EXPECT_FALSE(s.reopen());
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
class SessionOnlyOtaStore : public OtaStore {
|
||||
public:
|
||||
bool begin(uint32_t) override { return false; }
|
||||
bool write(uint32_t, const uint8_t*, uint32_t) override { return false; }
|
||||
bool read(uint32_t, uint8_t*, uint32_t) const override { return false; }
|
||||
uint32_t capacity() const override { return 0; }
|
||||
uint32_t staged_size() const override { return 0; }
|
||||
void clear() override { cleared = true; }
|
||||
|
||||
bool cleared = false;
|
||||
};
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST(OtaStoreDiscardTest, ConservativeDefaultCannotClaimDurableInvalidation) {
|
||||
SessionOnlyOtaStore store;
|
||||
EXPECT_FALSE(store.discard());
|
||||
EXPECT_TRUE(store.cleared);
|
||||
}
|
||||
|
||||
// --- merkle proof GENERATION (server side) matches the Python oracle ---------------------------
|
||||
|
||||
TEST(OtaMerkle, GenProofMatchesPythonAndVerifies) {
|
||||
@@ -1605,6 +1659,21 @@ TEST(OtaServe, ClearPrimaryInvalidatesCallerOwnedView) {
|
||||
EXPECT_FALSE(manager.remove_source(&folder));
|
||||
}
|
||||
|
||||
TEST(OtaServe, FetchSessionResetPreservesIndependentPrimaryView) {
|
||||
OtaManager manager;
|
||||
manager.begin(0, nullptr, nullptr);
|
||||
ASSERT_TRUE(manager.serve(SIM_MOTA, SIM_MOTA_LEN));
|
||||
ASSERT_EQ(manager.servedCount(), 1);
|
||||
ASSERT_TRUE(manager.servedEntry(0)->is_self);
|
||||
|
||||
// `ota cancel` resets only the receive side. A served image (and its
|
||||
// caller-owned backing buffer) must remain registered until the serving
|
||||
// side explicitly calls clear_primary().
|
||||
manager.reset_session();
|
||||
ASSERT_EQ(manager.servedCount(), 1);
|
||||
EXPECT_TRUE(manager.servedEntry(0)->is_self);
|
||||
}
|
||||
|
||||
TEST(OtaTransfer, TwoManagersFullTransfer) {
|
||||
g_q.clear();
|
||||
OtaManager server, client;
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
#include <cstring>
|
||||
|
||||
#include "helpers/ota/OtaFlashLayout_nrf52.h"
|
||||
#include "helpers/ota/OtaFlashLayout_esp32.h"
|
||||
#include "helpers/ota/OtaStoreQspiNrf52.h"
|
||||
|
||||
using namespace mesh::ota;
|
||||
@@ -318,6 +319,307 @@ TEST(OtaFlashPlan, ReopenBoundsUntrustedTotalBeforeManifestRead) {
|
||||
8u + 197u + 5u));
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
struct FakeStagedHeader {
|
||||
uint32_t address;
|
||||
uint32_t total;
|
||||
bool present;
|
||||
bool invalidate_ok;
|
||||
bool invalidated;
|
||||
};
|
||||
|
||||
struct FakeStagedFlash {
|
||||
FakeStagedHeader* headers;
|
||||
size_t count;
|
||||
uint32_t invalidate_calls;
|
||||
};
|
||||
|
||||
bool fake_read_staged_header(void* context, uint32_t address,
|
||||
uint32_t& total) {
|
||||
FakeStagedFlash* flash = static_cast<FakeStagedFlash*>(context);
|
||||
for (size_t i = 0; i < flash->count; ++i) {
|
||||
FakeStagedHeader& header = flash->headers[i];
|
||||
if (header.address == address && header.present) {
|
||||
total = header.total;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
bool fake_invalidate_staged_header(void* context, uint32_t address) {
|
||||
FakeStagedFlash* flash = static_cast<FakeStagedFlash*>(context);
|
||||
++flash->invalidate_calls;
|
||||
for (size_t i = 0; i < flash->count; ++i) {
|
||||
FakeStagedHeader& header = flash->headers[i];
|
||||
if (header.address != address || !header.present) continue;
|
||||
if (!header.invalidate_ok) return false;
|
||||
header.present = false;
|
||||
header.invalidated = true;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST(OtaFlashDiscard, FreshIdleObjectStillInvalidatesPersistedHeader) {
|
||||
// The callback context models flash only: there is deliberately no live
|
||||
// OtaStore session or staged_size. Cancellation after reboot must still
|
||||
// find the bottom-aligned persistent header.
|
||||
FakeStagedHeader headers[] = {
|
||||
{0xE2000u, EXPANDED - 0xE2000u, true, true, false},
|
||||
};
|
||||
FakeStagedFlash flash{headers, 1u, 0u};
|
||||
uint32_t invalidated = 99u;
|
||||
|
||||
EXPECT_TRUE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, APP_END_V6, EXPANDED, &flash, fake_read_staged_header,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 1u);
|
||||
EXPECT_EQ(flash.invalidate_calls, 1u);
|
||||
EXPECT_TRUE(headers[0].invalidated);
|
||||
|
||||
// Durable discard is idempotent: a new scan cannot reopen or re-invalidate
|
||||
// the header that the first call consumed.
|
||||
invalidated = 99u;
|
||||
EXPECT_TRUE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, APP_END_V6, EXPANDED, &flash, fake_read_staged_header,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 0u);
|
||||
EXPECT_EQ(flash.invalidate_calls, 1u);
|
||||
}
|
||||
|
||||
TEST(OtaFlashDiscard, InvalidatesEveryCurrentAndLegacyLayoutHeader) {
|
||||
FakeStagedHeader headers[] = {
|
||||
// Two expanded-ceiling containers from transfers of different sizes.
|
||||
{0xE2000u, EXPANDED - 0xE2000u, true, true, false},
|
||||
{0xD8000u, EXPANDED - 0xD8000u, true, true, false},
|
||||
// A retained container from the older D4000 ceiling geometry.
|
||||
{0xC0000u, LEGACY - 0xC0000u, true, true, false},
|
||||
// Magic at a page whose total does not bottom-align there is not a
|
||||
// reopenable store header and must remain untouched.
|
||||
{0xD9000u, 0x8000u, true, true, false},
|
||||
};
|
||||
FakeStagedFlash flash{headers, 4u, 0u};
|
||||
uint32_t invalidated = 0;
|
||||
|
||||
EXPECT_TRUE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, APP_END_V6, EXPANDED, &flash, fake_read_staged_header,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 3u);
|
||||
EXPECT_EQ(flash.invalidate_calls, 3u);
|
||||
EXPECT_TRUE(headers[0].invalidated);
|
||||
EXPECT_TRUE(headers[1].invalidated);
|
||||
EXPECT_TRUE(headers[2].invalidated);
|
||||
EXPECT_TRUE(headers[3].present);
|
||||
EXPECT_FALSE(headers[3].invalidated);
|
||||
}
|
||||
|
||||
TEST(OtaFlashDiscard, ReportsFailureButContinuesInvalidatingOtherHeaders) {
|
||||
FakeStagedHeader headers[] = {
|
||||
{0xE2000u, EXPANDED - 0xE2000u, true, false, false},
|
||||
{0xD8000u, EXPANDED - 0xD8000u, true, true, false},
|
||||
};
|
||||
FakeStagedFlash flash{headers, 2u, 0u};
|
||||
uint32_t invalidated = 0;
|
||||
|
||||
EXPECT_FALSE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, APP_END_V6, EXPANDED, &flash, fake_read_staged_header,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 1u);
|
||||
EXPECT_EQ(flash.invalidate_calls, 2u);
|
||||
EXPECT_TRUE(headers[0].present);
|
||||
EXPECT_TRUE(headers[1].invalidated);
|
||||
}
|
||||
|
||||
TEST(OtaFlashDiscard, UnsafeBoundsFailBeforeAnyFlashCallback) {
|
||||
FakeStagedHeader headers[] = {
|
||||
{0xE2000u, EXPANDED - 0xE2000u, true, true, false},
|
||||
};
|
||||
FakeStagedFlash flash{headers, 1u, 0u};
|
||||
uint32_t invalidated = 99u;
|
||||
|
||||
EXPECT_FALSE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, EXPANDED + 1u, EXPANDED, &flash, fake_read_staged_header,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 0u);
|
||||
EXPECT_EQ(flash.invalidate_calls, 0u);
|
||||
EXPECT_TRUE(headers[0].present);
|
||||
|
||||
EXPECT_FALSE(mota_nrf52_discard_staged_headers(
|
||||
APP_V6, APP_END_V6, EXPANDED, &flash, nullptr,
|
||||
fake_invalidate_staged_header, &invalidated));
|
||||
EXPECT_EQ(flash.invalidate_calls, 0u);
|
||||
}
|
||||
|
||||
TEST(OtaEsp32FlashPlan, FullAndDeltaPlacementStaySectorAligned) {
|
||||
constexpr uint32_t partition = 2u * 1024u * 1024u;
|
||||
constexpr uint32_t sector = 4096u;
|
||||
constexpr uint32_t meta_capacity = 65536u;
|
||||
MotaEsp32StageLayout full;
|
||||
ASSERT_TRUE(mota_esp32_stage_layout(
|
||||
partition, sector, meta_capacity, true, 1024u * 1024u, 40000u,
|
||||
1024u * 1024u, full));
|
||||
EXPECT_EQ(full.total, 40000u + 1024u * 1024u + 5u);
|
||||
EXPECT_EQ(full.meta_flush, 40960u);
|
||||
EXPECT_EQ(full.meta_part, partition - full.meta_flush);
|
||||
EXPECT_EQ(full.meta_part % sector, 0u);
|
||||
EXPECT_EQ(full.pay_part0, 0u);
|
||||
|
||||
MotaEsp32StageLayout delta;
|
||||
ASSERT_TRUE(mota_esp32_stage_layout(
|
||||
partition, sector, meta_capacity, false, 900000u, 1234u,
|
||||
100000u, delta));
|
||||
EXPECT_EQ(delta.total, 101239u);
|
||||
EXPECT_EQ(delta.meta_span, sector);
|
||||
EXPECT_EQ(delta.meta_part, delta.write_start);
|
||||
EXPECT_EQ(delta.write_start % sector, 0u);
|
||||
EXPECT_EQ(delta.pay_part0, delta.write_start + sector);
|
||||
}
|
||||
|
||||
TEST(OtaEsp32FlashPlan, RejectsOverflowAndUnfittableMetadata) {
|
||||
MotaEsp32StageLayout layout;
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
4097u, 4096u, 65536u, true, 1u, 200u, 1u, layout));
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
2u * 1024u * 1024u, 4096u, 65536u, true, 1u, UINT32_MAX,
|
||||
1u, layout));
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
32768u, 4096u, 65536u, true, 1u, 40000u, 1u, layout));
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
2u * 1024u * 1024u, 4096u, 65536u, false,
|
||||
2u * 1024u * 1024u, 1000u, 1000u, layout));
|
||||
}
|
||||
|
||||
TEST(OtaEsp32FlashPlan, RejectsMalformedOrOversizedFullPayloadBeforeErase) {
|
||||
constexpr uint32_t partition = 2u * 1024u * 1024u;
|
||||
constexpr uint32_t sector = 4096u;
|
||||
constexpr uint32_t meta_capacity = 65536u;
|
||||
MotaEsp32StageLayout layout;
|
||||
|
||||
// FULL means that the payload is exactly the final application image.
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
partition, sector, meta_capacity, true, 100000u, 2000u,
|
||||
99999u, layout));
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
partition, sector, meta_capacity, true, 100000u, 2000u,
|
||||
100001u, layout));
|
||||
|
||||
// The logical container and rounded metadata/payload regions must both fit
|
||||
// wholly inside the inactive slot before OtaStoreFlashEsp32::begin().
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
partition, sector, meta_capacity, true, partition, 2000u,
|
||||
partition, layout));
|
||||
EXPECT_FALSE(mota_esp32_stage_layout(
|
||||
8192u, sector, meta_capacity, true, 4097u, 1u, 4097u,
|
||||
layout));
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
struct FakeEspStagedHeader {
|
||||
uint32_t offset;
|
||||
bool reopenable;
|
||||
bool probe_ok;
|
||||
bool invalidate_ok;
|
||||
bool invalidated;
|
||||
};
|
||||
|
||||
struct FakeEspPartition {
|
||||
FakeEspStagedHeader* headers;
|
||||
size_t count;
|
||||
uint32_t probe_calls;
|
||||
uint32_t invalidate_calls;
|
||||
};
|
||||
|
||||
bool fake_probe_esp_header(void* context, uint32_t offset,
|
||||
bool& reopenable) {
|
||||
FakeEspPartition* partition = static_cast<FakeEspPartition*>(context);
|
||||
++partition->probe_calls;
|
||||
reopenable = false;
|
||||
for (size_t i = 0; i < partition->count; ++i) {
|
||||
FakeEspStagedHeader& header = partition->headers[i];
|
||||
if (header.offset != offset || header.invalidated) continue;
|
||||
reopenable = header.reopenable;
|
||||
return header.probe_ok;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool fake_invalidate_esp_header(void* context, uint32_t offset) {
|
||||
FakeEspPartition* partition = static_cast<FakeEspPartition*>(context);
|
||||
++partition->invalidate_calls;
|
||||
for (size_t i = 0; i < partition->count; ++i) {
|
||||
FakeEspStagedHeader& header = partition->headers[i];
|
||||
if (header.offset != offset || header.invalidated) continue;
|
||||
if (!header.invalidate_ok) return false;
|
||||
header.invalidated = true;
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
TEST(OtaEsp32FlashDiscard, FreshIdleScanInvalidatesEveryReopenableHeader) {
|
||||
constexpr uint32_t sector = 4096u;
|
||||
constexpr uint32_t partition_size = 6u * sector;
|
||||
FakeEspStagedHeader headers[] = {
|
||||
{5u * sector, true, true, true, false},
|
||||
{3u * sector, false, true, true, false},
|
||||
{1u * sector, true, true, true, false},
|
||||
};
|
||||
FakeEspPartition partition{headers, 3u, 0u, 0u};
|
||||
uint32_t invalidated = 99u;
|
||||
|
||||
EXPECT_TRUE(mota_esp32_discard_staged_headers(
|
||||
partition_size, sector, &partition, fake_probe_esp_header,
|
||||
fake_invalidate_esp_header, &invalidated));
|
||||
EXPECT_EQ(partition.probe_calls, 6u);
|
||||
EXPECT_EQ(partition.invalidate_calls, 2u);
|
||||
EXPECT_EQ(invalidated, 2u);
|
||||
EXPECT_TRUE(headers[0].invalidated);
|
||||
EXPECT_FALSE(headers[1].invalidated);
|
||||
EXPECT_TRUE(headers[2].invalidated);
|
||||
}
|
||||
|
||||
TEST(OtaEsp32FlashDiscard, IoAndInvalidateFailuresAreReportedAfterFullScan) {
|
||||
constexpr uint32_t sector = 4096u;
|
||||
constexpr uint32_t partition_size = 6u * sector;
|
||||
FakeEspStagedHeader headers[] = {
|
||||
{4u * sector, false, false, true, false},
|
||||
{3u * sector, true, true, false, false},
|
||||
{1u * sector, true, true, true, false},
|
||||
};
|
||||
FakeEspPartition partition{headers, 3u, 0u, 0u};
|
||||
uint32_t invalidated = 0u;
|
||||
|
||||
EXPECT_FALSE(mota_esp32_discard_staged_headers(
|
||||
partition_size, sector, &partition, fake_probe_esp_header,
|
||||
fake_invalidate_esp_header, &invalidated));
|
||||
EXPECT_EQ(partition.probe_calls, 6u);
|
||||
EXPECT_EQ(partition.invalidate_calls, 2u);
|
||||
EXPECT_EQ(invalidated, 1u);
|
||||
EXPECT_FALSE(headers[1].invalidated);
|
||||
EXPECT_TRUE(headers[2].invalidated);
|
||||
}
|
||||
|
||||
TEST(OtaEsp32FlashDiscard, InvalidScanArgumentsFailWithoutCallbacks) {
|
||||
FakeEspPartition partition{nullptr, 0u, 0u, 0u};
|
||||
uint32_t invalidated = 99u;
|
||||
EXPECT_FALSE(mota_esp32_discard_staged_headers(
|
||||
4095u, 4096u, &partition, fake_probe_esp_header,
|
||||
fake_invalidate_esp_header, &invalidated));
|
||||
EXPECT_FALSE(mota_esp32_discard_staged_headers(
|
||||
4097u, 4096u, &partition, fake_probe_esp_header,
|
||||
fake_invalidate_esp_header, &invalidated));
|
||||
EXPECT_EQ(invalidated, 0u);
|
||||
EXPECT_EQ(partition.probe_calls, 0u);
|
||||
EXPECT_EQ(partition.invalidate_calls, 0u);
|
||||
}
|
||||
|
||||
// out_start is only written on success - a rejected plan must not clobber the caller's variable.
|
||||
TEST(OtaFlashPlan, LeavesOutputUntouchedOnReject) {
|
||||
uint32_t start = 0x1234ABCD;
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+788
-105
File diff suppressed because it is too large
Load Diff
+1020
-145
File diff suppressed because it is too large
Load Diff
Executable
+638
@@ -0,0 +1,638 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Exhaustively select a compact legacy-ceiling RAK3401 mOTA route.
|
||||
|
||||
Cache entries are keyed by source SHA, target SHA, and workspace size rather
|
||||
than inventory indices. A cache generated for an older inventory can therefore
|
||||
be migrated safely when new candidate images are inserted.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
from concurrent.futures import ProcessPoolExecutor, as_completed
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import re
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
REPO_ROOT = SCRIPT_DIR.parent.parent
|
||||
sys.path.insert(0, str(REPO_ROOT / "tools" / "mota"))
|
||||
|
||||
import motalib # noqa: E402
|
||||
|
||||
APP_BASE = 0x26000
|
||||
STAGE_CEILING = 0xD4000
|
||||
PAGE = 4096
|
||||
SEGMENT = PAGE
|
||||
FIXED_MEMORY = 0x98000
|
||||
AVAILABLE_PAGES = (STAGE_CEILING - APP_BASE) // PAGE
|
||||
EXPECTED_TARGET_ID = 0x2FA509C1
|
||||
EXPECTED_HARDWARE = "RAK_3401"
|
||||
FIELDS = [
|
||||
"source", "target", "source_sha256", "target_sha256", "memory",
|
||||
"payload", "container", "stage_start", "margin", "feasible", "error",
|
||||
]
|
||||
|
||||
|
||||
class RouteSearchError(RuntimeError):
|
||||
"""An invalid input or internally inconsistent cache."""
|
||||
|
||||
|
||||
def align_up(value: int, unit: int = PAGE) -> int:
|
||||
return (value + unit - 1) // unit * unit
|
||||
|
||||
|
||||
def align_down(value: int, unit: int = PAGE) -> int:
|
||||
return value // unit * unit
|
||||
|
||||
|
||||
def container_size(payload_size: int) -> int:
|
||||
return payload_size + 210 + 4 * ((payload_size + 1023) // 1024)
|
||||
|
||||
|
||||
def truth(value: object) -> bool:
|
||||
return value is True or str(value).lower() == "true"
|
||||
|
||||
|
||||
def sha256_file(path: Path) -> str:
|
||||
digest = hashlib.sha256()
|
||||
with path.open("rb") as source:
|
||||
while chunk := source.read(1024 * 1024):
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest()
|
||||
|
||||
|
||||
def _integer(value: object, label: str) -> int:
|
||||
if isinstance(value, int) and not isinstance(value, bool):
|
||||
result = value
|
||||
elif isinstance(value, str) and re.fullmatch(r"[0-9]+", value):
|
||||
result = int(value)
|
||||
else:
|
||||
raise RouteSearchError(f"{label} must be an integer")
|
||||
if result < 0:
|
||||
raise RouteSearchError(f"{label} cannot be negative")
|
||||
return result
|
||||
|
||||
|
||||
def load_inventory(path: Path) -> list[dict[str, object]]:
|
||||
try:
|
||||
document = json.loads(path.read_text(encoding="ascii"))
|
||||
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
|
||||
raise RouteSearchError(f"cannot read image inventory {path}: {exc}") from exc
|
||||
images = document.get("images") if isinstance(document, dict) else None
|
||||
if not isinstance(images, list) or len(images) < 2:
|
||||
raise RouteSearchError(f"invalid image inventory: {path}")
|
||||
|
||||
result: list[dict[str, object]] = []
|
||||
seen: set[str] = set()
|
||||
for index, raw in enumerate(images):
|
||||
if not isinstance(raw, dict):
|
||||
raise RouteSearchError(f"inventory image {index} is not an object")
|
||||
record = dict(raw)
|
||||
if _integer(record.get("node", index), f"inventory node {index}") != index:
|
||||
raise RouteSearchError(
|
||||
"inventory nodes must be ordered, contiguous, and zero-based"
|
||||
)
|
||||
raw_path = record.get("path")
|
||||
if not isinstance(raw_path, str) or not raw_path:
|
||||
raise RouteSearchError(f"inventory image {index} has no path")
|
||||
image_path = Path(raw_path)
|
||||
if not image_path.is_absolute():
|
||||
image_path = path.parent / image_path
|
||||
image_path = image_path.resolve()
|
||||
if not image_path.is_file():
|
||||
raise RouteSearchError(f"missing inventory image: {image_path}")
|
||||
try:
|
||||
image = image_path.read_bytes()
|
||||
except OSError as exc:
|
||||
raise RouteSearchError(f"cannot read inventory image: {image_path}") from exc
|
||||
actual_sha = hashlib.sha256(image).hexdigest()
|
||||
expected_sha = record.get("sha256")
|
||||
if not isinstance(expected_sha, str) or len(expected_sha) != 64 or any(
|
||||
character not in "0123456789abcdefABCDEF" for character in expected_sha
|
||||
):
|
||||
raise RouteSearchError(f"image {index} SHA-256 is invalid")
|
||||
expected_sha = expected_sha.lower()
|
||||
if actual_sha != expected_sha:
|
||||
raise RouteSearchError(f"image {index} SHA mismatch")
|
||||
if expected_sha in seen:
|
||||
raise RouteSearchError(f"duplicate image SHA at node {index}")
|
||||
seen.add(expected_sha)
|
||||
actual_size = len(image)
|
||||
if _integer(record.get("size"), f"image {index} size") != actual_size:
|
||||
raise RouteSearchError(f"image {index} size mismatch")
|
||||
if not motalib.has_endf(image):
|
||||
raise RouteSearchError(f"image {index} has no valid EndF")
|
||||
ident = motalib.parse_endf_ident(image)
|
||||
if (
|
||||
ident is None
|
||||
or ident.target_id != EXPECTED_TARGET_ID
|
||||
or ident.hw_id != EXPECTED_HARDWARE
|
||||
):
|
||||
raise RouteSearchError(f"image {index} has the wrong RAK3401 identity")
|
||||
_body, body_hash = motalib.parse_endf(image)
|
||||
declared_body_hash = record.get("body_hash")
|
||||
if (
|
||||
not isinstance(declared_body_hash, str)
|
||||
or not re.fullmatch(r"[0-9a-f]{16}", declared_body_hash)
|
||||
or declared_body_hash != body_hash.hex()
|
||||
):
|
||||
raise RouteSearchError(f"image {index} body hash mismatch")
|
||||
version = motalib.unpack_version(ident.fw_version)
|
||||
if record.get("version") != version:
|
||||
raise RouteSearchError(f"image {index} version mismatch")
|
||||
source_commit = record.get("source_commit")
|
||||
if source_commit is not None and (
|
||||
not isinstance(source_commit, str)
|
||||
or not re.fullmatch(r"[0-9a-f]{40}", source_commit)
|
||||
):
|
||||
raise RouteSearchError(f"image {index} source commit is invalid")
|
||||
if index == 1 and _integer(
|
||||
record.get("baseline_container_size"),
|
||||
"node 1 baseline_container_size",
|
||||
) <= 0:
|
||||
raise RouteSearchError("node 1 baseline_container_size must be positive")
|
||||
record.update(node=index, path=str(image_path), sha256=expected_sha, size=actual_size)
|
||||
result.append(record)
|
||||
return result
|
||||
|
||||
|
||||
def snapshot_inventory(source: Path, destination: Path) -> Path:
|
||||
"""Copy an inventory and its images before starting parallel patch work."""
|
||||
try:
|
||||
document = json.loads(source.read_text(encoding="ascii"))
|
||||
except (OSError, UnicodeError, json.JSONDecodeError) as exc:
|
||||
raise RouteSearchError(f"cannot read image inventory {source}: {exc}") from exc
|
||||
images = document.get("images") if isinstance(document, dict) else None
|
||||
if not isinstance(images, list) or len(images) < 2:
|
||||
raise RouteSearchError(f"invalid image inventory: {source}")
|
||||
image_directory = destination.parent / "images"
|
||||
image_directory.mkdir(parents=True, exist_ok=False)
|
||||
for index, raw in enumerate(images):
|
||||
if not isinstance(raw, dict):
|
||||
raise RouteSearchError(f"inventory image {index} is not an object")
|
||||
raw_path = raw.get("path")
|
||||
if not isinstance(raw_path, str) or not raw_path:
|
||||
raise RouteSearchError(f"inventory image {index} has no path")
|
||||
image_source = Path(raw_path)
|
||||
if not image_source.is_absolute():
|
||||
image_source = source.parent / image_source
|
||||
image_destination = image_directory / f"image-{index:03d}.bin"
|
||||
try:
|
||||
shutil.copy2(image_source, image_destination, follow_symlinks=True)
|
||||
except OSError as exc:
|
||||
raise RouteSearchError(
|
||||
f"cannot snapshot inventory image {index}: {image_source}: {exc}"
|
||||
) from exc
|
||||
raw["path"] = str(Path("images") / image_destination.name)
|
||||
destination.parent.mkdir(parents=True, exist_ok=True)
|
||||
destination.write_text(
|
||||
json.dumps(document, indent=2, sort_keys=True) + "\n",
|
||||
encoding="ascii",
|
||||
)
|
||||
# Validate the copied bytes against every declared content and EndF pin.
|
||||
load_inventory(destination)
|
||||
return destination
|
||||
|
||||
|
||||
def insert_bridges_before_endpoint(
|
||||
document: dict[str, object], bridges: list[dict[str, object]]
|
||||
) -> dict[str, object]:
|
||||
"""Insert ordered bridge records immediately before the existing endpoint."""
|
||||
existing = document.get("images")
|
||||
if (
|
||||
not isinstance(existing, list)
|
||||
or len(existing) < 2
|
||||
or not all(isinstance(item, dict) for item in existing)
|
||||
):
|
||||
raise RouteSearchError("inventory document has no valid start and endpoint")
|
||||
if not all(isinstance(item, dict) for item in bridges):
|
||||
raise RouteSearchError("inventory bridges must be objects")
|
||||
# Copy every record, preserving the last existing record as the endpoint.
|
||||
ordered = [dict(item) for item in existing[:-1]]
|
||||
ordered.extend(dict(item) for item in bridges)
|
||||
ordered.append(dict(existing[-1]))
|
||||
images = []
|
||||
for node, record in enumerate(ordered):
|
||||
record["node"] = node
|
||||
images.append(record)
|
||||
result = dict(document)
|
||||
result["images"] = images
|
||||
return result
|
||||
|
||||
|
||||
def cache_key(source_sha: str, target_sha: str, memory: int) -> tuple[str, str, int]:
|
||||
return source_sha.lower(), target_sha.lower(), memory
|
||||
|
||||
|
||||
def migrate_csv(
|
||||
inventory_path: Path, csv_path: Path
|
||||
) -> dict[tuple[str, str, int], dict[str, object]]:
|
||||
"""Migrate numeric CSV indices through the inventory that created them."""
|
||||
images = load_inventory(inventory_path)
|
||||
migrated: dict[tuple[str, str, int], dict[str, object]] = {}
|
||||
try:
|
||||
source = csv_path.open(newline="", encoding="ascii")
|
||||
except OSError as exc:
|
||||
raise RouteSearchError(f"cannot read geometry cache {csv_path}: {exc}") from exc
|
||||
with source:
|
||||
reader = csv.DictReader(source)
|
||||
required_fields = {"source", "target", "memory"}
|
||||
if reader.fieldnames is None or not required_fields.issubset(reader.fieldnames):
|
||||
raise RouteSearchError(f"geometry cache has missing columns: {csv_path}")
|
||||
for line, row in enumerate(reader, 2):
|
||||
source_node = _integer(row["source"], f"source at {csv_path}:{line}")
|
||||
target_node = _integer(row["target"], f"target at {csv_path}:{line}")
|
||||
if not (0 <= source_node < target_node < len(images)):
|
||||
raise RouteSearchError(f"bad nodes at {csv_path}:{line}")
|
||||
source_sha = str(images[source_node]["sha256"])
|
||||
target_sha = str(images[target_node]["sha256"])
|
||||
memory = _integer(row["memory"], f"memory at {csv_path}:{line}")
|
||||
for column, expected in (
|
||||
("source_sha256", source_sha), ("target_sha256", target_sha)
|
||||
):
|
||||
raw_recorded = row.get(column, "")
|
||||
if raw_recorded is None:
|
||||
raw_recorded = ""
|
||||
if not isinstance(raw_recorded, str):
|
||||
raise RouteSearchError(
|
||||
f"{column} is invalid at {csv_path}:{line}"
|
||||
)
|
||||
recorded = raw_recorded.lower()
|
||||
if recorded and recorded != expected:
|
||||
raise RouteSearchError(
|
||||
f"{column} disagrees with inventory at {csv_path}:{line}"
|
||||
)
|
||||
try:
|
||||
payload = int(row.get("payload", ""))
|
||||
total = int(row.get("container", ""))
|
||||
stage_start = int(row.get("stage_start", ""))
|
||||
margin = int(row.get("margin", ""))
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise RouteSearchError(
|
||||
f"invalid numeric geometry at {csv_path}:{line}"
|
||||
) from exc
|
||||
if payload < 0:
|
||||
raise RouteSearchError(
|
||||
f"failed cached geometry cannot prove an exhaustive search at "
|
||||
f"{csv_path}:{line}; regenerate it"
|
||||
)
|
||||
if row.get("error", ""):
|
||||
raise RouteSearchError(
|
||||
f"successful cached geometry carries an error at "
|
||||
f"{csv_path}:{line}; regenerate it"
|
||||
)
|
||||
expected_total = container_size(payload)
|
||||
expected_stage = align_down(STAGE_CEILING - expected_total)
|
||||
expected_margin = expected_stage - (APP_BASE + memory)
|
||||
if (total, stage_start, margin) != (
|
||||
expected_total, expected_stage, expected_margin
|
||||
):
|
||||
raise RouteSearchError(
|
||||
f"inconsistent cached geometry at {csv_path}:{line}"
|
||||
)
|
||||
if truth(row.get("feasible", False)) != (payload >= 0 and margin >= 0):
|
||||
raise RouteSearchError(
|
||||
f"cached feasibility is inconsistent at {csv_path}:{line}"
|
||||
)
|
||||
key = cache_key(source_sha, target_sha, memory)
|
||||
normalized = {field: row.get(field, "") for field in FIELDS}
|
||||
normalized.update(source=source_node, target=target_node,
|
||||
source_sha256=source_sha, target_sha256=target_sha,
|
||||
memory=memory)
|
||||
previous = migrated.get(key)
|
||||
comparable = ("payload", "container", "stage_start", "margin", "feasible", "error")
|
||||
if previous is not None and any(
|
||||
str(previous.get(item, "")) != str(normalized.get(item, ""))
|
||||
for item in comparable
|
||||
):
|
||||
raise RouteSearchError(f"conflicting cached geometry for {key}")
|
||||
migrated[key] = normalized
|
||||
return migrated
|
||||
|
||||
|
||||
Job = tuple[int, int, int, str, str, str, str]
|
||||
|
||||
|
||||
def all_jobs(images: list[dict[str, object]]) -> list[Job]:
|
||||
"""Enumerate every valid page-aligned workspace for every forward edge."""
|
||||
jobs: list[Job] = []
|
||||
endpoint = len(images) - 1
|
||||
# Node 0 may only use the byte-identical, physically qualified 0->1 package.
|
||||
# Node 1 still runs the fixed-workspace receiver, so measure each of its
|
||||
# possible forward transitions at exactly that legacy workspace.
|
||||
for target in range(2, endpoint + 1):
|
||||
jobs.append((1, target, FIXED_MEMORY, str(images[1]["path"]),
|
||||
str(images[target]["path"]), str(images[1]["sha256"]),
|
||||
str(images[target]["sha256"])))
|
||||
for source in range(2, endpoint):
|
||||
source_size = int(images[source]["size"])
|
||||
for target in range(source + 1, endpoint + 1):
|
||||
target_size = int(images[target]["size"])
|
||||
first_page = align_up(max(source_size + 2 * PAGE, target_size)) // PAGE
|
||||
for memory_page in range(first_page, AVAILABLE_PAGES):
|
||||
jobs.append((source, target, memory_page * PAGE,
|
||||
str(images[source]["path"]), str(images[target]["path"]),
|
||||
str(images[source]["sha256"]), str(images[target]["sha256"])))
|
||||
return jobs
|
||||
|
||||
|
||||
def geometry_job(args: tuple[int, int, int, str, str, str, str, str]) -> dict[str, object]:
|
||||
try:
|
||||
import detools
|
||||
except ImportError as exc:
|
||||
raise RouteSearchError(
|
||||
"detools is required to generate geometry; run this tool in the "
|
||||
"detools pipx environment"
|
||||
) from exc
|
||||
source, target, memory, from_raw, to_raw, source_sha, target_sha, patches_raw = args
|
||||
patch = Path(patches_raw) / f"{source_sha}-{target_sha}-{memory // PAGE:03d}.patch"
|
||||
try:
|
||||
detools.create_patch_filenames(
|
||||
from_raw, to_raw, str(patch), compression="crle", patch_type="in-place",
|
||||
algorithm="bsdiff", suffix_array_algorithm="divsufsort",
|
||||
memory_size=memory, segment_size=SEGMENT, use_mmap=True,
|
||||
)
|
||||
payload = patch.stat().st_size
|
||||
total = container_size(payload)
|
||||
stage_start = align_down(STAGE_CEILING - total)
|
||||
margin = stage_start - (APP_BASE + memory)
|
||||
error = ""
|
||||
except Exception as exc: # detools reports several backend exception types
|
||||
detail = f"{type(exc).__name__}: {exc}"[-500:].replace("\n", " ")
|
||||
raise RouteSearchError(
|
||||
f"detools geometry failed for {source}->{target} at 0x{memory:X}: {detail}"
|
||||
) from exc
|
||||
finally:
|
||||
patch.unlink(missing_ok=True)
|
||||
return {"source": source, "target": target, "source_sha256": source_sha,
|
||||
"target_sha256": target_sha, "memory": memory, "payload": payload,
|
||||
"container": total, "stage_start": stage_start, "margin": margin,
|
||||
"feasible": payload >= 0 and margin >= 0, "error": error}
|
||||
|
||||
|
||||
def project_cache(cache: dict[tuple[str, str, int], dict[str, object]],
|
||||
images: list[dict[str, object]],
|
||||
required: set[tuple[str, str, int]]) -> list[dict[str, object]]:
|
||||
index = {str(record["sha256"]): int(record["node"]) for record in images}
|
||||
rows: list[dict[str, object]] = []
|
||||
for key in sorted(required, key=lambda item: (index[item[0]], index[item[1]], item[2])):
|
||||
if key not in cache:
|
||||
raise RouteSearchError(f"required cache geometry is missing: {key}")
|
||||
row = dict(cache[key])
|
||||
if int(row.get("payload", -1)) < 0 or str(row.get("error", "")):
|
||||
raise RouteSearchError(
|
||||
f"required cache geometry is not a successful measurement: {key}"
|
||||
)
|
||||
row.update(source=index[key[0]], target=index[key[1]],
|
||||
source_sha256=key[0], target_sha256=key[1], memory=key[2])
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def write_atomic_text(path: Path, contents: str) -> None:
|
||||
"""Replace one regular output without ever following its pathname symlink."""
|
||||
if path.is_symlink() or (path.exists() and not path.is_file()):
|
||||
raise RouteSearchError(f"output path is not a regular file: {path}")
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
descriptor, temporary_name = tempfile.mkstemp(
|
||||
prefix=f".{path.name}.", dir=path.parent
|
||||
)
|
||||
temporary = Path(temporary_name)
|
||||
descriptor_open = True
|
||||
try:
|
||||
with os.fdopen(
|
||||
descriptor, "w", encoding="ascii", newline="\n"
|
||||
) as output:
|
||||
descriptor_open = False
|
||||
output.write(contents)
|
||||
output.flush()
|
||||
os.fsync(output.fileno())
|
||||
temporary.chmod(0o600)
|
||||
os.replace(temporary, path)
|
||||
except BaseException:
|
||||
if descriptor_open:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
temporary.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
|
||||
def write_csv(path: Path, rows: list[dict[str, object]]) -> None:
|
||||
if path.is_symlink() or (path.exists() and not path.is_file()):
|
||||
raise RouteSearchError(f"output path is not a regular file: {path}")
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
descriptor, temporary_name = tempfile.mkstemp(
|
||||
prefix=f".{path.name}.", dir=path.parent
|
||||
)
|
||||
temporary = Path(temporary_name)
|
||||
descriptor_open = True
|
||||
try:
|
||||
with os.fdopen(descriptor, "w", newline="", encoding="ascii") as output:
|
||||
descriptor_open = False
|
||||
writer = csv.DictWriter(
|
||||
output, fieldnames=FIELDS, lineterminator="\n", extrasaction="ignore"
|
||||
)
|
||||
writer.writeheader()
|
||||
writer.writerows(rows)
|
||||
output.flush()
|
||||
os.fsync(output.fileno())
|
||||
temporary.chmod(0o600)
|
||||
os.replace(temporary, path)
|
||||
except BaseException:
|
||||
if descriptor_open:
|
||||
try:
|
||||
os.close(descriptor)
|
||||
except OSError:
|
||||
pass
|
||||
temporary.unlink(missing_ok=True)
|
||||
raise
|
||||
|
||||
|
||||
def select_route(rows: list[dict[str, object]], images: list[dict[str, object]],
|
||||
baseline_size: int, output: Path, complete: bool) -> dict[str, object]:
|
||||
endpoint = len(images) - 1
|
||||
baseline_stage = align_down(STAGE_CEILING - baseline_size)
|
||||
baseline_margin = baseline_stage - (APP_BASE + FIXED_MEMORY)
|
||||
if baseline_margin < 0:
|
||||
raise RouteSearchError("pinned first package does not fit")
|
||||
edges: dict[tuple[int, int], dict[str, object]] = {
|
||||
(0, 1): {"source": 0, "target": 1, "memory": FIXED_MEMORY,
|
||||
"container": baseline_size, "stage_start": baseline_stage,
|
||||
"margin": baseline_margin, "feasible": True}
|
||||
}
|
||||
for row in rows:
|
||||
if int(row["source"]) == 0:
|
||||
# Older evidence tables measured these irrelevant shortcuts. Keep
|
||||
# them countable for archive verification, but never admit them to
|
||||
# the route graph; node 0 is pinned unconditionally to node 1.
|
||||
continue
|
||||
if not truth(row.get("feasible", False)):
|
||||
continue
|
||||
key = int(row["source"]), int(row["target"])
|
||||
rank = (int(row["container"]), -int(row["margin"]), int(row["memory"]))
|
||||
old = edges.get(key)
|
||||
if old is None or rank < (int(old["container"]), -int(old["margin"]), int(old["memory"])):
|
||||
edges[key] = row
|
||||
|
||||
best: dict[int, tuple[int, int, list[int]]] = {0: (0, 0, [0])}
|
||||
hop_distance = [sys.maxsize] * len(images)
|
||||
route_count = [0] * len(images)
|
||||
hop_distance[0], route_count[0] = 0, 1
|
||||
outgoing: dict[int, list[tuple[int, dict[str, object]]]] = {}
|
||||
for (source, target), row in edges.items():
|
||||
outgoing.setdefault(source, []).append((target, row))
|
||||
for source in range(endpoint):
|
||||
if source not in best:
|
||||
continue
|
||||
for target, row in outgoing.get(source, []):
|
||||
candidate = (best[source][0] + 1,
|
||||
best[source][1] + int(row["container"]),
|
||||
best[source][2] + [target])
|
||||
if target not in best or candidate < best[target]:
|
||||
best[target] = candidate
|
||||
distance = hop_distance[source] + 1
|
||||
if distance < hop_distance[target]:
|
||||
hop_distance[target], route_count[target] = distance, route_count[source]
|
||||
elif distance == hop_distance[target]:
|
||||
route_count[target] += route_count[source]
|
||||
|
||||
common = {"schema": 2, "app_base": f"0x{APP_BASE:X}",
|
||||
"stage_ceiling": f"0x{STAGE_CEILING:X}", "node_count": len(images),
|
||||
"search_complete": complete, "candidate_geometries": len(rows),
|
||||
"feasible_edges": len(edges),
|
||||
"objective": "minimum packages, then minimum total container bytes"}
|
||||
if endpoint not in best:
|
||||
result = {**common, "status": "unreachable", "reachable_nodes": sorted(best),
|
||||
"endpoint_node": endpoint,
|
||||
"endpoint_incoming_feasible": sorted(
|
||||
source for source, target in edges if target == endpoint
|
||||
)}
|
||||
else:
|
||||
nodes = best[endpoint][2]
|
||||
steps = []
|
||||
for source, target in zip(nodes, nodes[1:]):
|
||||
row = edges[source, target]
|
||||
steps.append({"source_node": source, "target_node": target,
|
||||
"inplace_memory": f"0x{int(row['memory']):X}",
|
||||
"reuse_baseline_package": source == 0 and target == 1,
|
||||
"expected_container_size": int(row["container"]),
|
||||
"expected_staging_margin": int(row["margin"]),
|
||||
"expected_target_sha256": images[target]["sha256"],
|
||||
"expected_target_version": images[target].get("version", "unknown")})
|
||||
result = {**common, "status": "reachable", "nodes": nodes,
|
||||
"shortest_package_count": hop_distance[endpoint],
|
||||
"shortest_route_count": route_count[endpoint],
|
||||
"selected_total_bytes": best[endpoint][1], "steps": steps}
|
||||
write_atomic_text(
|
||||
output, json.dumps(result, indent=2, sort_keys=True) + "\n"
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--inventory", type=Path, required=True)
|
||||
parser.add_argument("--reuse", action="append", default=[], metavar="IMAGES_JSON=GEOMETRY_CSV")
|
||||
parser.add_argument("--work-dir", type=Path, required=True)
|
||||
parser.add_argument("--workers", type=int, default=min(8, os.cpu_count() or 1))
|
||||
parser.add_argument("--no-generate", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
if args.workers < 1:
|
||||
parser.error("--workers must be positive")
|
||||
args.work_dir.mkdir(parents=True, exist_ok=True)
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix=".frozen-inventory-", dir=args.work_dir
|
||||
) as frozen_raw:
|
||||
frozen = Path(frozen_raw)
|
||||
inventory_path = snapshot_inventory(
|
||||
args.inventory, frozen / "current" / "images.json"
|
||||
)
|
||||
images = load_inventory(inventory_path)
|
||||
baseline_size = _integer(
|
||||
images[1].get("baseline_container_size"),
|
||||
"node 1 baseline_container_size",
|
||||
)
|
||||
cache: dict[tuple[str, str, int], dict[str, object]] = {}
|
||||
comparable = (
|
||||
"payload", "container", "stage_start", "margin", "feasible", "error"
|
||||
)
|
||||
for reuse_number, value in enumerate(args.reuse, 1):
|
||||
left, separator, right = value.partition("=")
|
||||
if not separator:
|
||||
parser.error("--reuse requires IMAGES_JSON=GEOMETRY_CSV")
|
||||
reuse_root = frozen / f"reuse-{reuse_number:03d}"
|
||||
reuse_inventory = snapshot_inventory(
|
||||
Path(left), reuse_root / "images.json"
|
||||
)
|
||||
reuse_csv = reuse_root / "geometry.csv"
|
||||
try:
|
||||
shutil.copy2(Path(right), reuse_csv, follow_symlinks=True)
|
||||
except OSError as exc:
|
||||
raise RouteSearchError(
|
||||
f"cannot snapshot geometry cache {right}: {exc}"
|
||||
) from exc
|
||||
for key, row in migrate_csv(reuse_inventory, reuse_csv).items():
|
||||
old = cache.get(key)
|
||||
if old is not None and any(
|
||||
str(old.get(field, "")) != str(row.get(field, ""))
|
||||
for field in comparable
|
||||
):
|
||||
raise RouteSearchError(f"conflicting reuse inputs for {key}")
|
||||
cache[key] = row
|
||||
|
||||
jobs = all_jobs(images)
|
||||
required = {cache_key(job[5], job[6], job[2]) for job in jobs}
|
||||
remaining = [
|
||||
job for job in jobs
|
||||
if cache_key(job[5], job[6], job[2]) not in cache
|
||||
]
|
||||
print(
|
||||
f"nodes={len(images)} geometries={len(jobs)} "
|
||||
f"reused={len(required & cache.keys())} remaining={len(remaining)}"
|
||||
)
|
||||
if remaining and not args.no_generate:
|
||||
# Patch payloads are disposable worker scratch. Keep them in a
|
||||
# uniquely owned directory so cleanup can never remove a caller's
|
||||
# pre-existing `work-dir/patches` tree.
|
||||
with tempfile.TemporaryDirectory(
|
||||
prefix=".route-patches-", dir=args.work_dir
|
||||
) as patches_raw:
|
||||
patches = Path(patches_raw)
|
||||
with ProcessPoolExecutor(max_workers=args.workers) as pool:
|
||||
futures = [
|
||||
pool.submit(geometry_job, (*job, str(patches)))
|
||||
for job in remaining
|
||||
]
|
||||
for done, future in enumerate(as_completed(futures), 1):
|
||||
row = future.result()
|
||||
key = cache_key(
|
||||
str(row["source_sha256"]),
|
||||
str(row["target_sha256"]),
|
||||
int(row["memory"]),
|
||||
)
|
||||
cache[key] = row
|
||||
if done % 100 == 0 or done == len(futures):
|
||||
print(f"measured={done}/{len(remaining)}", flush=True)
|
||||
completed = required & cache.keys()
|
||||
rows = project_cache(cache, images, completed)
|
||||
write_csv(args.work_dir / "geometry.csv", rows)
|
||||
complete = completed == required
|
||||
result = select_route(
|
||||
rows, images, baseline_size, args.work_dir / "route.json",
|
||||
complete=complete,
|
||||
)
|
||||
print(json.dumps(result, indent=2, sort_keys=True))
|
||||
return 0 if complete else 3
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,688 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Focused tests for variable-node compact RAK3401 bundle inputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import build_rak3401_compact_bundle as compact
|
||||
|
||||
|
||||
def firmware(body: bytes, version: str) -> bytes:
|
||||
hardware = compact.EXPECTED_HARDWARE.encode("ascii").ljust(32, b"\0")
|
||||
return (
|
||||
body
|
||||
+ b"EndF"
|
||||
+ struct.pack("<I", len(body))
|
||||
+ hashlib.sha256(body).digest()[:8]
|
||||
+ struct.pack(
|
||||
"<II", compact.motalib.pack_version(version), compact.EXPECTED_TARGET_ID
|
||||
)
|
||||
+ hardware
|
||||
)
|
||||
|
||||
|
||||
def uf2(image: bytes, payload_size: int = 256) -> bytes:
|
||||
padded_size = compact.align_up(len(image), payload_size)
|
||||
encoded = image + b"\xFF" * (padded_size - len(image))
|
||||
block_count = len(encoded) // payload_size
|
||||
blocks = []
|
||||
for number in range(block_count):
|
||||
block = bytearray(compact.UF2_BLOCK_SIZE)
|
||||
struct.pack_into(
|
||||
"<IIIIIIII", block, 0,
|
||||
compact.UF2_MAGIC_START0,
|
||||
compact.UF2_MAGIC_START1,
|
||||
compact.UF2_FLAG_FAMILY_ID,
|
||||
compact.APP_BASE + number * payload_size,
|
||||
payload_size,
|
||||
number,
|
||||
block_count,
|
||||
compact.UF2_NRF52840_FAMILY_ID,
|
||||
)
|
||||
block[compact.UF2_DATA_OFFSET:compact.UF2_DATA_OFFSET + payload_size] = (
|
||||
encoded[number * payload_size:(number + 1) * payload_size]
|
||||
)
|
||||
struct.pack_into(
|
||||
"<I", block, compact.UF2_BLOCK_SIZE - 4, compact.UF2_MAGIC_END
|
||||
)
|
||||
blocks.append(bytes(block))
|
||||
return b"".join(blocks)
|
||||
|
||||
|
||||
class CompactRouteInputTests(unittest.TestCase):
|
||||
def schema2_route(self) -> dict[str, object]:
|
||||
return {
|
||||
"schema": 2,
|
||||
"status": "reachable",
|
||||
"search_complete": True,
|
||||
"app_base": "0x26000",
|
||||
"stage_ceiling": "0xD4000",
|
||||
"node_count": 4,
|
||||
"objective": "minimum packages, then minimum total container bytes",
|
||||
"candidate_geometries": 100,
|
||||
"feasible_edges": 4,
|
||||
"shortest_package_count": 2,
|
||||
"shortest_route_count": 1,
|
||||
"selected_total_bytes": 300,
|
||||
"steps": [
|
||||
{
|
||||
"source_node": 0,
|
||||
"target_node": 1,
|
||||
"inplace_memory": "0x98000",
|
||||
"reuse_baseline_package": True,
|
||||
"expected_container_size": 100,
|
||||
"expected_staging_margin": 1,
|
||||
"expected_target_sha256": "1" * 64,
|
||||
"expected_target_version": "1.16.7.9",
|
||||
},
|
||||
{
|
||||
"source_node": 1,
|
||||
"target_node": 3,
|
||||
"inplace_memory": "0x98000",
|
||||
"reuse_baseline_package": False,
|
||||
"expected_container_size": 200,
|
||||
"expected_staging_margin": 2,
|
||||
"expected_target_sha256": "2" * 64,
|
||||
"expected_target_version": "1.17.1.5",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
def test_existing_schema1_route_remains_accepted(self) -> None:
|
||||
route = Path(__file__).with_name("rak3401_compact_route.json")
|
||||
steps, document = compact.read_route(route)
|
||||
self.assertEqual(document["schema"], 1)
|
||||
self.assertEqual(document["endpoint_node"], 30)
|
||||
self.assertEqual(len(steps), 9)
|
||||
|
||||
def test_schema2_route_is_normalized_for_bundle_provenance(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
path = Path(directory) / "route.json"
|
||||
path.write_text(json.dumps(self.schema2_route()), encoding="ascii")
|
||||
steps, document = compact.read_route(path)
|
||||
self.assertEqual([step["target_node"] for step in steps], [1, 3])
|
||||
self.assertEqual(document["endpoint_node"], 3)
|
||||
self.assertTrue(document["search"]["search_complete"])
|
||||
self.assertEqual(document["search"]["selected_total_bytes"], 300)
|
||||
|
||||
def test_public_route_search_drops_schema1_private_fields(self) -> None:
|
||||
route = Path(__file__).with_name("rak3401_compact_route.json")
|
||||
_steps, document = compact.read_route(route)
|
||||
document["search"].update({
|
||||
"password": "must-not-be-archived",
|
||||
"local_path": "/tmp/private/route.json",
|
||||
"nested": {"api_token": "must-not-be-archived"},
|
||||
})
|
||||
|
||||
public = compact.public_route_search(1, document)
|
||||
|
||||
self.assertNotIn("password", public)
|
||||
self.assertNotIn("local_path", public)
|
||||
self.assertNotIn("nested", public)
|
||||
self.assertEqual(
|
||||
set(public),
|
||||
{
|
||||
"objective", "shortest_package_count", "selected_total_bytes",
|
||||
"page_size", "candidate_pairs", "candidate_geometries",
|
||||
"feasible_edges", "shortest_route_count",
|
||||
},
|
||||
)
|
||||
|
||||
def test_schema2_route_must_be_a_complete_reachable_search(self) -> None:
|
||||
for field, value, message in (
|
||||
("status", "unreachable", "does not reach"),
|
||||
("search_complete", False, "incomplete"),
|
||||
):
|
||||
with self.subTest(field=field), tempfile.TemporaryDirectory() as directory:
|
||||
document = self.schema2_route()
|
||||
document[field] = value
|
||||
path = Path(directory) / "route.json"
|
||||
path.write_text(json.dumps(document), encoding="ascii")
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, message):
|
||||
compact.read_route(path)
|
||||
|
||||
|
||||
class CompactImageInventoryTests(unittest.TestCase):
|
||||
def test_schema2_snapshot_freezes_manifest_and_referenced_images(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source"
|
||||
source.mkdir()
|
||||
image = firmware(b"frozen body", "1.17.1.5")
|
||||
image_path = source / "image.bin"
|
||||
image_path.write_bytes(image)
|
||||
record = {
|
||||
"node": 0,
|
||||
"path": image_path.name,
|
||||
"size": len(image),
|
||||
"sha256": hashlib.sha256(image).hexdigest(),
|
||||
"body_hash": hashlib.sha256(image[:-56]).digest()[:8].hex(),
|
||||
"version": "1.17.1.5",
|
||||
}
|
||||
manifest = source / "images.json"
|
||||
manifest.write_text(json.dumps({"images": [record]}), encoding="ascii")
|
||||
frozen = compact.snapshot_schema2_inventory(
|
||||
manifest, root / "frozen" / "images.json"
|
||||
)
|
||||
|
||||
image_path.write_bytes(b"changed")
|
||||
manifest.write_text("{}", encoding="ascii")
|
||||
images, _sources = compact.read_image_inventory(frozen, 1)
|
||||
|
||||
self.assertEqual(images[0], image)
|
||||
|
||||
def test_schema1_snapshot_freezes_manifest_sibling_zips(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source"
|
||||
source.mkdir()
|
||||
archive = source / "bridge.zip"
|
||||
archive.write_bytes(b"frozen zip")
|
||||
manifest = source / "bridges.json"
|
||||
manifest.write_text(
|
||||
json.dumps({"targets": [{"zip": archive.name}]}),
|
||||
encoding="ascii",
|
||||
)
|
||||
frozen = compact.snapshot_schema1_manifest(
|
||||
manifest, root / "frozen" / manifest.name
|
||||
)
|
||||
|
||||
archive.write_bytes(b"changed")
|
||||
manifest.write_text("{}", encoding="ascii")
|
||||
self.assertEqual(
|
||||
json.loads(frozen.read_text(encoding="ascii")),
|
||||
{"targets": [{"zip": "bridge.zip"}]},
|
||||
)
|
||||
self.assertEqual(
|
||||
(frozen.parent / "bridge.zip").read_bytes(), b"frozen zip"
|
||||
)
|
||||
|
||||
def test_inventory_validates_order_hash_size_identity_and_version(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
records = []
|
||||
expected = {}
|
||||
for node, version in enumerate(("1.16.7.0", "1.16.7.9", "1.17.1.5")):
|
||||
image = firmware(bytes([node + 1]) * (128 + node), version)
|
||||
image_path = root / f"image-{node:02d}.bin"
|
||||
image_path.write_bytes(image)
|
||||
expected[node] = image
|
||||
records.append({
|
||||
"node": node,
|
||||
"path": image_path.name,
|
||||
"size": len(image),
|
||||
"sha256": hashlib.sha256(image).hexdigest(),
|
||||
"body_hash": hashlib.sha256(image[:-56]).digest()[:8].hex(),
|
||||
"version": version,
|
||||
})
|
||||
manifest = root / "images.json"
|
||||
manifest.write_text(json.dumps({"images": records}), encoding="ascii")
|
||||
images, sources = compact.read_image_inventory(manifest, 3)
|
||||
self.assertEqual(images, expected)
|
||||
self.assertTrue(sources[1].startswith("image-sha256:"))
|
||||
|
||||
def test_inventory_rejects_a_stale_image_pin(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
image = firmware(b"body", "1.17.1.5")
|
||||
(root / "image-00.bin").write_bytes(image)
|
||||
record = {
|
||||
"node": 0,
|
||||
"path": "image-00.bin",
|
||||
"size": len(image),
|
||||
"sha256": "0" * 64,
|
||||
"body_hash": hashlib.sha256(b"body").digest()[:8].hex(),
|
||||
"version": "1.17.1.5",
|
||||
}
|
||||
manifest = root / "images.json"
|
||||
manifest.write_text(
|
||||
json.dumps({"images": [record, record, record]}), encoding="ascii"
|
||||
)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "SHA-256 mismatch"):
|
||||
compact.read_image_inventory(manifest, 3)
|
||||
|
||||
def test_bundled_inventory_drops_machine_local_paths(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source.json"
|
||||
destination = root / "bundled.json"
|
||||
source.write_text(
|
||||
json.dumps({
|
||||
"baseline_mota": "/tmp/private/step.mota",
|
||||
"admin_password": "must-not-be-archived",
|
||||
"api_token": "must-not-be-archived",
|
||||
"notes": "must-not-be-archived",
|
||||
"images": [{
|
||||
"node": 0,
|
||||
"path": "/tmp/private/image.bin",
|
||||
"size": 123,
|
||||
"sha256": "1" * 64,
|
||||
"body_hash": "2" * 16,
|
||||
"version": "1.16.7.0",
|
||||
"latitude": 47.0,
|
||||
"nested": {
|
||||
"source_path": "/tmp/private/source.bin",
|
||||
"private_key": "must-not-be-archived",
|
||||
"keep": 1,
|
||||
},
|
||||
}],
|
||||
}),
|
||||
encoding="ascii",
|
||||
)
|
||||
compact.write_inventory_provenance(source, destination)
|
||||
bundled = json.loads(destination.read_text(encoding="ascii"))
|
||||
self.assertNotIn("path", bundled["images"][0])
|
||||
self.assertNotIn("baseline_mota", bundled)
|
||||
self.assertNotIn("admin_password", bundled)
|
||||
self.assertNotIn("api_token", bundled)
|
||||
self.assertNotIn("notes", bundled)
|
||||
self.assertEqual(
|
||||
bundled,
|
||||
{
|
||||
"schema": 2,
|
||||
"images": [{
|
||||
"node": 0,
|
||||
"size": 123,
|
||||
"sha256": "1" * 64,
|
||||
"body_hash": "2" * 16,
|
||||
"version": "1.16.7.0",
|
||||
}],
|
||||
},
|
||||
)
|
||||
|
||||
def test_inventory_rejects_wrong_declared_body_hash(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
records = []
|
||||
for node in range(3):
|
||||
image = firmware(bytes([node + 1]) * 32, f"1.16.7.{node}")
|
||||
image_path = root / f"image-{node}.bin"
|
||||
image_path.write_bytes(image)
|
||||
records.append({
|
||||
"node": node, "path": image_path.name, "size": len(image),
|
||||
"sha256": hashlib.sha256(image).hexdigest(),
|
||||
"body_hash": "0" * 16, "version": f"1.16.7.{node}",
|
||||
})
|
||||
manifest = root / "images.json"
|
||||
manifest.write_text(json.dumps({"images": records}), encoding="ascii")
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "body hash mismatch"):
|
||||
compact.read_image_inventory(manifest, 3)
|
||||
|
||||
|
||||
class CompactSimulatorTests(unittest.TestCase):
|
||||
def test_exact_required_simulators_are_accepted(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
preview5 = Path(directory) / "preview5"
|
||||
current = Path(directory) / "current"
|
||||
preview5.write_bytes(b"p5")
|
||||
current.write_bytes(b"current")
|
||||
hashes = {
|
||||
preview5: compact.REQUIRED_SIMULATORS["preview5"]["sha256"],
|
||||
current: compact.REQUIRED_SIMULATORS["current"]["sha256"],
|
||||
}
|
||||
with mock.patch.object(compact.common, "sha256_file", side_effect=hashes.get):
|
||||
result = compact.parse_simulators([
|
||||
f"preview5={preview5}", f"current={current}"
|
||||
])
|
||||
self.assertEqual([label for label, _path in result], ["preview5", "current"])
|
||||
|
||||
def test_missing_or_wrong_simulator_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
simulator = Path(directory) / "sim"
|
||||
simulator.write_bytes(b"wrong")
|
||||
for values, digest in (([], None), ([f"preview5={simulator}"], "0" * 64)):
|
||||
with self.subTest(values=values), mock.patch.object(
|
||||
compact.common, "sha256_file", return_value=digest
|
||||
):
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "require exactly"):
|
||||
compact.parse_simulators(values)
|
||||
|
||||
|
||||
class CompactUf2Tests(unittest.TestCase):
|
||||
def test_exact_app_only_uf2_is_accepted(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
image = firmware(b"endpoint" * 41, "1.17.1.5")
|
||||
path = Path(directory) / "endpoint.uf2"
|
||||
path.write_bytes(uf2(image))
|
||||
compact.validate_uf2_firmware(path, image, "test UF2")
|
||||
|
||||
def test_wrong_family_and_non_app_address_are_rejected(self) -> None:
|
||||
image = firmware(b"endpoint", "1.17.1.5")
|
||||
for label, offset, value, message in (
|
||||
("family", 28, 0xDEADBEEF, "nRF52840"),
|
||||
("address", 12, compact.APP_BASE - 256, "app-only"),
|
||||
):
|
||||
with self.subTest(label=label), tempfile.TemporaryDirectory() as directory:
|
||||
raw = bytearray(uf2(image))
|
||||
struct.pack_into("<I", raw, offset, value)
|
||||
path = Path(directory) / "endpoint.uf2"
|
||||
path.write_bytes(raw)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, message):
|
||||
compact.validate_uf2_firmware(path, image, "test UF2")
|
||||
|
||||
def test_duplicate_block_and_firmware_mismatch_are_rejected(self) -> None:
|
||||
image = firmware(b"x" * 600, "1.17.1.5")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
duplicate = bytearray(uf2(image))
|
||||
struct.pack_into("<I", duplicate, compact.UF2_BLOCK_SIZE + 20, 0)
|
||||
duplicate_path = root / "duplicate.uf2"
|
||||
duplicate_path.write_bytes(duplicate)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "repeats"):
|
||||
compact.validate_uf2_firmware(duplicate_path, image, "test UF2")
|
||||
|
||||
mismatch = bytearray(uf2(image))
|
||||
mismatch[compact.UF2_DATA_OFFSET] ^= 1
|
||||
mismatch_path = root / "mismatch.uf2"
|
||||
mismatch_path.write_bytes(mismatch)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "does not match"):
|
||||
compact.validate_uf2_firmware(mismatch_path, image, "test UF2")
|
||||
|
||||
def test_non_erased_final_padding_is_rejected(self) -> None:
|
||||
image = firmware(b"odd", "1.17.1.5")
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
raw = bytearray(uf2(image))
|
||||
raw[compact.UF2_DATA_OFFSET + len(image)] = 0
|
||||
path = Path(directory) / "padding.uf2"
|
||||
path.write_bytes(raw)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "non-erased"):
|
||||
compact.validate_uf2_firmware(path, image, "test UF2")
|
||||
|
||||
|
||||
class CompactToolProvenanceTests(unittest.TestCase):
|
||||
def test_required_versions_and_actual_launcher_hashes_are_recorded(self) -> None:
|
||||
motatool = Path("/tools/motatool")
|
||||
detools = Path("/tools/detools")
|
||||
hashes = {motatool: "1" * 64, detools: "2" * 64}
|
||||
with mock.patch.object(
|
||||
compact, "run", side_effect=["motatool 0.1.0\n", "0.53.0\n"]
|
||||
), mock.patch.object(compact.common, "sha256_file", side_effect=hashes.get):
|
||||
result = compact.tool_provenance(motatool, detools)
|
||||
self.assertEqual(result["motatool"]["version"], "0.1.0")
|
||||
self.assertEqual(
|
||||
result["motatool"]["asserted_source_commit"], compact.MOTATOOL_COMMIT
|
||||
)
|
||||
self.assertEqual(result["motatool"]["executable_sha256"], "1" * 64)
|
||||
self.assertEqual(result["detools"]["launcher_sha256"], "2" * 64)
|
||||
|
||||
def test_wrong_tool_versions_fail_closed(self) -> None:
|
||||
for outputs, message in (
|
||||
(["motatool 0.2.0\n"], "motatool version"),
|
||||
(["motatool 0.1.0\n", "0.54.0\n"], "detools version"),
|
||||
):
|
||||
with self.subTest(outputs=outputs), mock.patch.object(
|
||||
compact, "run", side_effect=outputs
|
||||
):
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, message):
|
||||
compact.tool_provenance(Path("motatool"), Path("detools"))
|
||||
|
||||
|
||||
class CompactGeometryEvidenceTests(unittest.TestCase):
|
||||
def make_evidence(self, root: Path) -> tuple[Path, Path, list[dict[str, object]]]:
|
||||
records = []
|
||||
for node, version in enumerate(("1.16.7.0", "1.16.7.9", "1.17.1.3", "1.17.1.5")):
|
||||
image = firmware(bytes([node + 1]) * (64 + node), version)
|
||||
image_path = root / f"image-{node:02d}.bin"
|
||||
image_path.write_bytes(image)
|
||||
records.append({
|
||||
"node": node, "path": image_path.name, "size": len(image),
|
||||
"sha256": hashlib.sha256(image).hexdigest(),
|
||||
"body_hash": hashlib.sha256(image[:-56]).digest()[:8].hex(),
|
||||
"version": version,
|
||||
**({"baseline_container_size": 100} if node == 1 else {}),
|
||||
})
|
||||
inventory_path = root / "inventory.json"
|
||||
inventory_path.write_text(json.dumps({"images": records}), encoding="ascii")
|
||||
inventory = compact.route_search.load_inventory(inventory_path)
|
||||
rows = []
|
||||
for job in compact.route_search.all_jobs(inventory):
|
||||
source, target, memory, _from, _to, source_sha, target_sha = job
|
||||
if source == 1 and target == 3:
|
||||
payload = 10
|
||||
else:
|
||||
# A successfully generated patch may be too large for staging;
|
||||
# that is valid infeasible evidence. A tool exception is not.
|
||||
payload = 700_000
|
||||
container = compact.route_search.container_size(payload)
|
||||
stage_start = compact.route_search.align_down(
|
||||
compact.STAGE_CEILING - container
|
||||
)
|
||||
margin = stage_start - (compact.APP_BASE + memory)
|
||||
feasible, error = margin >= 0, ""
|
||||
rows.append({
|
||||
"source": source, "target": target, "source_sha256": source_sha,
|
||||
"target_sha256": target_sha, "memory": memory, "payload": payload,
|
||||
"container": container, "stage_start": stage_start, "margin": margin,
|
||||
"feasible": feasible, "error": error,
|
||||
})
|
||||
geometry_path = root / "geometry.csv"
|
||||
compact.route_search.write_csv(geometry_path, rows)
|
||||
route_path = root / "route.json"
|
||||
compact.route_search.select_route(rows, inventory, 100, route_path, complete=True)
|
||||
return inventory_path, route_path, rows
|
||||
|
||||
def test_complete_geometry_reproduces_declared_route(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, _rows = self.make_evidence(root)
|
||||
result = compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
self.assertEqual(result["nodes"], [0, 1, 3])
|
||||
|
||||
def test_legacy_source_zero_measurement_is_accepted_but_not_routed(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory_path, route_path, rows = self.make_evidence(root)
|
||||
inventory = compact.route_search.load_inventory(inventory_path)
|
||||
payload = 10
|
||||
container = compact.route_search.container_size(payload)
|
||||
stage_start = compact.route_search.align_down(
|
||||
compact.STAGE_CEILING - container
|
||||
)
|
||||
margin = stage_start - (compact.APP_BASE + compact.FIXED_WORKSPACE)
|
||||
rows.append({
|
||||
"source": 0,
|
||||
"target": 2,
|
||||
"source_sha256": inventory[0]["sha256"],
|
||||
"target_sha256": inventory[2]["sha256"],
|
||||
"memory": compact.FIXED_WORKSPACE,
|
||||
"payload": payload,
|
||||
"container": container,
|
||||
"stage_start": stage_start,
|
||||
"margin": margin,
|
||||
"feasible": margin >= 0,
|
||||
"error": "",
|
||||
})
|
||||
compact.route_search.write_csv(root / "geometry.csv", rows)
|
||||
compact.route_search.select_route(
|
||||
rows, inventory, 100, route_path, complete=True
|
||||
)
|
||||
|
||||
result = compact.validate_geometry_results(
|
||||
root / "geometry.csv", inventory_path, route_path
|
||||
)
|
||||
|
||||
self.assertEqual(result["nodes"], [0, 1, 3])
|
||||
self.assertEqual(result["candidate_geometries"], len(rows))
|
||||
|
||||
def test_geometry_tamper_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, rows = self.make_evidence(root)
|
||||
rows[0]["container"] = 123
|
||||
compact.route_search.write_csv(root / "geometry.csv", rows)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "inconsistent"):
|
||||
compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
|
||||
def test_missing_and_extra_geometry_are_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, rows = self.make_evidence(root)
|
||||
for label, altered, message in (
|
||||
("missing", rows[:-1], "missing"),
|
||||
("extra", rows + [{**rows[0], "memory": int(rows[0]["memory"]) + 1}], "extra"),
|
||||
):
|
||||
with self.subTest(label=label):
|
||||
compact.route_search.write_csv(root / "geometry.csv", altered)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, message):
|
||||
compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
|
||||
def test_route_metric_or_step_mismatch_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, _rows = self.make_evidence(root)
|
||||
document = json.loads(route.read_text(encoding="ascii"))
|
||||
document["selected_total_bytes"] += 1
|
||||
route.write_text(json.dumps(document), encoding="ascii")
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "selected_total_bytes"):
|
||||
compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
|
||||
def test_failed_patch_job_cannot_be_exhaustive_evidence(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, rows = self.make_evidence(root)
|
||||
rows[0].update(
|
||||
payload=-1, container=-1, stage_start=-1, margin=-1,
|
||||
feasible=False, error="MemoryError: test",
|
||||
)
|
||||
compact.route_search.write_csv(root / "geometry.csv", rows)
|
||||
with self.assertRaisesRegex(
|
||||
compact.CompactBuildError, "cannot prove an exhaustive route search"
|
||||
):
|
||||
compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
|
||||
def test_short_geometry_row_fails_cleanly(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory, route, _rows = self.make_evidence(root)
|
||||
(root / "geometry.csv").write_text(
|
||||
",".join(compact.route_search.FIELDS) + "\n1,2\n",
|
||||
encoding="ascii",
|
||||
)
|
||||
with self.assertRaisesRegex(
|
||||
compact.CompactBuildError, "integer|malformed|invalid"
|
||||
):
|
||||
compact.validate_geometry_results(root / "geometry.csv", inventory, route)
|
||||
|
||||
|
||||
class CompactPhysicalValidationTests(unittest.TestCase):
|
||||
def evidence(self) -> tuple[dict[str, object], dict[str, object]]:
|
||||
expected = {
|
||||
"chain_sha256": "1" * 64,
|
||||
"start_sha256": "2" * 64,
|
||||
"endpoint_sha256": "3" * 64,
|
||||
"endpoint_body_hash": "a" * 16,
|
||||
"endpoint_version": "1.17.1.5",
|
||||
"output_rows": [{"target_body_hash": "5" * 16}],
|
||||
"validation_steps": [{
|
||||
"mota_sha256": "6" * 64,
|
||||
"target_sha256": "7" * 64,
|
||||
}],
|
||||
}
|
||||
document: dict[str, object] = {
|
||||
"schema": 1,
|
||||
"kind": compact.PHYSICAL_VALIDATION_KIND,
|
||||
"status": "passed",
|
||||
"chain_sha256": expected["chain_sha256"],
|
||||
"start_sha256": expected["start_sha256"],
|
||||
"endpoint_sha256": expected["endpoint_sha256"],
|
||||
"endpoint_body_hash": expected["endpoint_body_hash"],
|
||||
"endpoint_version": expected["endpoint_version"],
|
||||
"step_count": 1,
|
||||
"steps": [{
|
||||
"step": 1,
|
||||
"status": "passed",
|
||||
"mota_sha256": "6" * 64,
|
||||
"target_sha256": "7" * 64,
|
||||
"target_body_hash": "5" * 16,
|
||||
}],
|
||||
"final_swd": {"status": "passed", "app_sha256": "3" * 64},
|
||||
}
|
||||
return document, expected
|
||||
|
||||
def test_exact_record_is_accepted_and_canonicalized(self) -> None:
|
||||
document, expected = self.evidence()
|
||||
document["endpoint_body_hash"] = str(document["endpoint_body_hash"]).upper()
|
||||
result = compact.validate_physical_validation_record(document, **expected)
|
||||
self.assertEqual(result["status"], "passed")
|
||||
self.assertEqual(result["endpoint_body_hash"], "a" * 16)
|
||||
|
||||
def test_stale_package_or_incomplete_step_list_is_rejected(self) -> None:
|
||||
for label, mutate, message in (
|
||||
(
|
||||
"stale",
|
||||
lambda document: document["steps"][0].__setitem__(
|
||||
"mota_sha256", "8" * 64
|
||||
),
|
||||
"does not match",
|
||||
),
|
||||
(
|
||||
"incomplete",
|
||||
lambda document: document.__setitem__("steps", []),
|
||||
"every chain step",
|
||||
),
|
||||
):
|
||||
with self.subTest(label=label):
|
||||
document, expected = self.evidence()
|
||||
mutate(document)
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, message):
|
||||
compact.validate_physical_validation_record(document, **expected)
|
||||
|
||||
def test_extra_transcript_or_secret_fields_are_rejected(self) -> None:
|
||||
document, expected = self.evidence()
|
||||
document["admin_password"] = "must-not-be-archived"
|
||||
with self.assertRaisesRegex(compact.CompactBuildError, "unexpected"):
|
||||
compact.validate_physical_validation_record(document, **expected)
|
||||
|
||||
|
||||
class CompactReproducibilityTests(unittest.TestCase):
|
||||
def test_zip_metadata_does_not_depend_on_source_mtime(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
archives = []
|
||||
for number in (1, 2):
|
||||
tree = root / f"tree-{number}" / "same-root"
|
||||
tree.mkdir(parents=True)
|
||||
item = tree / "payload.bin"
|
||||
item.write_bytes(b"identical")
|
||||
os.utime(item, (946684800 + number, 946684800 + number))
|
||||
output = root / f"archive-{number}.zip"
|
||||
compact.make_reproducible_zip(tree, output)
|
||||
archives.append(output.read_bytes())
|
||||
self.assertEqual(archives[0], archives[1])
|
||||
|
||||
def test_zip_output_inside_archived_tree_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
tree = Path(directory) / "tree"
|
||||
tree.mkdir()
|
||||
(tree / "payload.bin").write_bytes(b"payload")
|
||||
with self.assertRaisesRegex(
|
||||
compact.CompactBuildError, "outside the archived tree"
|
||||
):
|
||||
compact.make_reproducible_zip(tree, tree / "bundle.zip")
|
||||
|
||||
def test_zip_output_symlinked_inside_archived_tree_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
parent = Path(directory)
|
||||
tree = parent / "tree"
|
||||
tree.mkdir()
|
||||
(tree / "payload.bin").write_bytes(b"payload")
|
||||
alias = parent / "output-alias"
|
||||
alias.symlink_to(tree, target_is_directory=True)
|
||||
with self.assertRaisesRegex(
|
||||
compact.CompactBuildError, "outside the archived tree"
|
||||
):
|
||||
compact.make_reproducible_zip(tree, alias / "bundle.zip")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+1587
-10
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,381 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import csv
|
||||
import hashlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import struct
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
import rak3401_route_search as search
|
||||
|
||||
|
||||
def firmware(body: bytes, version: str) -> bytes:
|
||||
hardware = search.EXPECTED_HARDWARE.encode("ascii").ljust(32, b"\0")
|
||||
return (
|
||||
body
|
||||
+ b"EndF"
|
||||
+ struct.pack("<I", len(body))
|
||||
+ hashlib.sha256(body).digest()[:8]
|
||||
+ struct.pack(
|
||||
"<II", search.motalib.pack_version(version), search.EXPECTED_TARGET_ID
|
||||
)
|
||||
+ hardware
|
||||
)
|
||||
|
||||
|
||||
class RouteSearchTests(unittest.TestCase):
|
||||
def write_inventory(
|
||||
self, root: Path, payloads: list[bytes], *, baseline_size: int = 100,
|
||||
versions: list[str] | None = None,
|
||||
) -> tuple[Path, list[dict[str, object]]]:
|
||||
records = []
|
||||
for node, payload in enumerate(payloads):
|
||||
version = versions[node] if versions is not None else f"1.0.0.{node}"
|
||||
encoded = firmware(payload, version)
|
||||
image = root / f"image-{node:02d}.bin"
|
||||
image.write_bytes(encoded)
|
||||
record: dict[str, object] = {
|
||||
"node": node,
|
||||
"path": image.name,
|
||||
"size": len(encoded),
|
||||
"sha256": hashlib.sha256(encoded).hexdigest(),
|
||||
"body_hash": hashlib.sha256(payload).digest()[:8].hex(),
|
||||
"version": version,
|
||||
}
|
||||
if node == 1:
|
||||
record["baseline_container_size"] = baseline_size
|
||||
records.append(record)
|
||||
manifest = root / "images.json"
|
||||
manifest.write_text(json.dumps({"images": records}), encoding="ascii")
|
||||
return manifest, records
|
||||
|
||||
def test_cache_migration_survives_inserted_node_and_remaps_indices(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
old_dir, new_dir = root / "old", root / "new"
|
||||
old_dir.mkdir()
|
||||
new_dir.mkdir()
|
||||
old_manifest, old = self.write_inventory(
|
||||
old_dir, [b"A", b"B", b"C"],
|
||||
versions=["1.0.0.0", "1.0.0.1", "1.0.0.2"],
|
||||
)
|
||||
cache = old_dir / "geometry.csv"
|
||||
with cache.open("w", newline="", encoding="ascii") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=search.FIELDS)
|
||||
writer.writeheader()
|
||||
writer.writerow({
|
||||
"source": 1, "target": 2, "memory": search.FIXED_MEMORY,
|
||||
"payload": 7,
|
||||
"container": search.container_size(7),
|
||||
"stage_start": search.align_down(
|
||||
search.STAGE_CEILING - search.container_size(7)
|
||||
),
|
||||
"margin": search.align_down(
|
||||
search.STAGE_CEILING - search.container_size(7)
|
||||
) - (search.APP_BASE + search.FIXED_MEMORY),
|
||||
"feasible": True, "error": "",
|
||||
})
|
||||
new_manifest, _ = self.write_inventory(
|
||||
new_dir, [b"A", b"X", b"B", b"C"],
|
||||
versions=["1.0.0.0", "1.0.0.99", "1.0.0.1", "1.0.0.2"],
|
||||
)
|
||||
migrated = search.migrate_csv(old_manifest, cache)
|
||||
key = search.cache_key(str(old[1]["sha256"]), str(old[2]["sha256"]),
|
||||
search.FIXED_MEMORY)
|
||||
self.assertIn(key, migrated)
|
||||
projected = search.project_cache(
|
||||
migrated, search.load_inventory(new_manifest), {key}
|
||||
)
|
||||
self.assertEqual((projected[0]["source"], projected[0]["target"]), (2, 3))
|
||||
|
||||
def test_all_jobs_covers_every_valid_page_workspace(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
manifest, _ = self.write_inventory(
|
||||
Path(directory), [b"a", b"b", b"c" * 10000, b"d" * 12000]
|
||||
)
|
||||
jobs = search.all_jobs(search.load_inventory(manifest))
|
||||
fixed = [job for job in jobs if job[0] < 2]
|
||||
dynamic = [job for job in jobs if job[0] >= 2]
|
||||
self.assertEqual(len(fixed), 2)
|
||||
self.assertTrue(all(job[2] == search.FIXED_MEMORY for job in fixed))
|
||||
self.assertTrue(all(job[0] == 1 for job in fixed))
|
||||
inventory = search.load_inventory(manifest)
|
||||
first_page = search.align_up(
|
||||
max(
|
||||
int(inventory[2]["size"]) + 2 * search.PAGE,
|
||||
int(inventory[3]["size"]),
|
||||
)
|
||||
) // search.PAGE
|
||||
self.assertEqual(
|
||||
[job[2] for job in dynamic],
|
||||
[page * search.PAGE for page in range(first_page, search.AVAILABLE_PAGES)],
|
||||
)
|
||||
self.assertEqual(len(jobs), 2 + search.AVAILABLE_PAGES - first_page)
|
||||
|
||||
def test_route_ignores_geometry_that_bypasses_pinned_first_bridge(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
manifest, _ = self.write_inventory(root, [b"0", b"1", b"2"])
|
||||
images = search.load_inventory(manifest)
|
||||
result = search.select_route(
|
||||
[
|
||||
{
|
||||
"source": 0, "target": 2, "memory": search.FIXED_MEMORY,
|
||||
"container": 1, "stage_start": 0xC0000,
|
||||
"margin": 10, "feasible": True,
|
||||
},
|
||||
{
|
||||
"source": 1, "target": 2, "memory": search.FIXED_MEMORY,
|
||||
"container": 10, "stage_start": 0xC0000,
|
||||
"margin": 10, "feasible": True,
|
||||
},
|
||||
],
|
||||
images, 100, root / "route.json", True,
|
||||
)
|
||||
self.assertEqual(result["nodes"], [0, 1, 2])
|
||||
|
||||
def test_cache_migration_rejects_inconsistent_numeric_geometry(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
manifest, _ = self.write_inventory(root, [b"A", b"B", b"C"])
|
||||
cache = root / "geometry.csv"
|
||||
with cache.open("w", newline="", encoding="ascii") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=search.FIELDS)
|
||||
writer.writeheader()
|
||||
writer.writerow({
|
||||
"source": 1, "target": 2, "memory": search.FIXED_MEMORY,
|
||||
"payload": 7, "container": 999, "stage_start": 800000,
|
||||
"margin": 1, "feasible": True, "error": "",
|
||||
})
|
||||
with self.assertRaisesRegex(
|
||||
search.RouteSearchError, "inconsistent cached geometry"
|
||||
):
|
||||
search.migrate_csv(manifest, cache)
|
||||
|
||||
def test_cache_migration_rejects_failed_geometry(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
manifest, _ = self.write_inventory(root, [b"A", b"B", b"C"])
|
||||
cache = root / "geometry.csv"
|
||||
with cache.open("w", newline="", encoding="ascii") as output:
|
||||
writer = csv.DictWriter(output, fieldnames=search.FIELDS)
|
||||
writer.writeheader()
|
||||
writer.writerow({
|
||||
"source": 1, "target": 2, "memory": search.FIXED_MEMORY,
|
||||
"payload": -1, "container": -1, "stage_start": -1,
|
||||
"margin": -1, "feasible": False, "error": "out of memory",
|
||||
})
|
||||
with self.assertRaisesRegex(
|
||||
search.RouteSearchError, "cannot prove an exhaustive search"
|
||||
):
|
||||
search.migrate_csv(manifest, cache)
|
||||
|
||||
def test_geometry_tool_error_aborts_instead_of_becoming_infeasible(self) -> None:
|
||||
class BrokenDetools:
|
||||
@staticmethod
|
||||
def create_patch_filenames(*_args: object, **_kwargs: object) -> None:
|
||||
raise OSError("disk full")
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory, mock.patch.dict(
|
||||
sys.modules, {"detools": BrokenDetools}
|
||||
):
|
||||
with self.assertRaisesRegex(search.RouteSearchError, "disk full"):
|
||||
search.geometry_job((
|
||||
1, 2, search.FIXED_MEMORY,
|
||||
str(Path(directory) / "from.bin"),
|
||||
str(Path(directory) / "to.bin"),
|
||||
"1" * 64, "2" * 64, directory,
|
||||
))
|
||||
|
||||
def test_snapshot_inventory_is_immune_to_later_source_mutation(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
source = root / "source"
|
||||
source.mkdir()
|
||||
manifest, _ = self.write_inventory(source, [b"A", b"B", b"C"])
|
||||
frozen = search.snapshot_inventory(
|
||||
manifest, root / "frozen" / "images.json"
|
||||
)
|
||||
source_image = source / "image-02.bin"
|
||||
source_image.write_bytes(b"changed")
|
||||
images = search.load_inventory(frozen)
|
||||
self.assertNotEqual(Path(images[2]["path"]).read_bytes(), b"changed")
|
||||
|
||||
def test_generation_never_removes_a_preexisting_patches_directory(self) -> None:
|
||||
class ImmediateFuture:
|
||||
def __init__(self, value: dict[str, object]):
|
||||
self._value = value
|
||||
|
||||
def result(self) -> dict[str, object]:
|
||||
return self._value
|
||||
|
||||
class ImmediatePool:
|
||||
def __init__(self, **_kwargs: object):
|
||||
pass
|
||||
|
||||
def __enter__(self) -> "ImmediatePool":
|
||||
return self
|
||||
|
||||
def __exit__(self, *_args: object) -> None:
|
||||
pass
|
||||
|
||||
def submit(self, function: object, *args: object) -> ImmediateFuture:
|
||||
return ImmediateFuture(function(*args)) # type: ignore[operator]
|
||||
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
inventory_root = root / "inventory"
|
||||
inventory_root.mkdir()
|
||||
manifest, records = self.write_inventory(
|
||||
inventory_root, [b"start", b"bridge", b"endpoint"]
|
||||
)
|
||||
work = root / "work"
|
||||
existing = work / "patches"
|
||||
existing.mkdir(parents=True)
|
||||
sentinel = existing / "keep.txt"
|
||||
sentinel.write_text("caller-owned", encoding="ascii")
|
||||
source_sha = str(records[1]["sha256"])
|
||||
target_sha = str(records[2]["sha256"])
|
||||
job = (
|
||||
1, 2, search.FIXED_MEMORY,
|
||||
str(inventory_root / str(records[1]["path"])),
|
||||
str(inventory_root / str(records[2]["path"])),
|
||||
source_sha, target_sha,
|
||||
)
|
||||
|
||||
def measured(args: tuple[object, ...]) -> dict[str, object]:
|
||||
source, target, memory, *_rest = args
|
||||
payload = 7
|
||||
total = search.container_size(payload)
|
||||
stage = search.align_down(search.STAGE_CEILING - total)
|
||||
return {
|
||||
"source": source, "target": target,
|
||||
"source_sha256": source_sha,
|
||||
"target_sha256": target_sha,
|
||||
"memory": memory, "payload": payload,
|
||||
"container": total, "stage_start": stage,
|
||||
"margin": stage - (search.APP_BASE + int(memory)),
|
||||
"feasible": True, "error": "",
|
||||
}
|
||||
|
||||
with (
|
||||
mock.patch.object(search, "all_jobs", return_value=[job]),
|
||||
mock.patch.object(search, "geometry_job", side_effect=measured),
|
||||
mock.patch.object(search, "ProcessPoolExecutor", ImmediatePool),
|
||||
mock.patch.object(search, "as_completed", side_effect=lambda values: values),
|
||||
):
|
||||
self.assertEqual(
|
||||
search.main([
|
||||
"--inventory", str(manifest),
|
||||
"--work-dir", str(work),
|
||||
"--workers", "1",
|
||||
]),
|
||||
0,
|
||||
)
|
||||
self.assertEqual(sentinel.read_text(encoding="ascii"), "caller-owned")
|
||||
|
||||
def test_route_minimizes_hops_then_bytes(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
manifest, _ = self.write_inventory(root, [b"0", b"1", b"2", b"3", b"4"])
|
||||
images = search.load_inventory(manifest)
|
||||
|
||||
def edge(source: int, target: int, size: int) -> dict[str, object]:
|
||||
return {
|
||||
"source": source, "target": target, "memory": 0x1000,
|
||||
"container": size, "stage_start": 0xC0000,
|
||||
"margin": 100, "feasible": True,
|
||||
}
|
||||
|
||||
# Both endpoint routes have three hops including the pinned 0->1.
|
||||
# The path through node 3 is smaller and must win even though it is
|
||||
# encountered after the path through node 2.
|
||||
rows = [edge(1, 2, 100), edge(2, 4, 100),
|
||||
edge(1, 3, 40), edge(3, 4, 50)]
|
||||
result = search.select_route(rows, images, 100, root / "route.json", True)
|
||||
self.assertEqual(result["nodes"], [0, 1, 3, 4])
|
||||
self.assertEqual(result["shortest_package_count"], 3)
|
||||
self.assertEqual(result["shortest_route_count"], 2)
|
||||
self.assertEqual(result["selected_total_bytes"], 190)
|
||||
|
||||
# A direct 1->4 edge wins on hop count even when it costs more.
|
||||
rows.append(edge(1, 4, 1000))
|
||||
result = search.select_route(rows, images, 100, root / "route2.json", True)
|
||||
self.assertEqual(result["nodes"], [0, 1, 4])
|
||||
self.assertEqual(result["selected_total_bytes"], 1100)
|
||||
|
||||
def test_fixed_outputs_never_follow_preexisting_symlinks(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
victim = root / "victim.txt"
|
||||
victim.write_text("caller-owned", encoding="ascii")
|
||||
geometry = root / "geometry.csv"
|
||||
geometry.symlink_to(victim)
|
||||
with self.assertRaisesRegex(search.RouteSearchError, "regular file"):
|
||||
search.write_csv(geometry, [])
|
||||
self.assertEqual(victim.read_text(encoding="ascii"), "caller-owned")
|
||||
|
||||
manifest, _ = self.write_inventory(
|
||||
root, [b"start", b"bridge", b"endpoint"]
|
||||
)
|
||||
route = root / "route.json"
|
||||
route.symlink_to(victim)
|
||||
with self.assertRaisesRegex(search.RouteSearchError, "regular file"):
|
||||
search.select_route(
|
||||
[], search.load_inventory(manifest), 100, route, True
|
||||
)
|
||||
self.assertEqual(victim.read_text(encoding="ascii"), "caller-owned")
|
||||
|
||||
def test_invalid_inventory_is_rejected(self) -> None:
|
||||
with tempfile.TemporaryDirectory() as directory:
|
||||
root = Path(directory)
|
||||
manifest, records = self.write_inventory(root, [b"one", b"two"])
|
||||
cases = []
|
||||
bad = json.loads(manifest.read_text())
|
||||
bad["images"][1]["node"] = 3
|
||||
cases.append((bad, "contiguous"))
|
||||
bad = json.loads(manifest.read_text())
|
||||
bad["images"][1]["sha256"] = bad["images"][0]["sha256"]
|
||||
cases.append((bad, "SHA mismatch"))
|
||||
bad = json.loads(manifest.read_text())
|
||||
bad["images"][1]["path"] = "missing.bin"
|
||||
cases.append((bad, "missing inventory image"))
|
||||
for number, (document, message) in enumerate(cases):
|
||||
candidate = root / f"bad-{number}.json"
|
||||
candidate.write_text(json.dumps(document), encoding="ascii")
|
||||
with self.subTest(message=message), self.assertRaisesRegex(
|
||||
search.RouteSearchError, message
|
||||
):
|
||||
search.load_inventory(candidate)
|
||||
|
||||
def test_bridges_are_inserted_before_preserved_endpoint(self) -> None:
|
||||
original = {
|
||||
"schema": 1,
|
||||
"images": [
|
||||
{"node": 0, "kind": "start"},
|
||||
{"node": 1, "kind": "old-bridge"},
|
||||
{"node": 2, "kind": "endpoint", "path": "endpoint.bin"},
|
||||
],
|
||||
}
|
||||
result = search.insert_bridges_before_endpoint(
|
||||
original,
|
||||
[{"node": 99, "kind": "new-bridge-1"}, {"kind": "new-bridge-2"}],
|
||||
)
|
||||
self.assertEqual(
|
||||
[record["node"] for record in result["images"]], [0, 1, 2, 3, 4]
|
||||
)
|
||||
self.assertEqual(
|
||||
[record["kind"] for record in result["images"]],
|
||||
["start", "old-bridge", "new-bridge-1", "new-bridge-2", "endpoint"],
|
||||
)
|
||||
self.assertEqual(result["images"][-1]["path"], "endpoint.bin")
|
||||
self.assertEqual(len(original["images"]), 3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user