The manifest said "rebuild these for every espressif32 platform bump" and nothing
enforced it. mbedtls_4k.py verified the staged archives against the manifest's own
hashes, which proves the pair agrees with itself and nothing more: bump the
platform without rebuilding and every check still passes while the link takes
mbedTLS built against a different IDF. That fails at runtime on struct-layout
drift, not at the link, which is the failure the mechanism claimed to prevent.
Fingerprint the framework's own mbedTLS archives — the ones ours displace — as
stock: lines in the manifest and check them before the build. If the framework
moves, the staged pair is stale by construction and the build stops with the
replacement hashes printed ready to paste. Stronger than comparing a version
string: framework-arduinoespressif32 versions independently of the platform, and
its archives are what actually has to match.
The lib directory is resolved by trying the layouts espressif32 has used rather
than hardcoding one, and failing closed if none holds all four archives. The fetch
script ignores the new lines; its known-arches hint skips them so they cannot be
reported as architectures.
Three defects found reviewing the preceding commits.
1. reconnectSlotClient() stranded a STOPPED client, reintroducing the very bug
this branch fixes. It only rebuilt when isStarted() was true and otherwise
fell through to reconnect(), which is a documented no-op on a stopped client
— so nothing restarted it, at any rung, including the breaker probe. The
WiFi-transition teardown reaches exactly this state: it calls the hard
disconnect(), clearing _started while initial_connect_done stays set, so
after WiFi returned the slot could never come back. Now a stopped client is
started with connect() before the rebuild/reuse decision is considered.
The post-NTP credential refresh had the same exposure — it called
reconnect() directly — so it now goes through the helper too, still reusing
the transport since its fault is stale credentials, not the transport.
2. Allocating the neighbors buffer on first use let a stopped bridge allocate.
A neighbour discovery started before a stop can complete after it, and
neither caller rechecks bridge state, so requestPublishNeighbors() would
allocate 4 KB after releaseRuntimeBuffers() had already run and strand
_neighbors_publish_pending with no task to consume it. end() then returns
early on !_initialized, retaining the buffer until a later begin/end or a
reboot. Guarded on isRunning(), the same flag end() checks.
The release/acquire handoff itself was confirmed sound: the allocation and
copy precede the release store, and the task loop reads the pointer only
after its acquire load, so a half-published pointer is not observable.
3. The post-link map check failed open, contradicting the fail-closed claim in
its own commit message. A missing map, an unrecognised map format, or a
partial archive list each warned and passed; and it hardcoded firmware.map
while the post-action target used ${PROGNAME}, so a renamed program could
inspect a stale or absent file and still succeed. All four now fail the
build, and it requires every one of the four archives to appear rather than
at least one.
Rebuilt Heltec_v3_repeater_observer_mqtt, Heltec_v3_repeater and
heltec_v4_repeater_observer_mqtt; the opt-in path still reports all 4 archives
linked from .mbedtls-4k/.
(cherry picked from commit 5b5f076e5e165997e8050f2be061c7c67340fcf7)
The reduced-TLS work was validated on hardware but only reachable through
PLATFORMIO_BUILD_FLAGS pointing at an absolute path in a developer's home
directory, so nothing outside that machine could reproduce it.
Distribute the archives as a release asset instead of committing them: ~6 MB
per architecture, and they must be rebuilt for every espressif32 bump, so
committing would grow history permanently and go stale without any signal.
scripts/mbedtls_4k_manifest.txt per-arch sha256 of each archive
scripts/fetch_mbedtls_4k.sh fetch into .mbedtls-4k/<arch>/, verify
scripts/mbedtls_4k.py pre-build wiring and post-link proof
Off by default. The script is attached to esp32_base but returns immediately
unless MESHCORE_REDUCED_TLS=1, so ordinary builds need no artifact and are
byte-for-byte unaffected — confirmed by building with it absent.
Both ways this can fail silently produce a firmware that looks fine and lacks
the change, so the opt-in path refuses to guess:
- a -L at a missing or partial directory: the linker ignores an unusable
search path and resolves mbedTLS from the framework. Now a hard error.
- archives left over from an earlier platform version: now a sha256
mismatch against the manifest, naming both hashes.
- a -L that is present but outranked, leaving the flag inert: after the
link, firmware.map must resolve every libmbed*.a into .mbedtls-4k/, or
the build fails and prints the offending paths.
That last check earned its place immediately — it caught its own first
implementation comparing a relative map path against an absolute one, and an
earlier build flag in this investigation was accepted by the compiler while
no source read it. A flag reaching the compiler proves nothing about the link.
Verified all four paths on Heltec_v3_repeater_observer_mqtt: default build
unaffected; opted in with archives present links all four from .mbedtls-4k/
and says so; archives absent fails with a fetch hint; a single appended byte
fails on sha256.
Also removes platformio.local.ini.hold, which held the superseded approach of
pointing platform_packages at a whole custom framework. That installs over the
shared framework package and changes mbedTLS for every other ESP32 project on
the machine; the -L path keeps the change scoped to one env.
Note the inbound record buffer stays at 16 KiB, so this lowers per-connection
footprint by ~12 KiB but does not move the contiguous allocation a handshake
needs. It buys headroom, not a lower floor.
(cherry picked from commit a87faff6ff170c328fdd0550f4b4dd9089aa2ea0)
Two findings from review, both real, both mine.
The CLI could read secrets the portal has never exposed. CommonCLI splits its
surface by CALLER, not by command: a serial caller (sender_timestamp 0, physical
access) reads secrets in plaintext, a remote one gets "******** (serial only)".
Its own comments say so — "Serial only (WiFi creds grant LAN access); remote
sees set/unset". execCommand passes 0, which is what makes `erase`, `stats-*`
and `set freq` reachable at all, and with it the terminal inherited the serial
console's plaintext answers for an HTTP request: `get prv.key` returned this
node's identity, `get wifi.pwd` the operator's network.
Worse in setup mode, which authenticates by proximity to an open AP — and `start
webconfig ap` can be run on an already-configured node, so the secrets are real
by then, not blank.
I had reasoned that the AP was the trust boundary either way because the wizard
can already rewrite these. That conflated two capabilities: replacing a WiFi
password does not reveal the current one, and replacing an identity does not
reveal the existing private key. /api/config has always masked these on read
(wcIsSecretKey); the CLI simply broke that rule. Now only the READ is masked —
the command surface stays whole — in CommonCLI's own words, keeping the
set/unset signal that is the useful part.
Onboarding could also skip the mandatory password. handleConfigPost refuses to
arm a reboot during initial setup without one; the CLI only warned in the
browser, which a pasted script or a direct POST ignores, so a node could reboot
onto the LAN still holding the factory credential. Same rule now applies at
POST. It is satisfied by a `password` command anywhere in the session rather
than only in the same request, so the natural two-step console flow still works
— the form batch always sends both together and never needed that memory.
wcIsSecretReadCommand lives in WebConfigKeys.h beside the rest of the secret
classification, pinned by three host tests: what must be masked, what must not,
and that only reads are touched. 17 keys + 24 batch tests pass; the audit checks
a masked read round-trips as masked.
The full embedded string is v1.16.0.5-observer-beta-dev-a1b2c3d. The -observer
tag is on every observer build and the commit is for machines, so neither tells
a person anything at a glance. Both the page header and the console banner now
show base + published build number + channel, paired with the build date the
way `ver` pairs them:
v1.16.0.5-dev (6 Jun 2026)
v1.16.0.5 (6 Jun 2026)
`ver` still prints the whole string, commit included, for when that is what you
need. The channel suffix follows the release filenames rather than the embedded
tag — build.sh writes FILENAME_CHANNEL_TAG "-dev" for the same builds it tags
"-observer-beta-dev" internally, so "-dev" is the name these already carry.
Carrying the build date meant /api/status had to report it; WebConfigServer now
takes FIRMWARE_BUILD_DATE alongside FIRMWARE_VERSION, from the same defines
`ver` reads.
A local build has neither build number nor channel to show, so the fact worth
knowing about it moves to the second line: "local build, OTA not configured".
build.sh deliberately leaves OTA_MANIFEST_BASE undefined there, and a bare
version number gives no hint that the node cannot update itself.
The console showed "v1.16.0" — the version was there but buried in the header
line beside role and board, and on the build under test it genuinely had no more
to show: `pio run` never goes through build.sh, so no build number, no commit,
no OTA config.
The banner now prints FIRMWARE_VERSION whole and on its own line. Nothing was
truncating it; build.sh composes base[.build][-observer][-channel]-hash, so a CI
build already carries the published build number as a 4th component and the
commit as the trailing token — the two things that actually identify a build.
It also names the channel, which the version string encodes but does not spell
out (OTA_CHANNEL_TAG=beta-dev -> "-observer-beta-dev-"):
v1.16.0.5-observer-beta-dev-a1b2c3d (dev channel)
v1.16.0.5-observer-beta-a1b2c3d (beta channel)
v1.16.0.5-observer-a1b2c3d (release channel)
v1.16.0 (local build — not from CI, OTA not
configured)
That last one earns its wording: build.sh deliberately leaves OTA_MANIFEST_BASE
undefined on local builds so such a node cannot update itself, and nothing about
a bare version number says so.
The mock reports a build.sh-shaped version now (--fw-version switches channel),
and `ver` answers from the same string /api/status does, as both do on-device.
Five findings from review, all confirmed against the source.
Failure classification (P2). Testing replies for an "Err" prefix passed five
other shapes off as success: "Unknown command", "unknown config: x", "??: x",
"Can't find GPS", "(ERR: clock cannot go backwards)" and "File system erase:
Err". They rendered green, and worse, left _batch_all_ok true — so a queued
reboot went ahead after commands that had failed, defeating the gate entirely.
Rather than lengthen one guess, the two questions are now asked separately,
each erring safe:
- colour asks "does this look like a failure", against every shape CommonCLI
actually emits, enumerated in WebConfigBatch.h and pinned by a host test
that uses the literal strings. Getting this wrong is cosmetic.
- the reboot gate asks something narrower and answerable: "did every setting
I asked for take". Only `set`/`password` gate it, and only on the "OK"
prefix every setter keeps. Diagnostics no longer gate a reboot at all, so a
harmless `memory` cannot strand one and no guess is made about "> value".
Reboot deferral (P2). CommonCLI dispatches on a six-byte prefix, so `reboot
now` and `rebooted` reach Board::reboot() too. Matching exactly meant those
variants skipped both the confirmation and the deferral and took the node down
mid-drain — the precise failure deferral exists to prevent. Both sides now
anchor the way the firmware dispatches, and the UI's risk matcher with them.
Three commands the portal cannot honestly serve are refused at POST with a
reason, and dropped from autocomplete, instead of running and lying:
- `start ota` builds a second AsyncWebServer on port 80 with no bind check
and answers "Started" regardless; the portal already holds that port, so it
could only leak the allocation and inhibit sleep.
- `clock sync` takes its time from the caller's timestamp, which a web
request has none of, so CommonCLI always rejected it. `time <epoch>` works
and remains offered.
- bare `log` and `get acl` write their real output to Serial and hand back a
stub the terminal showed as success; `log` also streams a whole file from
the loop task, stalling the mesh and radio while it does.
The mock now emits the same failure shapes it used to fake as successes, so
these are reproducible off-hardware. 24 batch + 14 keys tests pass; audit
reports 119/119 answered, 0 missing, 4/4 refused with a reason.
Two things hardware turned up.
The whole terminal came back red. The endpoint decided a command had succeeded
by testing its reply for an "OK" prefix — the convention the config batch relies
on, and a safe one there because every allowlisted setter uses it. The CLI
reaches the whole surface, where success has no single shape: setters answer
"OK...", getters answer "> value", `erase` answers "File system erase: OK". Only
failure is uniform ("Err", "ERR:", "Error:"), so that is what the CLI now tests
for.
Colour was the visible half. The other half was worse: _batch_all_ok went false
the moment a sequence contained a `get`, so a script ending in `reboot` was told
some commands had failed and the reboot was withheld.
Replies are green now and red means the node said no, which is what red should
have meant all along. The "> " a getter prefixes its value with is dropped for
display — on the serial console it sets the value apart, here it collides with
the prompt glyph that means "you typed this". The mock emits that marker too;
had it done so from the start, this would have shown up before the flash.
Second: discover.neighbors and discover.scopes did not autocomplete, because
MyMesh::handleCommand intercepts a few commands before delegating to CommonCLI
and the table was built by reading CommonCLI alone. setperm, `get acl` and
`shutdown` were missing for the same reason.
The audit could not have caught that: it drove every command the table offered
and checked the mock answered, which only finds gaps in one direction. It now
also reads the command literals the firmware dispatches on — across CommonCLI,
CommonCLI_Observer and MyMesh — and fails on any the table does not offer. That
check found `shutdown` immediately.
122 commands, all answered, none missing. 22 batch + 14 keys tests pass.
The terminal has been driving the mock since it was built. This is the firmware
side, so it works on hardware.
Same 202 + reqid + poll contract as a config save, for the same reason:
CommonCLI touches prefs, the radio and the filesystem, none of which may be
reached from the async_tcp task. Commands go into the deferred slot and tick()
drains them on the loop task. Unlike a save this is not allowlisted — reaching
what the serial console reaches is the point, and execCommand() already passes
sender_timestamp 0, so the terminal gets exactly the serial console's
privilege. Authentication is the boundary, as it is there.
The CLI shares the config batch's slot rather than owning a second MAX_BATCH
array: both drain on the loop task, both are single-slot, and a duplicate would
cost ~8 KB of permanently resident RAM. Sharing also makes a save and a CLI run
mutually exclusive, which they must be. Each reader checks the kind, so neither
can serve the other's results.
Three things the mock could not have taught us:
- Board::reboot() does not return, so a drained `reboot` would take the node
down before the client read a single result. It is answered rather than
executed, and the batch arms the existing deferred-reboot path once the
results have been read — withheld if any command failed, exactly as a save
withholds one. clkreboot/poweroff/ota update do real work on the way down
and cannot be faked, so they still drop the connection; the UI warns first.
- `password <new>` echoes the new password in its reply. The config path
already scrubbed that by key; a CLI entry has no key, so it is matched on
the command. CLI commands are also kept out of the serial log entirely —
the browser session and the serial console are different audiences.
- MAX_BATCH is 24, not the 64 the page assumed. It is reported as
status.max_cmds instead of hardcoded, so the cap cannot drift.
Results stream and page (kCliResultPage = 8), and "done" means the client has
been handed every result, not merely that execution finished — otherwise a
client that stops polling at "done" loses the last page. Commands are never
echoed back: they may carry a secret, and the client matches by index.
New decisions live in WebConfigBatch.h with the rest, covered by three host
tests. Builds clean for heltec_v4_repeater_observer_mqtt; 22 batch + 14 keys
tests pass; the CLI audit reports 119/119 against the updated mock.
`get radio.fem.rxgain` returned "unknown config key" from the mock, which reads
as the terminal offering a command that does not exist. It does exist: CommonCLI
implements get and set for it, gated at runtime by Board::canControlLoRaFemLna()
rather than compiled out, so the command is present in every build and the board
answers for itself — "Error: unsupported" where there is no front-end module.
Auditing the whole table found 31 of 70 config keys unanswered, all the ones no
portal form drives: alert.*, bridge.*, owner.info, path.hash.mode, dutycycle and
the rest. Plus 14 verbs (gps, powersaving, sensor, region, clock sync) with no
handler at all. They now live in a "cli" section of the mock config, typed
through the existing lookup tables and stripped from /api/config, which does not
carry them.
Two real bugs behind that:
- the `set` path gated on whether a key was *readable*, so write-only and
computed keys (prv.key, dutycycle, radio.fem.rxgain) were rejected as
unknown. apply_set now owns that decision alone.
- apply_set accepted anything it did not recognise and replied OK. That
leniency is what let the gap hide: a CLI `set` on an unknown key looked
like it worked. It is strict now — verified against every key in
WC_ALLOWED_SET_KEYS so the form batch is unaffected.
Also mqtt.neighbors / mqtt.neighbors.interval, which the MQTT tab binds but the
mock's config never carried, so that toggle could not round-trip.
webconfig_cli_audit.py keeps the two honest: it drives every command the
autocomplete table offers through /api/cli and fails on anything unanswered.
119 commands, all answered.
The generator gzipped webui/index.html verbatim, so the page's comments — and
this file is commented heavily by house style — were paying flash rent. A
line-based pass now drops comments, indentation and blank lines before
compressing. The source stays as readable as it was.
Conservative on purpose: only a comment that starts its own line is removed, so
a `//` inside a URL or a `/*` inside a regex can never be mistaken for one.
Line breaks survive, which leaves JS statement boundaries (and the space a
newline contributes between HTML inline elements) exactly as written.
This ships to thousands of devices, so it is not taken on trust:
- check_stripped() fails the build if the page's structure changed or the
output shrank implausibly
- the pass lives in its own module, shared with the mock backend's new
--minify flag, so the bytes exercised in a browser are the bytes that get
embedded rather than a second implementation that could drift
- webconfig_minify.py joins the generator in the freshness hash, so editing
the stripper forces a regenerate
Today's page: 22,678 -> 17,671 bytes gzipped.
Design prototype, driven entirely by the mock backend — nothing here runs
on-device yet.
The portal's form batch is deliberately allowlisted (WebConfigKeys.h), which
leaves everything the serial console can do unreachable from a browser. This
adds a fifth tab holding a real terminal: monospace white-on-black in either
colour scheme, autocomplete over the full ~270-command surface, in-session
history, and a confirmation step for pasted command sequences.
Autocomplete goes past the flasher's <datalist>: rows carry descriptions, Tab
extends to the longest shared prefix before committing to a match, and once
`set <key> ` is complete it switches to completing the VALUE — enums from the
command table, broker presets from /api/presets, packet-type names per CSV
segment. The table is generated from a key list rather than written out per
slot, so mqttN.* tracks active_slots instead of being duplicated six times.
Pasting several lines never mangles the prompt: the lines are parsed (comments,
blank lines and pasted `>` prompts stripped), listed back numbered, and run only
after an explicit confirm. Commands that restart, erase, reflash or move the
node off its network get the same confirmation singly. History is memory-only —
`set wifi.pwd` and `password` pass through it.
/api/cli mirrors the config-save contract (202 + reqid, poll for results) for
the same reason: commands run on the node's main loop, not in the request. The
one difference is that results stream, so a long sequence fills the window as
it executes rather than landing all at once.
The tab is hidden in setup mode, where the portal authenticates by proximity
and no admin password exists yet.
Add support for named packet types in per-slot filters, allowing users
to specify packet types using descriptive names alongside numeric values.
This improves usability and clarity in configuring MQTT slot filters.
Updates include modifications to the parsing logic, WebConfig interface,
and related documentation to reflect the new naming conventions.
Introduce per-slot packet filters to allow users to specify which
packet types are uploaded for each MQTT slot. This feature enhances
the flexibility of the MQTT bridge by enabling users to configure
allowlists for packet types, improving the efficiency of data
transmissions. The implementation includes updates to the WebConfig
interface, internal handling of packet filters, and necessary
modifications to the MQTT preferences structure.
Adds an admin-password field to the setup wizard and the LAN editor, so a
node's password can be set during onboarding and rotated later without a
serial console.
The key maps to the top-level `password` CLI command rather than a `set`
handler, so it is classified separately from WC_ALLOWED_SET_KEYS. It is the
only key granted that treatment, which keeps the allowlist the sole route to
`set` and leaves no general path from a batch to arbitrary CLI commands.
Accepted in both modes: MODE_OFF is refused earlier in handleConfigPost, LAN
required a login to get that far, and the setup AP implies physical proximity.
Restricting rotation to the AP would have forced a bridge outage (`set bridge
off` + `start webconfig ap`) just to change a password.
First onboarding is gated server-side: while the setup AP is up and no WiFi is
configured, a batch that reboots or sets wifi.ssid must also carry a password,
so neither the Advanced editor nor a crafted request can save WiFi and strand
the node on the factory password. The flag is latched at AP start, so a save
that fails partway cannot drop the requirement on retry.
The CLI's `password` command echoes the new secret back in its reply, and
replies are served to the client over the open setup AP, so the reply is
overwritten with "OK" before it can be serialized.
UI: the field lives with the other NodePrefs settings (wizard step 2, and the
Node card on the Radio tab) rather than beside the WiFi password, which is a
different credential. Confirm fields mirror their password twin and are cleared
whenever it is, so a stale confirm value cannot fail a later save as a spurious
mismatch. Validation runs ahead of the WiFi-changed split so a password-only
save is still checked, and reveals the Radio tab before reporting, since the
save bar spans every tab.
Add validation for request IDs in the web configuration server to ensure
they conform to the expected format. Enhance error responses for invalid
or unknown request IDs, improving the robustness of request handling.
Update the web UI to reflect these changes, ensuring that clients can
properly handle errors related to request ID mismatches.
Include detailed instructions for local testing of observer and WiFi
functionality without hardware. Document the use of a mock backend and
Wokwi ESP32-S3 simulation for easier development and testing.
Enhance the MQTT implementation documentation to improve developer
experience and facilitate testing workflows.
Introduce a web configuration portal for easier node management and
provisioning without serial CLI. Enhance MQTT functionality with
improved IATA code validation, dynamic slot management, and
background NTP synchronization. Update web UI elements for better
user experience and security notes regarding open AP usage.
Implemented functionality to generate and compare partition-table
signatures during OTA updates. This enhancement ensures that the
target build's partition layout matches the device's actual layout,
improving the reliability of OTA updates and preventing issues
related to partition changes.
- Switch from PubSub to PsychicMqttClient for async operations and websockets support
- Add support for US and EU Let's Mesh Analyzer servers with JWT authentication.
- Introduce CLI commands to enable/disable analyzer servers.
- Update NodePrefs to store analyzer server settings.
- Modify MQTTBridge to publish status and packet data to analyzer servers via WebSocket MQTT.
- Enhance documentation to reflect new features and configuration options.
agessaman