Add a new classification mechanism for MQTT slot activation to inform
users whether a slot will connect on the current hardware. This includes
detailed handling of enabled slots and their limits, improving the CLI's
ability to provide accurate feedback during configuration. Additionally,
unit tests are introduced to validate the new functionality across various
scenarios, enhancing overall robustness.
Replace multiple snprintf calls with a single replyAppendf function to
improve buffer management and prevent overflow. This change enhances
the readability and maintainability of the code while ensuring that
the reply buffer is handled safely across various MQTT status and
diagnostic replies.
Update commit logic to remove existing destination before renaming
the temporary file on ESP32. This change addresses the SPIFFS_ERR_CONFLICTING_NAME
issue, ensuring that the new preferences are published correctly.
Add tests to validate the new behavior and confirm that the
temporary file is handled appropriately.
- Allow mqtt.neighbors and mqtt.neighbors.interval in the WebConfigKeys set-key
allowlist (the CLI enforces the PSRAM guard; the stub reply handles non-PSRAM).
- Emit neighbors + neighbors_interval (hours) in the WebConfigServer config JSON.
- Add a "Publish neighbors" toggle and a "Neighbors interval (hours)" field
(12-336) to the Publishing card, with getVal() cases in webui/index.html.
- Cover both keys in test_webconfig_keys.
WebConfigHtml.h is a gitignored build artifact regenerated by the pre-build
hook from index.html, so it is not committed. Verified: test_webconfig_keys
passes and the T_Beam_S3_Supreme observer_mqtt firmware builds [SUCCESS].
Port the periodic-neighbors publication from mqtt-bridge-implementation-flex,
adapted to this branch's structure:
- Add MQTT_PUBLICATION_NEIGHBORS ("neighbors") to the pure MQTTTopicRouter
instead of flex's messageTypeSuffix() helper (this branch already routes
every publication type through mqttBuildPublicationTopic()). Neighbors is a
MeshCore/custom publication type, so it resolves to
meshcore/{iata}/{device}/neighbors and honors custom templates.
- Deliberately do NOT port flex's "all message types to MeshRank" change:
this branch documents and host-tests a packets-only MeshRank contract
(MQTTPresets.h, MQTT_IMPLEMENTATION.md, MeshRankContractIsPacketsOnly). So
neighbors follows status/raw and is rejected on MeshRank slots.
- WITH_MQTT_NEIGHBORS guard (PSRAM + MAX_NEIGHBOURS) gates all new surface.
- MSG_NEIGHBORS message type + enum-drift static_assert.
- Persistent ~10KB PSRAM neighbors buffer allocated/freed via the existing
MQTTRuntimeBufferLifecycle path (allocate/release), not the ctor as flex did.
- Core1->Core0 handoff: requestPublishNeighbors() (mesh) fills the buffer with
a release store; the MQTT task consumes it with an acquire load, publishes
via publishNeighbors() (QoS1, retain = preset->allow_retain, custom=false),
and clears the pending flag. A second snapshot is dropped while one is
in flight.
- setNeighborsSchedule()/NeighborsPhase let the mesh report the timer summary;
formatMqttStatusReply() gains a "nbr: <when>/<last>" field via formatDuration.
Also fix the on-connect status publish (publishStatusToSlot) to honor
preset->allow_retain instead of hardcoding retain=true, matching the periodic
publishStatus() path. Brokers with allow_retain=false (e.g. the waev MeshCore
preset) reject retained publishes, so the on-connect status was being dropped
there. This is flex followup 028a5dca, reconciled to this branch's custom-slot
default of non-retained.
Extends the host topic-router test to cover the neighbors type across all
routes and freezes the new enum value. Bridge itself is on-target only.
Add buildNeighborsMessage to the pure MQTTPayloadBuilder core and a thin
delegating wrapper + NeighborsMessageEntry alias on MQTTMessageBuilder, so
the neighbors topic is built by the same firmware-facing API as status/
packet/raw while the layout logic stays exercisable by native tests.
The document is bounded to the publish buffer: entries arrive ordered most-
to least-useful and the tail is dropped once the next entry would overflow,
so a fixed PSRAM buffer can never be handed truncated JSON.
Uses ArduinoJson v7 idioms (.to<JsonObject>()/.add<JsonObject>()) to stay
warning-clean under -Werror, unlike the deprecated createNested* forms.
Adds three test_mqtt_payload_builder cases: self+entry round-trip, empty
table / null scopes, and bounded-growth tail-drop under a tight buffer.
Append mqtt_neighbors_enabled(u8) + mqtt_neighbors_interval(u32) to the
observer tail of MQTTPrefs. The layout is kept byte-identical to the flex
neighbors build: the enable flag lands in the old struct's zeroed trailing
padding (offset 2857) and the interval begins exactly at the former baseline
(2860), so sizeof grows 2860 -> 2864 (net +4 bytes). offsetof static_asserts
lock the layout so a mismatch fails the build.
The codec now accepts three v1 payload sizes: register 2860 as a
"pre-neighbors" Current payload so an in-lineage upgrade reads its existing
/mqtt_prefs and defaults the neighbors tail (off / 24h). Because 2864 is the
shared Current baseline, a /mqtt_prefs written by either the flex build or this
firmware is interchangeable.
Add the 12/24/336h interval constants, neighbors defaults, and a load-time
interval clamp that keeps persisted values inside the signed-delta millis()
scheduling window. Extend the host codec suite with a pre-neighbors migration
case and neighbors round-trip coverage.
First upstream merge since the 2026-06-06 base (191 upstream commits). 14 files
conflicted; resolutions below.
Fleet-critical check (Constraint 1): upstream reordered NodePrefs members
(rx_boosted_gain / path_hash_mode moved to the struct tail) but did NOT change
/com_prefs. Persistence is written field-by-field at explicit offsets, so member
order is in-memory only. Verified the fork's writeCommonPrefsImage() is
byte-identical to upstream's inline writer at every offset (79 pad, 121, 122,
290-294). No migration needed.
Resolutions:
- CommonCLI.h: kept the fork's NodePrefs (superset) and adopted upstream's
setRxBoostedGain(bool)->bool signature change, which CommonCLI.cpp now uses to
report unsupported. Corrected a stale comment claiming rx_boosted_gain lives at
offset 79 (it is a pad; the field is at 290).
- CommonCLI.cpp: kept the fork's legacy /com_prefs migration and the extracted
writeCommonPrefsImage() call.
- UITask.cpp: three-way merge - upstream's drawTextCentered + powering-off
screen, plus the fork's WITH_WEBCONFIG portal/reboot screens.
- ESP32Board.cpp, MeshCore.h, platformio.ini: kept both sides (fork OTA additions
alongside upstream powerOff/enterDeepSleep and Packet.cpp).
- MicroNMEALocationProvider.h: took upstream's claim/release and added the
_claims member they depend on.
- MyMesh.cpp/.h (repeater + room server): kept the fork's superset defaults.
- Removed duplicate declarations auto-merge produced: RadioLibWrapper::_cad_enabled
and MyMesh::getCADEnabled().
Verification: native suite 15/15 (incl. upstream's new test_mesh_tables), both
MQTT smoke builds green, ArduinoJson pin check passes. Hardware validation next.
The WebConfig POST/result/reboot/stop batch state machine was the largest
remaining Phase 6 coverage gap (all inline in WebConfigServer.cpp, coupled to
AsyncWebServer/ArduinoJson and untestable on host). Extract its decision + timing
CORE into a pure, dependency-free spec mirroring MQTTLifecycle.h:
- src/helpers/WebConfigBatch.h: classifyPost (replay/busy/accept/no-changes with
the DONE-vs-PENDING reqid asymmetry), drain pacing (signed 25 ms gate, sticky
all_ok, 30 s reboot fallback), result classification + arm-once 3 s reboot,
signed-wrap-safe reboot-due / isRebootPending, and stop gating (finalize when
refs==0, warn-once, never force teardown). Constants verbatim from the source.
- test/test_webconfig_batch/: full host coverage incl. exact boundaries and
millis() rollover.
Spec-first, exactly like Phase 4's MQTTLifecycle.h: this is NOT yet wired into
WebConfigServer.cpp. That server is hardware-tuned (debugged against real iOS
captive-portal + HTTP-caching + route-ordering behavior), so making the spec
load-bearing is a deliberately separate, hardware-validated follow-up.
Faithfulness independently reviewed against WebConfigServer.cpp; native suite
green (14 dirs). No production behavior change.
Close the host-testable gaps named in Phase 6 of STABILITY_TESTABILITY_HANDOFF.md
by moving the last inline decision logic into the pure, host-tested policy seams:
- WiFi STA reconnect backoff: extract the inline ladder + wrap-safe timing from
handleWiFiConnection() into MQTTConnectionPolicy::{wifiReconnectBackoffMs,
wifiReconnectDue,nextWifiBackoffAttempt}. Behavior-preserving (elapsedMs is the
wrap-safe form of the old ULONG_MAX branch); ladder/clamp/attempt-cap unchanged.
- Publication outcome pairing: name the (packet, raw) -> delivered contract as
MQTTPacketQueuePolicy::queuedPacketPublished() and wire both queue-drain sites;
partial success = completed, not retried.
- Freeze MQTTPublicationType enum values in a test (the bridge-side MQTTMessageType
alignment is already enforced by a compile-time static_assert).
Adds host tests for all three (exact boundaries + millis() rollover). Native suite
green (13 dirs); non-PSRAM observer firmware smoke build compiles.
WebConfig batch/reboot/stop state-machine extraction and queue-orchestration
coverage remain open (tracked in the Phase 6 status).
Fork-owned, host-tested MQTT bridge lifecycle state machine and the narrow
dependency seam to drive it deterministically, plus the cross-core ownership
model. This is the Phase 4 "ownership and teardown test seams" safety net that
must land before the Phase 5 cooperative-shutdown refactor.
- src/helpers/MQTTLifecycle.h: pure state machine
(Stopped->Starting->Running->StopRequested->Stopping->Stopped) + injected
Ops seam (clock / task / resource owner / OTA barrier) + Coordinator with a
bounded stop timeout. No Arduino/FreeRTOS/WiFi deps.
- test/test_mqtt_lifecycle/: 18 GoogleTest cases covering the phase's teardown
matrix (stop during every activity, callback timing, duplicate/early stop,
restart, timeout fallback, no-access-after-release) and the OTA-barrier
scenarios.
- MQTT_OWNERSHIP.md: one owner per mutable runtime domain, current cross-core
hazards with file:line references, target primitives, and Phase-0-pending
(hardware-characterization) items.
- STABILITY_TESTABILITY_HANDOFF.md: Phase 4 status and verified premise
refinements.
Scope: MQTTBridge.cpp is intentionally untouched. The production rewiring
(plain-data snapshot publication, volatile-handshake replacement, cooperative
shutdown) is deferred to Phase 5.
Rework the /mqtt_prefs load/save path so preference migrations are
crash-safe and, for the first time, unit-testable on the host.
Most of this is extraction. The multi-format migration that previously
lived inline in CommonCLI.cpp (and could only run on-device) is moved
into three dependency-free headers so it can be exercised without
Arduino, a filesystem, or the radio stack:
- MQTTPrefsStorage.h frozen layout structs for every /mqtt_prefs
format ever shipped, with static_asserts that
fail the build if any on-flash offset changes.
- MQTTPrefsCodec.h pure format classification, field-copy
migration, and plausibility validation.
- MQTTPrefsAtomicStore.h transactional writer plus the power-cut
upgrade gate, both host-testable.
New behavior, beyond the refactor:
- Atomic writes: /mqtt_prefs is written to /mqtt_prefs.tmp, verified,
then published with an atomic rename; the writer never removes the
existing file. A failed or interrupted save leaves the current
config intact.
- Power-cut ordering: LegacyUpgradeGate guarantees /mqtt_prefs is
durably committed before the legacy /com_prefs (or /node_prefs)
carrying the observer tail is compacted or removed, so an
interrupted two-file upgrade retries on the next boot without
losing settings.
- Corrupt, unsupported-version, and newer-than-known files are
preserved and the device boots on in-RAM defaults, rather than
overwriting a file this firmware cannot fully decode.
- Headerless legacy formats are validated for plausibility before
they are trusted and rewritten (raw prefs carry no checksum).
The full historical format matrix is migrated forward to the versioned
v1 layout: pre-slot (including pre-wifi-power), 3-slot (base and
token/topic tails), and headerless 6-slot (base, audience, rx, ntp).
Scope note: only /mqtt_prefs and the one-time /node_prefs -> /com_prefs
name migration use the atomic path. Ordinary /com_prefs saves remain a
direct rewrite, unchanged by this commit.
Tests: adds two host GoogleTest suites (pio test -e native).
- test_mqtt_prefs_codec: format classification, migration fixtures,
v1 header integrity, downgrade preservation.
- test_mqtt_prefs_atomic_store: transactional writes, short-write
detection, begin/finish/rename failure cleanup, original-file
preservation.
Replace direct JSON construction in MQTTMessageBuilder with calls to
MQTTPayloadBuilder for building status, packet, and raw messages.
This change improves code maintainability and reduces duplication by
centralizing message formatting logic. Additionally, update platformio.ini
to include ArduinoJson dependency for JSON handling.
Improve the mqttSubstituteTopic function to ensure that it reports
overflow when the output buffer is full, preventing silent truncation
of topics. Update related tests to verify the new behavior for
literal overflow and exact fit scenarios, enhancing robustness and
correctness of topic handling.
Add validation for request IDs in the web configuration server to ensure
they conform to the expected format. Enhance error responses for invalid
or unknown request IDs, improving the robustness of request handling.
Update the web UI to reflect these changes, ensuring that clients can
properly handle errors related to request ID mismatches.
Include detailed instructions for local testing of observer and WiFi
functionality without hardware. Document the use of a mock backend and
Wokwi ESP32-S3 simulation for easier development and testing.
Enhance the MQTT implementation documentation to improve developer
experience and facilitate testing workflows.
hasSeen() was simultaneously a predicate and a mutator — it inserted the
packet hash on every miss, making five call sites that only wanted to mark
a packet as sent call it with the return value discarded.
Split into:
- wasSeen() — pure predicate, no side effects
- markSeen() — explicit insert
All query sites now call markSeen() immediately after wasSeen() returns
false, preserving identical runtime behaviour. The five mark-only send
sites (sendFlood, sendDirect, sendZeroHop x2) now call markSeen directly.
Also fixes three bridge sites (BridgeBase, ESPNowBridge, RS232Bridge)
that had the same query+implicit-insert pattern.
Tests: add test/test_mesh_tables/ covering wasSeen purity, markSeen,
dup stats, and clear. Update SHA256 mock to produce deterministic output
(previously finalize() was a no-op). Add Packet.cpp to native build filter.
agessaman