Replace the sparse Heltec V4 R8 observer home screen with a padded dark
analytics dashboard, add manual display control, and make blanking a runtime
setting.
Dashboard (DISPLAY_ACTIVITY_DASHBOARD, the four R8 TFT observer envs):
- RadioActivityWindow: 20 one-minute buckets of valid RX packets, no heap.
The caller's 32-bit millis() is extended to a monotonic 64-bit clock, so
nothing downstream has a rollover case; an always-on node passes 2^32 ms
after ~49.7 days, which would otherwise re-enter warm-up and divide 20
minutes of traffic by seconds. Rates use 19 whole minutes plus the elapsed
part of the current one rather than a fixed 1200 s.
- ObserverDashboard: header, radio strip, headline totals, a 20-bar
packets-per-minute graph and RF/status footers, with separate portrait and
landscape layouts. A text row is a fixed 16 px, which is 3.2 logical units
in portrait but 4.27 in landscape, so one shared grid would overlap.
Text is trimmed by character budget, not measured width: getTextWidth()
reports an over-long string at the portrait driver's fallback scale, so
DisplayDriver::drawTextEllipsized() under-trims and the row renders at half
height.
- Six per-row signatures computed from what is actually drawn, so only the
rows whose pixels changed repaint. No startFrame(), no whole-screen clear.
Link state moved out of the full-frame signature, so a DHCP renewal or WiFi
flap repaints one footer row instead of the panel.
- Dark theme by retuning the UIColor statics at runtime, which needs no
display-driver edit and carries boot, setup, reboot and power-off with it.
Touch and button (DISPLAY_TOUCH_TOGGLE):
- CHSC6X at I2C 0x2E, polled; TP_INT is unusable (optional R13, and GPIO 43
is U0TXD). The point-count byte is tested against a valid count, never
against non-zero: an idle read returns 0xFF, which reads as a finger held
down forever and latches the tap detector after one event.
- turnOff() no longer parks PIN_TFT_RST low on this board. GPIO 21 is a
shared LCD_RST/TP_RST net, so doing that held the touch controller in
reset for as long as the display was off. Verified against Heltec's
expansion-board and mainboard schematics and the V4-R8 datasheet pinout,
which also correct the pin comment in HeltecV4R8Board.cpp.
- The USER button click now toggles the display too; it previously did
nothing whenever the display was already on.
display.timeout:
- `set display.timeout <secs>` / `get display.timeout`, 0 = stay on, 60 s
default, 3600 max. Read live, so a change applies without a reboot and
restarts the countdown rather than firing on the old deadline.
- Stored in MQTTPrefs (/mqtt.json), keeping NodePrefs aligned with upstream.
Runtime-only: LegacyV1MQTTPrefs and the four frozen binary payload sizes
are unchanged. No JSON format-version bump - the loader skips keys no
def() claims, so older firmware reads newer files and this firmware reads
older ones with the default applied. Both directions are covered by tests.
- Joins the observer atomic-setter contract, so a failed save rolls the live
value back instead of only claiming to.
New periodic work uses a wrap-safe deadline check; `millis() >= deadline`
fires every loop for a whole interval before each rollover.
Adds test_radio_activity_window, test_observer_dashboard (driving the real
renderer against a recording DisplayDriver in both orientation profiles) and
test_touch_tap_detector. 440 native cases pass.
Picks up upstream MeshCore 1.17.1.
Notable upstream content:
- 1.17.1 version/build-date bump in the example MyMesh headers.
- nRF52: combine radio entropy with CC310 RNG.
- Companion FEM prefs: load/save of fem_ properties commented out until they
can be set from the client.
- Scoped reply routing: replies no longer dropped when flood.max.unscoped is
low (RoutingPolicy + unit tests).
- nRF52 unused-pin sweep (T1, T-Echo Lite, MeshPocket).
No conflicts.
Picks up upstream MeshCore 1.17.0 and the dev commits that followed it.
Notable upstream content:
- 1.17.0 version/build-date bump in the example MyMesh headers.
- anon_req hardening: reply_path_len is now uint8_t with an isValidPathLen()
bounds check and a 0xFF sentinel; reply_path_hash_size is gone.
- LR2021 support (Meshnology W12, Seeed MeshTracker X1) incl. side detectors,
multi-SF and the new `extra.sf` get/set CLI.
- CustomLFS 0.2.2 -> 0.2.3 (GD25Q64C support).
- kiss_modem envs for several nRF52 variants; ThinkNode M6 GPS/flash fixes.
Conflicts resolved:
- platformio.ini: took upstream's CustomLFS 0.2.3, kept our lib_ignore for
the vendored PsychicMqttClient.
- CommonCLI.h: kept both upstream's USE_LR2021 configSideDetectors() hook and
our fault-alert callbacks.
NodePrefs gains upstream's extra_sf[4], which is not registered with
ConfigSerializer, so /prefs.json layout is unchanged.
Verified: 275/275 native host tests pass; Heltec_v3_repeater_observer_mqtt,
Heltec_v3_repeater and heltec_v4_repeater_observer_mqtt all build.
A neighbour heard before the clock is set carries the firmware's unset-clock
default (1715770351, 15 May 2024). Subtracting that from an NTP-synced clock
published ages of ~806 days for neighbours that had just answered a live scope
query, and a backwards clock step reported 0, i.e. "heard just now".
- finishNeighborDiscover() reports the age as unknown ("heard_secs_ago": null)
when the stored stamp and the current clock come from different epochs, or
when the clock has stepped backwards
- handleNeighborDiscoverResponse() re-stamps heard_timestamp on a zero-hop scope
reply, in both the snapshot and the live table, so entries heal once per
discovery cycle instead of waiting for the neighbour's next advert
- publish ordering places usable ages ahead of unknown ones so a poisoned stamp
cannot displace fresh entries when the JSON buffer truncates
- document the null case, and the always-present total_neighbors /
queried_neighbors / truncated fields the payload sample omitted
- add UPSTREAM_BUGS.md, tracking the monotonic-uptime fix to propose upstream
plus the unclamped subtraction in the companion and CLI readouts
Containment only: upstream still stamps neighbours from the wall clock at
packet-reception time, which on a cold boot always precedes NTP.
Absorbs 106 upstream commits. Seven files conflicted; the substantive one
was upstream's new JSON ConfigSerializer (PR #2982), which replaces the
binary /com_prefs layout with /prefs.json and makes NodePrefs a
ConfigSerializer subclass.
Prefs migration
- Adopt upstream's ConfigSerializer. writeCommonPrefsImage() and its
documented offsets (0-294) are deleted, along with the now-unreachable
saveCommonPrefsImageAtomically()/CommonPrefsFileStore atomic rename path.
- Load order is /prefs.json, then /com_prefs, then /node_prefs. Upstream
dropped the /node_prefs fallback; it is restored here so devices that
never advanced past that filename keep their config.
- Legacy files are never removed, so migration cannot destroy its own
source and a deferred or failed save simply retries next boot.
- /com_prefs is treated as a format migration only, not an "upgrade", so
it does not trip the bridge.source tx->rx flip on existing nodes.
- The MQTTPrefsAtomicStore legacy gate is retained: the observer tail
recovered from an old-format file still commits to /mqtt_prefs before
/prefs.json is written.
- MQTTPrefs and /mqtt_prefs are untouched; savePrefs keeps its save_mqtt
parameter and now returns upstream's bool.
Fixes to upstream code
- RadioPrefs::structure() bound both "rxgain" and "fem_rxgain" to
rx_boosted_gain, so radio_fem_rxgain was never persisted. Bound to the
correct field.
- discovery_mod_timestamp was dropped from structure(); it gates
'since'-filtered DISCOVER replies and is set on every config change, so
losing it would silently stop discovery responses after a reboot. Added
as "disc_mod".
Merge artifacts repaired
- Restored bblanchon/ArduinoJson to the native test env; a clean but wrong
auto-merge at the lib_deps block boundary dropped it and broke all 19
host test suites.
- Migrated the fork's WebConfig UITask screens off the removed
DisplayDriver::Color enum to upstream's UIColor element types.
- Removed duplicate getCADEnabled() definitions in companion MyMesh.cpp
and simple_sensor SensorMesh.cpp that both sides had added.
- Dropped memset(&_prefs, 0, ...) in the four example meshes; NodePrefs now
has a vtable. guard gains an initializer that memset used to provide.
Other resolutions
- simple_room_server keeps both the fork's discover.* commands and
upstream's new room.post.
- docs/payloads.md taken from upstream, undoing content earlier merges had
reverted (Control data section, split login tables).
Verified: 273/273 host tests pass across native and native_kiss_modem;
Heltec v3 repeater, repeater_observer_mqtt, room_server_observer_mqtt,
sensor, and companion_radio_ble all build clean.
Enhance the neighbor discovery JSON structure by introducing a
default_scope field, which indicates the region name this node
floods to by default. This change improves clarity in the
neighbor discovery process and aligns with the unscoped flood
behavior when no default region is set. Updates include
modifications to the MyMesh class and related message building
functions to accommodate the new field.
Introduce new methods to manage neighbor discovery JSON budget and
entries in MyMesh. This includes tracking the number of queried and
published neighbors, measuring JSON sizes, and handling truncation
when the buffer limit is reached. These improvements optimize the
neighbor discovery process and ensure efficient JSON message
construction for MQTT communications.
Add functionality to manage neighbor discovery requests in MyMesh.
This includes handling the status of requests, updating the state
upon transmission success or failure, and implementing a timeout
mechanism for neighbor discovery queries. The changes improve the
efficiency and reliability of neighbor discovery operations in
MQTT-enabled environments.
Improve the response message for the `alert test` command to clarify
that a successful test send does not guarantee automatic alerts are
enabled. This change helps prevent confusion for operators regarding
the status of the alert system after testing.
Port the neighbor-discovery state machine from mqtt-bridge-implementation-flex,
adapted to this branch's MyMesh (WebConfig members shifted the insertion points;
applied by content).
- Two-stage periodic refresh in loop() driven by mqtt_neighbors_interval:
stage 1 is a zero-hop sendNodeDiscoverReq() (reuses the existing 60s window),
stage 2 (startNeighborDiscover) fires one anon-regions scope query per heard
neighbour, then finishNeighborDiscover() builds the table JSON via
MQTTMessageBuilder::buildNeighborsMessage and hands it to
bridge->requestPublishNeighbors().
- Peer overlay at NEIGHBOR_DISCOVER_PEER_BASE lets scope-query RESPONSE packets
from non-ACL neighbours decrypt: searchPeersByHash prepends heard neighbours
(bounded by MAX_CLIENTS), getPeerSharedSecret derives the secret on the fly,
and onPeerDataRecv routes both overlay-index and ACL-client-that-is-a-neighbour
responses into handleNeighborDiscoverResponse.
- Entries ordered most- to least-useful (recent, then stronger SNR) so the JSON
builder's tail-drop keeps the useful head.
- `discover.scopes` CLI command (manual trigger), with a WITH_MQTT_BRIDGE stub
replying "requires PSRAM" on non-PSRAM builds.
- Reports schedule to the bridge each loop via setNeighborsSchedule().
- Uses ArduinoJson v7 JsonDocument (not deprecated DynamicJsonDocument).
All gated on WITH_MQTT_NEIGHBORS. Reuses the existing MQTTBridge* member.
Verified: T_Beam_S3_Supreme_SX1262_repeater_observer_mqtt builds [SUCCESS].
First upstream merge since the 2026-06-06 base (191 upstream commits). 14 files
conflicted; resolutions below.
Fleet-critical check (Constraint 1): upstream reordered NodePrefs members
(rx_boosted_gain / path_hash_mode moved to the struct tail) but did NOT change
/com_prefs. Persistence is written field-by-field at explicit offsets, so member
order is in-memory only. Verified the fork's writeCommonPrefsImage() is
byte-identical to upstream's inline writer at every offset (79 pad, 121, 122,
290-294). No migration needed.
Resolutions:
- CommonCLI.h: kept the fork's NodePrefs (superset) and adopted upstream's
setRxBoostedGain(bool)->bool signature change, which CommonCLI.cpp now uses to
report unsupported. Corrected a stale comment claiming rx_boosted_gain lives at
offset 79 (it is a pad; the field is at 290).
- CommonCLI.cpp: kept the fork's legacy /com_prefs migration and the extracted
writeCommonPrefsImage() call.
- UITask.cpp: three-way merge - upstream's drawTextCentered + powering-off
screen, plus the fork's WITH_WEBCONFIG portal/reboot screens.
- ESP32Board.cpp, MeshCore.h, platformio.ini: kept both sides (fork OTA additions
alongside upstream powerOff/enterDeepSleep and Packet.cpp).
- MicroNMEALocationProvider.h: took upstream's claim/release and added the
_claims member they depend on.
- MyMesh.cpp/.h (repeater + room server): kept the fork's superset defaults.
- Removed duplicate declarations auto-merge produced: RadioLibWrapper::_cad_enabled
and MyMesh::getCADEnabled().
Verification: native suite 15/15 (incl. upstream's new test_mesh_tables), both
MQTT smoke builds green, ArduinoJson pin check passes. Hardware validation next.
An operator who triggers `ota update` via remote management only sees the
command's reply; the actual OTA runs ~2.5s later and reboots on success,
so the start/fail notifications land outside the reply window. Mirror the
key OTA milestones onto the configured alert channel (in addition to the
Serial log):
- START ("OTA update starting") from beginDeferredOtaUpdate(), i.e. while
the loop still runs -- a flood queued at the deferred fire could never
transmit before the flash blocks the loop / reboots on success.
- FAIL ("OTA aborted: ...") at both abort points (teardown barrier
withheld flashing; preflight/download error).
Success has no message: a successful flash reboots into the new image, so
the node returning on the new version is the signal.
New MyMesh::otaAlert() gates on the `alert on/off` master switch and rides
the configured alert scope (AlertReporter::sendText -> sendChannel ->
resolveAlertScope); no-op when alerts are off or no channel is set. Only
these start/fail milestones -- routine slot connect/disconnect is
unaffected (stays in AlertReporter's fault logic). Documented in ALERTS.md.
Both observer firmwares build.
Introduce a web configuration portal for easier node management and
provisioning without serial CLI. Enhance MQTT functionality with
improved IATA code validation, dynamic slot management, and
background NTP synchronization. Update web UI elements for better
user experience and security notes regarding open AP usage.
Continues restoring features dropped by the 22eb9b87 revert. Both were
upstream-tested code, not intentional fork removals.
CAD (hardware Channel Activity Detection / listen-before-talk before TX),
fully restored and functional:
- NodePrefs.cad_enabled + `set cad on/off` / `get cad` CLI (default off)
- RadioLibWrapper: _cad_enabled + setCADEnabled() + the scanChannel()/CAD
branch in isChannelActive() (Phase 1 already restored the Dispatcher hook)
- getCADEnabled() overrides in the repeater/room/sensor apps (return the
pref) and companion (always on, matching upstream)
radio_fem_rxgain: the NodePrefs field + /com_prefs persistence are restored
here at upstream's exact offsets (293 fem, 294 cad), which makes /com_prefs
byte-identical to upstream through the tail. The field is persisted and
defaults on (upstream default), but the per-board LNA *driving* + the
`radio.fem.rxgain` CLI are deferred to the FEM-hardware change (they depend
on board methods and want per-board bench testing).
The new-format /com_prefs tail grows from 3 to 5 bytes; the old-format
detection threshold and the host migration harness are updated accordingly
(all scenarios pass, incl. the non-MQTT-build variant).
Builds: Heltec_v3 repeater, repeater-observer, room-observer. (sensor /
plain-room / companion fail only on the pre-existing Timezone.h include
issue, unrelated to these changes.)
The `recv_pkt_region` is set when processing a flood packet in `filterRecvFloodPacket`
but direct/non-flood packets would never pass through that function, so the pointer was
not cleared for them.
`sendFloodReply` would then later use it blindly, which meant that the response would
either inherit the region from the last flood packet, or refer to a non-initialised pointer
if no region floods had been received yet.
Refactored the handling of observer-related settings by moving them from
NodePrefs to a new MQTTPrefs structure. This change centralizes MQTT,
WiFi, timezone, SNMP, and alert configurations, improving code organization
and maintainability. The new structure allows for better separation of
concerns and prepares the codebase for future enhancements.
Implemented new commands for configuring and diagnosing NTP server
settings in the MQTT bridge. Users can now set a custom NTP server
and probe connectivity to configured servers. This enhancement
improves time synchronization reliability for JWT authentication
and provides better diagnostics for NTP connectivity issues.
Added support for deferred OTA updates in the MyMesh class, allowing
the system to schedule firmware updates to occur after a confirmation
reply is sent. This change improves the user experience by ensuring
that the update process does not block the main application loop,
allowing for smoother operation during firmware updates.
Updated the fault alert functionality to include an optional region name for scoping alert floods, allowing operators to override the default scope. Introduced a list of banned channels (e.g., Public PSK, `#test`, `#bot`) to prevent spamming community channels with alerts. The implementation ensures that alerts are only sent to private PSKs or non-banned hashtags. Relevant changes were made across multiple files, including updates to the CLI for setting and retrieving the new `alert.region` preference.
Implemented a new fault alert system that broadcasts notifications over LoRa when WiFi or MQTT connections are down for a specified duration. The alerts are configurable via CLI commands, allowing operators to set private PSKs or hashtags for alert channels. Default settings for alert thresholds and intervals are established, and the system ensures that alerts do not spam the public channel. Updated relevant files to integrate this feature into the MyMesh implementations and CLI handling.
Introduced a new CLI command to view and set the radio watchdog interval, allowing users to configure the watchdog timeout in minutes (0 to disable, 1-120 for active). Updated the documentation to reflect this addition and clarified the parameters for the `neighbor.remove` command. Default watchdog interval is set to 5 minutes in the codebase.
* room server: added RegionMap, and new CommonCLI wiring, default_scope handling
* sensor: only minimal RegionMap wiring. Still needs work to handle default-scope
Implement a new method `formatRadioDiagReply` in MyMesh to format and return radio diagnostic information. Update CommonCLI to handle the new command for retrieving radio diagnostics. Enhance the Dispatcher class with a radio watchdog feature to monitor and recover from silent radio states. Introduce new error flags and methods in the Dispatcher and RadioLibWrapper classes to support these diagnostics, improving overall radio management and troubleshooting capabilities.
Reverted prior dev merge, then merged current meshcore-dev dev. Resolved
CommonCLI by restoring MQTT prefs flow with upstream com_prefs layout
(rx_boosted_gain at offset 79), rxgain CLI, poweroff/shutdown, and FEM
variant files from upstream.
Made-with: Cursor
agessaman