Neighbors publication was gated on BOARD_HAS_PSRAM. Removing that gate alone
was not enough: the feature built but was inert without PSRAM, because three
allocation sites asked for MALLOC_CAP_SPIRAM (which returns null with no
PSRAM), the bridge's persistent buffer was allocated inside a BOARD_HAS_PSRAM
block, and neighborDiscoverReady() rejected every pass at runtime on
psramFound(). The entry table also did not fit: finishNeighborDiscover put
pubkey_hex[50][65] plus entries[50] on the stack, a 4752-byte frame against
the mesh loop task's 8 KB.
- Gate on MAX_NEIGHBOURS plus PSRAM or an explicit per-variant
MQTT_NEIGHBORS_WITHOUT_PSRAM opt-in.
- Move the entry table and its hex strings into one heap block sized to the
pass; the frame drops from 4752 to 304 bytes.
- Prefer PSRAM and fall back to internal DRAM in the mesh-side allocations
and the ArduinoJson pool; hoist the bridge's persistent buffer out of the
BOARD_HAS_PSRAM block (psram_malloc already falls back).
- Keep the runtime psramFound() check only where the buffers are sized for
PSRAM, so a board whose PSRAM failed to init still refuses.
- Size for internal DRAM without PSRAM: 4 KB text buffer and 20 entries per
publish, keeping the pool to a single block and the peak near 13 KB rather
than ~35 KB. Oversized tables truncate and report total_neighbors as before.
Enabled on the ESP32-S3 observer envs (Heltec V3/WSL3, RAK3112, Heltec
Tracker v1.1/v2). Left off for the classic ESP32 T-LoRa V2.1-1.6, which is
already limited to one active TLS slot.
Costs ~7.4 KB static DRAM on repeaters and ~9.6 KB on room servers. The
prefs layout is unchanged, so this is neutral for existing devices.
Adds ThinkNode_M7_repeater_observer_mqtt and
ThinkNode_M7_room_server_observer_mqtt, following the Station G3 observer
overlay: adafruit-full cert bundle, MQTT bridge + SNMP, pinned observer
lib_deps, and the quieter default logging profile.
WiFi-only for now. The M7 has an onboard CH390 Ethernet controller (used by
ThinkNode_M7_companion_radio_ethernet, and an lwIP netif so the MQTT data path
would work over it), but the bridge's link management is bound to the WiFi
station API, so Ethernet cannot carry MQTT yet.
The board has PSRAM, so MAX_NEIGHBOURS=50 enables WITH_MQTT_NEIGHBORS —
verified present in both images. Builds use 46.5% of the 3.19 MB app slot at
13.7% RAM, and build.sh emits the full artifact set (.bin, -merged.bin,
.partsig). The partition table is inherited from [ThinkNode_M7] unchanged, so
no merged first flash is needed; its signature is byte-identical to the T-Beam
Supreme observer builds (both default_8MB.csv).
Also fixes two pre-existing build failures on this board:
- [ThinkNode_M7] re-added a blanket helpers/*.cpp after arduino_base excludes
the MQTT-only sources, so every M7 env failed on a missing Timezone.h. The
glob was otherwise fully redundant with the base, so it is dropped rather
than re-excluding the two files.
- ThinkNode_M7_companion_radio_ethernet sets DISPLAY_CLASS=NullDisplayDriver
but never compiled NullDisplayDriver.cpp, which defines UIColor::window_bkg,
so it failed at link. The ble and wifi envs both already list it.
All nine M7 envs now build. Observer env discovery picks up both new envs, and
the ArduinoJson pin check covers both new sections.
A neighbour heard before the clock is set carries the firmware's unset-clock
default (1715770351, 15 May 2024). Subtracting that from an NTP-synced clock
published ages of ~806 days for neighbours that had just answered a live scope
query, and a backwards clock step reported 0, i.e. "heard just now".
- finishNeighborDiscover() reports the age as unknown ("heard_secs_ago": null)
when the stored stamp and the current clock come from different epochs, or
when the clock has stepped backwards
- handleNeighborDiscoverResponse() re-stamps heard_timestamp on a zero-hop scope
reply, in both the snapshot and the live table, so entries heal once per
discovery cycle instead of waiting for the neighbour's next advert
- publish ordering places usable ages ahead of unknown ones so a poisoned stamp
cannot displace fresh entries when the JSON buffer truncates
- document the null case, and the always-present total_neighbors /
queried_neighbors / truncated fields the payload sample omitted
- add UPSTREAM_BUGS.md, tracking the monotonic-uptime fix to propose upstream
plus the unclamped subtraction in the companion and CLI readouts
Containment only: upstream still stamps neighbours from the wall clock at
packet-reception time, which on a cold boot always precedes NTP.
Adds Station_G3_ESP32_repeater_observer_mqtt and
Station_G3_ESP32_room_server_observer_mqtt, mirroring the known-good
Station G2 observer envs. The two boards are the same family — ESP32-S3,
qio_opi, PSRAM, 16 MB flash, SH1106 display — so the overlay ports across
unchanged: default_16MB.csv partitions, adafruit-full cert bundle,
MQTT bridge + SNMP, and the same lib_deps pinning.
Verified the G3 partition signature is byte-identical to the G2 observer's,
so the OTA partition-compatibility gate treats them the same. Both envs
build clean (1.67 MB of the 6.25 MB app slot, 28% RAM) and produce the full
CI artifact set via build.sh: .bin, -merged.bin and .partsig. Observer env
discovery now finds 32 envs (was 30) and the ArduinoJson pin check covers
both new sections.
Docs: added the G3 build commands and the two partition-table rows to
MQTT_IMPLEMENTATION.md.
Port the observer doc cleanup from observer-firmware (4beb5472) onto dev,
keeping dev-specific content intact.
- move internal mechanics into MQTT_INTERNALS.md: deferred construction,
runtime slot memory, backoff/circuit breaker, message building, command
namespacing, and the legacy prefs key mapping
- drop the duplicated First-Time Setup, Command Architecture, and SNMP
Monitoring sections, folding their unique steps into Quick Start
- condense the partition/NVS prose and merge the five custom-broker
examples into one Custom Brokers section
- promote the preset list to a top-level Broker Presets section with an
Extra setup column, and document the presets the table was missing:
meshcore-fi, corecomms, mesh-chaun14, wcmesh, and meshtexas
- fix the coloradomesh port (443, not 1883)
- point flasher and changelog links at observer.gessaman.com
- restore the nbr: status field, which the code still emits, and correct
the neighbors topic to QoS 0 retained per allow_retain
Dev-specific content is preserved as-is: per-slot packet filters, the Web
Configuration Portal, local testing without hardware, MeshRank's raw
exclusion, and neighbors default_scope.
MeshRank slots previously took packets only. The broker tolerates status and
neighbors (and its maintainer intends to look at using the neighbors data), so
those now publish under meshrank/uplink/{token}/{device}/{type}, using the same
type suffixes as the MeshCore layout. Raw stays excluded: it is the
highest-volume topic and the broker does not consume it, so `set mqtt.raw on`
has no effect on a MeshRank slot.
All MeshRank gating funnels through the topic router, so relaxing that single
guard was sufficient - publishStatusToSlot, publishStatus, publishRaw,
publishNeighbors, and eligiblePacketSlots already skip slots that cannot form a
topic. A per-slot token is still required. Because eligiblePacketSlots resolves
topic support before serialising, raw JSON is never built for a MeshRank slot
rather than built and discarded.
observer-firmware sends raw to MeshRank; this supersedes that decision, so raw
must stay excluded when the branches merge. MeshRankTakesEveryTypeExceptRaw
fails if a merge re-enables it.
Enhance the neighbor discovery JSON structure by introducing a
default_scope field, which indicates the region name this node
floods to by default. This change improves clarity in the
neighbor discovery process and aligns with the unscoped flood
behavior when no default region is set. Updates include
modifications to the MyMesh class and related message building
functions to accommodate the new field.
Enhance the MQTT preferences handling by establishing a downgrade
contract that ensures compatibility between different firmware versions.
This contract allows nodes to read settings from newer builds while
safeguarding against data loss during downgrades. The implementation
includes updates to the classification logic, ensuring that longer
payloads from newer versions are handled correctly without rejecting
files, thus preserving critical WiFi credentials and broker settings.
Add support for named packet types in per-slot filters, allowing users
to specify packet types using descriptive names alongside numeric values.
This improves usability and clarity in configuring MQTT slot filters.
Updates include modifications to the parsing logic, WebConfig interface,
and related documentation to reflect the new naming conventions.
Introduce per-slot packet filters to allow users to specify which
packet types are uploaded for each MQTT slot. This feature enhances
the flexibility of the MQTT bridge by enabling users to configure
allowlists for packet types, improving the efficiency of data
transmissions. The implementation includes updates to the WebConfig
interface, internal handling of packet filters, and necessary
modifications to the MQTT preferences structure.
Include detailed instructions for local testing of observer and WiFi
functionality without hardware. Document the use of a mock backend and
Wokwi ESP32-S3 simulation for easier development and testing.
Enhance the MQTT implementation documentation to improve developer
experience and facilitate testing workflows.
Introduce a web configuration portal for easier node management and
provisioning without serial CLI. Enhance MQTT functionality with
improved IATA code validation, dynamic slot management, and
background NTP synchronization. Update web UI elements for better
user experience and security notes regarding open AP usage.
Credit: @yellowcooln (PR #12). Adds Wireless Tracker v1.1 board
def and MQTT observer envs for v1.1/v2; gate FEM control so v1.1
builds without KCT8103L pins.
Reverts merge ae045395 (feat/flex-ipv6, PR #25). Enabling IPv6 joins the
device into IPv6 multicast/ND processing; on multicast-heavy LANs (e.g.
with a Thread/Matter border router advertising a ULA prefix) inbound
bursts land in dynamic WiFi RX buffers in internal heap. Measured on a
Station G2 with 4 WSS brokers: min-free floor dropped 36 KB -> 18 KB,
largest free block pinned below the 16 KB publish threshold for minutes
at a time, 523 dropped publishes in 15 minutes. With IPv6 disabled the
floor and max-alloc recovered and publish skips stopped.
The feature only fed the wifi.status display line — no transport uses
IPv6 (all brokers connect over IPv4), so the fleet risk (network-
dependent degradation on unknown home LANs) buys nothing. Revisit after
the Arduino core 3.x / IDF 5.x move if IPv6 transport is ever needed;
the branch remains at feat/flex-ipv6.
Kept from the merge: the wifi.status uptime append now computes the
actual remaining space in the 160-byte reply buffer instead of assuming
a hardcoded 128, fixing a latent overflow of the snprintf bound.
The Files section (source layout, upstream integration seams, and
on-device settings migration) is developer-facing detail that sat near
the top of the otherwise user-facing MQTT_IMPLEMENTATION.md. Move it to
a new MQTT_INTERNALS.md and link it from the bottom of the main doc.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The NodePrefs->MQTTPrefs split (2eb41bae) left the promised one-time
migration of the old /com_prefs trailing block unimplemented, so users
upgrading an observer node silently lost SNMP, radio-watchdog, and
fault-alert configuration (alerts reset to off; PSK/hashtag/region wiped).
loadPrefsInt now detects an old-format /com_prefs by its size, skips the
legacy zero-filled MQTT gap (6-slot or 3-slot era), and recovers the
trailing observer block into a LegacyObserverTail (reusing the old
firmware's byte291/292 heuristic and per-field availability guards).
loadMQTTPrefs applies those values when the loaded /mqtt_prefs predates
the appended observer fields, and both files are rewritten once in the
current layout. rx_boosted_gain/flood_max_* are also recovered from the
correct offsets (previously read from inside the old gap and reset).
Verified with a host-side harness that round-trips the real old-firmware
savePrefs (from 2eb41bae^) through the new load path across upgrade,
fresh-install, upstream-format, 3-slot-era, truncated, and legacy-variant
cases, plus a non-MQTT-build variant. Updates the stale migration comments
and documents the CommonCLI_Observer seam in MQTT_IMPLEMENTATION.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Updated MQTT configuration to succeed when port is defined in mqttN.server but not explicitly set in mqttN.port. Updated documentation to specify that when a full
URL with a scheme is provided, the port setting is optional. Added an
example for local development using a plain WebSocket URL, enhancing
clarity for users configuring custom MQTT brokers.
Implemented new commands for configuring and diagnosing NTP server
settings in the MQTT bridge. Users can now set a custom NTP server
and probe connectivity to configured servers. This enhancement
improves time synchronization reliability for JWT authentication
and provides better diagnostics for NTP connectivity issues.
Enhanced the firmware versioning system by appending a build number
suffix when available, allowing for better tracking of published builds.
This change improves the OTA update process by providing clearer
versioning information in the embedded firmware string.
Introduced a radio watchdog that detects when the LoRa radio is stuck in RX mode without activity. The firmware can now idle and restart the radio after a configured silence interval, helping MQTT observers recover from missed interrupts. Added CLI commands to get and set the watchdog interval, with a default of 5 minutes. Updated documentation to include usage examples and configuration notes.
- Changed the connection details for the `meshat.se` preset from MQTT over TLS to WSS with JWT authentication.
- Updated related documentation to reflect the new connection method and authentication type.
- Removed fixed username/password mention for the `meshat.se` preset in the documentation.
- Introduced a new MQTT preset for NZ Analyzer, including its connection details and JWT authentication.
- Updated documentation to reflect the addition of the NZ Analyzer preset in the slot commands.
- Increased the MQTT preset count to accommodate the new entry.
- Introduced a new MQTT preset for INW Mesh with username/password authentication.
- Updated documentation to include the new preset and its configuration requirements.
- Enhanced credential handling in MQTTBridge to support dynamic username/password assignment for presets requiring per-slot credentials.
- Improved feedback in CLI for users to set credentials when needed.
- Introduced two new MQTT presets: `meshcore-ca-1` and `meshcore-ca-2`.
- Updated documentation to reflect compile-time configuration options for MQTT defaults, including slot presets, IATA, and timezone settings.
- Removed legacy default MQTT settings from the code, now relying on the new configuration approach.
- Added guidance for using a single active WSS broker preset in the T-LoRa V2.1–1.6 MQTT observer environments, clarifying that dual TLS sessions may fail due to memory constraints.
- Updated partition table descriptions to reflect the new recommendation for the `LilyGo_TLora_V2_1_1_6_repeater_observer_mqtt` and `LilyGo_TLora_V2_1_1_6_room_server_observer_mqtt` targets.
- Enhanced platformio.ini comments to emphasize the single broker usage and its implications for memory management.
Updated the alert PSK implementation to remove base64 support, now requiring a 32-character hex format for private channel secrets. Adjusted related CLI commands, error messages, and internal handling to ensure consistency with the new format. This change enhances clarity and aligns with the mobile app's "Share Channel" output. Relevant updates made across multiple files, including documentation and preference handling.
Enhanced the CLI command for setting the alert PSK to support both base64 (24/44 chars) and hex (32/64 chars) formats. This change allows for greater flexibility in key input, accommodating the mobile app's "share" output. Updated error messages for clarity and ensured that previously derived hashtags are cleared when a new PSK is set. Relevant adjustments made in CommonCLI and MQTT implementation documentation.
Added support for `LilyGo_TBeam_1W_repeater_observer_mqtt` and `LilyGo_TBeam_1W_room_server_observer_mqtt` in the MQTT implementation documentation. Updated partition table references to `default_16MB.csv` for these observers, clarifying the requirements for flashing firmware and the implications for stored settings. Enhanced the platformio configuration for both observer types, including build flags and dependencies.
Updated the fault alert functionality to include an optional region name for scoping alert floods, allowing operators to override the default scope. Introduced a list of banned channels (e.g., Public PSK, `#test`, `#bot`) to prevent spamming community channels with alerts. The implementation ensures that alerts are only sent to private PSKs or non-banned hashtags. Relevant changes were made across multiple files, including updates to the CLI for setting and retrieving the new `alert.region` preference.
agessaman