[P1] A failed setup no longer strands the slot. setupSlot() returns bool and the
startup loops count only successful activations, so a slot that fails on a client
allocation neither consumes an active-slot position (starving a later healthy
broker on capped hardware) nor sits dead forever: maintainSlotConnections()
previously skipped clientless slots and the reconnect ladder is gated on
initial_connect_done, so nothing retried it. It now retries an enabled but
unactivated slot on a 60 s timer, one per cycle, gated on the same
_slots_setup_done ordering so the NTP-deferred setup sequence is preserved.
[P2] JWT setup no longer proceeds without a usable token. Both the preset and
custom-audience paths returned after ignoring createSlotAuthToken()'s result,
then called connect() and latched initial_connect_done -- so the token-allocation
failure introduced by the previous commit produced an unauthenticated attempt
exactly when memory was exhausted. They now return false and let the retry path
handle it.
[P2] ensureSlotAuthToken() no longer clears an existing token. It cleared
unconditionally, so every renewal wiped the current token before JWTHelper ran;
a renewal that then failed left an empty password where the inline buffer used to
preserve working credentials (JWTHelper writes only on success). Only freshly
allocated buffers are initialised now.
[P2] Raw publications reuse the shared document. buildRawJSON() reached
MQTTPayloadBuilder::buildRawMessage(), which constructed its own default
JsonDocument and therefore malloc'd and freed an internal-heap variant pool per
message -- on the highest-rate topic. The document is threaded through both
builders and the bridge passes _json_scratch_doc.
[P3] The writeTo() guard validates the source fields, not just the destination.
A corrupt payload_len of MAX_PACKET_PAYLOAD + 1 still leaves getRawLength()
inside MAX_TRANS_UNIT, so writeTo() read past packet->payload. Sizing and
validation moved to a pure MQTTWireScratch header with host tests covering the
accept/reject edges, matching the MQTTPacketFilter/MQTTConnectionPolicy pattern.
Two findings fell out: MAX_PATH_SIZE one-byte hops is not encodable (the hop
count is 6 bits, so 64 & 63 == 0; 32 two-byte hops is the widest real path), and
a zero-payload packet serializes but does not survive readFrom() -- pinned as a
test because it constrains any future wire-only queue.
[P3] Corrected the pool-size comment: these targets are 32-bit, so
ARDUINOJSON_SLOT_ID_SIZE is 2 and a pool block is 128 slots / 1024 bytes, not
4096. The 4096 figure came from a pre-existing comment near
NEIGHBORS_DOC_POOL_BUDGET, which is left alone -- its byte measurements are
empirical and still stand, only the block-size attribution is wrong.
Activation is now centralized in activatedSlotCount()/canActivateSlot(), used by
both startup loops, the retry path, and applySlotPreset(). That closes the
pre-existing divergence where a live preset change called setupSlot() without
consulting _max_active_slots, letting a non-PSRAM board reach three concurrent
TLS sessions against a cap of two. BEHAVIOUR CHANGE: a reconfigure that would
exceed the cap now logs and leaves the slot inactive instead of connecting.
Reconfiguring an already-active slot still works, because teardownSlot() releases
its position first.
272/272 native tests pass (5 new); both observer envs and an nRF52 repeater build
clean. Flash 1593249 B non-PSRAM, 1555625 B PSRAM.
Five independent, behavior-preserving reductions. Measured on
Heltec_v3_repeater_observer_mqtt (non-PSRAM) and
ThinkNode_M7_repeater_observer_mqtt (PSRAM), 267/267 native tests green.
Drop the unused static-task bookkeeping. StaticTask_t _mqtt_task_tcb (344 B)
and StackType_t* _mqtt_task_stack were never used: there is no
xTaskCreateStatic call, the pointer was assigned nullptr immediately before
xTaskCreatePinnedToCore, and the two psram_free() calls on it were dead.
Size the wire-format scratch buffers from the protocol maximum. raw_hex[1024]
at three sites becomes 2*MAX_TRANS_UNIT+1, and raw_buf/reconstructed[512]
become MAX_TRANS_UNIT. Both writeTo() sites now check getRawLength() first:
writeTo() does not bounds-check and returns uint8_t, so the old 512-byte
buffers were the only thing absorbing a malformed payload_len, and the
post-hoc "raw_len > sizeof(buf)" test ran after the overrun.
Pass the already-known serialized length into publishToSlot() instead of
re-running strlen() per destination slot (up to 2 KB per packet per slot, and
NEIGHBORS_JSON_BUFFER_SIZE per neighbor snapshot). Same for the direct
publish in publishStatusToSlot().
Share one JSON buffer and one document across packet, raw, and status. All
publish paths serialize on the bridge task, so the separate status buffer and
document were never concurrent. Status keeps STATUS_JSON_BUFFER_SIZE as its
serialization ceiling, so which oversized status documents get dropped is
unchanged.
Route the document's pools through a PSRAM-preferring allocator. Under
ArduinoJson 7 StaticJsonDocument<N> is a deprecated empty subclass of
JsonDocument whose template argument only feeds capacity(); the object is
64 B and each pool block (4096 B here) came from plain malloc(), i.e. the
internal DRAM the mbedTLS working set needs. Mirrors NeighborsDocAllocator.
The old comment claiming an inline pool has been corrected.
Measured:
sizeof(MQTTBridge) non-PSRAM 13208 -> 12032 B (-1176, internal heap)
PSRAM 10492 -> 10080 B (-412, plus one fewer
768 B PSRAM allocation)
stack, non-PSRAM buildPacketJSON[FromRaw] 1264 -> 736 B
buildRawJSON 1136 -> 608 B
publishPacket 688 -> ~432 B
packetToHex 560 -> ~304 B
deepest publish chain ~2.6 -> ~1.8 KB of 8 KB
flash 1592513 -> 1592573 B (+60)
static RAM 74656 B unchanged -- MQTTBridge is heap-allocated, so
these savings are internal heap, not the linker figure
Deferred from the review: the QueuedPacket wire-only redesign (reward is
1.56 KB on non-PSRAM only, and Packet::readFrom() rejects payload_len == 0),
demand-driven slot clients/JWT tokens, and pool retention across publishes --
JsonDocument::to<T>() always calls clear(), which destroys pools, so
"retain pools by clearing the root object" needs a string-pool lifetime
analysis first.
Three fork-introduced dependencies had leaked into non-ESP32 builds, breaking
every nRF52 and RP2040 target since 7e4f75c9 (2026-04-10):
- The `memory` CLI command called ESP.getFreeHeap()/heap_caps_* unguarded from
shared CommonCLI.cpp. ESP32 output is unchanged; other platforms now report
newlib arena stats, omitting min-ever-free and largest-free-block rather than
substituting numbers that mean something different.
- The vendored PsychicMqttClient (ESP-IDF esp-mqtt) was pulled in by the LDF on
nRF52; nrf52_base now lib_ignores it, matching how RP2040 variants ignore BLE.
- JWTHelper.cpp and MQTTMessageBuilder.cpp are excluded in arduino_base, but 22
variants re-glob helpers/*.cpp after inheriting it, which undoes the exclusion.
Guarding the file contents on WITH_MQTT_BRIDGE is robust against any variant's
filter, and matches helpers/esp32/WebConfigServer.cpp.
Verified: RAK4631 repeater + room server, Heltec T114, T1000-E, Wio WM1110,
Xiao nRF52 and ThinkNode M1 all build. ESP32 observer builds are byte-identical
to before (RAM and flash), and the native suite stays at 267/267.
RP2040 still fails separately: four boards declare the pre-force_ap
startOTAUpdate signature. Not addressed here.
Enhance the neighbor discovery JSON structure by introducing a
default_scope field, which indicates the region name this node
floods to by default. This change improves clarity in the
neighbor discovery process and aligns with the unscoped flood
behavior when no default region is set. Updates include
modifications to the MyMesh class and related message building
functions to accommodate the new field.
Introduce new methods to manage neighbor discovery JSON budget and
entries in MyMesh. This includes tracking the number of queried and
published neighbors, measuring JSON sizes, and handling truncation
when the buffer limit is reached. These improvements optimize the
neighbor discovery process and ensure efficient JSON message
construction for MQTT communications.
Add checks to prevent null or zero-length hex buffers in the
bytesToHex and packetToHex functions. This guarantees that an
empty string is returned instead of an uninitialized buffer,
improving the safety of JSON serialization.
Add buildNeighborsMessage to the pure MQTTPayloadBuilder core and a thin
delegating wrapper + NeighborsMessageEntry alias on MQTTMessageBuilder, so
the neighbors topic is built by the same firmware-facing API as status/
packet/raw while the layout logic stays exercisable by native tests.
The document is bounded to the publish buffer: entries arrive ordered most-
to least-useful and the tail is dropped once the next entry would overflow,
so a fixed PSRAM buffer can never be handed truncated JSON.
Uses ArduinoJson v7 idioms (.to<JsonObject>()/.add<JsonObject>()) to stay
warning-clean under -Werror, unlike the deprecated createNested* forms.
Adds three test_mqtt_payload_builder cases: self+entry round-trip, empty
table / null scopes, and bounded-growth tail-drop under a tight buffer.
Replace direct JSON construction in MQTTMessageBuilder with calls to
MQTTPayloadBuilder for building status, packet, and raw messages.
This change improves code maintainability and reduces duplication by
centralizing message formatting logic. Additionally, update platformio.ini
to include ArduinoJson dependency for JSON handling.
Remove unused MQTTMessageBuilder members (getPacketTypeString,
formatTimestamp/Time/Date stubs, JSON_BUFFER_SIZE constant) for a
small flash saving with no behavior change.
Replace the per-byte snprintf("%02X") in bytesToHex with a nibble
lookup table, avoiding a format-string parse up to ~512x per publish
on the MQTT task. Output is byte-for-byte identical uppercase hex.
Updated the MQTTMessageBuilder to include microsecond precision in the
timestamp formatting. The formatIsoTimestampForMqtt function now accepts
a microsecond parameter, allowing for more accurate time representation.
Modified related functions in MQTTBridge to utilize the new timestamp
formatting, ensuring consistency across MQTT messages.
- Introduced a new method `formatIsoTimestampForMqtt` to centralize ISO-like timestamp formatting for MQTT messages, applying timezone preferences.
- Updated `buildStatusMessage`, `buildPacketJSON`, `buildPacketJSONFromRaw`, and `buildRawJSON` methods to utilize the new timestamp formatting function, improving code clarity and reducing redundancy.
- Adjusted `publishStatusToSlot` and `publishStatus` in MQTTBridge to use the new timestamp formatting method, ensuring consistent timestamp handling across status messages.
Updated the MQTT message structure to add a new field for repeat status, allowing the indication of forwarding status as "on" or "off". This change includes modifications to the MQTTMessageBuilder and related documentation to reflect the new parameter. Additionally, updated CLI command documentation to clarify flooding behavior in different firmware versions.
Reintroduced the 'origin' field in the buildPacketMessage function to its original position within the JSON object. This adjustment ensures consistency in the message format and aligns with previous structural changes made to enhance clarity.
Updated the buildPacketMessage function to improve the structure of the JSON object being built. The 'origin_id' field has been moved to a later position, and the 'hash' field has been added to enhance the packet's metadata. This change aims to streamline the message format for better clarity and consistency.
- Introduced `getPacketsRecvErrors()` method in `Dispatcher` to track receive errors.
- Updated `buildStatusMessage()` in `MQTTMessageBuilder` to include `recv_errors` parameter.
- Modified `publishStatus()` in `MQTTBridge` to retrieve and send receive error statistics.
- Adjusted `RadioLibWrappers` to override `getPacketsRecvErrors()` for consistency.
- Added support for automatic stats collection in MQTTBridge, allowing for detailed status messages including battery voltage, uptime, error flags, and air time metrics.
- Updated MQTTMessageBuilder to accommodate additional stats in the status message, increasing buffer size for JSON documents.
- Modified CommonCLI to display MQTT status interval in minutes and adjusted command handling for setting the interval.
- Introduced new methods in MQTTBridge for setting stats sources
- Implemented size-adaptive JSON document allocation in MQTTMessageBuilder to reduce memory fragmentation.
- Increased buffer sizes for raw hex conversions to accommodate larger packets.
- Updated MQTTBridge to store raw radio data with each packet, improving data integrity during transmission.
- Enhanced packet queue management by clearing structures to prevent stale data and ensuring safe memory handling.
- Added debug logging for packet processing to aid in troubleshooting.
- Introduce timezone settings in NodePrefs for persistent storage
- Update CLI commands to get/set timezone and offset
- Modify MQTT message builder to utilize timezone for accurate timestamps
- Implement timezone handling in MQTTBridge for local time conversion
- Include timezone library dependency in platformio.ini
- Added support for automatic stats collection in MQTTBridge, allowing for detailed status messages including battery voltage, uptime, error flags, and air time metrics.
- Updated MQTTMessageBuilder to accommodate additional stats in the status message, increasing buffer size for JSON documents.
- Modified CommonCLI to display MQTT status interval in minutes and adjusted command handling for setting the interval.
- Introduced new methods in MQTTBridge for setting stats sources
- Implemented size-adaptive JSON document allocation in MQTTMessageBuilder to reduce memory fragmentation.
- Increased buffer sizes for raw hex conversions to accommodate larger packets.
- Updated MQTTBridge to store raw radio data with each packet, improving data integrity during transmission.
- Enhanced packet queue management by clearing structures to prevent stale data and ensuring safe memory handling.
- Added debug logging for packet processing to aid in troubleshooting.
- Introduce timezone settings in NodePrefs for persistent storage
- Update CLI commands to get/set timezone and offset
- Modify MQTT message builder to utilize timezone for accurate timestamps
- Implement timezone handling in MQTTBridge for local time conversion
- Include timezone library dependency in platformio.ini
agessaman