next_check and next_gps_update stored a future millis() value in a signed
long and compared with a naive '>'. After the ~24.8-day millis() sign flip
the deadline sits above the wrapped millis(), so the block never runs again
and GPS->RTC time-sync (and the location cache refresh) stall permanently
until reboot. Switch to unsigned deadlines with the wrap-safe signed-
difference compare '(long)(millis() - deadline) > 0', matching the idiom in
Dispatcher::millisHasNowPassed.
Also: reorder the MicroNMEALocationProvider ctor init-list to declaration
order (silences -Wreorder) and drop the always-true 'if (_claims > 0)' guard
in claim() (claim() always runs after _claims++, so it is >= 1).
Integrates Mike Carper's transmission-reliability work (direct retries with
SNR-adaptive backoff and CR, flood retry controls, alt-path replies, flood
channel gates) plus its upstream/dev base (as of 2026-07-06) on top of the
observer stack. The flex branch is untouched; this branch is the experimental
integration line.
Conflict-resolution decisions, for future re-syncs (git rerere is enabled and
has recorded these):
- NodePrefs adopts keymind/upstream member order + retry/flood tail. Member
order is in-memory only: /com_prefs stays field-by-field with the same
canonical file order both sides already share through offset 294; keymind
appends the retry tail at 295+ (new canonical size 676). Flex fleet files
load unchanged; retry fields default via direct_retry_prefs_magic.
- loadPrefsInt keeps the fork's legacy MQTT-gap recovery but moves its
detection boundary from 'extra > 5' to 'extra > LEGACY_MQTT_GAP_3SLOT (864)'
so keymind-size tails take the normal read path. COM_PREFS_TAIL_BYTES
retired.
- Dispatcher/RadioLibWrappers: fork watchdog additions + keymind TX overrides
and CAD busy counter are unioned; RxReservePacketManager::queueOutbound
follows upstream's new bool return (false when shedding).
- CommonCLI: fork observer dispatch + versioned /mqtt_prefs machinery kept;
keymind CLI (retry/flood/radioat) taken; duplicate CAD/FEM handlers and
sanitise lines deduped (kept keymind placements to minimize future diff).
- MyMesh (repeater/room): fork alerter/OTA/observer wiring kept; keymind
scheduled-radio system replaces the old pending_* temp-radio members;
applySavedRadioParams() replaces raw setParams at startup.
- ESP32Board: fork manifest-OTA + keymind powerOff/deepSleep and stopOTAUpdate
unioned; startOTAUpdate keeps fork's STA-aware IP with keymind's idempotency
guard.
- build.sh taken wholesale from keymind (fleet builds use GH Actions).
Verified: Heltec_v3_repeater_observer_mqtt, Heltec_v3_repeater,
heltec_v4_repeater_observer_mqtt, Heltec_v3_room_server_observer_mqtt build;
host migration harness (13 MQTT + non-MQTT scenarios) passes with updated
expectations (676-byte /com_prefs; odd-size files normalize on save, not
load); native unit tests 13/13.
Device testing at 'set dutycycle 1' on a busy mesh showed the node becoming
un-administrable within ~2 minutes: the shed policy dropped its own CLI
responses along with repeats, and parked retransmissions (which never expire)
absorbed every budget refill.
RxReservePacketManager now sheds by priority below the RX reserve — only
pri > 1 outbound (multi-hop flood repeats, adverts, trace) is refused, so the
node's own responses/ACKs (pri 0) and login/PATH replies (pri 1) still queue;
below an emergency floor (reserve/2) everything is shed to protect capture.
Queued packets untransmitted 30 s past their scheduled time are expired at
dequeue via a pointer-keyed age table (the pool is a fixed set of packets, so
pool_size slots cover every key). Under normal load the queue drains in
milliseconds and neither policy triggers.
Load-testing the restored token bucket at 'set dutycycle 1' showed MQTT
capture dropping to exactly the TX rate. Queued retransmissions hold static-
pool packets with no expiry, so throttling parks the whole pool in the send
queue; Dispatcher::checkRecv() then discards received packets before logRx()
ever feeds the bridge — each completed TX frees exactly one packet for
exactly one more RX.
Observer builds now use RxReservePacketManager (fork-owned header): once the
free pool drops below a quarter of the pool, outbound packets are refused and
freed, so RX allocation and MQTT capture continue at full rate while the node
sheds repeat load it has no TX budget for anyway. Non-observer builds keep
upstream pool behavior via the same factory; StaticPoolPacketManager stays
byte-identical to upstream.
performChannelScan was restored as protected non-virtual but upstream declares
it public virtual — match upstream verbatim so the hunk disappears from the
merge surface. Also replace the bare 'extra > 5' tail threshold with
COM_PREFS_TAIL_BYTES, tied by comment to the trailing writes in savePrefs(),
so the next upstream field append updates one named constant.
The unknown-version path kept defaults at boot but any later savePrefs()
(every CLI set command) rewrote /mqtt_prefs as v1 with defaults, destroying
the newer config after a firmware downgrade. Latch _mqtt_prefs_hold when an
unsupported version is seen and refuse to write while it is set — checked
before the NRF52/STM32 open path, which deletes the file first.
Also pin the frozen legacy /mqtt_prefs layouts (472/1464/2904 bytes + 8-byte
header) with static_asserts so every target build re-verifies the deployed
fleet's file offsets, and null-check _obs in AlertReporter::onLoop.
Completes the FEM RX-gain restoration begun in the CAD/prefs change, which
persisted radio_fem_rxgain but didn't yet drive the hardware. Also dropped
by the 22eb9b87 revert; restored to match upstream.
- MainBoard: setLoRaFemLnaEnabled()/canControlLoRaFemLna()/isLoRaFemLnaEnabled()
virtuals (default: can't control — non-FEM boards report unsupported)
- heltec_v4: board overrides driving loRaFEMControl; LoRaFEMControl gains the
isLNAEnabled() getter (it already tracked lna_enabled and drove the FEM)
- CLI: `set radio.fem.rxgain on/off` / `get radio.fem.rxgain` (guarded by
canControlLoRaFemLna, so it reports "unsupported" on non-FEM boards)
- app startup applies the persisted pref: board.setLoRaFemLnaEnabled(
_prefs.radio_fem_rxgain), beside setRxBoostedGainMode
Default is ON (upstream), so on FEM boards the LNA is enabled after upgrade —
a real reception behavior change to confirm on hardware. The other FEM
variants (heltec_t096/tower_v2/tracker_v2) need the same small board-override
addition; until then `radio.fem.rxgain` reports unsupported there (no
regression — status quo).
Builds: heltec_v4 repeater-observer + room-observer (FEM board), Heltec_v3
repeater (non-FEM, base virtuals no-op). NEEDS on-device validation on a
Heltec V4.
Continues restoring features dropped by the 22eb9b87 revert. Both were
upstream-tested code, not intentional fork removals.
CAD (hardware Channel Activity Detection / listen-before-talk before TX),
fully restored and functional:
- NodePrefs.cad_enabled + `set cad on/off` / `get cad` CLI (default off)
- RadioLibWrapper: _cad_enabled + setCADEnabled() + the scanChannel()/CAD
branch in isChannelActive() (Phase 1 already restored the Dispatcher hook)
- getCADEnabled() overrides in the repeater/room/sensor apps (return the
pref) and companion (always on, matching upstream)
radio_fem_rxgain: the NodePrefs field + /com_prefs persistence are restored
here at upstream's exact offsets (293 fem, 294 cad), which makes /com_prefs
byte-identical to upstream through the tail. The field is persisted and
defaults on (upstream default), but the per-board LNA *driving* + the
`radio.fem.rxgain` CLI are deferred to the FEM-hardware change (they depend
on board methods and want per-board bench testing).
The new-format /com_prefs tail grows from 3 to 5 bytes; the old-format
detection threshold and the host migration harness are updated accordingly
(all scenarios pass, incl. the non-MQTT-build variant).
Builds: Heltec_v3 repeater, repeater-observer, room-observer. (sensor /
plain-room / companion fail only on the pre-existing Timezone.h include
issue, unrelated to these changes.)
Commit 22eb9b87 ("Revert 'Merge remote-tracking branch origin/dev...'")
reverted an entire upstream merge to escape a bad merge state, dropping
860 lines across 66 files. Among the collateral never reconciled on a
later re-merge was eb4fa032's token-bucket duty-cycle enforcement — the
mechanism that keeps nodes under a configured airtime budget (and EU
868 MHz nodes under the legally-mandated duty cycle). The fork had fallen
back to fixed per-packet spacing (getAirtimeBudgetFactor reverted to 2.0),
losing the windowed enforcement.
This was never an intentional design choice, so restoring it re-aligns the
fork with upstream and REDUCES the merge-conflict surface: Dispatcher.{h,cpp}
now diverge from upstream by watchdog additions only (77 insertions, 0
deletions) instead of rewriting checkSend()/loop().
Restored from upstream: updateTxBudget/tx_budget_ms/duty_cycle_window_ms/
getRemainingTxBudget/getDutyCycleWindowMs and the windowed budget logic in
Dispatcher; getOutboundTotal() and the 0xFFFFFFFF count-all sentinel in
StaticPoolPacketManager; the getOutboundTotal() call in StatsFormatHelper.
Re-applied the fork's MQTT radio-watchdog on top as pure additions
(#ifdef WITH_MQTT_BRIDGE), keeping formatRadioDiag.
Stored airtime_factor settings keep their meaning: fork's t*factor spacing
and upstream's 1/(1+factor) windowed budget yield the same steady-state
duty cycle; upstream additionally allows short bursts within the window.
Phase 2 (CAD / radio_fem_rxgain, which touch NodePrefs persistence and
per-board FEM wiring) is documented in RESTORE_UPSTREAM_NOTES.md, not done
here. Builds: Heltec_v3 observer + plain repeater. NEEDS ON-DEVICE
duty-cycle validation before merge.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The /mqtt_prefs format was detected purely by struct size, which is fragile
(size collisions across eras) and forced the vestigial `_legacy_*` fields to
be retained forever to preserve byte offsets. With ~several thousand observer
devices deployed from the rolling `observer-mqtt-latest` flasher, this is the
last safe moment to fix it before the observer-settings split ships.
/mqtt_prefs now leads with an 8-byte MQTTPrefsHeader (magic {0xF5,'M','Q','P'},
version, payload_len) followed by the raw payload. The magic's non-ASCII lead
byte cannot collide with a legacy file (whose payload starts with the
mqtt_origin string), so versioned and headerless files are cleanly separable.
An unrecognized (newer) version leaves the file untouched and falls back to
defaults rather than misreading it.
MQTTPrefs is compacted: the six `_legacy_*` fields are removed. Every deployed
headerless layout — pre-slot (OldMQTTPrefs), 3-slot (ThreeSlotMQTTPrefs), and
the shipped 6-slot flex layout (new Legacy6SlotMQTTPrefs) — is field-copied
into the compact struct and re-saved with the header once, on first boot.
Future fields append to the payload and stay backward compatible.
Verified with a host harness that generates a byte-exact deployed flex
/mqtt_prefs (the flex MQTTPrefs is confirmed identical to Legacy6SlotMQTTPrefs,
2904 bytes) and round-trips it through the new load path, plus pre-slot/3-slot
migration, unknown-version safety, magic/origin collision-safety, and the
combined /com_prefs observer-tail recovery. Observer + room-server + no-MQTT
ESP32 targets build.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The NodePrefs->MQTTPrefs split (2eb41bae) left the promised one-time
migration of the old /com_prefs trailing block unimplemented, so users
upgrading an observer node silently lost SNMP, radio-watchdog, and
fault-alert configuration (alerts reset to off; PSK/hashtag/region wiped).
loadPrefsInt now detects an old-format /com_prefs by its size, skips the
legacy zero-filled MQTT gap (6-slot or 3-slot era), and recovers the
trailing observer block into a LegacyObserverTail (reusing the old
firmware's byte291/292 heuristic and per-field availability guards).
loadMQTTPrefs applies those values when the loaded /mqtt_prefs predates
the appended observer fields, and both files are rewritten once in the
current layout. rx_boosted_gain/flood_max_* are also recovered from the
correct offsets (previously read from inside the old gap and reset).
Verified with a host-side harness that round-trips the real old-firmware
savePrefs (from 2eb41bae^) through the new load path across upgrade,
fresh-install, upstream-format, 3-slot-era, truncated, and legacy-variant
cases, plus a non-MQTT-build variant. Updates the stale migration comments
and documents the CommonCLI_Observer seam in MQTT_IMPLEMENTATION.md.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mikecarper