HaloKeymind/RESTORE_UPSTREAM_NOTES.md

# Restoring accidentally-reverted upstream features

## Background

On 2026-03-20, commit `22eb9b87` — *Revert "Merge remote-tracking branch 'origin/dev' into mqtt-bridge-implementation"* — reverted an entire upstream merge to escape a bad merge state, deleting 860 lines across 66 files. That was not intentional feature removal; it wholesale dropped a batch of upstream progress. When upstream was later re-merged, some collateral came back (MicroNMEA `claim()/release()`, the GAT562 board) but several upstream features were never reconciled and remained missing.

This branch restores them. They are **pure upstream code the fork accidentally dropped**, so restoring re-aligns the fork with upstream and *reduces* the future merge-conflict surface rather than adding to it.

## Phase 1 — duty-cycle enforcement (done on this branch)

Restored the token-bucket duty-cycle enforcement and its cluster:

- `src/Dispatcher.{h,cpp}` — restored upstream's `updateTxBudget()`, `tx_budget_ms`,
  `duty_cycle_window_ms`, `getRemainingTxBudget()`, `getDutyCycleWindowMs()`, and the
  windowed `checkSend()`/`loop()` budget logic. The fork's MQTT **radio watchdog** was
  re-applied on top as pure additions (behind `#ifdef WITH_MQTT_BRIDGE`).
- `src/helpers/StaticPoolPacketManager.{h,cpp}` — restored `getOutboundTotal()` and the
  `0xFFFFFFFF` count-all sentinel in `countBefore()`.
- `src/helpers/StatsFormatHelper.h` — restored the `getOutboundTotal()` call; kept the
  fork's `formatRadioDiag` template.

Net effect: these files now diverge from upstream by **watchdog additions only (0 deletions)**
instead of rewriting upstream's TX logic.

### Why this is safe (no reinterpretation of stored settings)

`getAirtimeBudgetFactor()` changed *meaning* between fork and upstream, but the resulting
duty cycle is the **same formula**:
- Fork: `next_tx = t · factor` → duty ≈ `1/(1+factor)`
- Upstream: `duty = 1/(1+factor)` enforced over a rolling window

So a device's stored `airtime_factor` (a `NodePrefs` field, unchanged) keeps its meaning.
The only behavioral difference is upstream enforces it over a 1-hour window (allowing
short bursts) instead of rigid per-packet spacing — a strict improvement, and the
mechanism that keeps EU 868 MHz nodes under the legally-mandated duty cycle.

### Must be validated on-device before merge

This touches core TX timing. Confirm on hardware — **all device-confirmed 2026-07-09/10
on a Heltec V4.2 (busy live mesh + off-frequency bench):**
- ✅ Normal traffic still flows (repeater forwards, observer uplinks).
- ✅ Sustained TX is throttled to the configured duty cycle: at `set dutycycle 1` on a
  busy mesh, TX pinned to ~1% while MQTT capture continued at full rate (after the
  `RxReservePacketManager` fixes below). Remote admin remains usable under throttle
  with the priority-shed + stale-expiry policy.
- ✅ CAD enabled under load: TX, capture, and CLI all normal.
- ✅ The observer radio watchdog fires and recovers non-destructively: with
  `radio.watchdog 1` on a silent frequency, `err_flags` bit 8 set, radio remained in
  RX state through repeated recovery cycles, and packets were still received after.
  Note: firings are invisible on release builds (no MESH_DEBUG) — check
  `stats-radio-diag` `err_flags`; the watchdog arms only after first radio activity
  (`last_active > 0`), so a radio wedged from boot is deliberately not covered.

Remaining before undraft: passive soak at fleet-default `airtime_factor` (duty
convergence, flat heap, adverts still advertised).

### Known interaction: throttling starves MQTT capture (mitigated)

Found during the on-device duty-cycle load test (`set dutycycle 1`): queued
retransmissions hold static-pool packets with no expiry, so heavy throttling parks the
whole pool in the send queue. `Dispatcher::checkRecv()` then drops received packets
before `logRx()` runs, capping MQTT capture at the TX rate (each TX frees one packet
for one RX). Parked repeats also absorb every budget refill, starving the node's own
CLI responses — device-confirmed: a 1%-duty node on a busy mesh became
un-administrable within ~2 minutes. This is inherent upstream behavior — the old
fork's `next_tx` spacing had the same steady-state drain — but it defeats the
observer's purpose. Mitigated on observer builds by `RxReservePacketManager`
(`src/helpers/RxReservePacketManager.h`): priority-aware shedding below a pool
reserve (own responses/ACKs stay queueable) plus 30 s expiry of stale queued
outbound. See MQTT_INTERNALS.md "Capture vs. duty-cycle throttling".

## Phase 2 — CAD and FEM RX gain (NOT done; needs care + device testing)

Still missing at HEAD, also dropped by `22eb9b87`, still present upstream:

- **`cad_enabled`** — hardware Channel Activity Detection (listen-before-talk) before TX.
  The `Dispatcher`/`Radio` interface (`setCADEnabled`/`getCADEnabled`) is restored by
  Phase 1, so CAD currently stays **off by default** (unchanged behavior). To make it
  configurable again, restore:
  - `NodePrefs.cad_enabled` field, its CLI get/set, and its persistence in
    `CommonCLI.cpp` (`loadPrefsInt`/`savePrefs`). **Offset care:** this branch's
    `/com_prefs` layout is carefully managed — add `cad_enabled` following the same
    append-and-size-guard pattern used for `rx_boosted_gain`/`flood_max_*`, and add a
    host-side round-trip test (see `scratchpad/migtest`).
  - The `MyMesh::getCADEnabled()` override returning `_prefs.cad_enabled`.
  - `RadioLibWrappers::setCADEnabled()` override so the hardware CAD is actually driven
    (the fork's wrapper currently doesn't override it).
- **`radio_fem_rxgain`** — LoRa front-end-module RX gain. Restore `NodePrefs.radio_fem_rxgain`
  + CLI + persistence (same offset care), plus the per-board FEM wiring reverted across
  ~20 `variants/*/target.cpp` and the `heltec_tracker_v2/LoRaFEMControl.{cpp,h}` files.
  This is board-specific and only affects FEM-equipped hardware.

Phase 2 is lower urgency than duty-cycle enforcement (CAD/FEM are capability gaps, not a
compliance regression) and is best done as its own change with per-board hardware testing.