docs: publish the system design and architecture decision records

DESIGN.md is the reconciled high-level design after the 2026-09 restructure:
system context, roles, build-time composition, layers and dependency rules
(with the known exceptions), threading and event loop, end-to-end flows,
cross-cutting concerns and the upstream alignment policy. docs/adr/ holds the
nine decision records (ESP32 boot layout, S3 native USB, overlay precedence,
BLE privacy, Zephyr main pin, SX126x duty cycle and RX-busy gating, adaptive
contention window, storage backend, WiFi companion boards).
ARCHITECTURE.md stays the detailed component reference and links both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
liquidraverandClaude Opus 5.5 committed 2026-09-25 22:01:28 +02:00
1 parent 159c086485
commit 9fc1eaf18a
12 files changed
+707

No files matched your search

+1
View File
@@ -1,6 +1,7 @@
# ZephCore Architecture Guide
> Comprehensive developer reference for the ZephCore codebase — a Zephyr RTOS port of the Arduino MeshCore LoRa mesh networking firmware.
> For the system view (layers, rules, threading, upstream policy) start with [DESIGN.md](DESIGN.md); decisions are in [adr/](adr/README.md).
---
+363
View File
@@ -0,0 +1,363 @@
# ZephCore — System Design
> Version 1.0, 2026-09-25: the outcome of the 2026-09 restructure (twelve slices over build, core, radio,
> companion, server roles, event loops, connectivity, storage, peripherals, UI and the repository).
> This is the system view: what ZephCore is made of and the rules that keep it coherent. Component detail is in
> [ARCHITECTURE.md](ARCHITECTURE.md); decisions and their history are in [adr/](adr/README.md); the CLI is in
> [Repeater_CLI_commands.md](Repeater_CLI_commands.md). Where a **rule** is stated, known exceptions are listed
> in §5.3.
---
## 1. Purpose and scope
ZephCore is a Zephyr RTOS port of [Arduino MeshCore](https://github.com/meshcore-dev/MeshCore): LoRa mesh firmware
that interoperates on-air and app-side with stock MeshCore nodes, while running on Zephyr's driver model across
multiple MCU families.
### 1.1 Goals (in priority order)
1. **Interoperability** — byte-compatible with Arduino MeshCore on the air and on the companion protocol; prefs
stored in upstream's `prefs.json` format.
2. **Robustness in the field** — a node must never silently go deaf, mute, or lose its identity; every stall has a
bounded recovery (§8.7).
3. **Upstream portability** — most new mesh features arrive from Arduino MeshCore, so code that mirrors upstream
stays structurally identical to it (§9).
4. **Hardware breadth** — one codebase across nRF52840, nRF54L15, the ESP32 family, EFR32MG24, STM32WL and native
Linux, with four radio families.
5. **Resource discipline** — static allocation for the packet path; companion builds are RAM-bound.
### 1.2 Non-goals
- Replacing the MeshCore protocol or phone apps. ZephCore extends behind stable interfaces; it does not fork them.
- A hardware watchdog (decided; §8.7).
- Dynamic plugin loading. All variability is build-time (§4).
---
## 2. System context
```mermaid
flowchart LR
subgraph Node["ZephCore node"]
FW[ZephCore firmware]
end
Mesh(("LoRa mesh<br/>(ZephCore + Arduino MeshCore nodes)"))
App["MeshCore phone / desktop app"]
Admin["Operator<br/>(serial CLI / remote admin)"]
Broker["MQTT broker<br/>(observer / uplink)"]
Flasher["Flasher / updater<br/>(UF2, esptool, WiFi OTA, web flasher)"]
HW["GNSS, sensors, RTC,<br/>display, buttons"]
Mesh <-- "MeshCore wire protocol v1" --> FW
App <-- "Companion protocol<br/>BLE / USB CDC / TCP (WiFi) / UART" --> FW
Admin -- "Text CLI<br/>USB, or over the mesh" --> FW
FW -- "MeshCore JSON over MQTT (WiFi)" --> Broker
Flasher -- "firmware image" --> FW
HW <-- "I2C / UART / GPIO / ADC" --> FW
```
| External actor | Interface | Compatibility rule |
|---|---|---|
| Other mesh nodes | MeshCore wire protocol v1 | **Frozen** to upstream. Never change on-air format unilaterally. |
| Phone / desktop app | Companion binary protocol (opcodes, push codes), CayenneLPP telemetry | Upstream is authoritative; ZephCore additions only in unused code space. Telemetry layout as upstream (battery, MCU temperature and GPS on channel 1, one channel per sensor from 2). |
| Operator | Text CLI | Upstream command names, semantics and reply texts are authoritative; additions documented. |
| MQTT broker | MeshCore JSON | Matches community ingest. |
| Flasher | UF2 / MCUboot image / merged `.bin` / WiFi OTA | Per platform ([ADR 0001](adr/0001-esp32-boot-and-flash-layout.md), [0002](adr/0002-esp32s3-native-usb-companion.md)). |
---
## 3. Roles
A **role** is a firmware personality chosen at build time. Exactly one role per image.
| Role | Purpose | Talks to | Arduino counterpart | Persisted root |
|---|---|---|---|---|
| Companion | Phone-paired chat node: contacts, channels, offline queue | App (BLE/USB/TCP/UART) | `examples/companion_radio` | `/lfs` (+ `/ext`) |
| Repeater | Autonomous relay: ACL, regions, neighbours, admin CLI | Operator | `examples/simple_repeater` | `/lfs/repeater/` |
| Room server | Store-and-forward board, per-client sync cursor | Operator + logged-in clients | `examples/simple_room_server` | `/lfs/repeater/` |
| Observer | Listen-only; publishes packets to MQTT (ESP32) | Broker | none (ZephCore-only) | `/lfs/repeater/` |
Roles share the mesh core; they differ in policy. **Switching between the companion and a server role formats the
volume** (the identity is lost unless exported first); repeater, room server and observer share one layout.
---
## 4. Build-time composition
A build is the tuple **(board, role, radio family, UI kind, optional features)**; the board fixes platform, radio
and UI, the user picks role and features.
```mermaid
flowchart LR
B["board string"] --> P["platform = board directory<br/>(cmake/zephcore_board.cmake)"]
P --> C["Kconfig chain<br/>prj.conf → zephcore_common → &lt;platform&gt;_common<br/>→ esp32c3s3_common → board.conf → user extras"]
P --> D["Devicetree<br/>board.overlay → conf-paired overlays → partitions.overlay"]
P --> Z["Zephyr patches at configure<br/>patches/zephyr + zephyr-new drivers"]
R["role / feature confs"] --> C
C --> K["resolved Kconfig"]
D --> K
K --> S["source selection"]
Z --> S
S --> I["image: UF2 / hex / MCUboot-signed / merged bin"]
```
| Dimension | Values | Selected by |
|---|---|---|
| Platform | nRF52840, nRF54L15, ESP32 classic, ESP32-S3/C3/C6, EFR32MG24, STM32WL, native Linux | board directory |
| Radio | SX126x family (default), LR1110, LR2021, SX127x | devicetree / board conf |
| Role | companion (default), repeater, room server, observer | user `EXTRA_CONF_FILE` |
| UI | none, button UI, joystick/keypad UI (companion) | board Kconfig select |
| Features | debug, WiFi companion, WiFi OTA, uplink, packet logging, PM | user confs; some auto-added |
The board list is one manifest per board (`zephcore.yml`), read by `build.sh`, the provider catalog and
`docs/supported_boards.md`.
**Rules**
- R4.1 A board directory declares hardware only (pins, peripherals, partitions, identity), never role policy.
- R4.2 CMake auto-includes a conf only where the combination is otherwise broken (repeater → WiFi OTA on
S3/C-series; S3 companion → native USB; WiFi-capable companion → WiFi companion).
- R4.3 Later layers override earlier ones, for Kconfig and devicetree alike
([ADR 0003](adr/0003-devicetree-overlay-precedence.md)).
- R4.4 Each upstream Zephyr file has exactly one owning patch in `patches/`.
---
## 5. Logical architecture
### 5.1 Layers
```mermaid
flowchart TB
subgraph L5["L5 Composition roots — src/main_companion.cpp, src/server_main_common.cpp, app/main_observer.cpp"]
MAIN["construct objects, wire callbacks, run the event loop"]
end
subgraph L4["L4 Roles and app policy — app/"]
ROLES["CompanionMesh · RepeaterMesh · RoomServerMesh · ObserverMesh · PowerPolicy"]
end
subgraph UI["Presentation — helpers/ui*"]
UIX["display, button UI, joystick UI, buzzer, LEDs"]
end
subgraph L2["L2 Protocol helpers — helpers/ (mirrors Arduino src/helpers)"]
HELP["BaseChatMesh · CommonCLI · ClientACL · RegionMap · ConfigSerializer · SensorManager · MultiSerialInterface · MeshTimeSync"]
end
subgraph L1["L1 Mesh core — src/, include/mesh (mirrors Arduino src/)"]
CORE["Dispatcher · Mesh · Packet · Identity · PacketManager · ContentionTracker<br/>abstract: Radio · MainBoard · RNG · RTCClock · MillisecondClock"]
end
subgraph L3["L3 Adapters — adapters/ (Zephyr implementations of L1/L2 interfaces)"]
AD["radio · ble · usb · transport · datastore · gps · board · clock · rng · sensors · wifi · mqtt · ota"]
end
subgraph L0["L0 Platform — Zephyr + patches + our radio drivers"]
ZE["kernel · drivers · BT · LittleFS · USB · net"]
end
MAIN --> ROLES & UIX & AD
ROLES --> HELP --> CORE
ROLES --> AD
UIX --> HELP
AD -.implements.-> CORE
AD --> ZE
```
| Layer | Directory | Mirrors upstream | May depend on | Must not depend on |
|---|---|---|---|---|
| L0 Platform | `zephyr/`, `patches/` | — | — | anything above |
| L1 Mesh core | `src/` (excl. mains), `include/mesh/` | `src/` | L1 interfaces, `lib/monocypher` | adapters, helpers, roles, UI |
| L2 Protocol helpers | `helpers/` | `src/helpers/` | L1 | roles, UI |
| L3 Adapters | `adapters/` | platform classes (`NRF52Board`, sensor managers, …) | L0, L1/L2 interfaces | roles, UI |
| L4 Roles | `app/` | `examples/*/MyMesh` | L1, L2, adapter interfaces | UI internals |
| Presentation | `helpers/ui*` | `examples/companion_radio/ui-*` | L2, the role-facing UI interface | concrete role classes (exception: §5.3) |
| L5 Composition roots | mains | `examples/*/main.cpp` | everything | — |
### 5.2 Dependency rules
- R5.1 Dependencies point downward. The composition roots are the only place that names concrete adapter types.
- R5.2 L1 and L2 stay structurally aligned with upstream (§9); Zephyr specifics enter through interfaces, never by
editing shared logic inline.
- R5.3 Cross-cutting services (GPS, display, OTA, RTC) reach L2/L4 through narrow interfaces, injected by L5.
- R5.4 The UI talks to the mesh through events in (`ui_notify_*`) and actions out (`ui_mesh_actions`, applied on
the main thread).
- R5.5 No include cycles.
### 5.3 Known exceptions
| # | Exception | Status |
|---|---|---|
| V2 | `helpers/CommonCLI.cpp` includes the GPS manager, `ZephyrBoard`, WiFi OTA and UI headers (it serves `gps sync`, `start dfu`, `start ota`, display and input rotation). Sensors and GPS settings go through upstream's `SensorManager` since 2026-09-25. | Accepted: the remaining includes serve ZephCore-only commands. |
| V3 | `helpers/boot_prefs.h` includes the GPS manager (applies GPS prefs at boot, as upstream's `applyGpsPrefs()`). | Accepted. |
| V4 | The joystick UI calls `CompanionMesh` directly. | **Accepted by decision**: it runs on the main thread, so the calls are safe, and upstream's UITask reads `the_mesh` the same way. One access point (`JoystickUITask::getMesh()`). |
| V5 | The MQTT adapter includes `app/observer_creds.h`. | Open, small. |
### 5.4 Component inventory
| Component | Layer | Responsibility |
|---|---|---|
| Dispatcher, Mesh, Packet, PacketManager | L1 | TX/RX state machine, CAD/LBT gate, duty cycle, routing, dedup, adverts; 32-slot static pool |
| ContentionTracker | L1 (ZephCore) | adaptive flood jitter and reactive backoff ([ADR 0007](adr/0007-adaptive-contention-window.md)) |
| Identity / Utils | L1 | Ed25519/X25519 keys, AES/HMAC helpers |
| `LoRaRadio` + per-family ops tables | L3 | one `mesh::Radio` adapter over SX126x / LR11xx / LR20xx / SX127x drivers; RX-busy gate, noise floor, CAD ([ADR 0006](adr/0006-sx126x-rx-duty-cycle-and-busy-gating.md)) |
| BaseChatMesh, CompanionMesh | L2 / L4 | contacts, channels, messages; companion protocol (one handler per opcode), offline queue, v-contact |
| RepeaterMesh, RoomServerMesh, ObserverMesh | L4 | forwarding policy, ACL, regions, posts, MQTT |
| CommonCLI (+ ClientACL, RegionMap, TransportKeyStore) | L2 | the text CLI for every role, in upstream's shape |
| Companion transports | L3 | BLE, USB/UART, TCP as upstream `BaseSerialInterface`s in one `MultiSerialInterface` (broadcast) |
| WiFi / MQTT / OTA | L3 | ESP32 networking, uplink, HTTP OTA ([ADR 0009](adr/0009-wifi-companion-boards.md)) |
| DataStore (companion, server) | L3 / L4 | identity, `prefs.json` (upstream `ConfigSerializer`), contacts, channels, blobs ([ADR 0008](adr/0008-storage-and-settings-backend.md)) |
| ZephyrBoard, power-off | L3 | battery (cached), LEDs, reboot, bootloader, `powerOff()`, reset/shutdown reasons, crash record |
| GPS manager | L3 | main-thread state machine, module power and configuration (three files) |
| ZephyrSensorManager | L3 | upstream `SensorManager`: telemetry (`querySensors`), the `gps` setting |
| PowerPolicy | L4 | low-battery auto-shutdown and battery alert, every companion |
| UI (display, button UI, joystick UI) | Presentation | local interaction |
---
## 6. Runtime view
### 6.1 Threading model
The mesh is **single-threaded by design**: all mesh state (Dispatcher, Mesh, role objects, packet pool, GPS
state, the joystick UI) is owned by the Zephyr main thread, which blocks in `k_event_wait()` and never polls.
Everything else (radio DIO1 work, BLE host, USB, GNSS parsing, button UI, WiFi) runs in its own context and hands
off to the main thread through events and queues.
```mermaid
sequenceDiagram
participant ISR as Radio DIO1 (ISR → driver work queue)
participant RB as SPSC ring buffer
participant EV as k_event
participant MT as Main thread
participant T as Transport (BLE/USB/TCP)
ISR->>RB: write raw frame
ISR->>EV: post MESH_EVENT_LORA_RX
T->>EV: queue + post MESH_EVENT_BLE_RX
EV-->>MT: k_event_wait unblocks
MT->>RB: drain (Dispatcher::checkRecv)
MT->>MT: Mesh::onRecvPacket → role hooks
MT->>T: writeFrame (every connected interface)
MT->>MT: Dispatcher::checkSend → radio TX
```
**Rules**
- R6.1 Mesh state is touched only on the main thread. Other contexts hand off via `k_msgq`, SPSC ring buffers,
atomics + `k_event`. The GNSS callbacks only validate and snapshot a fix; the fix is delivered on the main thread.
- R6.2 ISR and driver work-queue code never calls into C++ mesh objects except the documented radio callbacks,
which only enqueue and signal.
- R6.3 No busy-wait or unbounded blocking on the main thread.
- R6.4 The radio is RX 24/7; it is never PM-suspended.
### 6.2 Execution contexts
| Context | Stack | Runs |
|---|---|---|
| Main thread | 6 KB companion, 8 KB servers (4 KB observer, 6 KB STM32WL) | role event loop, all mesh state, GPS state machine, joystick UI loop |
| System work queue | 4 KB | timers, BLE/USB callbacks, button UI rendering, GNSS parsing, the hardware-RTC write |
| Radio DIO1 queues | per driver | IRQ decode, RX-busy latch, watchdogs |
| `lora_tx_wait` | 2 KB | TX-done wait (bounded), RX restart |
| `buzzer_wq`, `mqtt_pub_thread`, TCP listener | small / 12 KB / — | melodies; MQTT publish; TCP companion accept |
### 6.3 Main event loop
The main thread waits on `MESH_EVENT_ALL`; the shared bits are in `src/mesh_events.h` (LoRa RX 0, TX done 1,
transport or CLI RX 2, housekeeping/maintenance 3, GPS action 4, TX drain 5; bit 6 free; role bits from 7).
**Scheduling differs per role by design**: the repeater and room server are deadline-driven (the loop arms a
one-shot for the next due item, ~0.07 idle wakes/s), the companion keeps a periodic 5 s housekeeping tick because
it polls external state. Repeater and room server share one loop (`server_main_common.cpp`).
---
## 7. Key end-to-end flows
| Flow | Path |
|---|---|
| Receive and deliver | DIO1 → driver → ring buffer → `Dispatcher::checkRecv` → `Mesh::onRecvPacket` → decrypt → role hook → app frame / offline queue |
| Forward (repeater) | `onRecvPacket` → forwarding policy (disable_fwd, flood_max, region, rate) → path append → delayed retransmit with adaptive jitter |
| Send (companion) | app opcode → `handleCmdFrame` → per-opcode handler → `BaseChatMesh::sendMessage` → encrypt → `checkSend` → CAD/LBT → TX |
| Remote admin | ANON_REQ login → ACL → REQ with CLI text → `CommonCLI` → RESPONSE |
| Telemetry | REQ → permissions (guest: base only; inverse mask honoured) → battery, `sensors.querySensors()`, MCU temperature → CayenneLPP |
| Advert | periodic / on demand → signed advert → flood; receivers verify, update contacts/neighbours, feed MeshTimeSync |
| Persist | dirty contacts/channels flushed lazily; identity, prefs, channels by atomic replace |
---
## 8. Cross-cutting concerns
### 8.1 Memory
- The packet path is **heap-free**: 32-slot static pool, fixed tables.
- Companion RAM is the binding constraint; the levers are `MAX_CONTACTS` and the offline queue size. On PSRAM
ESP32-S3 boards the WiFi heap and the `CompanionMesh` object live in PSRAM.
- R8.1: a new feature states its RAM cost per role.
### 8.2 Time
- Two clocks: monotonic milliseconds (all protocol timers) and wall-clock RTC (timestamps only).
- Invariant: wall-clock set paths are **forward-only**, except the GPS fix and `clkreboot`; a backward step mutes
a node mesh-wide.
- Sources: GPS / manual set > mesh consensus (`MeshTimeSync`, opt-in) > hardware RTC. Every set also writes the
hardware RTC when the board has one.
### 8.3 Power
- The radio never sleeps (RX 24/7; optional RX duty-cycle sniff inside the driver).
- One power-off path (`zephcore_power_off()`: loads off, LoRa in reset, wake button armed, power latch last),
reached by `poweroff`, the UI and the low-battery policy.
- Low-battery policy for every companion (`PowerPolicy`): three low readings 30 s apart, never on external power,
never on a reading under 2000 mV (no battery).
- GPS: duty-cycled; holds the SoC light-sleep lock only while acquiring.
### 8.4 Security
- On-air: Ed25519 identities, X25519 ECDH, AES-128 + truncated HMAC (upstream scheme).
- Companion link: BLE LE Secure Connections with passkey; pairing is reactive, never a proactive SMP request;
privacy per platform ([ADR 0004](adr/0004-ble-privacy.md)).
- Server roles: password login, four permission levels, rate limits on login, anonymous requests and discovery.
- Entropy: hardware RNG with a seeded fallback.
### 8.5 Persistence
- LittleFS `/lfs` (internal) and optional `/ext` (QSPI). BLE bonds in NVS, isolated from LittleFS on nRF52/ESP32.
- Prefs are upstream's `prefs.json` (ZephCore fields under `zc`); the legacy binary layouts are migrated once and
kept for downgrades.
- Atomic replace for identity, prefs and channels; contacts on `/ext` only.
### 8.6 Observability
- Zephyr logging per module; `debug.conf`; RTT on nRF, console on ESP32.
- CLI `get`/`stats-*`, `get pwrmgt.bootreason` (reset cause, shutdown reason, and the last fatal error with its pc).
- Host regression suite (`tests/`, CI with ASan/UBSan) over the production sources.
### 8.7 Fault handling and recovery
- **No hardware watchdog** (deliberate). Every stall-prone state machine owns a bounded software recovery in the
layer that owns the state (radio parked-RX, RX-latch deadline, CAD timeout, BLE TX timeout, advertising
watchdog, parser resyncs).
- R8.2: a new blocking wait or latch ships with its bound and its recovery.
- Fatal errors reboot (`fatal_reboot.c`) and leave a record for the next boot.
---
## 9. Upstream alignment policy
| Class | Meaning | Examples | Policy |
|---|---|---|---|
| **Shared** | ported from Arduino, same responsibility | Mesh, Dispatcher, Packet, Identity, BaseChatMesh, ClientACL, RegionMap, TransportKeyStore, MultiSerialInterface, ConfigSerializer, SensorManager, LPPDataHelpers (listed in `.editorconfig`) | Upstream text verbatim; every ZephCore change fenced `// ZEPHCORE:`, so a diff against upstream shows only intentional divergence. Arduino APIs come from `helpers/compat/`. |
| **Upstream-shaped** | ZephCore files built in upstream's shape | CompanionMesh, RepeaterMesh, RoomServerMesh, CommonCLI | Upstream names, order and replies where upstream has the feature; ZephCore additions marked. |
| **Extension** | ZephCore-only mesh behaviour | ContentionTracker, MeshTimeSync, ObserverMesh, RX-busy gating, PowerPolicy | Own files, attached through hooks. |
| **Platform** | replaces upstream's Arduino/board layer | adapters, mains, drivers, UI | Free to follow Zephyr idioms. |
Where a ZephCore-owned port can be more accurate than upstream at no compatibility cost, it is (the compat
CayenneLPP encoder rounds where the library truncates); upstream bugs found this way are fixed in our copy
(fenced) and reported.
---
## 10. Decisions
Recorded as ADRs in [adr/](adr/README.md): ESP32 boot and flash layout (0001), S3 native-USB companion (0002),
overlay precedence (0003), BLE privacy (0004), the Zephyr `main` pin (0005), SX126x duty cycle and RX-busy gating
(0006), adaptive contention window (0007), storage and settings backend (0008), WiFi companion boards (0009).
Principles recorded in this document rather than as ADRs: the single-threaded mesh (§6.1), the heap-free packet
path (§8.1), no hardware watchdog (§8.7), forward-only wall clock (§8.2), one owner per patched Zephyr file (R4.4),
upstream alignment (§9).
---
## 11. Open structural items
1. `CompanionMesh.cpp` (~3.5k lines) and `CommonCLI.cpp` (~2k) remain the largest files, though now in upstream's
shape (one handler per opcode; upstream command layout).
2. V5 (§5.3).
3. Merging the GPS standby and shutdown pin sequences needs current measurements first.
4. ESP32 reset cause is not reported (`get pwrmgt.bootreason` shows `Unknown`).
@@ -0,0 +1,68 @@
# ADR 0001 — ESP32 boot: simple boot for plain builds, MCUboot for OTA builds and releases
Status: accepted (recorded 2026-09-25)
## Context
**ESP32 flash** — Zephyr uses `CONFIG_ESP_SIMPLE_BOOT` by default (no MCUBoot). The standard build produces a
self-contained `zephyr.bin` that the ESP32 ROM bootloader loads directly from `0x0`. Just `west flash` — no
`--sysbuild`, no ceremony, works on a bare chip:
```bash
west build -b <board>/esp32s3/procpu zephcore --pristine
west flash --esp-device COMX
```
**Exception — WiFi AP OTA** (`wifi_ota.conf`): the HTTP OTA updater writes to MCUBoot slot1 and requires
MCUBoot. When building with `wifi_ota.conf`, add `--sysbuild` and flash once to seat MCUBoot:
```bash
west build -b <board>/esp32s3/procpu zephcore --pristine --sysbuild -- \
-DEXTRA_CONF_FILE="boards/common/wifi_ota.conf"
west flash --esp-device COMX
```
Subsequent OTA-capable builds still need `--sysbuild`; plain builds without `wifi_ota.conf` do not.
**`repeater.conf` pulls `wifi_ota.conf` in by itself on S3/C-series boards, so ESP32 repeater builds always need
`--sysbuild`.** The ESP32 auto-include block in `CMakeLists.txt` appends it whenever the platform is `esp32`, the
build is classified as repeater (from the user conf *names* — `repeater.conf` — or
`-DCONFIG_ZEPHCORE_ROLE_REPEATER`; checked against Kconfig after `find_package`), and the board is not a classic
`/esp32/` one. Without `--sysbuild` the app still comes out built *for* the MCUboot slot
(`CONFIG_BOOTLOADER_MCUBOOT=y`, `CONFIG_FLASH_LOAD_OFFSET=0x10000`) but is never signed: `west flash` writes it to
`0x10000`, leaves `0x0` untouched, and whatever MCUboot is already there rejects the unsigned header. The symptom
is not a build or flash error — both report success and the hash verifies — but the board stops enumerating USB
entirely, and only the BOOT button gets it back. Companion builds on the same boards really are simple-boot
(`CONFIG_ESP_SIMPLE_BOOT=y`, load offset `0x0`), which is why this bites only when switching a board to the
repeater role.
**GitHub Release artifacts diverge from the plain-build default.** `build.sh` always builds S3/C-series boards
(`heltec_wifi_lora32_v3/v4/v43/v4_r8`, `station_g2`, `xiao_esp32s3`, `xiao_esp32c6`, `lilygo_tlora_c6`,
`heltec_wireless_tracker`, `thinknode_m9`) with `--sysbuild`, regardless of `wifi_ota.conf` — `sysbuild.conf` at
the repo root forces `SB_CONFIG_BOOTLOADER_MCUBOOT=y` whenever `--sysbuild` is passed. So the published release
for these boards is always an MCUboot layout (MCUboot @ `0x0`, signed app @ `0x10000`), never the simple-boot
`zephyr.bin`. (The app slot is at `0x10000` — right after the 64 KB MCUboot region, matching the
Arduino/ESP-IDF app offset; it was `0x20000` before 2026-07-09. The flash map lives in per-board
`boards/esp32/<board>/partitions.overlay`, fed to **both** the app and the MCUboot image via
`sysbuild/CMakeLists.txt` so their slots match. Before that split, `board.overlay` reached only the app, so
MCUboot used the upstream base table and looked for OTA images at a different `image-1` address than the app
wrote them to, silently reverting every WiFi-OTA update on the slot-customizing boards.) Only `-merged.bin`
(MCUboot + signed app, esptool `merge-bin`) is published for them — the signed app alone isn't bootable
standalone and isn't shipped (see [GH #42](https://github.com/liquidraver/ZephCore/issues/42): publishing it as a
"plain .bin" caused bricked boards when flashed like classic-ESP32's self-contained image). Classic ESP32 boards
(T-Beam, PICO-D4 — chip name `esp32`, not `esp32s3`/`esp32c*`) still use simple-boot and ship a genuinely
self-contained plain `.bin` at the `0x1000` ROM bootloader offset. Flashing `-merged.bin` via
`west flash`/`esptool write_flash 0x0 <file>` only touches MCUboot + the app slot — it does not reach
`storage_partition`/`lfs_partition` (identity, prefs, contacts, BLE bonds), so it's safe for routine updates as
well as first flash.
## Decision
Plain local builds of S3/C-series companions stay simple-boot; repeater builds and every release build use
MCUboot (`--sysbuild`), with one per-board `partitions.overlay` shared by app and MCUboot.
## Consequences
- A repeater build without `--sysbuild` bricks USB silently (see above): always pass it.
- Releases publish only `-merged.bin` for S3/C-series boards.
- `sysbuild.conf` sets `SB_CONFIG_BOOT_SIGNATURE_TYPE_NONE` (the RSA default overflows MCUboot's 64 KB slot).
@@ -0,0 +1,31 @@
# ADR 0002 — ESP32-S3 companions use native USB; `start dfu` restores USB-Serial-JTAG
Status: accepted (recorded 2026-09-25)
## Context
**ESP32-S3 companions speak the companion protocol over native USB by default** (since 2026-08-28).
`zephcore/CMakeLists.txt` auto-includes `boards/common/esp32s3_usb.conf` for **companion** builds on every S3
board whose `board.overlay` includes `esp32s3_usb_otg.dtsi` — `heltec_wifi_lora32_v4/v43/v4_r8`,
`heltec_wireless_tracker_v2`, `xiao_esp32s3`, `station_g2`, `lilygo_t3s3`, `meshnology_w12`.
Repeater/observer/room-server and **debug** builds are excluded (they keep the port for the console; ESP32 has no
RTT). This is in CMakeLists rather than `build.sh` on purpose, so a locally built companion is the same firmware
as the release. Cost: +14 KB DRAM (worst board lands at 94%), and the port is no longer USB-Serial-JTAG, **so
esptool's auto-reset into download mode does not work** — use `start dfu` or an Arduino-style 1200-baud touch
(see `ZephyrBoard::rebootToBootloader`), or the BOOT button. Both paths set `FORCE_DOWNLOAD_BOOT` **and** clear
the RTC-domain USB PHY mux (`RTC_CNTL_USB_CONF_REG` `SW_HW_USB_PHY_SEL`/`SW_USB_PHY_SEL`), so the ROM comes back
on USB-Serial-JTAG (`303a:1001`) and ordinary `esptool write-flash` works again with its normal auto-reset.
Clearing the mux is **not** optional: Zephyr's DWC2 quirk layer claims the internal PHY for USB OTG at init,
those bits live in the same always-on RTC domain as `FORCE_DOWNLOAD_BOOT` and survive the reboot, so before
2026-09-04 `start dfu` landed in ROM **USB-OTG** download mode (`303a:0009`) instead. That state is flashable
(esptool reports `USB mode: USB-OTG` and loads the stub) but needs `--before no-reset`, and browser flashers
cannot drive it at all because esptool-js special-cases only PID `0x1001`.
## Decision
Native USB CDC for S3 companions (not debug/server builds), selected in CMakeLists; the reboot-to-bootloader path
restores both the download flag and the PHY mux.
## Consequences
Flash an S3 companion with a 1200-baud touch (or `start dfu`), then esptool on the ROM's USB-Serial-JTAG port.
@@ -0,0 +1,20 @@
# ADR 0003 — Devicetree overlays: board.overlay first, conf-paired overlays after it
Status: accepted (recorded 2026-09-25)
## Context
`board.overlay` goes into `EXTRA_DTC_OVERLAY_FILE` **first**, so overlays paired with later confs
(`repeater`/`pm_esp32`/`esp32s3_usb`/`no_display`) override it — matching the conf chain's own "user extras last"
rule. This was the opposite order until 2026-08-28, which silently broke `esp32s3_usb.overlay`'s console reroute
on the boards that choose a console in `board.overlay`, and `pm_esp32.overlay`'s SX1262 NSS sleep-hold on
`heltec_wireless_tracker`. `partitions.overlay` stays last.
## Decision
Overlay order mirrors the conf order: board first, conf-paired overlays next, `partitions.overlay` last.
## Consequences
A conf-paired overlay can always override a board default; a board that must win has to change its board.conf
pairing, not the order.
+24
View File
@@ -0,0 +1,24 @@
# ADR 0004 — BLE privacy: off on nRF/MG24, on for ESP32-S3 (with identity advertising)
Status: accepted (recorded 2026-09-25)
## Context
BLE privacy (`CONFIG_BT_PRIVACY`): **keep disabled on nRF / MG24** (they already advertise the identity address,
so both iOS and Android work). **ESP32-S3 enables privacy** (`boards/common/esp32_common.conf`) because the
Espressif controller's privacy-OFF Secure-Connections path MIC-fails against iOS (HCI disconnect `0x3d` at
encryption start; root cause captured 2026-06-15). Privacy ON keeps the
controller on its working SC path. **Android "connect from app" is preserved despite privacy ON** because
advertising uses `BT_LE_ADV_OPT_USE_IDENTITY` (`adapters/ble/ZephyrBLE.cpp` `start_adv`) — we expose the stable
identity address, never an RPA.
## Decision
Privacy per platform as above; advertising always uses the identity address.
## Consequences
Do **not** drop `USE_IDENTITY` while ESP32 privacy is on, or Android connect-from-app regresses. If Espressif ever
fixes the privacy-OFF controller path, the ESP32 privacy override can be dropped (`USE_IDENTITY` stays harmless
everywhere). Never send an SMP Security Request proactively: pairing is triggered reactively via ATT error 0x05
(Apple §55).
+41
View File
@@ -0,0 +1,41 @@
# ADR 0005 — Track Zephyr `main`, pinned to a commit
Status: accepted (recorded 2026-09-25)
## Context
The Zephyr tree is pinned to a specific commit on **`main`**, not on a release tag. It tracked `v4.4-branch` from
2026-06 until the 2026-09-19 jump back to main. That jump needed a few non-obvious changes; they should not be
undone:
- LoRa API 0.9.0: `lora_recv_duty_cycle_async`, const `lora_modem_config`.
- `uart_irq_update()` returns void. The UART ISRs loop
`for(;;){ uart_irq_update(); if (uart_irq_is_pending() <= 0) break; … }`.
- Ring buffers use `ring_buf_get_ptr`/`ring_buf_consume`; `CONFIG_RING_BUFFER` now means the legacy API and is not set.
- Work queues are named through `k_work_queue_config.name`.
- Every flash partition we define carries `compatible = "zephyr,mapped-partition"`.
- `CONFIG_PSA_CRYPTO=y` is set for BT-off roles.
- `sysbuild.conf` sets `SB_CONFIG_BOOT_SIGNATURE_TYPE_NONE`, because the RSA default makes MCUboot overflow its
64 KB slot without any build error.
- `wio_tracker_l1` is **upstream's** in-tree board (`boards/seeed/wio_tracker_l1`) plus our `board.overlay`. It is
no longer a custom board.
- **nRF 32.768 kHz source is devicetree, not Kconfig**: the split clock drivers read it from the node. A board
with no LF crystal sets `&lfclk { k32src = "rc"; k32src-accuracy-ppm = <…>; };` and may keep
`CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC_CALIBRATION=y`. The old `CONFIG_CLOCK_CONTROL_NRF_K32SRC_RC` / `_XTAL` /
`_<n>PPM` choices are **silently dropped** (a Kconfig warning, no build error; a crystal-less board ends up on a
floating LFXO: BLE drops, timer skew). `promicro_sx1262` and `me25ls02` are the RC boards. nRF54L boards must
also enable `&xo` and `&lfclk`.
Patch `0003` (SX126x) is rebased behaviour-preserving (step A: upstream's own duty-cycle machinery removed in
favour of ours), then reduced (step B, 2026-09-19: upstream's DC command helpers, `duty_cycle` fields and
`sx126x_duty_cycle_stop()` reused; each piece benched). Switching to upstream's `SX126X_STATE_RX_DUTY_CYCLE`
state model was deliberately not done — see the patch preamble.
## Decision
Pin a `main` commit in `west.yml`; advance by the west update procedure (update `revision:`,
`west update`, verify the patches apply with a build).
## Consequences
Every bump re-checks the patch watch list.
@@ -0,0 +1,57 @@
# ADR 0006 — SX126x: RX duty cycle from datasheet limits; four-layer RX-busy gating
Status: accepted (recorded 2026-09-25; code in `patches/zephyr/0003-lora-sx126x-native.patch`)
## Context
**RX duty cycle (sniff mode)** — window timing is computed in `LoRaRadio::startReceive()` from primary-source
constraints (SX1261/2 DS §13.1.7 + AN1200.36): deaf time per cycle (sleep +
`hwWakeupTimeUs()` transition) ≤ `(P − 2D − 1)·Tsym` catch budget, and `2R + S ≥ (P + 14)·Tsym` so an early
preamble detect never times out mid-packet. `D = CONFIG_ZEPHCORE_LORA_DC_MIN_SYMBOLS` (8, per AN1200.36).
Drivers accept explicit periods only (no auto-compute). The SX126x patch additionally: pins RX gain (0x08AC) and
TX modulation (0x0889) into the warm-start retention list **once at chip init**, not per DC re-arm (DS §9.6 —
mandatory, else every wake after the first sleep loses 3 dB; the list lives in retention memory and survives
those wakes, so rewriting it per packet only added SPI to the DC deaf window).
`CONFIG_ZEPHCORE_LORA_RETENTION_DEBUG`, enabled by `boards/common/debug.conf`, reads the list and 0x08AC back at
install and after every DC packet, before `restart_rx` rewrites the gain, and warns if either was lost. The patch
also sets `StopTimerOnPreamble=1`, and runs a parked-RX watchdog (two-strike, sampling period
`2×(preamble+8)` symbols floored at 250 ms) that re-arms the cycle after a false preamble detect parks the chip
in RX. Presets whose preamble can't cover the budget (16 symbols at SF≥9, or TCXO transition too large) fall
back to continuous RX. `get dc.restarts` counts false-preamble re-arms.
**RX-busy gating (TX-during-RX prevention)** lives entirely in the SX126x patch and works in four layers:
1. **`rx_packet_active` latch** on `struct sx126x_data` — set by the work handler when `HEADER_VALID` fires on
DIO1; cleared on `RX_DONE` / `CRC_ERR` / `RX_TX_TIMEOUT` / every RX (re)start / TX-state entry. The latch is
the primary "we are receiving" signal across the entire payload phase. **Bounded by a payload deadline**
(patch 0013, `header_seen_at_ms` + `sx126x_max_payload_ms()`): continuous RX has no symbol timer, so a
`HEADER_VALID` whose packet never completes would otherwise pin the TX gate forever and silently mute the
node. The bound is max-length-packet airtime at the current SF/BW +25% +100 ms, deliberately generous — it is
a stuck-state safety net, not a timing mechanism.
2. **Preamble grace** in `sx126x_is_receiving()` — `PREAMBLE_DETECTED` is masked off DIO1 (would fire on noise)
but visible in the IRQ register. When the poll observes it, a timestamp is recorded; the gate stays busy until
either `HEADER_VALID` promotes the latch or an SF-aware grace `(preamble_len + 8) × 2^SF / BW` ms elapses.
Foreign sync words release after the grace; real packets always promote the latch inside it. The poll path
clears sticky reception bits only to *release* an expired preamble grace or a blown payload deadline (all
three drivers); otherwise it is read-only.
3. **CAD-busy in-driver RX restore** — `sx126x_lora_send_async` accepts entry CAS from both `REST_STATE → TX` and
`RX → TX`, so the LBT branch tracks `was_rx` and has `sx126x_restart_rx` put the chip back in RX before
`-EBUSY` propagates to the C++ caller. LR11xx and LR20xx mirror this in their own `send_async` LBT branches.
`sx126x_lora_recv_async` has an idempotent fast-path so the C++ failure-path `startReceive()` is a no-op when
the driver already restored RX.
4. **CAD-timeout recovery** — `mesh::Radio::recoverRxState()` (default no-op, overridden in `LoRaRadio` as
`hwCancelReceive → _in_recv_mode=0 → _config_cached=false → startReceive`). `Dispatcher::checkSend()`'s 4 s
CAD-timeout block calls it and re-wakes the loop instead of falling through to TX.
The C++ adapter virtual is `hwIsReceiving()` (formerly `hwIsPreambleDetected` — renamed when the latch made the
old name a lie). It calls the family's `*_is_receiving()` through the ops table (`LoRaRadioOps.h`); that function
may clear bits only for those two releases — never a bit a pending DIO1 event still needs.
## Decision
As above; the other radio families mirror layers 3-4.
## Consequences
Any change to preamble length, DC timing or the IRQ handling must keep the two duty-cycle inequalities and the
four gating layers; bench-check with `get dc.restarts` and CAD stats.
@@ -0,0 +1,32 @@
# ADR 0007 — Adaptive contention window instead of Arduino's static tx/rx delays
Status: accepted (recorded 2026-09-25; ZephCore-only)
## Context
Replaces Arduino's static `txdelay`/`rxdelay` with three mechanisms:
- **EMA-based flood delay factor**: scales flood TX retransmit jitter adaptively based on local contention
history. Jitter is double-capped: `min(2000ms, 6·airtime)` for repeaters — the airtime-scaled ceiling keeps
SF7/narrow-BW configs from wasting time in oversized windows, while the absolute cap bounds per-hop latency in
dense areas. Ring buffer holds 24 concurrent tracked retransmits (sized for ~50-neighbor hilltops).
- **Per-dupe reactive backoff**: each heard duplicate of a pending TX packet delays transmission by
`backoff_multiplier * airtime` from NOW. Capped at `min(2000ms, 12·airtime)` total reactive extension per
packet. After the cap, CAD handles channel-busy detection like Arduino.
- **Initial-flood jitter (companion-only)**: companions don't forward floods (when `client_repeat=0`) and use a
fixed `20..150ms` window for originated TX — small enough to feel responsive, large enough to avoid lockstep
collisions with nearby repeaters. Passive flood tracking is disabled in that mode (no EMA warming needed). When
`client_repeat=1`, the EMA is warmed naturally via the forwarding path and `getRetransmitDelay()` falls back to
the same adaptive math as repeaters.
- **Direct packets**: use minimal fixed jitter only (no adaptive scaling).
## Decision
`set backoff.multiplier X` controls per-dupe delay (default 0.2; 0.0 disables reactive backoff). The old CLI
commands (`txdelay`, `rxdelay`, `direct.txdelay`) are accepted for prefs compatibility but ignored; `get txdelay`
shows the adaptive state.
## Consequences
A divergence from upstream by design (memory `acw-reactive-backoff`); ports touching `Mesh`/`Dispatcher` timing
must keep it.
@@ -0,0 +1,30 @@
# ADR 0008 — Storage layout, power-loss writes, BLE bonds in NVS
Status: accepted (recorded 2026-09-25)
## Context
- `/lfs` — internal flash: identity (`_main.id`), prefs (`prefs.json`, upstream's `ConfigSerializer` format;
the legacy binary `new_prefs` is migrated once and kept for downgrades)
- `/ext` — QSPI external flash (when present): contacts, channels, blob LRU
- **Power-loss writes:** identity, prefs, and `channels2` use atomic replace (`.tmp` + `fs_sync` + `fs_rename`,
one implementation: `adapters/datastore/ZephyrFsUtil.c`); `contacts3` only on `/ext` (two copies do not fit a
128 KB `/lfs`); repeater `/lfs/repeater/` identity and prefs use the same pattern. BLE bonds are handled by the
Zephyr settings subsystem, not `ZephyrDataStore`, and the backend is **platform-dependent**:
`zephcore_common.conf` defaults to the file backend at `/lfs/settings`, but `nrf52_common.conf` and
`esp32_common.conf` both override it to `CONFIG_SETTINGS_NVS=y` (`SETTINGS_FILE=n`), putting bonds in a
dedicated `storage_partition` NVS region isolated from the `/lfs` LittleFS volume, so a busy bond store cannot
corrupt user data. The file backend therefore only applies to platforms that do not override it. This moved in
1.16.2; `/lfs/settings` surviving on disk is the pre-1.16.2 marker that
`ZephyrDataStore::hasOldSettingsFile()` tests for.
- The shutdown-reason marker `/lfs/shutdn` (one byte) is written by `adapters/board/zephyr_poweroff.c`.
## Decision
As above: atomic replace for everything that must survive a power cut mid-write; bonds isolated in NVS on
nRF52/ESP32.
## Consequences
A role switch reformats `/lfs` (identity lost unless exported first); a merged-bin ESP32 flash never touches
`storage_partition`/`lfs_partition` (ADR 0001).
+24
View File
@@ -0,0 +1,24 @@
# ADR 0009 — WiFi companion: which boards, at what contact count
Status: accepted (recorded 2026-09-25)
## Context
**WiFi companion** (`CONFIG_ZEPHCORE_COMPANION_WIFI`, `boards/common/wifi_companion.conf`): auto-added to
companion builds of boards whose `zephcore.yml` declares `capabilities: wifi: true`. On the PSRAM ESP32-S3
boards the WiFi heap, network buffers and the `CompanionMesh` object go to PSRAM. Four no-PSRAM boards also
declare it (Heltec V3 and Wireless Tracker at 140/130 contacts, both C6 boards at 280; the offline queue stays
256): each sets `ZEPHCORE_MAX_CONTACTS` in `board.conf` to the largest count that links, and an S3 without PSRAM
also gets bounded WiFi buffers and a measured 72 KB heap (Kconfig, `HEAP_MEM_POOL_IGNORE_MIN`). Tracker V2, C3
and the classic ESP32 boards do not fit WiFi + BLE (DRAM). Upstream's
`set/get wifi.ssid|pwd|enabled`, `set wifi.clear`, `get wifi.status|ip`; applied on reboot; TCP port
`CONFIG_ZEPHCORE_TCP_PORT` (5000, as upstream). The Zephyr ESP32 WiFi stack needs patches `0017`/`0018` and
hal_espressif `0001` to survive traffic bursts (§13).
## Decision
WiFi + BLE on every board that can hold both; fewer contacts on the no-PSRAM ones rather than dropping WiFi.
## Consequences
Watch patches `0017`/`0018` and hal_espressif `0001` on every west update.
+16
View File
@@ -0,0 +1,16 @@
# Architecture decision records
One decision per file: Context / Decision / Consequences / Status. Recorded 2026-09-25 at the end of the
restructure; most describe decisions taken earlier. The system view is in [../DESIGN.md](../DESIGN.md).
| # | Decision |
|---|---|
| [0001](0001-esp32-boot-and-flash-layout.md) | ESP32 boot: simple boot for plain builds, MCUboot for OTA builds and releases |
| [0002](0002-esp32s3-native-usb-companion.md) | ESP32-S3 companions use native USB; `start dfu` restores USB-Serial-JTAG |
| [0003](0003-devicetree-overlay-precedence.md) | Devicetree overlays: board.overlay first, conf-paired after it |
| [0004](0004-ble-privacy.md) | BLE privacy off on nRF/MG24, on for ESP32-S3 (identity advertising) |
| [0005](0005-zephyr-main-pin.md) | Track Zephyr `main`, pinned; the migration changes not to undo |
| [0006](0006-sx126x-rx-duty-cycle-and-busy-gating.md) | SX126x RX duty cycle and four-layer RX-busy gating |
| [0007](0007-adaptive-contention-window.md) | Adaptive contention window instead of static tx/rx delays |
| [0008](0008-storage-and-settings-backend.md) | Storage layout, power-loss writes, BLE bonds in NVS |
| [0009](0009-wifi-companion-boards.md) | WiFi companion boards and contact counts |