ZephCore

dandri/ZephCore

Fork 0

mirror of https://github.com/liquidraver/ZephCore.git synced 2026-10-15 14:19:56 +00:00

Commit Graph

Author SHA1 Message Date

Author	SHA1	Message	Date
liquidraver	d2d786517b	power & storage: ESP32 light sleep that sleeps and wakes; less flash wear ESP32 light sleep (devdocs/lld/13-power-management.md) Light sleep on the S3 repeaters never worked as designed, for two independent reasons: - No S3 repeater ever slept. Every one carries WiFi OTA, the WiFi driver starts WiFi at boot (NULL mode), and a started WiFi library holds its modem PM lock. Measured on the XIAO: zero sleep entries with WiFi compiled in. helpers/pm_esp32_wake.c now stops WiFi at boot on PM builds; `start ota` and a station connect start it again (verified: AP up, sleep blocked for the session, sleeping after `stop ota`). - DIO1 was not a wake source. GPIO_INT_WAKEUP means EXT1 in Zephyr's ESP32 driver: RTC pads only, one trigger polarity for all pins on the S3, and pads left on the RTC mux after every sleep. Since 1.17.1 pm_esp32.overlay also armed the active-low user button; gpio-keys configures first, so the driver refused the active-high DIO1. The new helper arms the radio IRQ line and the user button (sw0) as per-pin level GPIO wakes (ESP-IDF gpio_wakeup_enable, any pad) for the length of each sleep, masks their CPU interrupt while in level mode, and re-raises the interrupt a sleep swallowed through status_w1ts, so the radio's dio1_isr and gpio-keys run as usual. EXT1 is no longer used: patch 0003 configures DIO1 as a plain input again and drops the `d7cc4cb` pad-restore notifier (preamble section rewritten). Non-RTC DIO1 boards (XIAO GPIO39, Station G2 GPIO48) can now light-sleep. Each light sleep is also bounded by the next esp_timer alarm (pm_policy_next_custom_ticks, CONFIG_PM_CUSTOM_TICKS_HOOK), as ESP-IDF does; the BLE controller's modem-sleep wake timer lives there. Light sleep isolates every pad that is not held, RTC or digital, while the radio keeps receiving. Holds (ESP32_GPIO_SLEEP_HOLD_EN, inert without PM) now live in each board's own files: NSS and RESET on all PM-capable boards, VFEM_Ctrl (GPIO7) and Vext on V4/V4.3/V4 R8/WTv2, CPS on V4, RXEN on the XIAO. The shared overlay keeps only &rtc_timer and the GPIO wake source; the wrong RTC_PERIPH explanations are gone. Also: - `powersaving` (upstream's command and replies) gates light sleep; light-sleep builds default to on, and a stored 0 without the new pwr_sav_set marker (the field did nothing before) becomes on once. `powersaving off` persists and keeps the node (and its USB) awake. - `get pm`: asleep %, sleeps, wake causes, per-pin wake counts. - The console window reopens on every user-button press. - XIAO ESP32-S3: GPIO21 is both the kit's user button and the board's led0; the heartbeat drove it low and every sleep woke as a button press. led0 is deleted, the button gets a pull-up. - Station G2: button active-low with pull-up, as upstream reads it (was active-high). Untested on hardware. Verified on the XIAO ESP32-S3 as a light-sleep repeater on the live mesh (repeat off, 2 dBm) against a T1000-E on the same channel: awake 20 min, ratio of floods heard 0.95; asleep 30.5 min (96.6 % of uptime), 119 vs 119, ratio 1.00. TX: sent == flood + direct. Heltec boards: build-only (no unit on the bench). Companion light sleep stays off by default; the esp_timer bound makes it possible. Spike on the XIAO (pm_esp32.conf by hand, BLE only): a held connection survives, 35 min with a poll every minute and no miss, the SoC asleep ~91 % between connection events, LoRa RX 130 vs ~130 on the T1000-E. Pairing while asleep fails; USB CDC and UI still need locks before companions can ship with it. Flash wear (storage) The companion marked contacts dirty on is_new, but upstream's is_new means "not in contacts[]": auto-add declined the node (type filter, hop limit, full table). Every advert from such a node forced a 5 s full rewrite of an unchanged contacts3, while a contact that really was just added only got the liveness deadline. onAdvertRecv now snapshots the stored record first. is_new never dirties. An addition or a changed name, type or position flushes in 5 s, and a plain re-advert waits the liveness deadline, now 1 h instead of 10 min. A new weak hook, zephcore_persist_before_off(), flushes the deferred writes before every clean reboot and power-off. It is called from zephcore_power_off(), the ZephyrBoard reboot, bootloader and BLE-OTA paths, the UI save-and-restart, the joystick restart and wifi_ota. Off the main thread it posts MESH_EVENT_PERSIST and waits up to 5 s. Before this, only CMD_REBOOT flushed. The repeater saved its ACL on every admin login. It now saves only when the role or secret changed; the secret is the ECDH of two fixed keys, so re-logins store nothing new. The room server likewise skips the lazy write unless the role, sync_since or secret changed. zephcore_prefs_json_save() compares the serialised JSON with the file and skips an identical rewrite. Not tested on hardware. Builds and tests: role_builds.sh 10/10; the ESP32 PM set on the combined tree (five Heltec repeaters, V4 R8, Station G2, XIAO repeater and companion) 9/9, repeaters at 69-71 % dram0; board_manifest check clean; host tests quick 114/114. heltec_companion dram0 at 99.80 % (776 B left, +88 B for both changes together).	2026-09-26 16:49:17 +02:00
liquidraver	38f815a0a1	server roles, event loops, companion transports, storage and prefs.json Server roles (slice 7) - CommonCLI split into upstream's shape; region CLI moved into CommonCLI; ClientACL, RegionMap, TransportKeyStore and both server meshes in upstream shape over an Arduino compat shim (helpers/compat: Arduino.h, Stream/Print, Serial, SHA256, FILESYSTEM/File over Zephyr fs) - room server processAck port; `get acl` prints as upstream - room: default name Room-XXXX, display/input rotation, live radio UI refresh Event loops (slice 7b) - repeater and room server share server_main_common with a ServerRole descriptor; shared event bits in src/mesh_events.h - room server deadline-driven (idle wakes 2.2/s -> 0.08/s); companion loop() runs on housekeeping (closes the lazy contact-flush gap) - `get loop.wakes` in debug builds (CONFIG_ZEPHCORE_LOOP_WAKE_STATS) Companion transports and WiFi (slice 8) - upstream MultiSerialInterface: BLE, USB/UART and TCP are each a BaseSerialInterface, every connected transport gets every frame; CompanionSerial holds a lossless reply; shared frame_txq for BLE/TCP - SerialCompanionTransport and LinuxTCPTransport replaced by TcpCompanionTransport; ZephyrBLE split (gatt layout, DFU) - WiFi companion (upstream wifi.ssid/pwd/enabled CLI, TCP port 5000) on boards declaring `capabilities: wifi` in zephcore.yml: the PSRAM ESP32-S3 boards, plus Heltec V3 (140 contacts), Wireless Tracker (130) and both C6 boards (280); ESP32-S3 without PSRAM gets bounded WiFi buffers and a measured 72 KB heap; PSRAM boards' internal heap 72 KB - TCP queues: own depth (ZEPHCORE_TCP_QUEUE_SIZE=8), RX waits instead of dropping - RAM: companion ClientACL dropped; main/BT-TX/TCP stacks sized from measured peaks (-7 KB nRF companions, -9 KB ESP32 companions) - ESP32 WiFi stopped working under ~200 frames/s unicast. Fixed by: patches/zephyr/0017 never block the WiFi task in the RX callback; patches/zephyr/0018 lock shared_multi_heap (heap corruption with the WiFi heap in PSRAM); patches/modules/hal_espressif/0001 backport of hal_espressif af97fb02df (PSRAM realloc) Storage and prefs (slice 9) - prefs are now prefs.json through upstream's ConfigSerializer (ported verbatim), with upstream's key names for shared fields and ZephCore's own under "zc"; on first boot the binary new_prefs / repeater prefs is migrated and kept, so a downgrade still boots with the old settings - helpers/PrefsCodec: both legacy binary layouts (companion 272 B, server 311 B) as Zephyr-free codecs; sanitizeNodePrefs is the only validator - fix: server prefs could be replaced by a truncated file when a write failed; identity, prefs and channel writes go through one checked atomic replace (ZephyrFsUtil) - fix: a corrupt cad_auto byte turned adaptive CAD off instead of the default on - fix: a deliberate 300 s GPS duty on a repeater was rewritten to 48 h on every boot (that migration now runs on legacy files only) - one identity loader for all roles (IdentityFile); an unusable key file is always parked as .bad before regenerating - server store no longer stamps repeater defaults on room server and observer; companion first-boot volume policy moved into the store (adoptVolume) - ZephyrDataStore 1435 -> ~850 lines	2026-09-25 08:34:21 +02:00

liquidraver

d2d786517b

power & storage: ESP32 light sleep that sleeps and wakes; less flash wear

ESP32 light sleep (devdocs/lld/13-power-management.md)

Light sleep on the S3 repeaters never worked as designed, for two
independent reasons:

- No S3 repeater ever slept. Every one carries WiFi OTA, the WiFi
  driver starts WiFi at boot (NULL mode), and a started WiFi library
  holds its modem PM lock. Measured on the XIAO: zero sleep entries
  with WiFi compiled in. helpers/pm_esp32_wake.c now stops WiFi at
  boot on PM builds; `start ota` and a station connect start it again
  (verified: AP up, sleep blocked for the session, sleeping after
  `stop ota`).
- DIO1 was not a wake source. GPIO_INT_WAKEUP means EXT1 in Zephyr's
  ESP32 driver: RTC pads only, one trigger polarity for all pins on
  the S3, and pads left on the RTC mux after every sleep. Since 1.17.1
  pm_esp32.overlay also armed the active-low user button; gpio-keys
  configures first, so the driver refused the active-high DIO1.

The new helper arms the radio IRQ line and the user button (sw0) as
per-pin level GPIO wakes (ESP-IDF gpio_wakeup_enable, any pad) for the
length of each sleep, masks their CPU interrupt while in level mode,
and re-raises the interrupt a sleep swallowed through status_w1ts, so
the radio's dio1_isr and gpio-keys run as usual. EXT1 is no longer
used: patch 0003 configures DIO1 as a plain input again and drops the
d7cc4cb pad-restore notifier (preamble section rewritten). Non-RTC
DIO1 boards (XIAO GPIO39, Station G2 GPIO48) can now light-sleep.
Each light sleep is also bounded by the next esp_timer alarm
(pm_policy_next_custom_ticks, CONFIG_PM_CUSTOM_TICKS_HOOK), as ESP-IDF
does; the BLE controller's modem-sleep wake timer lives there.

Light sleep isolates every pad that is not held, RTC or digital, while
the radio keeps receiving. Holds (ESP32_GPIO_SLEEP_HOLD_EN, inert
without PM) now live in each board's own files: NSS and RESET on all
PM-capable boards, VFEM_Ctrl (GPIO7) and Vext on V4/V4.3/V4 R8/WTv2,
CPS on V4, RXEN on the XIAO. The shared overlay keeps only &rtc_timer
and the GPIO wake source; the wrong RTC_PERIPH explanations are gone.

Also:
- `powersaving` (upstream's command and replies) gates light sleep;
  light-sleep builds default to on, and a stored 0 without the new
  pwr_sav_set marker (the field did nothing before) becomes on once.
  `powersaving off` persists and keeps the node (and its USB) awake.
- `get pm`: asleep %, sleeps, wake causes, per-pin wake counts.
- The console window reopens on every user-button press.
- XIAO ESP32-S3: GPIO21 is both the kit's user button and the board's
  led0; the heartbeat drove it low and every sleep woke as a button
  press. led0 is deleted, the button gets a pull-up.
- Station G2: button active-low with pull-up, as upstream reads it
  (was active-high). Untested on hardware.

Verified on the XIAO ESP32-S3 as a light-sleep repeater on the live
mesh (repeat off, 2 dBm) against a T1000-E on the same channel: awake
20 min, ratio of floods heard 0.95; asleep 30.5 min (96.6 % of
uptime), 119 vs 119, ratio 1.00. TX: sent == flood + direct. Heltec
boards: build-only (no unit on the bench).

Companion light sleep stays off by default; the esp_timer bound makes
it possible. Spike on the XIAO (pm_esp32.conf by hand, BLE only): a
held connection survives, 35 min with a poll every minute and no
miss, the SoC asleep ~91 % between connection events, LoRa RX 130 vs
~130 on the T1000-E. Pairing while asleep fails; USB CDC and UI still
need locks before companions can ship with it.

Flash wear (storage)

The companion marked contacts dirty on is_new, but upstream's is_new
means "not in contacts[]": auto-add declined the node (type filter,
hop limit, full table). Every advert from such a node forced a 5 s
full rewrite of an unchanged contacts3, while a contact that really
was just added only got the liveness deadline. onAdvertRecv now
snapshots the stored record first. is_new never dirties. An addition
or a changed name, type or position flushes in 5 s, and a plain
re-advert waits the liveness deadline, now 1 h instead of 10 min.

A new weak hook, zephcore_persist_before_off(), flushes the deferred
writes before every clean reboot and power-off. It is called from
zephcore_power_off(), the ZephyrBoard reboot, bootloader and BLE-OTA
paths, the UI save-and-restart, the joystick restart and wifi_ota.
Off the main thread it posts MESH_EVENT_PERSIST and waits up to 5 s.
Before this, only CMD_REBOOT flushed.

The repeater saved its ACL on every admin login. It now saves only
when the role or secret changed; the secret is the ECDH of two fixed
keys, so re-logins store nothing new. The room server likewise skips
the lazy write unless the role, sync_since or secret changed.
zephcore_prefs_json_save() compares the serialised JSON with the file
and skips an identical rewrite. Not tested on hardware.

Builds and tests: role_builds.sh 10/10; the ESP32 PM set on the
combined tree (five Heltec repeaters, V4 R8, Station G2, XIAO repeater
and companion) 9/9, repeaters at 69-71 % dram0; board_manifest check
clean; host tests quick 114/114. heltec_companion dram0 at 99.80 %
(776 B left, +88 B for both changes together).

2026-09-26 16:49:17 +02:00

liquidraver

38f815a0a1

server roles, event loops, companion transports, storage and prefs.json

Server roles (slice 7)
- CommonCLI split into upstream's shape; region CLI moved into CommonCLI;
  ClientACL, RegionMap, TransportKeyStore and both server meshes in
  upstream shape over an Arduino compat shim (helpers/compat: Arduino.h,
  Stream/Print, Serial, SHA256, FILESYSTEM/File over Zephyr fs)
- room server processAck port; `get acl` prints as upstream
- room: default name Room-XXXX, display/input rotation, live radio UI refresh

Event loops (slice 7b)
- repeater and room server share server_main_common with a ServerRole
  descriptor; shared event bits in src/mesh_events.h
- room server deadline-driven (idle wakes 2.2/s -> 0.08/s); companion
  loop() runs on housekeeping (closes the lazy contact-flush gap)
- `get loop.wakes` in debug builds (CONFIG_ZEPHCORE_LOOP_WAKE_STATS)

Companion transports and WiFi (slice 8)
- upstream MultiSerialInterface: BLE, USB/UART and TCP are each a
  BaseSerialInterface, every connected transport gets every frame;
  CompanionSerial holds a lossless reply; shared frame_txq for BLE/TCP
- SerialCompanionTransport and LinuxTCPTransport replaced by
  TcpCompanionTransport; ZephyrBLE split (gatt layout, DFU)
- WiFi companion (upstream wifi.ssid/pwd/enabled CLI, TCP port 5000) on
  boards declaring `capabilities: wifi` in zephcore.yml: the PSRAM
  ESP32-S3 boards, plus Heltec V3 (140 contacts), Wireless Tracker (130)
  and both C6 boards (280); ESP32-S3 without PSRAM gets bounded WiFi
  buffers and a measured 72 KB heap; PSRAM boards' internal heap 72 KB
- TCP queues: own depth (ZEPHCORE_TCP_QUEUE_SIZE=8), RX waits instead
  of dropping
- RAM: companion ClientACL dropped; main/BT-TX/TCP stacks sized from
  measured peaks (-7 KB nRF companions, -9 KB ESP32 companions)
- ESP32 WiFi stopped working under ~200 frames/s unicast. Fixed by:
  patches/zephyr/0017 never block the WiFi task in the RX callback;
  patches/zephyr/0018 lock shared_multi_heap (heap corruption with the
  WiFi heap in PSRAM); patches/modules/hal_espressif/0001 backport of
  hal_espressif af97fb02df (PSRAM realloc)

Storage and prefs (slice 9)
- prefs are now prefs.json through upstream's ConfigSerializer (ported
  verbatim), with upstream's key names for shared fields and ZephCore's
  own under "zc"; on first boot the binary new_prefs / repeater prefs is
  migrated and kept, so a downgrade still boots with the old settings
- helpers/PrefsCodec: both legacy binary layouts (companion 272 B,
  server 311 B) as Zephyr-free codecs; sanitizeNodePrefs is the only
  validator
- fix: server prefs could be replaced by a truncated file when a write
  failed; identity, prefs and channel writes go through one checked
  atomic replace (ZephyrFsUtil)
- fix: a corrupt cad_auto byte turned adaptive CAD off instead of the
  default on
- fix: a deliberate 300 s GPS duty on a repeater was rewritten to 48 h
  on every boot (that migration now runs on legacy files only)
- one identity loader for all roles (IdentityFile); an unusable key
  file is always parked as .bad before regenerating
- server store no longer stamps repeater defaults on room server and
  observer; companion first-boot volume policy moved into the store
  (adoptVolume)
- ZephyrDataStore 1435 -> ~850 lines

2026-09-25 08:34:21 +02:00