The dominant garble was NOT the mic: T:RAWMIC/recorder ran full-duplex (start_loopback), so
the T-Deck played the decoded loopback round-trip out its OWN speaker, inches from the mic ->
acoustic feedback that built into "oscillating static that gets slowly louder". Gate the
playback ring-write off while the PSRAM recorder is active (i2s_playback.cpp) so raw-mic
captures are feedback-free -> speech becomes intelligible (Whisper recovers ~85% of words).
Also in this change:
- Recorder stores BOTH TDM channels (main.cpp) for offboard de-interleave; confirmed CH0 is
the mic, CH1 dead -> de-interleave was always correct.
- MCLK raised to an exact APLL-locked 12.288MHz (es7210.cpp MCLK_DIV_FRE=768 + the
{12288000,16000} coeff; i2s_capture.cpp use_apll=true/fixed_mclk) -- cleaner than the
fractional-N 4.096MHz, though the feedback (not jitter) was the real cause.
Residual (under investigation): signal-correlated distortion ("static reacts to speech") +
50Hz mains hum (USB-ground pickup, removed by the existing 300Hz HPF in the call path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Confirmed fixes:
- LVGL hybrid-allocator threshold 1024->256 (lib/lv_mem_hybrid.h): moves ~45KB of
small LVGL objects to PSRAM, fixing internal-heap starvation that made the
call/loopback pipeline intermittently fail to allocate and reboot mid-call
(internal free 71->116KB, largest contiguous block 61->106KB).
- ES7210 mic PGA gain 21dB->12dB (lxst_audio.cpp): 21dB saturated the ADC -- an
rms-7003 noise floor in silence + 0x8000 negative-rail spikes + a spectral-peak
shift that masqueraded as a "+17% pitch warp". At 12dB idle silence is clean (rms 48).
Diagnostic harness (PYXIS_TEST_HOOKS, test-only): T:RAWMIC[stage], T:REG, and
T:RECORD/T:DUMPREC add a raw-mic UDP tap plus a reliable PSRAM recorder with a
checksummed serial transfer, and T:LOOPBACK wiring, for objective mic-capture
analysis (offboard Whisper STT scoring).
Exploratory / still unresolved: captured speech remains garbled (oscillating static,
likely an ES7210 sigma-delta capture instability -- under investigation). Includes a
16kHz/main-PLL clock path + MICBIAS exploration; the "warp" was the gain artifact, not
a clock issue, so the 16kHz path can later be simplified back to 8kHz.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
- test_patch_nimble.py: the refactor made the script import its sibling
_build_helpers; the standalone-exec test ran it with a temp PROJECT_DIR, so add
the script's real directory to the shim's sys.path (CI pytest failure fix).
- sync_file_libdeps.py: remove the now-dead ENV_NAME binding.
- _build_helpers.py: reword the docstring (the "never literal env" line conflicted
with the helper's own "tdeck" fallback) and note inline that "tdeck" is a
never-hit default (PlatformIO always injects PIOENV).
- main.cpp: DRY the OTA hostname into OTA_HOSTNAME used by both setHostname() and
the log line.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
OTA hardening follow-up to the patch_nimble.py hardcoded-"tdeck" bug that broke
tdeck-ota linking (undefined reference to nimble_host_reset_reason):
- Add _build_helpers.env_libdeps_dir(env, *parts): the single per-environment
libdeps path resolver (.pio/libdeps/<PIOENV>/...). Converted all five pre-scripts
(patch_nimble/msgpack/filestore/littlefs_paths, sync_file_libdeps) to use it, so
the env component can no longer be hand-rolled/hardcoded wrong per script.
- "OTA: Ready" log -> "OTA: wireless flash service started (pyxis-tdeck:3232)":
ArduinoOTA.begin() is void and can't confirm a ready state, so log the target
instead of claiming readiness we can't verify.
Verified: both tdeck and tdeck-ota build, and the NimBLE patch lands in EACH env's
own libdeps tree (nimble_host_reset_reason count 2 in both).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Settings/clock (user-reported):
- Announce interval: default 1h (was 60s); the field is now MINUTES (label
"Announce Interval (min):", populate /60, read *60).
- Prop sync interval: default 4h (was 1h); the field is now HOURS (label
"Prop Sync Interval (hrs):", populate /3600, read *3600); log prints hours.
- Settings clock now ticks live: SettingsScreen::tick() (throttled ~1s) is hooked
into UIManager::update() like the announce-list tick, so the Time/GPS/system
readouts refresh while the screen is open instead of being a static snapshot.
OTA repair (was broken two ways; diagnosed via an ultracode workflow):
- patch_nimble.py hardcoded the `tdeck` libdeps path, so building any OTHER env
(tdeck-ota gets its own libdeps tree) never received the NimBLE patch that
DEFINES nimble_host_reset_reason -> undefined-reference link error. Made it
env-aware (env.get("PIOENV",...)), matching the four sibling pre-scripts. Also
restores 3 NimBLE stability patches that were silently missing from OTA builds.
- ArduinoOTA onStart synchronously called TCPClientInterface::stop() (blocks up to
30s), overrunning espota's ~10s connect-back window -> "No response from device"
(phase 2), and its un-fed watchdog tripped reboots. Made onStart non-blocking.
- [env:tdeck-ota] upload_command invoked a non-existent tools/espota.py -> switched
to upload_protocol=espota.
- Do NOT WiFi.setSleep(false): the ESP32 requires WiFi modem-sleep when WiFi+BT
coexist (aborts at boot otherwise) -- documented inline.
Verified on device: tdeck-ota links; OTA over WiFi completes 100% (Result: OK) and
the device reboots clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Addresses Greptile review on #36:
- Arizona is inside the Mountain longitude band but doesn't observe DST (and it
straddles the Mountain/Pacific boundary), so MST7MDT wrongly showed MDT in
summer. It now gets an explicit MST7 (no-DST) carve-out by lat/lon box before
the longitude bands. (The DST-observing Navajo Nation is not separately handled.)
- The Settings "Time:" display used a >= 2016 threshold, which let the ESP32's
~2016-01-01 first-boot default render as a real date instead of "not set".
Raised to >= 2024 so an unsynced clock is visibly unsynced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
The device intermittently stamped messages ~19 years in the future. Root cause is
a GPS week-number rollover (1024 weeks ~= 19.6y): an un-resolved/cold-start fix
emits a rollover date (e.g. 2046), and sync_time_from_gps accepted any date with
year >= 2024 (no upper bound) and wrote it to the system clock, which microLXMF
then stamps onto every outgoing message.
- Quality+stability gate: set the clock only from a real 3D fix (>= 4 satellites,
HDOP <= 5) whose calendar date has been stable for >= 3s. Rollover/cold-start
bogus dates are weak and don't hold steady, so they're rejected. No hardcoded
future cutoff (which would itself age out). Date stability is tracked off the
main loop's continuous gps.encode() feed.
- Re-sync fixed: the periodic retry called sync_time_from_gps(0), which returned
immediately without syncing, and only ran until the first sync. It's now
genuinely non-blocking (sets the clock iff the current fix passes the gate) and
keeps retrying, so the clock self-corrects when a fix appears (e.g. basement ->
window) with no reboot.
- DST + double-mktime (ported from abandoned branch 8d4a047): TZ=UTC0 is set
before the single mktime() (the old double call corrupted the struct and skewed
the UTC clock by the DST offset), and continental-US timezones use POSIX strings
with DST rules so summer shows EDT/CDT/MDT/PDT instead of standard time.
Field-verified: from a window fix it synced to the correct 2026-06-22, no rollover.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
- Backstop ARMS its timer on the first disconnected tick instead of firing
immediately, so it can't issue a redundant begin() ~1s into boot and reset the
in-progress association (or setAutoReconnect's own retry). Re-armed on each
(re)connect so every drop gets a fresh ~15s grace.
- WiFi.persistent(false): begin() no longer writes creds to NVS on every call, so
the periodic backstop during a long outage doesn't wear flash. Creds are
already persisted by the app's Preferences store; setAutoReconnect is in-RAM.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
connect_wifi() never enabled auto-reconnect, and the loop's "WiFi reconnect
check" only re-attempted on the manual Settings -> Reconnect button. So a dropped
association left the device offline (no TCP, no AutoInterface, "no connection" in
the status bar) until a reboot. Enable WiFi.setAutoReconnect(true) + persistent,
and add a non-blocking backstop in the main loop that re-issues WiFi.begin()
every ~15s while disconnected -- setAutoReconnect alone doesn't cover every
disconnect reason. Verified the SSID/AP are fine (connects at boot); closes the
stay-offline-until-reboot gap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
stop()'s join had a fixed deadline (CONNECT_TIMEOUT_MS + 2s); a slow DNS could
keep the task inside connect() past it, so stop() would free the object while the
task still referenced `this`. Extend the deadline well beyond any connect()+DNS,
and if it still expires, vTaskDelete(_task_handle) the task so it can't touch
`this` after return. (The task's own self-delete path sets _task_done first, so
this branch only runs when it has not self-deleted — no double delete.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Storing _reconnected before _conn_state=CONNECTED left a seq-cst window where the
main loop could observe _reconnected==true while still CONNECTING. check_reconnected()
would then clear the flag and announce on an offline interface (loop() returns
early), so no announce fired once actually connected. Store CONNECTED first; seq-cst
then guarantees _reconnected is only ever observed true on an online interface.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
task_loop() set `_online = true` during the CONNECTING window, which races with
loop()'s `_online = false` on the main loop (plain bool, no synchronizes-with).
It's redundant: the main loop sets `_online = true` when it observes CONNECTED.
Removing it eliminates the race with no behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Both are written by task_loop() (core 0) and read by stop() (core 1); volatile
gives no cross-core ordering. Use std::atomic<bool> to match the other shared
flags (_conn_state, _reconnected, _last_connect_attempt) and give stop()'s join
a well-defined happens-before.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
With _last_connect_attempt == 0, task_loop()'s first
`now - _last_connect_attempt >= RECONNECT_WAIT_MS` check only passes once
millis() >= RECONNECT_WAIT_MS, delaying the very first connect up to 15s after
boot. Seed it to millis() - RECONNECT_WAIT_MS in start() so the first attempt
fires immediately (unsigned wraparound keeps it correct when millis() < the wait).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
- stop() now waits on a _task_done flag the task sets right before exiting,
instead of a fixed sleep. Closes a use-after-free window where an in-flight
connect() overrunning CONNECT_TIMEOUT_MS (slow DNS) could touch `this` after
~TCPClientInterface() freed it.
- _last_connect_attempt is now std::atomic<uint32_t> — it's read/written by
task_loop() (core 0) and handle_disconnect() (core 1).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
The blocking WiFiClient.connect() ran on the main loop and stalled it for the
lwIP default (~18.5s) when the host was unreachable -- including the DNS lookup --
freezing the UI, so the screen took ~10s to wake. The 2-arg connect() also
ignored CONNECT_TIMEOUT_MS (that only bounds reads).
Move only the blocking connect() to a dedicated FreeRTOS task. read/write/frame
stay on the main loop exactly as before (unchanged low-latency data path -- the
link/Resource timing is untouched). An atomic _conn_state hands _client ownership
between the task (while CONNECTING) and the main loop (while CONNECTED) so they
never touch the socket concurrently. Bound the connect via the 3-arg connect()
and back off retries to 15s.
tests/hardware: wait_for_tcp_link() matched "started", keying on interface
startup rather than the actual connect. With the async connect that let the
harness drive the announce before the link was up, so the device's announce was
lost and the first direct message (bz2-probe) failed. Match "connected to".
Verified on a T-Deck: screen wake instant (connect off the main loop); e2e smoke
5/5 including bz2-on-receive.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
The offset was computed as unix_ms - millis() (32-bit) but ltime() adds it to a
64-bit (high32<<32|low32) counter. Once high32 != 0 (a millis() rollover, or an
erratic screen-off/wake tripping ltime()'s roll-over check) OS::time() was wrong
by high32*49.7 days -- surfacing as garbage "Nw ago" timestamps in the
conversation list. Compute the offset from OS::ltime()-getTimeOffset() at both
the GPS and NTP sync sites; add a diagnostic logging OS::time() after each sync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Repins microReticulum + microLXMF onto the upstream-0.4.1 graft and adapts
pyxis to the new src/microReticulum/ layout and 0.4.x APIs. The far-diverged
0.3.0 fork's Resource/Transport/Identity work is subsumed by upstream's
reimplementation; only the still-needed fixes ride on the pinned branches
(PKCS7/HMAC/X25519 crypto -- proven byte-identical to python RNS 1.3.1 --
Packet link-proof callback, Identity short-sig guard, and the bz2 layer +
decompress-on-receive in Resource::assemble()).
Consumer-side changes:
- platformio.ini: pin microReticulum @2f21fee (pyxis-fixes-on-0.4.1) and
microLXMF @33760d0 (chore/microreticulum-0.4.1-layout); bump microStore
ceea8f5 -> c5fb69d (0.4.x requires the new BasicFileStore::init API);
-std=gnu++11 -> gnu++17 (upstream requires C++17).
- Namespace all microReticulum includes (angle + quote) to <microReticulum/...>
for the relocated layout; shim-local Utilities/Stream.h|Print.h preserved.
- Interface::send_outgoing now returns bool: update TCP/BLE/SX1262/Auto
overrides with correct success/failure returns.
- SDArchiveFileSystem::init(bool reformatOnFail=true) to match new microStore.
- Static Transport::get_path_table() -> path_table(); instance getter unchanged.
- Remove duplicate shim Cryptography/BZ2 (microReticulum provides it now; keep
lib/libbz2 as the ESP32 bzlib provider).
- patch_littlefs_paths.py: normalize microStore's LittleFS adapter paths to a
leading "/" -- ESP32 Arduino LittleFS rejects "./"-prefixed paths, which
silently broke the path store (no peer paths learned, all messaging blocked).
Validated on T-Deck Plus: builds (RAM 27.5% / Flash 77.7%), boots stable
(no WDT/panic), and a full on-device LXMF e2e (DIRECT + OPPORTUNISTIC +
bz2-compressed-Resource receive) passes 5/5.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
LVGL_LOCK() was held for the entire base64 serial dump (~18s for a 320x240
frame), blocking the LVGL render task the whole time and tripping the 5s
LVGLLock recursive-take timeout assert in debug builds (crash on every
screenshot). lv_snapshot_take() copies the pixels into its own buffer, so the
lock is only needed for the snapshot itself -- scope it there, dump the copy
unlocked, and re-take the lock only to free the snapshot.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
The `!auto_interface_impl` guard meant start_auto_interface() only ran on the
first WiFi-connect edge; on every reconnect the block was skipped, so
AutoInterface kept stale multicast sockets and peers never rediscovered until
reboot. start_auto_interface() is idempotent (its else-if(!online()) branch
rebinds the sockets), so drop the guard and call it on every connect edge.
TCPClientInterface self-reconnects in its own loop(), so only AutoInterface
needed this.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
main.cpp: split on_wifi_connected NTP sync into kick-off +
pump_ntp_sync_if_pending poller. Previously the synchronous
getLocalTime retry loop blocked loopTask for up to 10 s on first
WiFi-associate, stalling RNS packet ingestion / LXMF delivery /
SX1262 RX FIFO drain. Now configTzTime kicks off the SNTP task
non-blocking and the periodic loop probes for completion until
NTP_TIMEOUT_MS elapses.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- main.cpp: split T:RX `count=` from ring-buffer index so soak-test
harness sees true received-message count past TEST_RX_RING=32
- sync_file_libdeps.py: drop hardcoded ~/repos/microReticulum, read
PYXIS_MICRORETICULUM_DIR env var for opt-in local-override workflow
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The previous hardcoded LAN IP in platformio.ini identified a specific
deployment (Mac rnsd at a 10.0.0.x address). Move host:port to env
vars read at build time via PIO's ${sysenv.X} interpolation, with
a .env.example template + .env gitignored. Production builds (no
PYXIS_TEST_HOOKS) skip the macros entirely.
Changes:
- platformio.ini (both env blocks): replace literal IP/port with
${sysenv.PYXIS_TEST_TCP_HOST} / ${sysenv.PYXIS_TEST_TCP_PORT}.
Both quoted so an unset env var expands to an empty string
literal "" rather than an empty token (the latter would break
the PORT > 0 numeric compare in main.cpp).
- src/main.cpp test-hooks block: guard host/port assignment so
empty values fall back to NVS-stored settings. atoi() handles
the port string side. Missing env var won't silently brick
test mode — it just becomes "use whatever's in NVS".
- .env.example: checked-in template with placeholder values
(TEST-NET-1 192.0.2.x range, not a real LAN IP).
- .gitignore: add `.env` so the populated local copy stays out
of source.
Build: verified clean with env vars UNSET (NVS-fallback path) and
SET (override path); 27.4% RAM, 79.8% Flash in both cases.
New testing/docs surface for grabbing the active LVGL screen as a
PNG over USB-CDC. Useful both for documentation (round-trip capture
of every public screen via T:SHOW <name> + T:SCREENSHOT) and
automated UI regression tests.
On-device:
- lib/lv_conf.h — enable LV_USE_SNAPSHOT (~5 KB code; uses PSRAM
via the existing hybrid allocator so internal RAM is unaffected)
- src/main.cpp — T:SCREENSHOT handler takes an lv_snapshot_take()
of lv_scr_act() under LVGL_LOCK(), dumps a delimited base64
stream over CDC. Inlines a tiny base64 encoder (no new dep).
- src/main.cpp — T:SHOW <name> dispatches to UIManager::show_*()
for the six publicly-navigable screens (conversation_list,
compose, announces, status, settings, propagation_nodes).
Wire format:
T:SCREENSHOT BEGIN W=320 H=240 FMT=rgb565<be|le> BYTES=153600
<base64 line, 76 chars>
...
T:SCREENSHOT END
Host side:
- screenshot.py — auto-detects the pyxis port via T:ID probe,
sends T:SCREENSHOT, reads until END, filters out interleaved
log lines (heap heartbeats / BLE stats can splice in
mid-dump), validates byte count matches header, decodes RGB565
with the documented byte order, expands channels via 5→8 / 6→8
high-bit replication, saves PNG. Pillow + pyserial.
Catalog:
- docs/serial_commands.md — full reference for all T:* commands
accumulated so far (identity/paths/send/receive, propagation,
voice, BLE, UI). New commands should land here when added.
Throughput: ~205 KB base64 over CDC at 115200 → ~18 s/shot. Fine
for docs and automated tests, not video. Bumping baud or zlib-
compressing on-device is queued in the doc as future work.
Two related changes:
1. NimBLE advertising overflow
At boot pyxis was logging "NimBLEAdvertisementData: Data length
exceeded" twice. The 128-bit Reticulum service UUID is 18 bytes
once you include the AD type+length headers; the device name
"TD-XXXXXX" is another 9-11 bytes; flags eat 3 bytes. That's
already over the 31-byte legacy adv-packet limit, so NimBLE was
silently truncating the advertisement and dropping the service
UUID. Android Columba's BleScanner filters by ServiceUuid at the
Android BLE driver layer (ScanFilter.Builder().setServiceUuid),
so without the UUID in the primary adv data, pyxis was invisible
to Columba.
Fix: call enableScanResponse(true) BEFORE addServiceUUID +
setName. NimBLE then routes the long device name into the
secondary 31-byte scan-response payload that active scanners
request, leaving the primary adv data with just flags + the
service UUID — under budget and visible to the filter.
Verified: with the fix, Android system Bluetooth reads pyxis's
name as "TD-46cbcf" and Columba's BleGattServer logs
"Central connected: FC:69:15:9C:B2:C9" (pyxis as central). The
connection holds for ~40s before HCI_CONN_TIMEOUT — separate
issue not addressed here, just the unblock so the link can be
established at all.
2. T:BLE on|off harness hook
Mirrors T:CALL_PROFILE / T:ANNLXST / T:LXSTDEST: persists the
ble_en NVS key and starts/stops the interface live so the LXMF
harness can flip BLE on/off the same way it drives any other
subsystem. Idempotent for "already on" / "already off". Useful
for upcoming pyxis ↔ Android Columba BLE smoke tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AutoInterface init at boot is gated on \`WiFi.status() == WL_CONNECTED\`,
but WiFi association typically takes 2-5s and finishes well after the
boot block runs. The result: with auto_enabled=true in NVS, boot logs
"AutoInterface enabled but WiFi not connected - skipping" and the
interface never starts even though the settings UI shows it on. Pyxis
ends up looking enabled but actually isn't peering with anyone.
TCP interface has the same race and handles it via start_tcp_interface()
called from both boot and on_wifi_connected(). Mirror that pattern with
a new start_auto_interface() helper:
- Idempotent — safe to call from boot AND post-WiFi.
- Creates the AutoInterface instance the first time and registers it
with Transport.
- Re-starts an existing-but-stopped instance (e.g. WiFi disconnect →
reconnect cycle).
Boot path now invokes start_auto_interface() inline if WiFi was
already up, otherwise logs "will retry from on_wifi_connected" — and
the post-WiFi handler in the main loop calls it once WiFi lands.
Verified: pyxis with auto_enabled=true now logs the full AutoInterface
startup sequence (multicast join + sockets bound) ~3-5s into runtime
instead of never. Multicast peering visibility is a separate issue
being chased now (pyxis joins the group successfully but its announces
don't reach rnsd or Sideband, and vice versa — likely an ESP32 lwIP
multicast TX/RX subscription issue, not the gating bug fixed here).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real-LXST 14s call at ULBW (Codec2-700C) was timing out T:CALL_QOS /
T:CALL_STATS responses ~10-15s in. Pyxis itself was still processing
audio fine; the host's serial reader was just overrun by debug-level
prints from three sources, all firing per-packet during voice traffic:
1. TCPClientInterface: per-frame "[TCP] Reading X bytes" / "[TCP]
First bytes: ..." / "[HDLC] Frame #N: ..." / "[TCP] Processing
frame" / 5s "[TCP] connected= ..." were unconditional Serial.printf.
Now gated behind `RNS::loglevel() >= LOG_DEBUG` and the snprintf
work skipped when it'd be discarded.
2. i2s_capture.cpp: "[CAP] rate=" fired every 2s regardless of
activity. Now only emits when ringDrops > 0 OR runningPeak > 1000
(something happened worth noting). Counters still update — only
the print is gated.
3. microStore upstream: "[ustore] get: key not found in index" fires
on every path-store miss, which RNS hits constantly during a call.
patch_filestore.py was already a registered pre-build script for
diagnostic patches; reactivate it (was commented out in
platformio.ini) and add a silence patch as the always-on default.
Diagnostic exists()/put() patches gated behind PYXIS_FILESTORE_DIAG=1
so they're easy to bring back when investigating path-store drift
without touching the script each time.
After this, the ULBW real-LXST call validator returns PASS with full
final stats (pyxis_tx=34 rx=119 decode_ok=151 decode_fail=0
pyxis_rms=4410). 1600bps/3200bps profiles still hit serial-timeout
patterns under sustained TX — likely CPU saturation in the main
loop, separate from this fix; tracked in #75 followup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two harness hooks for testing pyxis as the LXST callee:
- T:LXSTDEST: returns pyxis's lxst.telephony destination hash. The
caller-side bot (real LXST.Telephony.Telephone in this case) needs
this to dial pyxis. Backed by a new test_lxst_dest_hex() accessor on
UIManager that reads _lxst_destination and returns hex (or empty if
the destination isn't registered yet).
- T:ANNLXST: forces a fresh announce of the lxst.telephony destination.
The TCP-reconnect handler in main.cpp:963 only announces LXMF, so on
a fresh boot the lxst.telephony destination is missing from rnsd's
cache and link requests addressed to it get dropped with "no known
path to final destination" (rnsd debug log). The harness pings this
before each callee test to ensure rnsd has a fresh path.
Also adds INFO logging to announce_lxst() mirroring the LXMF announce
log (Announcing destination: <hash> ... announce sent), so it's
visible in tdeck-side traces when an announce actually went out vs.
silently no-oped.
Validated: bot dials, pyxis transitions IDLE -> INCOMING_RINGING (UI
shows incoming-call screen), harness sends T:CALL_ANSWER, state
becomes ACTIVE. (Audio path crashes shortly after via the same
Ed25519 announce-validation bug — fix landed in microReticulum
f4bad06, but PIO's libdeps cache had been holding a stale copy; a
manual rm -rf .pio/libdeps/tdeck/microReticulum was needed to pick it
up. Tracked in #73.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
WIRE TX raw + WIRE TX framed were INFO-level, ~150-180 bytes each, and
fired per outgoing packet. During an LXST voice call (~5 batches/sec
plus retries) that's ~2KB/s of pure debug log on USB CDC, on top of
existing call/heap/disp prints. The serial buffer saturated ~15s in
and T:CALL_QOS / T:CALL_STATS commands timed out at the 5s threshold,
even though the device itself was healthy.
Demote both to DEBUG and gate the hex-encoding work behind a runtime
loglevel check so the per-packet snprintf loop doesn't run when the
output would be discarded anyway. Re-enable by raising RNS log level
to DEBUG when actually debugging the wire format.
After this, a 14s real-LXST E2E call returns full stats every poll
with no timeouts and the harness validator runs to PASS.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two profile constants beyond the existing LXST_PROFILE_LBW (0x30,
Codec2-3200): LXST_PROFILE_VLBW (0x20, Codec2-1600) and
LXST_PROFILE_ULBW (0x10, Codec2-700C). Default is now ULBW — a 700C
frame fits comfortably inside an SF7-9 LoRa packet, which is the
target medium for pyxis voice. The previous 3200bps default was 4.5x
larger and unsuitable for the radio path.
Profile is selectable at runtime via T:CALL_PROFILE [hex]. Replaces
five hardcoded LXST_PROFILE_LBW sites: three audio-init paths in
call_process_signal and two profile-negotiation send_signal calls.
Adds T:CALL_ANSWER for harness pyxis-as-callee testing — sets the
same _call_answer_pending flag the UI button does so call_answer()
runs on the main loop in its proper context. Validates against
real LXST.Telephony.Telephone callers.
Validated: pyxis dialed real LXST upstream Telephone bot, negotiated
ULBW end-to-end, reached STATUS_ESTABLISHED, decoded frames cleanly
both directions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PlatformIO dep sh123/esp32_codec2_arduino@1.0.7 bundles codec2
v0.9.2. Mac-side pycodec2 v3.0.4 links libcodec2 v1.2.0. Years of
codec2 development between those releases.
Replace the upstream lib with a local vendor of drowe67/codec2 v1.2.0
under lib/codec2/. Trim the 191-file source tree down to the
~100 files actually needed for codec2 (drop FreeDV, OFDM, COHPSK,
FSK, FM-FSK, LDPC, Horus, CLI tools — pyxis only uses
codec2_create/destroy/encode/decode + samples_per_frame /
bytes_per_frame). Carry over the v0.9 codebook .c files since the
codebook contents matched (compared against v1.2's src/codebook/*.txt).
Define __EMBEDDED__ so the codebooks land in flash (.const) rather
than RAM. Without it the codebooks add ~127KB to BSS and the LVGL
task fails to start (RAM was 65% full vs 27% with __EMBEDDED__).
Provide trivial codec2_malloc/codec2_free wrappers in
codec2_alloc_esp32.c (codec2 v1.2 expects them when __EMBEDDED__ is
defined; ESP-IDF's malloc/free already pull from internal RAM).
Also explicitly add SD/FS to lib_deps and #include <SD.h> in
main.cpp — the previous esp32_codec2 dep transitively pulled SD
which let SDArchiveFileSystem.h get away with depending on it
implicitly. With chain+ ldf mode and no esp32_codec2 dep, we have
to declare the framework lib explicitly.
DOES NOT fix the ~30x speech-decode RMS asymmetry between pycodec2
self-tests (~5800) and pyxis decoding the same encoded bytes (~170).
Sine waves and 3-formant synthesis pass clean both directions; only
real TTS speech triggers it. Probably a separate codec-state
divergence (the encoder/decoder are independent codec2 instances
in pyxis, both fresh per call) or a wire-format quirk we still need
to track down. v1.2 is the right baseline regardless — same bug
class as several upstream fixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the LXST harness from frame-flow only (#62) and decoder QoS
only (#63) to a full content-fidelity test: a known 1kHz sine wave
flows in BOTH directions through the Codec2-3200 round-trip, and
the harness asserts the decoded RMS at each end matches expected
energy within tolerance (after Codec2's lossy speech-codec behavior).
Firmware additions (under PYXIS_TEST_HOOKS):
I2SCapture
setInjectSine(enabled, freq=1000, amp=0.5)
Replaces mic input with a phase-continuous synthesized sine.
Bypasses ES7210 capture and the voice filter chain so the
encoder sees pure samples — peer's decoded RMS validates that
pyxis's TX path delivers content.
I2SPlayback
pcmSampleCount(), pcmSumSquares()
Decoded-PCM energy accumulators, fed from each successful
Codec2 decode. uint64 sumsq holds ~2³⁴ frames before overflow,
far longer than any test call.
LXSTAudio + UIManager (test-only)
captureSetInjectSine, playbackPcmSampleCount, playbackPcmSumSquares
test_call_set_inject_sine, test_call_pcm_sample_count,
test_call_pcm_sum_squares
Serial T: hooks (main.cpp)
T:CALL_INJECT <on|off> [freq] [amp_pct]
Drive the capture-side injection from the harness.
T:CALL_QOS now also returns pcm_n + pcm_ss
Harness divides + sqrts to RMS for content validation.
Validated with /tmp/lxst_call_harness.py + /tmp/lxst_call_bot.py
(scripts vault-local per the no-PII rule):
pyxis_rms = 6363, bot_rms = 1930, decode_fail = 0
PASS: bidirectional audio + content-fidelity validated
The empirical RMS floor is 800 (pycodec2 self-test on 1kHz amp 0.5
yields ~1400; pyxis decoder hits ~6300; bot decoder hits ~1900 —
all far above the ~5-50 silence floor). Codec2 is a speech codec
so pure-tone round-trip is naturally lossy; the test gates on
"audio bytes carry actual content energy", not lossless round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the on-device test surface the LXST voice-call harness needs to
drive an end-to-end audio flow + QoS validation against a Mac-side
softphone bot. Both hooked behind PYXIS_TEST_HOOKS so production
firmware is unaffected.
T: serial commands (main.cpp):
T:CALL <hex> — initiate call (hex = peer's lxst.telephony dest)
T:CALL_STATE — current call FSM state name
T:CALL_HANGUP — tear down the active call
T:CALL_STATS — audio frame counters: tx, rx, state
T:CALL_QOS — Codec2 decoder QoS: decode_ok, decode_fail, state
T:HASIDENTITY <h> — bool, distinct from T:HASPATH because path_store
and known_destinations are populated by separate
code paths (announce can land in one before the
other; harness has to wait for both).
UIManager additions (under PYXIS_TEST_HOOKS):
test_call_initiate, test_call_hangup, test_call_state_name,
test_call_audio_tx_count, test_call_audio_rx_count,
test_call_decode_ok, test_call_decode_fail.
I2SPlayback / LXSTAudio additions (always on — counters are tiny):
decodeOkCount(), decodeFailCount(), resetCounters() on I2SPlayback
surface the Codec2 decode success/fail rate. LXSTAudio re-exports.
Each writeEncodedPacket call increments exactly one counter so the
ratio is "wire-level audio fidelity" of the peer's encoder.
Validated:
Frame-flow soak (12s call): pyxis tx=73 rx=72, bot tx=230 rx=72.
QoS soak (12s call): pyxis tx=73 rx=76, decode_ok=81 decode_fail=0,
bot tx=227 rx=73.
Harness scripts that drive these hooks live in the local Obsidian
vault (under 80 Assistant/Memory/pyxis/soak_scripts/) — they encode
LAN-specific state and aren't checked in. See the vault's
automated_soak_testing.md for the full procedure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Boot was 48.5s end-to-end on this hardware: 92% of that was two
synchronous waits — gps_sync (15s) and wifi_connect (30s). Both
have async paths already (gps.encode is fed in loop(); the periodic
status check sets up TCP when WL_CONNECTED appears post-boot), so
blocking at boot wasn't actually load-bearing. Drop both waits to
brief opportunistic checks (500ms / 1s) and let the main loop pick
up async events.
Three follow-on changes were needed to keep functionality:
- Extract on_wifi_connected() (NTP + OTA + UDP logging setup) from
setup_wifi so the post-boot WL_CONNECTED transition in loop() can
run it. Idempotent via _wifi_post_connect_done.
- Add periodic GPS-time-sync retry in loop() — every 30s while
unsynced, attempt a fast (timeout=0) sync_time_from_gps. The
cheap path returns immediately when TinyGPSPlus already holds a
valid date+time.
- Defer LVGL task start until AFTER setup_ui_manager. Previously
the LVGL task started immediately after lv_init and refreshed
its empty default screen on top of the boot splash, causing a
visible flash to black before the first real UI frame painted.
With the start moved to after the UIManager has built screens
and configured the active one, the splash stays on-screen until
the first real frame.
Net: boot drops from ~48s to ~3-5s.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Hardware::TDeck::SDArchiveFileSystem — a microStore::FileSystem
adapter that piggybacks on the SD card already mounted by SDAccess::init
and serializes every operation through the shared SPI bus mutex (so it
cooperates with display + LoRa traffic on HSPI).
In setup_lxmf, after MessageStore is constructed, attach the SD as the
archive tier at "/lxmf-archive". When the SD is missing the message
store falls back to delete-on-cull (still bounded, just no historical
scrollback). Without this the LittleFS partition (1.875MB) fills in
~30 min of sustained receive and trips lfs_alloc divide-by-zero.
Also moves the LVGL_LOCK in UIManager::on_message_received to AFTER
the save_message call. The 5s LVGL lock timeout was tripping when
LittleFS compaction stalled the save for several seconds, panicking
the loop watchdog.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a USB-CDC serial command interface gated behind PYXIS_TEST_HOOKS
that lets a host-side harness drive pyxis end-to-end without UI taps.
Built specifically to run /tmp/tdeck_harness.py and prove LXMF DIRECT,
OPPORTUNISTIC, and PROPAGATED delivery against a Mac-side echo bot
over the Mac's rnsd + lxmd.
Commands (all newline-terminated, replies T:OK or T:ERR):
T:DEST — pyxis's delivery dest hash (hex)
T:ID — pyxis's identity hash (hex)
T:ANN — force an announce
T:PATHS — count + dump in-memory path table
T:HASPATH <hex> — Transport::has_path + in-memory check
T:RECALL <hex> — Identity::recall_app_data hex
T:SEND <hex> <text> — outbound DIRECT LXMessage
T:SENDOPP <hex> <text> — outbound OPPORTUNISTIC LXMessage
T:SENDPROP <hex> <text> — outbound PROPAGATED LXMessage
T:SETPROP <hex> <stamp_cost> — set outbound propagation node
T:SYNCPROP — request_messages_from_propagation_node
T:SYNCSTATE — current PR_* sync state
T:STATE <msg_hash> — LXMessage state for a tracked send
T:RX — drain inbound RX ring
T:RXCLR — clear RX ring
Build-flag side:
-DPYXIS_TEST_HOOKS — gates all of the above
-DPYXIS_TEST_TCP_HOST="..." — hard-overrides NVS tcp_host so the
harness's rnsd is the only target
-DPYXIS_TEST_TCP_PORT=... — same for tcp_port
Also: replaces `lib_extra_dirs = deps/microReticulum` with an explicit
`file://~/repos/microReticulum` lib_dep. lib_extra_dirs
caused PIO to compile microReticulum twice (once through the extra
dir, once through microLXMF's transitive auto-fetch), producing two
copies of `Transport::_path_store` in BSS. Different translation
units linked against different statics, so put() and exists() landed
in different in-memory indexes. Symptom: `T:HASPATH` returned 0 even
when the previous announce's `[ustore] put: wrote key` log line was
visible. Single source path → single static → consistent reads.
`patch_filestore.py` is committed but commented out in extra_scripts
— used during diagnostic when the dual-static issue was being
triaged. Easy to re-arm if FileStore put/exists drift recurs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two adjustments to setup_reticulum() / setup_filesystem():
1. `Reticulum::transport_enabled(true)` — unlocks the path-store
init block at Transport.cpp:244 that was gated behind transport
mode. Without it, _new_path_table.put() always returned false at
TypedStore::isValid() (no filesystem assigned), every announce
surfaced as "Failed to add destination" spam, and the UI's
announce list stayed empty. Transport mode also enables relaying
for other nodes — tolerable on a T-Deck Plus with PSRAM headroom
and LittleFS-backed path persistence.
2. Switch the microStore adapter from SPIFFSFileSystem to
LittleFSFileSystem (paired with -DUSTORE_USE_LITTLEFS in
platformio.ini). LittleFS handles sustained writes; SPIFFS GC
stalls were causing every path-store put to fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Stops the TASK_WDT(6) reset loop that was firing every ~50-90s of
post-setup runtime on the graft (and on pre-graft pyxis). Decoded
backtrace pinned the cause to ESP-IDF's WiFi ppTask doing
pm_tx_data_done_process and starving the CPU0 idle task — a
WiFi-stack-internal that has nothing to do with user code.
Two attempts, both useful to record:
1. sdkconfig.defaults: set CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0=n.
Doesn't take effect — Arduino-ESP32 ships its framework PRE-BUILT
with the original sdkconfig (CPU0=y, timeout=5s) baked in.
sdkconfig.defaults overrides only apply if you rebuild the framework
from source. Kept the .defaults change for documentation; the
runtime escape uses esp_task_wdt_init.
2. esp_task_wdt_delete(xTaskGetIdleTaskHandleForCPU(0)) at boot.
Doesn't stick — Arduino-ESP32's runtime re-subscribes CPU0 idle
on the next loop iteration since CHECK_IDLE_TASK_CPU0 is still
set in the prebuilt config.
Final fix: esp_task_wdt_init(60, false). 60s timeout (was 30s) AND
panic=false (warnings logged, no reset). This makes WDT a soft
indicator instead of a hard reset trigger — appropriate while the
underlying WiFi-pm-starvation issue lives in ESP-IDF, not pyxis.
Verified runtime:
- 3 minutes continuous uptime, zero aborts/panics/resets
- 5 propagation nodes discovered from the network
(LXMF protocol layer working end-to-end on the graft)
- Heap stable around 46KB free, low-water 37KB
- 165KB UART log captured (vs prior ~30KB before reset)
This means pyxis on top of attermann/microReticulum @ 0.3.0 +
torlando-tech/microReticulum:pyxis-fixes-on-0.3.0 is now
**runtime-stable** — first time the graft has run uninterrupted
long enough to actually use.
Future: revisit panic=true once we can prove the WDT culprit is
moved to ESP-IDF's tracker (or pyxis is updated to a framework that
fixes the WiFi pm starvation). Until then, log-only avoids false
positives.
relocate shim to lib/; pyxis lib API renames
Compile-tier graft progress on top of 40e561f. Key changes:
- Re-vendor lib/microreticulum-shim/ (was src-shim/) from the *actual*
ca355e5 commit content rather than the stale "feat/t-deck HEAD"
/tmp clone the previous commit pulled from. Recovers process_sync()
on LXMRouter and MEMORY_MONITOR_POLL macro that were missing.
- Move src-shim/ → lib/microreticulum-shim/ + add library.json so
PlatformIO discovers the .cpp files and links them. Was previously
only on the include path; the .cpps weren't in the build.
(This unblocks the 30+ undefined-reference linker errors for LXMF
and Instrumentation symbols.)
- Drop -Isrc-shim/Utilities (/Cryptography/Instrumentation) from
build_flags — they were over-broad and put our Stream.h on the
GLOBAL header path, breaking Arduino's Wire.cpp which has
`class TwoWire: public Stream`. -Ilib/microreticulum-shim alone
resolves subdir lookups via <Cryptography/X.h>, <Utilities/Y.h>.
- UniversalFileSystem migrated to microStore::Adapters::SPIFFSFileSystem
(activated by -DUSTORE_USE_SPIFFS). Vanilla upstream microReticulum
@ 0.3.0 deleted RNS::FileSystem in favor of microStore. Pyxis's
lib/universal_filesystem/ is now dead code on this build path.
- pyxis lib API renames for the post-graft world:
SDLogger.cpp: RNS::setLogCallback -> RNS::set_log_callback
AnnounceListScreen: Transport::get_destination_table ->
Transport::get_path_table
Transport::DestinationEntry ->
RNS::Persistence::DestinationEntry
UIManager.cpp: _lxst_destination ctor explicit RNS::Type::NONE
(vanilla Destination has no default ctor)
Identity::mark_persistent calls disabled w/
restoration TODO
ConversationListScreen: Interface::get_rssi/get_stats calls
disabled (the methods are non-virtual
on BLEInterface/SX1262Interface post
de-virtualization in a0ff631)
Compile is clean against the fixed-cryptography submodule pin; current
failure layer is fork-only Type::Channel constants referenced by the
vendored shim's Buffer/ChannelData files. That's the next session's
problem — see pyxis_microReticulum_graft_spike_findings.md for the
plan options (most likely: remove Channel/Buffer/ChannelData/Ratchet
from the shim, since LXMF doesn't use Channel anyway per the 2026-05-04
investigation).
