The dominant garble was NOT the mic: T:RAWMIC/recorder ran full-duplex (start_loopback), so
the T-Deck played the decoded loopback round-trip out its OWN speaker, inches from the mic ->
acoustic feedback that built into "oscillating static that gets slowly louder". Gate the
playback ring-write off while the PSRAM recorder is active (i2s_playback.cpp) so raw-mic
captures are feedback-free -> speech becomes intelligible (Whisper recovers ~85% of words).
Also in this change:
- Recorder stores BOTH TDM channels (main.cpp) for offboard de-interleave; confirmed CH0 is
the mic, CH1 dead -> de-interleave was always correct.
- MCLK raised to an exact APLL-locked 12.288MHz (es7210.cpp MCLK_DIV_FRE=768 + the
{12288000,16000} coeff; i2s_capture.cpp use_apll=true/fixed_mclk) -- cleaner than the
fractional-N 4.096MHz, though the feedback (not jitter) was the real cause.
Residual (under investigation): signal-correlated distortion ("static reacts to speech") +
50Hz mains hum (USB-ground pickup, removed by the existing 300Hz HPF in the call path).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Confirmed fixes:
- LVGL hybrid-allocator threshold 1024->256 (lib/lv_mem_hybrid.h): moves ~45KB of
small LVGL objects to PSRAM, fixing internal-heap starvation that made the
call/loopback pipeline intermittently fail to allocate and reboot mid-call
(internal free 71->116KB, largest contiguous block 61->106KB).
- ES7210 mic PGA gain 21dB->12dB (lxst_audio.cpp): 21dB saturated the ADC -- an
rms-7003 noise floor in silence + 0x8000 negative-rail spikes + a spectral-peak
shift that masqueraded as a "+17% pitch warp". At 12dB idle silence is clean (rms 48).
Diagnostic harness (PYXIS_TEST_HOOKS, test-only): T:RAWMIC[stage], T:REG, and
T:RECORD/T:DUMPREC add a raw-mic UDP tap plus a reliable PSRAM recorder with a
checksummed serial transfer, and T:LOOPBACK wiring, for objective mic-capture
analysis (offboard Whisper STT scoring).
Exploratory / still unresolved: captured speech remains garbled (oscillating static,
likely an ES7210 sigma-delta capture instability -- under investigation). Includes a
16kHz/main-PLL clock path + MICBIAS exploration; the "warp" was the gain artifact, not
a clock issue, so the 16kHz path can later be simplified back to 8kHz.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Addresses Greptile review on #37. SettingsScreen has its OWN settings load
(separate from main.cpp's app_settings), which still defaulted announce to 60s and
sync to 3600s and overrode the intended defaults -- so the UI still showed the old
values. Set them to 3600s (1h) and 14400s (4h) to match. Also updated two stale
block comments in update_ui_from_settings / update_settings_from_ui that still said
"minutes" after the sync field was promoted to hours.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Settings/clock (user-reported):
- Announce interval: default 1h (was 60s); the field is now MINUTES (label
"Announce Interval (min):", populate /60, read *60).
- Prop sync interval: default 4h (was 1h); the field is now HOURS (label
"Prop Sync Interval (hrs):", populate /3600, read *3600); log prints hours.
- Settings clock now ticks live: SettingsScreen::tick() (throttled ~1s) is hooked
into UIManager::update() like the announce-list tick, so the Time/GPS/system
readouts refresh while the screen is open instead of being a static snapshot.
OTA repair (was broken two ways; diagnosed via an ultracode workflow):
- patch_nimble.py hardcoded the `tdeck` libdeps path, so building any OTHER env
(tdeck-ota gets its own libdeps tree) never received the NimBLE patch that
DEFINES nimble_host_reset_reason -> undefined-reference link error. Made it
env-aware (env.get("PIOENV",...)), matching the four sibling pre-scripts. Also
restores 3 NimBLE stability patches that were silently missing from OTA builds.
- ArduinoOTA onStart synchronously called TCPClientInterface::stop() (blocks up to
30s), overrunning espota's ~10s connect-back window -> "No response from device"
(phase 2), and its un-fed watchdog tripped reboots. Made onStart non-blocking.
- [env:tdeck-ota] upload_command invoked a non-existent tools/espota.py -> switched
to upload_protocol=espota.
- Do NOT WiFi.setSleep(false): the ESP32 requires WiFi modem-sleep when WiFi+BT
coexist (aborts at boot otherwise) -- documented inline.
Verified on device: tdeck-ota links; OTA over WiFi completes 100% (Result: OK) and
the device reboots clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Addresses Greptile review on #36:
- Arizona is inside the Mountain longitude band but doesn't observe DST (and it
straddles the Mountain/Pacific boundary), so MST7MDT wrongly showed MDT in
summer. It now gets an explicit MST7 (no-DST) carve-out by lat/lon box before
the longitude bands. (The DST-observing Navajo Nation is not separately handled.)
- The Settings "Time:" display used a >= 2016 threshold, which let the ESP32's
~2016-01-01 first-boot default render as a real date instead of "not set".
Raised to >= 2024 so an unsynced clock is visibly unsynced.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
- Add a live "Time:" line to Settings -> GPS showing the system clock (e.g.
"Time: 2026-06-22 16:32:12", or "not set" when unsynced), so the GPS time-sync
can be verified on-device at a glance -- a far-future year or "not set" flags a
rollover/unsynced clock without needing serial.
- Fix the HDOP readout: it divided by 100 a second time (hdop.hdop() already
returns the true value), so it always showed ~0.0 (Ideal). Now shows the real
geometry -- which matters since HDOP is the metric the new sync-gate keys off.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
- AnnounceListScreen::tick(): drop the pre-sort 64-item gather cap so the sort
always sees the true newest destinations (the path table is bounded by
USTORE_DEFAULT_MAX_RECS=400 and items allocate in PSRAM, so the gather is
bounded); only the render stays capped at MAX_DISPLAY.
- tick() now re-adds the freshly rendered item containers to the focus group and
focuses the first. The deferred gather meant show() (which adds widgets to the
group) ran before the containers existed and lv_obj_clean dropped the old ones,
so trackball navigation couldn't reach the list items.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Pins microReticulum to the fork's pyxis-fixes-on-0.4.1 @ b582986 (three 0.4.1-graft
regression fixes) and adds the pyxis-side pieces.
microReticulum fixes (via the pin):
- Identity::recall now accounts for the announce ratchet (matches validate_announce)
— it read app_data 32 bytes early for ratchet-bearing announces (Sideband/Columba),
garbling peer display names.
- DestinationEntry caps persisted random_blobs to PERSIST_RANDOM_BLOBS (16); the set
was never trimmed, growing path entries ~12B/re-announce past microStore's 1024B
ceiling so put() rejected them (empty path table) — and under a raised ceiling
bloated the FS until compaction crashed.
- Transport mirrors learned paths into _path_table so path_table() (the UI announce
list + T:PATHS) reflects them; the microStore migration stopped populating it.
pyxis pieces:
- -DUSTORE_DEFAULT_MAX_RECS=400 caps the live path-store record count (defense-in-depth).
- AnnounceListScreen::refresh() defers to a main-loop tick(): the gather (path-table
iteration + per-entry recall/app_data) runs off the LVGL lock, serialized with the
main-loop path-table writes, instead of racing them on the LVGL task and blocking
past the 5s lock timeout — which hung/crashed on open once the table was non-empty.
Verified on device: display names correct, T:PATHS 0 -> 11, no rejections, no
crashes, announce list opens cleanly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
on_full_message_close() runs from the Close button's own callback, and the button
is a descendant of the modal — so lv_obj_del(modal) freed the button mid-dispatch
(use-after-free). lv_obj_del_async defers the delete until the event completes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Long-press a bubble to open a scrollable full-message view with Copy/Close.
Bubbles render truncated for scroll performance, so the handler recovers the FULL
stored content (row -> hash -> item) for the view -- which also fixes Copy, which
had regressed to copying the truncated label text after the render cap.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
A message with multi-KB content (e.g. a large bz2-delivered payload) rendered
untruncated, so LVGL laid it out as a 50+ line wrapped bubble and re-drew the
whole thing while scrolling past it -- crawling the UI. Cap the *displayed* text
to MAX_DISPLAY_CHARS; the full content stays stored. (Decompression is unrelated:
it happens once in Resource::assemble() at receive, content is saved already
decompressed, and load_message_metadata never re-decompresses.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
on_scroll() loaded a full MESSAGES_PER_PAGE batch synchronously under the LVGL
lock, which froze scrolling. Make it trigger the same incremental
tick_background_fill() streaming as the open path instead. _bg_fill_active is now
std::atomic since on_scroll() (LVGL task) sets it while tick_background_fill()
(main loop) reads it; the target is written before the flag for visibility.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Open a conversation by rendering only the 3 newest messages synchronously (fast),
then stream the rest of the first page in BG_FILL_BATCH (2) at a time from
UIManager::update() via tick_background_fill(). Each step holds the LVGL lock
only briefly, so a large conversation no longer freezes the UI or trips
LVGLLock's 5s timeout (which previously asserted/crashed). Runs on the main loop
rather than a task because MessageStore shares one _json_doc between save and
load and is not safe for concurrent access.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
ChatScreen::refresh() loads + renders MESSAGES_PER_PAGE messages while the LVGL
mutex is held (the open path runs on the LVGL task's lv_task_handler). On a 32+
message conversation with a memory-pressured heap, 20 LittleFS reads + parses
exceeded LVGLLock's 5s timeout and asserted (crash). Cap to 10 so the under-lock
work stays well under budget; older messages load on scroll.
This is a mitigation. The real fix is to do the message I/O off the LVGL lock
(load lock-free on the main loop, render under the lock) so a large conversation
neither freezes the UI nor risks the timeout.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Repins microReticulum + microLXMF onto the upstream-0.4.1 graft and adapts
pyxis to the new src/microReticulum/ layout and 0.4.x APIs. The far-diverged
0.3.0 fork's Resource/Transport/Identity work is subsumed by upstream's
reimplementation; only the still-needed fixes ride on the pinned branches
(PKCS7/HMAC/X25519 crypto -- proven byte-identical to python RNS 1.3.1 --
Packet link-proof callback, Identity short-sig guard, and the bz2 layer +
decompress-on-receive in Resource::assemble()).
Consumer-side changes:
- platformio.ini: pin microReticulum @2f21fee (pyxis-fixes-on-0.4.1) and
microLXMF @33760d0 (chore/microreticulum-0.4.1-layout); bump microStore
ceea8f5 -> c5fb69d (0.4.x requires the new BasicFileStore::init API);
-std=gnu++11 -> gnu++17 (upstream requires C++17).
- Namespace all microReticulum includes (angle + quote) to <microReticulum/...>
for the relocated layout; shim-local Utilities/Stream.h|Print.h preserved.
- Interface::send_outgoing now returns bool: update TCP/BLE/SX1262/Auto
overrides with correct success/failure returns.
- SDArchiveFileSystem::init(bool reformatOnFail=true) to match new microStore.
- Static Transport::get_path_table() -> path_table(); instance getter unchanged.
- Remove duplicate shim Cryptography/BZ2 (microReticulum provides it now; keep
lib/libbz2 as the ESP32 bzlib provider).
- patch_littlefs_paths.py: normalize microStore's LittleFS adapter paths to a
leading "/" -- ESP32 Arduino LittleFS rejects "./"-prefixed paths, which
silently broke the path store (no peer paths learned, all messaging blocked).
Validated on T-Deck Plus: builds (RAM 27.5% / Flash 77.7%), boots stable
(no WDT/panic), and a full on-device LXMF e2e (DIRECT + OPPORTUNISTIC +
bz2-compressed-Resource receive) passes 5/5.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
refresh() called set_display_name() (a microStore/LittleFS write) while the
LVGL lock was held by UIManager::update(). On a cold-boot announce burst,
refresh() writes a name per newly-seen peer, serially stalling the LVGL render
task for the combined I/O time. Mirror the on_message_received fix: accumulate
the write-throughs in _pending_name_writes during refresh(), and flush them at
the top of UIManager::update() before it takes the LVGL lock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Greptile cleanup (greploop): universal_filesystem is no longer included by
main.cpp (migrated to microStore) -- drop the dead lib_dep so a clean build
doesn't pull its removed SPIFFS dependency. _stat_rx_packets_complete was
declared but never incremented or logged -- remove the unfinished counter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Companion to 21b0e96 which deleted BluedroidPlatform.{cpp,h} but
missed staging the three edited files that complete the cleanup:
- platformio.ini: remove the [env:tdeck-bluedroid] env block.
- .github/workflows/build-check.yml: drop tdeck-bluedroid from
the CI matrix.
- lib/ble_interface/BLEPlatform.cpp: drop the
USE_BLUEDROID-gated factory branches.
Same `git add` short-arg trap as the eridanus mishap earlier today
— specifying non-existent paths aborts the add before reaching the
M-file paths. Mental-model fix: stage M-files in a separate `git add`
from the staged-deletes.
The bluedroid BLE stack hasn't been the runtime path for a while;
NimBLE-Arduino is the canonical backend (lighter heap, more modern
API, what the live ble_interface uses). The bluedroid env+code were
still being maintained as a CI matrix entry, and just started
failing on the current branch — no value to keeping it green.
Changes:
- platformio.ini: remove the `[env:tdeck-bluedroid]` env block
entirely (was 159 lines, near-duplicate of [env:tdeck] modulo
`-DUSE_BLUEDROID`). Also remove the stale comment header that
used to sit above it.
- .github/workflows/build-check.yml: drop `tdeck-bluedroid` from
the build matrix.
- lib/ble_interface/platforms/BluedroidPlatform.{cpp,h} deleted
(2 files, ~81 KB / ~2000 LOC of dead code — all gated behind
`#if defined(USE_BLUEDROID)` which can no longer be defined).
- lib/ble_interface/BLEPlatform.cpp: drop USE_BLUEDROID-gated
factory branches (PlatformType::ESP_IDF case + the BluedroidPlatform.h
include + the "Bluedroid takes priority" detection clause).
Left in place:
- PlatformType::ESP_IDF enum member in BLETypes.h — dormant value,
not worth a coordinated removal sweep.
- USE_BLUEDROID build flag was already absent from [env:tdeck]'s
flags (this env always used NimBLE in production); just no
longer ever defined anywhere.
Build verified clean: `pio run -e tdeck` succeeds with the same
27.4% RAM / 79.8% Flash shape.
SDArchiveFileSystem.h: split listDirectory's SPI bus mutex hold into
per-entry acquire/release cycles. The previous all-or-nothing hold
scaled with archive size; a directory with thousands of entries
could keep the bus locked past LoRa/display's 500 ms acquire timeout
(LoRa TX would fail silently, display would tear, RX FIFO could
overflow under inbound flood). The new pattern records each entry's
name to a std::string, releases the bus, invokes the callback (or
appends), then re-acquires for the next entry. Callback runs outside
the critical section. SD cursor lives in the root File so other-CS
bus users (display CS, LoRa CS) between iterations don't clobber it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
SDArchiveFileSystem.h: guard release_bus() in FileImpl::close() on
the acquire_bus return value, matching the pattern every other
method already uses. Previously close() (which is also called from
~FileImpl) issued an unconditional xSemaphoreGive even when
acquire_bus(500) timed out, skewing the SPI bus mutex counter on
each over-release.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
sh123/esp32_codec2 PR #4 (merged Jan 6 2026) updated the bundled
codec2 to the modern stable line — addresses the v0.9.2-vs-pycodec2-
v1.2.0 interop gap that motivated vendoring back on May 8. With the
version gap closed, the standalone vendor in lib/codec2/ becomes
pure noise: 42 .c source files (~2.9 MB tree) sitting in the repo
when a single lib_deps line plus two build_flags get the same
behaviour.
Changes:
- platformio.ini lib_deps (both env blocks): `codec2` (vendored)
→ `sh123/esp32_codec2@^1.0.7`. Comment notes the version
history so future eyes don't re-vendor.
- platformio.ini build_flags (both env blocks): add
`-D__EMBEDDED__` and `-DMEMORY_CRITICAL` so codec2's codebook
tables land in flash (.const) instead of BSS. Moved here from
lib/codec2/library.json's `build.flags`. Without these, ~127 KB
of codebook data competes with LVGL's framebuffer in BSS.
- lib/codec2/ (vendored tree) deleted — 106 files, ~43,000 lines.
PIO will fetch sh123 into .pio/libdeps/tdeck/esp32_codec2/ on
first build.
Size impact:
- RAM: 27.4% → 27.4% (no change; codebooks were already in flash
via __EMBEDDED__).
- Flash: 77.0% → 79.8% (+~88 KB). sh123 ships the full codec2
source (69 .c files) vs the trimmed vendor (42 files): unused
paths like FreeDV, OFDM, COHPSK, FSK get linked in. Well below
the 100% partition limit; can revisit with `lib_ignore` if
flash pressure increases.
This is a real-functional change vs the rest of this PR which is
purely build-config rewiring. Audio fidelity should be unchanged
(sh123 carries the same codec2 SHA range as the vendor), but worth
revalidating with the codec2 fidelity test against pycodec2 before
merging.
microLXMF flipped public today; microReticulum's pyxis-fixes-on-0.3.0
fork branch has been public throughout. Both are now reachable
without machine-local clones, so the file:// pin + lib/microLXMF
symlink workarounds can go.
Changes:
- platformio.ini lib_deps: replace `file://~/repos/microReticulum`
with `https://github.com/torlando-tech/microReticulum.git#3ee2bd8`
(the same SHA the conformance bridge fetches).
- platformio.ini lib_deps: add `https://github.com/torlando-tech/microLXMF.git
#9876dff` (main HEAD after PR #4) as an explicit lib_dep.
- lib/microLXMF symlink to ~/repos/microLXMF removed — no longer
needed; PIO will fetch into .pio/libdeps/tdeck/microLXMF/ on
first build.
- platformio.ini build_flags: remove `-Ideps/microReticulum/src`
from both [env] sections. The hardcoded -I path was shadowing
PIO's auto-include of .pio/libdeps/tdeck/microReticulum/src/
(build_flags are searched first), letting the submodule's
potentially-stale headers win over the freshly-fetched git pin.
Now headers come from the same SHA as the .cpp source.
Pin-coordination note (in the inline comment): bump both SHAs in
tandem if either upstream changes, AND keep them in sync with
microLXMF/conformance-bridge/CMakeLists.txt's FetchContent_Declare
tag — otherwise the bridge tests against a different microReticulum
than the firmware does.
Build: `pio run -e tdeck` succeeds with the new deps, 27.4% RAM,
77.0% Flash — same shape as before. Verified both deps fetched
from github with `git config --get remote.origin.url` in
.pio/libdeps/tdeck/.
New testing/docs surface for grabbing the active LVGL screen as a
PNG over USB-CDC. Useful both for documentation (round-trip capture
of every public screen via T:SHOW <name> + T:SCREENSHOT) and
automated UI regression tests.
On-device:
- lib/lv_conf.h — enable LV_USE_SNAPSHOT (~5 KB code; uses PSRAM
via the existing hybrid allocator so internal RAM is unaffected)
- src/main.cpp — T:SCREENSHOT handler takes an lv_snapshot_take()
of lv_scr_act() under LVGL_LOCK(), dumps a delimited base64
stream over CDC. Inlines a tiny base64 encoder (no new dep).
- src/main.cpp — T:SHOW <name> dispatches to UIManager::show_*()
for the six publicly-navigable screens (conversation_list,
compose, announces, status, settings, propagation_nodes).
Wire format:
T:SCREENSHOT BEGIN W=320 H=240 FMT=rgb565<be|le> BYTES=153600
<base64 line, 76 chars>
...
T:SCREENSHOT END
Host side:
- screenshot.py — auto-detects the pyxis port via T:ID probe,
sends T:SCREENSHOT, reads until END, filters out interleaved
log lines (heap heartbeats / BLE stats can splice in
mid-dump), validates byte count matches header, decodes RGB565
with the documented byte order, expands channels via 5→8 / 6→8
high-bit replication, saves PNG. Pillow + pyserial.
Catalog:
- docs/serial_commands.md — full reference for all T:* commands
accumulated so far (identity/paths/send/receive, propagation,
voice, BLE, UI). New commands should land here when added.
Throughput: ~205 KB base64 over CDC at 115200 → ~18 s/shot. Fine
for docs and automated tests, not video. Bumping baud or zlib-
compressing on-device is queued in the doc as future work.
Pre-this the 10s heartbeat reported running/scanning/connected/peers
state but nothing about whether data was actually flowing. With the
counters added to BLEInterface and threaded into the heartbeat
snprintf, the line now also surfaces:
tx_pkt — outbound RNS packets attempted
tx_frag — BLE fragments actually written/notified
tx_b — total bytes written
tx_fail — platform write/notify returned false
rx_frag — BLE fragments handed to the reassembler
rx_b — total bytes received
That was enough to root-cause the Columba-side stalls observed
during the BLE end-to-end testing session: pyxis showed connected=1
but tx_pkt frozen, surfacing that the keepalive loop wasn't firing
for a peer whose handshake had completed but identity recording
raced. Cumulative-since-start, no reset; cheap to keep on always.
Two related changes:
1. NimBLE advertising overflow
At boot pyxis was logging "NimBLEAdvertisementData: Data length
exceeded" twice. The 128-bit Reticulum service UUID is 18 bytes
once you include the AD type+length headers; the device name
"TD-XXXXXX" is another 9-11 bytes; flags eat 3 bytes. That's
already over the 31-byte legacy adv-packet limit, so NimBLE was
silently truncating the advertisement and dropping the service
UUID. Android Columba's BleScanner filters by ServiceUuid at the
Android BLE driver layer (ScanFilter.Builder().setServiceUuid),
so without the UUID in the primary adv data, pyxis was invisible
to Columba.
Fix: call enableScanResponse(true) BEFORE addServiceUUID +
setName. NimBLE then routes the long device name into the
secondary 31-byte scan-response payload that active scanners
request, leaving the primary adv data with just flags + the
service UUID — under budget and visible to the filter.
Verified: with the fix, Android system Bluetooth reads pyxis's
name as "TD-46cbcf" and Columba's BleGattServer logs
"Central connected: FC:69:15:9C:B2:C9" (pyxis as central). The
connection holds for ~40s before HCI_CONN_TIMEOUT — separate
issue not addressed here, just the unblock so the link can be
established at all.
2. T:BLE on|off harness hook
Mirrors T:CALL_PROFILE / T:ANNLXST / T:LXSTDEST: persists the
ble_en NVS key and starts/stops the interface live so the LXMF
harness can flip BLE on/off the same way it drives any other
subsystem. Idempotent for "already on" / "already off". Useful
for upcoming pyxis ↔ Android Columba BLE smoke tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three changes, motivated by debugging "Sideband + pyxis on the same
WiFi don't hear each other's announces":
1. Always call \`mld6_joingroup_netif()\` in addition to \`setsockopt
IPV6_JOIN_GROUP\`. On ESP-IDF lwIP, the setsockopt path returns
success but doesn't reliably push the multicast hash into the
WiFi MAC filter — incoming multicast frames get silently dropped
at L2. Calling the netif's mld6 API directly programs the chip
filter. Joining twice on the netif is refcount-safe.
2. Set IPV6_MULTICAST_LOOP=1 so pyxis receives its own multicast
echoes. ESP-IDF lwIP defaults this off, which makes upstream's
"carrier lost / multicast echo timeout" warning fire even on a
functioning network. With LOOP=1, the initial-echo path actually
works on isolated test setups too. Logged as DEBUG if the
platform doesn't support the option.
3. Add a periodic \`AutoInterface: stats announce_tx=N tx_fail=N
disc_rx=N disc_self=N data_rx=N peers=N\` heartbeat (every 10s).
Without this it's hard to tell whether pyxis isn't sending,
isn't receiving, or is sending+receiving but rejecting the
tokens. Discovery-RX from non-self addresses with bad tokens
now also logs once with the hex prefix so token-mismatch cases
are visible (group_id drift, scope-suffix encoding mismatches).
Added _initial_echo_received update on first self-echo so the
firewall warning at startup_grace fires correctly.
After this, pyxis's own multicast loopback works (disc_self=N
matches announce_tx=N within 10s). Cross-LAN multicast against
rnsd / Sideband still doesn't make it through, which is an ESP32
WiFi multicast TX limitation — pyxis's frames aren't reaching the
AP. Not a fix here; the diagnostics make the boundary visible.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
on_message_received() called \`_conversation_list_screen->refresh()\`
unconditionally per message. Under propagation-sync flood (50+ queued
messages delivered back-to-back) that's 50 full LVGL list redraws,
each holding LVGL_LOCK across:
- lv_obj_clean(_list) + reload conversations from MessageStore
- per-conversation container construction
- per-peer Identity::recall_app_data calls for display names
- SPI display flush
The refreshes serialize behind LVGL_LOCK, the SPI bus stays saturated
flushing dirty regions, and pyxis's USB CDC TX buffer overflows
because the main loop is too busy with display work to drain the
serial-output FIFO. Harness commands time out as a side effect.
Replace the per-message refresh with a coalescing flag drained from
update():
- on_message_received only sets _pending_conversation_refresh
- update() refreshes at most once per 750ms
- update() also skips the refresh entirely when the user isn't on
the conversation list (show_conversation_list refreshes when they
navigate back, so nothing's lost — a chat-screen user gets quiet
background ingestion)
Validated under a 2-round LXMF soak: direct + opportunistic short and
medium messages all PASS round-trip with the coalescing in effect.
Propagation flake is unchanged (known timing issue between pyxis
upload and bot's 8s sync poll, not a UI regression).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Earlier commit gated this print behind \`peak > 1000 || ringDrops >
0\`. Backfired in callee mode: t-deck speaker plays incoming TTS,
mic picks it up via acoustic feedback, peak rises above 1000, the
print fires every 2s anyway, and serial saturates ~12s into the call
— pyxis stops responding to T:CALL_QOS / T:CALL_STATS even though
audio is still flowing fine.
Now only print on ring drops (an actual problem). Counters keep
updating internally for callers that need them.
After this, --callee mode validates clean: pyxis_tx=34 rx=115
decode_ok=120 decode_fail=0 pyxis_rms=4345 with TTS bot. Both
directions of pyxis ↔ real LXST.Telephone interop now PASS
end-to-end at ULBW (Codec2-700C, 0x10).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Real-LXST 14s call at ULBW (Codec2-700C) was timing out T:CALL_QOS /
T:CALL_STATS responses ~10-15s in. Pyxis itself was still processing
audio fine; the host's serial reader was just overrun by debug-level
prints from three sources, all firing per-packet during voice traffic:
1. TCPClientInterface: per-frame "[TCP] Reading X bytes" / "[TCP]
First bytes: ..." / "[HDLC] Frame #N: ..." / "[TCP] Processing
frame" / 5s "[TCP] connected= ..." were unconditional Serial.printf.
Now gated behind `RNS::loglevel() >= LOG_DEBUG` and the snprintf
work skipped when it'd be discarded.
2. i2s_capture.cpp: "[CAP] rate=" fired every 2s regardless of
activity. Now only emits when ringDrops > 0 OR runningPeak > 1000
(something happened worth noting). Counters still update — only
the print is gated.
3. microStore upstream: "[ustore] get: key not found in index" fires
on every path-store miss, which RNS hits constantly during a call.
patch_filestore.py was already a registered pre-build script for
diagnostic patches; reactivate it (was commented out in
platformio.ini) and add a silence patch as the always-on default.
Diagnostic exists()/put() patches gated behind PYXIS_FILESTORE_DIAG=1
so they're easy to bring back when investigating path-store drift
without touching the script each time.
After this, the ULBW real-LXST call validator returns PASS with full
final stats (pyxis_tx=34 rx=119 decode_ok=151 decode_fail=0
pyxis_rms=4410). 1600bps/3200bps profiles still hit serial-timeout
patterns under sustained TX — likely CPU saturation in the main
loop, separate from this fix; tracked in #75 followup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two harness hooks for testing pyxis as the LXST callee:
- T:LXSTDEST: returns pyxis's lxst.telephony destination hash. The
caller-side bot (real LXST.Telephony.Telephone in this case) needs
this to dial pyxis. Backed by a new test_lxst_dest_hex() accessor on
UIManager that reads _lxst_destination and returns hex (or empty if
the destination isn't registered yet).
- T:ANNLXST: forces a fresh announce of the lxst.telephony destination.
The TCP-reconnect handler in main.cpp:963 only announces LXMF, so on
a fresh boot the lxst.telephony destination is missing from rnsd's
cache and link requests addressed to it get dropped with "no known
path to final destination" (rnsd debug log). The harness pings this
before each callee test to ensure rnsd has a fresh path.
Also adds INFO logging to announce_lxst() mirroring the LXMF announce
log (Announcing destination: <hash> ... announce sent), so it's
visible in tdeck-side traces when an announce actually went out vs.
silently no-oped.
Validated: bot dials, pyxis transitions IDLE -> INCOMING_RINGING (UI
shows incoming-call screen), harness sends T:CALL_ANSWER, state
becomes ACTIVE. (Audio path crashes shortly after via the same
Ed25519 announce-validation bug — fix landed in microReticulum
f4bad06, but PIO's libdeps cache had been holding a stale copy; a
manual rm -rf .pio/libdeps/tdeck/microReticulum was needed to pick it
up. Tracked in #73.)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds two profile constants beyond the existing LXST_PROFILE_LBW (0x30,
Codec2-3200): LXST_PROFILE_VLBW (0x20, Codec2-1600) and
LXST_PROFILE_ULBW (0x10, Codec2-700C). Default is now ULBW — a 700C
frame fits comfortably inside an SF7-9 LoRa packet, which is the
target medium for pyxis voice. The previous 3200bps default was 4.5x
larger and unsuitable for the radio path.
Profile is selectable at runtime via T:CALL_PROFILE [hex]. Replaces
five hardcoded LXST_PROFILE_LBW sites: three audio-init paths in
call_process_signal and two profile-negotiation send_signal calls.
Adds T:CALL_ANSWER for harness pyxis-as-callee testing — sets the
same _call_answer_pending flag the UI button does so call_answer()
runs on the main loop in its proper context. Validates against
real LXST.Telephony.Telephone callers.
Validated: pyxis dialed real LXST upstream Telephone bot, negotiated
ULBW end-to-end, reached STATUS_ESTABLISHED, decoded frames cleanly
both directions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VoiceFilterChain::applyHighPass collapsed to a constant-gain
multiplier instead of a real high-pass response. The inner loop
read `samples[prevIdx]` as "previous input" — but that slot held
y[n-1] (output) since the previous iteration had overwritten it.
Substituting that into the formula
y[n] = α(y[n-1] + x[n] - x[n-1])
with x[n-1] := y[n-1] gave
y[n] = α(y[n-1] + x[n] - y[n-1]) = α · x[n]
i.e. just a fixed gain ≈ 0.81 at 300Hz cutoff / 8kHz. DC offsets
sailed through; the chain only kept signal levels reasonable
because the AGC stage downstream pulled the residual toward target.
The same bug exists in upstream LXST-kt's
native_audio_filters.cpp (filed as LXST-kt#13).
Fix: walk per-channel with explicit `xPrev` / `yPrev` variables so
input history isn't clobbered by the output write. Per-chunk save
of `lastInputs[ch]` now stores the actual last input, not the last
output, so the first sample of each new chunk uses the correct
x[n-1].
Test `dc_offset_attenuated_by_hpf` updated to assert tail RMS
< 0.01 (was < 0.5 — accommodating the broken behavior). Pass.
End-to-end acoustic test (Mac speaker → T-Deck mic): pyxis_rms
went up from 4378 → 5835, bot_rms from 1194 → 1517 — DC offset
removal lets clean signal through better.
Pyxis-side fix only. LXST-kt should pick up the same fix
upstream — see project_lxst_hpf_filter_bug.md in the vault.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PlatformIO dep sh123/esp32_codec2_arduino@1.0.7 bundles codec2
v0.9.2. Mac-side pycodec2 v3.0.4 links libcodec2 v1.2.0. Years of
codec2 development between those releases.
Replace the upstream lib with a local vendor of drowe67/codec2 v1.2.0
under lib/codec2/. Trim the 191-file source tree down to the
~100 files actually needed for codec2 (drop FreeDV, OFDM, COHPSK,
FSK, FM-FSK, LDPC, Horus, CLI tools — pyxis only uses
codec2_create/destroy/encode/decode + samples_per_frame /
bytes_per_frame). Carry over the v0.9 codebook .c files since the
codebook contents matched (compared against v1.2's src/codebook/*.txt).
Define __EMBEDDED__ so the codebooks land in flash (.const) rather
than RAM. Without it the codebooks add ~127KB to BSS and the LVGL
task fails to start (RAM was 65% full vs 27% with __EMBEDDED__).
Provide trivial codec2_malloc/codec2_free wrappers in
codec2_alloc_esp32.c (codec2 v1.2 expects them when __EMBEDDED__ is
defined; ESP-IDF's malloc/free already pull from internal RAM).
Also explicitly add SD/FS to lib_deps and #include <SD.h> in
main.cpp — the previous esp32_codec2 dep transitively pulled SD
which let SDArchiveFileSystem.h get away with depending on it
implicitly. With chain+ ldf mode and no esp32_codec2 dep, we have
to declare the framework lib explicitly.
DOES NOT fix the ~30x speech-decode RMS asymmetry between pycodec2
self-tests (~5800) and pyxis decoding the same encoded bytes (~170).
Sine waves and 3-formant synthesis pass clean both directions; only
real TTS speech triggers it. Probably a separate codec-state
divergence (the encoder/decoder are independent codec2 instances
in pyxis, both fresh per call) or a wire-format quirk we still need
to track down. v1.2 is the right baseline regardless — same bug
class as several upstream fixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codec2-3200 is a SPEECH codec — pure-tone round-trip retains only
~12% of input RMS. The 1kHz-sine variant of the LXST QoS test
worked but tested the codec on its worst-case input.
Switch I2SCapture::setInjectSine to generate a synthesized voice-
like signal instead of a pure sine: F1 at the freq arg (default
730Hz, "ah" formant), F2 at 1.5·F1, F3 at 3.3·F1, all summed with
weights 0.55 / 0.30 / 0.15, modulated by a 120Hz amplitude envelope
that emulates glottal pulses. Phase-continuous so the encoder never
sees a discontinuity.
Codec2 retains far more energy on this content (~5400 RMS vs the
~1900 the sine produced) — same gate, much wider quality margin.
Validated end-to-end:
pyxis_rms=4378 bot_rms=5438 decode_fail=0+0 PASS
The LXST harness drives this via the same T:CALL_INJECT command
(API unchanged); only the in-firmware generator changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the LXST harness from frame-flow only (#62) and decoder QoS
only (#63) to a full content-fidelity test: a known 1kHz sine wave
flows in BOTH directions through the Codec2-3200 round-trip, and
the harness asserts the decoded RMS at each end matches expected
energy within tolerance (after Codec2's lossy speech-codec behavior).
Firmware additions (under PYXIS_TEST_HOOKS):
I2SCapture
setInjectSine(enabled, freq=1000, amp=0.5)
Replaces mic input with a phase-continuous synthesized sine.
Bypasses ES7210 capture and the voice filter chain so the
encoder sees pure samples — peer's decoded RMS validates that
pyxis's TX path delivers content.
I2SPlayback
pcmSampleCount(), pcmSumSquares()
Decoded-PCM energy accumulators, fed from each successful
Codec2 decode. uint64 sumsq holds ~2³⁴ frames before overflow,
far longer than any test call.
LXSTAudio + UIManager (test-only)
captureSetInjectSine, playbackPcmSampleCount, playbackPcmSumSquares
test_call_set_inject_sine, test_call_pcm_sample_count,
test_call_pcm_sum_squares
Serial T: hooks (main.cpp)
T:CALL_INJECT <on|off> [freq] [amp_pct]
Drive the capture-side injection from the harness.
T:CALL_QOS now also returns pcm_n + pcm_ss
Harness divides + sqrts to RMS for content validation.
Validated with /tmp/lxst_call_harness.py + /tmp/lxst_call_bot.py
(scripts vault-local per the no-PII rule):
pyxis_rms = 6363, bot_rms = 1930, decode_fail = 0
PASS: bidirectional audio + content-fidelity validated
The empirical RMS floor is 800 (pycodec2 self-test on 1kHz amp 0.5
yields ~1400; pyxis decoder hits ~6300; bot decoder hits ~1900 —
all far above the ~5-50 silence floor). Codec2 is a speech codec
so pure-tone round-trip is naturally lossy; the test gates on
"audio bytes carry actual content energy", not lossless round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the on-device test surface the LXST voice-call harness needs to
drive an end-to-end audio flow + QoS validation against a Mac-side
softphone bot. Both hooked behind PYXIS_TEST_HOOKS so production
firmware is unaffected.
T: serial commands (main.cpp):
T:CALL <hex> — initiate call (hex = peer's lxst.telephony dest)
T:CALL_STATE — current call FSM state name
T:CALL_HANGUP — tear down the active call
T:CALL_STATS — audio frame counters: tx, rx, state
T:CALL_QOS — Codec2 decoder QoS: decode_ok, decode_fail, state
T:HASIDENTITY <h> — bool, distinct from T:HASPATH because path_store
and known_destinations are populated by separate
code paths (announce can land in one before the
other; harness has to wait for both).
UIManager additions (under PYXIS_TEST_HOOKS):
test_call_initiate, test_call_hangup, test_call_state_name,
test_call_audio_tx_count, test_call_audio_rx_count,
test_call_decode_ok, test_call_decode_fail.
I2SPlayback / LXSTAudio additions (always on — counters are tiny):
decodeOkCount(), decodeFailCount(), resetCounters() on I2SPlayback
surface the Codec2 decode success/fail rate. LXSTAudio re-exports.
Each writeEncodedPacket call increments exactly one counter so the
ratio is "wire-level audio fidelity" of the peer's encoder.
Validated:
Frame-flow soak (12s call): pyxis tx=73 rx=72, bot tx=230 rx=72.
QoS soak (12s call): pyxis tx=73 rx=76, decode_ok=81 decode_fail=0,
bot tx=227 rx=73.
Harness scripts that drive these hooks live in the local Obsidian
vault (under 80 Assistant/Memory/pyxis/soak_scripts/) — they encode
LAN-specific state and aren't checked in. See the vault's
automated_soak_testing.md for the full procedure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes from the same testing session:
1. Make SplashImage.h a checked-in artifact rather than a per-build
gitignored regen. The pre-build script generate_splash.py needs
cairosvg + Pillow in the PlatformIO python env, but its import
block silently skips on ImportError — every contributor missing
those deps would silently get a black-screen splash with no clear
signal. Checked-in artifact means a fresh checkout works
regardless of local python state; the script's
should_regenerate() only refreshes when pyxis-icon.svg is newer
than the header, so the committed copy stays in sync for
contributors who DO have the deps.
2. ChatScreen now uses the same three-tier display name resolution
as ConversationListScreen (live announce → MessageStore-cached →
truncated hash) and writes through to the persistent cache when
the live cache hits. Previously it always fell to the truncated
hash if Identity::recall_app_data was empty, even when the
MessageStore had a cached name from a prior session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related bugs in the conversation list:
1. Without GPS/NTP, Utilities::OS::time() returns uptime seconds — way
smaller than any real unix-epoch message timestamp. The
format_timestamp "diff < 0 → Future" branch then fired on every
row. Add a sane-epoch threshold (2024-01-01) below which we render
"?" instead, since "Future" is misleading when it just means
"we don't know what time it is."
2. Identity::recall_app_data is in-memory only and lost on reboot. The
conversation list always re-fell back to truncated hashes on cold
start. Wire the three-tier resolution flow: live announce →
MessageStore-persisted name → hash. When the live cache hits, write
through to the persisted side via MessageStore::set_display_name so
future cold boots get the name back immediately.
Pulls in microLXMF 5531a59 (MessageStore display-name cache impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TinyGPSPlus's `satellites` only updates from $GPGGA (sats USED in
the position fix). A module seeing the sky but not yet acquired
shows fix=0 — previously rendered as red "0", which read as
"GPS broken." It's actually "GPS healthy, waiting for lock."
Bind a TinyGPSCustom to $GPGSV field 3 (satellites in view) and
fall through to it when fix-sats is 0:
-- muted no GPS handle / no NMEA
? yellow NMEA flowing, no fix, no GSV count yet
?N yellow N satellites visible but not yet locked
N colored N satellites locked in fix
So a cold start now shows a meaningful number ticking up as the
module finds birds, then flips to a green N once it gets a fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The top status bar's LoRa indicator was hard-disabled (`if (false &&
_lora_interface)`) — every render printed "--" no matter what the
radio was doing. Re-enable it: SX1262Interface::get_rssi() is
non-virtual on the impl class, so we drop down to the InterfaceImpl*
via Interface::get() and static_cast to SX1262Interface.
Also adds a third tier to the GPS readout so "module connected but no
fix yet" is distinguishable from "no GPS hardware":
-- muted no GPS handle, or no NMEA bytes parsed
? yellow NMEA flowing but no $GPGGA sat-count yet
N colored satellite count valid
Before: a TinyGPSPlus that was happily streaming $GPGSV but hadn't
yet parsed a $GPGGA showed "--" indistinguishably from a missing
module. The "?" tier surfaces "alive, waiting for sky."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
