Adds two profile constants beyond the existing LXST_PROFILE_LBW (0x30,
Codec2-3200): LXST_PROFILE_VLBW (0x20, Codec2-1600) and
LXST_PROFILE_ULBW (0x10, Codec2-700C). Default is now ULBW — a 700C
frame fits comfortably inside an SF7-9 LoRa packet, which is the
target medium for pyxis voice. The previous 3200bps default was 4.5x
larger and unsuitable for the radio path.
Profile is selectable at runtime via T:CALL_PROFILE [hex]. Replaces
five hardcoded LXST_PROFILE_LBW sites: three audio-init paths in
call_process_signal and two profile-negotiation send_signal calls.
Adds T:CALL_ANSWER for harness pyxis-as-callee testing — sets the
same _call_answer_pending flag the UI button does so call_answer()
runs on the main loop in its proper context. Validates against
real LXST.Telephony.Telephone callers.
Validated: pyxis dialed real LXST upstream Telephone bot, negotiated
ULBW end-to-end, reached STATUS_ESTABLISHED, decoded frames cleanly
both directions.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
VoiceFilterChain::applyHighPass collapsed to a constant-gain
multiplier instead of a real high-pass response. The inner loop
read `samples[prevIdx]` as "previous input" — but that slot held
y[n-1] (output) since the previous iteration had overwritten it.
Substituting that into the formula
y[n] = α(y[n-1] + x[n] - x[n-1])
with x[n-1] := y[n-1] gave
y[n] = α(y[n-1] + x[n] - y[n-1]) = α · x[n]
i.e. just a fixed gain ≈ 0.81 at 300Hz cutoff / 8kHz. DC offsets
sailed through; the chain only kept signal levels reasonable
because the AGC stage downstream pulled the residual toward target.
The same bug exists in upstream LXST-kt's
native_audio_filters.cpp (filed as LXST-kt#13).
Fix: walk per-channel with explicit `xPrev` / `yPrev` variables so
input history isn't clobbered by the output write. Per-chunk save
of `lastInputs[ch]` now stores the actual last input, not the last
output, so the first sample of each new chunk uses the correct
x[n-1].
Test `dc_offset_attenuated_by_hpf` updated to assert tail RMS
< 0.01 (was < 0.5 — accommodating the broken behavior). Pass.
End-to-end acoustic test (Mac speaker → T-Deck mic): pyxis_rms
went up from 4378 → 5835, bot_rms from 1194 → 1517 — DC offset
removal lets clean signal through better.
Pyxis-side fix only. LXST-kt should pick up the same fix
upstream — see project_lxst_hpf_filter_bug.md in the vault.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The PlatformIO dep sh123/esp32_codec2_arduino@1.0.7 bundles codec2
v0.9.2. Mac-side pycodec2 v3.0.4 links libcodec2 v1.2.0. Years of
codec2 development between those releases.
Replace the upstream lib with a local vendor of drowe67/codec2 v1.2.0
under lib/codec2/. Trim the 191-file source tree down to the
~100 files actually needed for codec2 (drop FreeDV, OFDM, COHPSK,
FSK, FM-FSK, LDPC, Horus, CLI tools — pyxis only uses
codec2_create/destroy/encode/decode + samples_per_frame /
bytes_per_frame). Carry over the v0.9 codebook .c files since the
codebook contents matched (compared against v1.2's src/codebook/*.txt).
Define __EMBEDDED__ so the codebooks land in flash (.const) rather
than RAM. Without it the codebooks add ~127KB to BSS and the LVGL
task fails to start (RAM was 65% full vs 27% with __EMBEDDED__).
Provide trivial codec2_malloc/codec2_free wrappers in
codec2_alloc_esp32.c (codec2 v1.2 expects them when __EMBEDDED__ is
defined; ESP-IDF's malloc/free already pull from internal RAM).
Also explicitly add SD/FS to lib_deps and #include <SD.h> in
main.cpp — the previous esp32_codec2 dep transitively pulled SD
which let SDArchiveFileSystem.h get away with depending on it
implicitly. With chain+ ldf mode and no esp32_codec2 dep, we have
to declare the framework lib explicitly.
DOES NOT fix the ~30x speech-decode RMS asymmetry between pycodec2
self-tests (~5800) and pyxis decoding the same encoded bytes (~170).
Sine waves and 3-formant synthesis pass clean both directions; only
real TTS speech triggers it. Probably a separate codec-state
divergence (the encoder/decoder are independent codec2 instances
in pyxis, both fresh per call) or a wire-format quirk we still need
to track down. v1.2 is the right baseline regardless — same bug
class as several upstream fixes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codec2-3200 is a SPEECH codec — pure-tone round-trip retains only
~12% of input RMS. The 1kHz-sine variant of the LXST QoS test
worked but tested the codec on its worst-case input.
Switch I2SCapture::setInjectSine to generate a synthesized voice-
like signal instead of a pure sine: F1 at the freq arg (default
730Hz, "ah" formant), F2 at 1.5·F1, F3 at 3.3·F1, all summed with
weights 0.55 / 0.30 / 0.15, modulated by a 120Hz amplitude envelope
that emulates glottal pulses. Phase-continuous so the encoder never
sees a discontinuity.
Codec2 retains far more energy on this content (~5400 RMS vs the
~1900 the sine produced) — same gate, much wider quality margin.
Validated end-to-end:
pyxis_rms=4378 bot_rms=5438 decode_fail=0+0 PASS
The LXST harness drives this via the same T:CALL_INJECT command
(API unchanged); only the in-firmware generator changed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Extends the LXST harness from frame-flow only (#62) and decoder QoS
only (#63) to a full content-fidelity test: a known 1kHz sine wave
flows in BOTH directions through the Codec2-3200 round-trip, and
the harness asserts the decoded RMS at each end matches expected
energy within tolerance (after Codec2's lossy speech-codec behavior).
Firmware additions (under PYXIS_TEST_HOOKS):
I2SCapture
setInjectSine(enabled, freq=1000, amp=0.5)
Replaces mic input with a phase-continuous synthesized sine.
Bypasses ES7210 capture and the voice filter chain so the
encoder sees pure samples — peer's decoded RMS validates that
pyxis's TX path delivers content.
I2SPlayback
pcmSampleCount(), pcmSumSquares()
Decoded-PCM energy accumulators, fed from each successful
Codec2 decode. uint64 sumsq holds ~2³⁴ frames before overflow,
far longer than any test call.
LXSTAudio + UIManager (test-only)
captureSetInjectSine, playbackPcmSampleCount, playbackPcmSumSquares
test_call_set_inject_sine, test_call_pcm_sample_count,
test_call_pcm_sum_squares
Serial T: hooks (main.cpp)
T:CALL_INJECT <on|off> [freq] [amp_pct]
Drive the capture-side injection from the harness.
T:CALL_QOS now also returns pcm_n + pcm_ss
Harness divides + sqrts to RMS for content validation.
Validated with /tmp/lxst_call_harness.py + /tmp/lxst_call_bot.py
(scripts vault-local per the no-PII rule):
pyxis_rms = 6363, bot_rms = 1930, decode_fail = 0
PASS: bidirectional audio + content-fidelity validated
The empirical RMS floor is 800 (pycodec2 self-test on 1kHz amp 0.5
yields ~1400; pyxis decoder hits ~6300; bot decoder hits ~1900 —
all far above the ~5-50 silence floor). Codec2 is a speech codec
so pure-tone round-trip is naturally lossy; the test gates on
"audio bytes carry actual content energy", not lossless round-trip.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the on-device test surface the LXST voice-call harness needs to
drive an end-to-end audio flow + QoS validation against a Mac-side
softphone bot. Both hooked behind PYXIS_TEST_HOOKS so production
firmware is unaffected.
T: serial commands (main.cpp):
T:CALL <hex> — initiate call (hex = peer's lxst.telephony dest)
T:CALL_STATE — current call FSM state name
T:CALL_HANGUP — tear down the active call
T:CALL_STATS — audio frame counters: tx, rx, state
T:CALL_QOS — Codec2 decoder QoS: decode_ok, decode_fail, state
T:HASIDENTITY <h> — bool, distinct from T:HASPATH because path_store
and known_destinations are populated by separate
code paths (announce can land in one before the
other; harness has to wait for both).
UIManager additions (under PYXIS_TEST_HOOKS):
test_call_initiate, test_call_hangup, test_call_state_name,
test_call_audio_tx_count, test_call_audio_rx_count,
test_call_decode_ok, test_call_decode_fail.
I2SPlayback / LXSTAudio additions (always on — counters are tiny):
decodeOkCount(), decodeFailCount(), resetCounters() on I2SPlayback
surface the Codec2 decode success/fail rate. LXSTAudio re-exports.
Each writeEncodedPacket call increments exactly one counter so the
ratio is "wire-level audio fidelity" of the peer's encoder.
Validated:
Frame-flow soak (12s call): pyxis tx=73 rx=72, bot tx=230 rx=72.
QoS soak (12s call): pyxis tx=73 rx=76, decode_ok=81 decode_fail=0,
bot tx=227 rx=73.
Harness scripts that drive these hooks live in the local Obsidian
vault (under 80 Assistant/Memory/pyxis/soak_scripts/) — they encode
LAN-specific state and aren't checked in. See the vault's
automated_soak_testing.md for the full procedure.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two fixes from the same testing session:
1. Make SplashImage.h a checked-in artifact rather than a per-build
gitignored regen. The pre-build script generate_splash.py needs
cairosvg + Pillow in the PlatformIO python env, but its import
block silently skips on ImportError — every contributor missing
those deps would silently get a black-screen splash with no clear
signal. Checked-in artifact means a fresh checkout works
regardless of local python state; the script's
should_regenerate() only refreshes when pyxis-icon.svg is newer
than the header, so the committed copy stays in sync for
contributors who DO have the deps.
2. ChatScreen now uses the same three-tier display name resolution
as ConversationListScreen (live announce → MessageStore-cached →
truncated hash) and writes through to the persistent cache when
the live cache hits. Previously it always fell to the truncated
hash if Identity::recall_app_data was empty, even when the
MessageStore had a cached name from a prior session.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two related bugs in the conversation list:
1. Without GPS/NTP, Utilities::OS::time() returns uptime seconds — way
smaller than any real unix-epoch message timestamp. The
format_timestamp "diff < 0 → Future" branch then fired on every
row. Add a sane-epoch threshold (2024-01-01) below which we render
"?" instead, since "Future" is misleading when it just means
"we don't know what time it is."
2. Identity::recall_app_data is in-memory only and lost on reboot. The
conversation list always re-fell back to truncated hashes on cold
start. Wire the three-tier resolution flow: live announce →
MessageStore-persisted name → hash. When the live cache hits, write
through to the persisted side via MessageStore::set_display_name so
future cold boots get the name back immediately.
Pulls in microLXMF 5531a59 (MessageStore display-name cache impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TinyGPSPlus's `satellites` only updates from $GPGGA (sats USED in
the position fix). A module seeing the sky but not yet acquired
shows fix=0 — previously rendered as red "0", which read as
"GPS broken." It's actually "GPS healthy, waiting for lock."
Bind a TinyGPSCustom to $GPGSV field 3 (satellites in view) and
fall through to it when fix-sats is 0:
-- muted no GPS handle / no NMEA
? yellow NMEA flowing, no fix, no GSV count yet
?N yellow N satellites visible but not yet locked
N colored N satellites locked in fix
So a cold start now shows a meaningful number ticking up as the
module finds birds, then flips to a green N once it gets a fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The top status bar's LoRa indicator was hard-disabled (`if (false &&
_lora_interface)`) — every render printed "--" no matter what the
radio was doing. Re-enable it: SX1262Interface::get_rssi() is
non-virtual on the impl class, so we drop down to the InterfaceImpl*
via Interface::get() and static_cast to SX1262Interface.
Also adds a third tier to the GPS readout so "module connected but no
fix yet" is distinguishable from "no GPS hardware":
-- muted no GPS handle, or no NMEA bytes parsed
? yellow NMEA flowing but no $GPGGA sat-count yet
N colored satellite count valid
Before: a TinyGPSPlus that was happily streaming $GPGSV but hadn't
yet parsed a $GPGGA showed "--" indistinguishably from a missing
module. The "?" tier surfaces "alive, waiting for sky."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Hardware::TDeck::SDArchiveFileSystem — a microStore::FileSystem
adapter that piggybacks on the SD card already mounted by SDAccess::init
and serializes every operation through the shared SPI bus mutex (so it
cooperates with display + LoRa traffic on HSPI).
In setup_lxmf, after MessageStore is constructed, attach the SD as the
archive tier at "/lxmf-archive". When the SD is missing the message
store falls back to delete-on-cull (still bounded, just no historical
scrollback). Without this the LittleFS partition (1.875MB) fills in
~30 min of sustained receive and trips lfs_alloc divide-by-zero.
Also moves the LVGL_LOCK in UIManager::on_message_received to AFTER
the save_message call. The 5s LVGL lock timeout was tripping when
LittleFS compaction stalled the save for several seconds, panicking
the loop watchdog.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a USB-CDC serial command interface gated behind PYXIS_TEST_HOOKS
that lets a host-side harness drive pyxis end-to-end without UI taps.
Built specifically to run /tmp/tdeck_harness.py and prove LXMF DIRECT,
OPPORTUNISTIC, and PROPAGATED delivery against a Mac-side echo bot
over the Mac's rnsd + lxmd.
Commands (all newline-terminated, replies T:OK or T:ERR):
T:DEST — pyxis's delivery dest hash (hex)
T:ID — pyxis's identity hash (hex)
T:ANN — force an announce
T:PATHS — count + dump in-memory path table
T:HASPATH <hex> — Transport::has_path + in-memory check
T:RECALL <hex> — Identity::recall_app_data hex
T:SEND <hex> <text> — outbound DIRECT LXMessage
T:SENDOPP <hex> <text> — outbound OPPORTUNISTIC LXMessage
T:SENDPROP <hex> <text> — outbound PROPAGATED LXMessage
T:SETPROP <hex> <stamp_cost> — set outbound propagation node
T:SYNCPROP — request_messages_from_propagation_node
T:SYNCSTATE — current PR_* sync state
T:STATE <msg_hash> — LXMessage state for a tracked send
T:RX — drain inbound RX ring
T:RXCLR — clear RX ring
Build-flag side:
-DPYXIS_TEST_HOOKS — gates all of the above
-DPYXIS_TEST_TCP_HOST="..." — hard-overrides NVS tcp_host so the
harness's rnsd is the only target
-DPYXIS_TEST_TCP_PORT=... — same for tcp_port
Also: replaces `lib_extra_dirs = deps/microReticulum` with an explicit
`file://~/repos/microReticulum` lib_dep. lib_extra_dirs
caused PIO to compile microReticulum twice (once through the extra
dir, once through microLXMF's transitive auto-fetch), producing two
copies of `Transport::_path_store` in BSS. Different translation
units linked against different statics, so put() and exists() landed
in different in-memory indexes. Symptom: `T:HASPATH` returned 0 even
when the previous announce's `[ustore] put: wrote key` log line was
visible. Single source path → single static → consistent reads.
`patch_filestore.py` is committed but commented out in extra_scripts
— used during diagnostic when the dual-static issue was being
triaged. Easy to re-arm if FileStore put/exists drift recurs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Follows the project-wide SPIFFS → LittleFS migration. SPIFFS chokes
under sustained writes (GC stalls block writes for 100s of ms) and
its FileStore-backed path table was failing every put. LittleFS
tolerates the workload and is the new default. Boot profiler's tiny
log files don't care which backend they sit on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
auto_interface and ble_interface declared `"microReticulum": "*"` as
a PIO library dependency. With deep+ LDF mode, that triggered PIO to
auto-fetch a parallel microReticulum copy into
.pio/libdeps/tdeck/microReticulum/ alongside our intended
deps/microReticulum/ overlay. The linker would silently pick the
fetched copy, dropping any local fork's .cpp changes.
These libs only need microReticulum HEADERS (already provided by the
project-level `-Ideps/microReticulum/src` build_flag), not a separate
linkable library — the actual microReticulum .a is built once via
`lib_extra_dirs = deps/microReticulum`. Dropping the dependency
declaration prevents the duplicate fetch.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the vendored 11-file LXMF subtree under
lib/microreticulum-shim/LXMF/ with a symlink at lib/microLXMF
pointing at ~/repos/microLXMF. PlatformIO's lib_ldf_mode = deep+
follows the symlink, finds microLXMF/library.json, and treats it as
a regular library — no platformio.ini changes required.
Build verified: pio run -e tdeck succeeds, RAM 21.4%, Flash 71.3%
(comparable to pre-extract baseline).
The lib/microreticulum-shim/ "fork-only-glue" library remains in
place — it still owns BZ2 + libbz2 + Display + FileSystem +
Instrumentation + Utilities pieces that haven't been extracted.
For collaborators without ~/repos/microLXMF checked out, the
symlink will dangle. Once microLXMF has a tagged release we'll
swap the symlink for a `lib_deps` git URL pin.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fixes the StoreProhibited panics surfaced by hardware boot of fd2d3de:
StoreProhibited @ Bytes::assign (offset 0x1c on a null _data shared_ptr)
← Ed25519PublicKey ctor
← Ed25519PrivateKey::public_key()
← Link::Link(Destination&) at Link.cpp:79
Vanilla upstream microReticulum @ 0.3.0 Link's all-defaults ctor goes
through load_private_key() which dereferences _prv (a shared_ptr that's
still null on a freshly-default-constructed Link). Fork's Link probably
guarded against this; upstream doesn't. The explicit
Type::NoneConstructor branch (Link.h:153) leaves _object null and is
safe for default-init of pool/pre-allocated members.
Two sites needed the same fix:
- lib/microreticulum-shim/LXMF/LXMRouter.h DirectLinkSlot::link
(the fork's pre-allocated DIRECT-delivery link pool)
- lib/tdeck_ui/UI/LXMF/UIManager.h UIManager::_call_link
(the LXST call link state)
Both: `RNS::Link link;` -> `RNS::Link link{RNS::Type::NONE};`.
🎉 First successful runtime of pyxis on top of upstream + fixes:
- Heap 131KB free post-boot (vs ~45KB on pre-graft, then panic)
- Min low-water 119KB — never got tight
- Display flushing, no panic, no reboot loop
- Reset reason: PANIC (4) on prior runs is replaced by clean run
- microStore SPIFFSFileSystem mount works
- Identity loads from NVS with same hash as pre-graft (b5f09f9a833f4ef8)
- LoRa init, AutoInterface init, Transport::start() all succeed
- MessageStore loads 4 conversations from /lxmf
- LXMRouter constructs cleanly
- PropagationNodeManager initializes
Caveats for runtime test (per pyxis_microReticulum_graft_spike_findings):
- BLE + TCP + Transport mode currently disabled in user settings
(was needed to get a stable enough OTA window during initial flash —
can re-enable once we confirm pyxis-fixes-on-0.3.0 doesn't introduce
a BLE/TCP-specific WDT)
- Resource-form inbound LXMF dispatch is no-op'd
(static_resource_concluded_callback)
- Some UI stat fields disabled (LoRa RSSI, BLE peer count)
🎯 First successful build against vanilla upstream (+ our PKCS7/HMAC/X25519
fixes branch). Flash 71.3% / RAM 21.4% on tdeck env.
Final shim trim + API patches to get the build through:
- Drop fork-only files from lib/microreticulum-shim/ that aren't
referenced anywhere outside the shim itself:
Buffer.cpp/h, ChannelData.h — fork's Channel work, unused by LXMF
Cryptography/Ratchet.cpp/h — RNS 1.x compat, included `<X25519.h>`
directly; not used by anything pyxis
touches today
SegmentAccumulator.cpp/h — fork's Resource segmentation, depends
on Resource methods vanilla doesn't
expose; not used outside shim
- LXMessage default-init Destination members with {RNS::Type::NONE} so the
default LXMessage() ctor isn't implicitly deleted (vanilla Destination
has no default ctor).
- LXMF/LXMRouter::static_resource_concluded_callback stubbed to a no-op
+ ERROR log: vanilla Resource doesn't expose link(). Resource-form
inbound LXMF delivery is currently disabled (PROPAGATION ⇄ LXMessage
glue is broken until Resource API is reconciled). Tracked in spike doc.
- Link::pending_requests_count() -> .pending_requests().size().
What works on this build:
- Compile + link (firmware.elf produced, 2.24MB flash)
- Crypto path goes through our pyxis-fixes-on-0.3.0 branch which has
spec-correct PKCS7, HMAC, and X25519 clamping
- microStore-based SPIFFS persistence via the SPIFFSFileSystem adapter
- All BLE/SX1262/auto-interface adapters
- LXMF outbound + inbound for non-Resource (DIRECT-via-Link, OPPORTUNISTIC)
What's broken on this build (known, deliberate, tracked):
- Resource-form inbound LXMF delivery (propagation node sync doesn't
receive messages — outbound path still works for sending)
- Transport memory diagnostics ([TABLES] dumps replaced with placeholder)
- LoRa RSSI display + BLE peer-count display in the LXMF UI
- Identity::mark_persistent (the 5s fast-flush no longer exists; fall
back to microStore's dirty-tracking)
Next: runtime smoke test on T-Deck (10.0.0.177 OTA) to validate that the
above "works" surfaces actually work end-to-end against the python
reference impl. Conformance bridge already passes 52/85 against this
exact submodule pin.
relocate shim to lib/; pyxis lib API renames
Compile-tier graft progress on top of 40e561f. Key changes:
- Re-vendor lib/microreticulum-shim/ (was src-shim/) from the *actual*
ca355e5 commit content rather than the stale "feat/t-deck HEAD"
/tmp clone the previous commit pulled from. Recovers process_sync()
on LXMRouter and MEMORY_MONITOR_POLL macro that were missing.
- Move src-shim/ → lib/microreticulum-shim/ + add library.json so
PlatformIO discovers the .cpp files and links them. Was previously
only on the include path; the .cpps weren't in the build.
(This unblocks the 30+ undefined-reference linker errors for LXMF
and Instrumentation symbols.)
- Drop -Isrc-shim/Utilities (/Cryptography/Instrumentation) from
build_flags — they were over-broad and put our Stream.h on the
GLOBAL header path, breaking Arduino's Wire.cpp which has
`class TwoWire: public Stream`. -Ilib/microreticulum-shim alone
resolves subdir lookups via <Cryptography/X.h>, <Utilities/Y.h>.
- UniversalFileSystem migrated to microStore::Adapters::SPIFFSFileSystem
(activated by -DUSTORE_USE_SPIFFS). Vanilla upstream microReticulum
@ 0.3.0 deleted RNS::FileSystem in favor of microStore. Pyxis's
lib/universal_filesystem/ is now dead code on this build path.
- pyxis lib API renames for the post-graft world:
SDLogger.cpp: RNS::setLogCallback -> RNS::set_log_callback
AnnounceListScreen: Transport::get_destination_table ->
Transport::get_path_table
Transport::DestinationEntry ->
RNS::Persistence::DestinationEntry
UIManager.cpp: _lxst_destination ctor explicit RNS::Type::NONE
(vanilla Destination has no default ctor)
Identity::mark_persistent calls disabled w/
restoration TODO
ConversationListScreen: Interface::get_rssi/get_stats calls
disabled (the methods are non-virtual
on BLEInterface/SX1262Interface post
de-virtualization in a0ff631)
Compile is clean against the fixed-cryptography submodule pin; current
failure layer is fork-only Type::Channel constants referenced by the
vendored shim's Buffer/ChannelData files. That's the next session's
problem — see pyxis_microReticulum_graft_spike_findings.md for the
plan options (most likely: remove Channel/Buffer/ChannelData/Ratchet
from the shim, since LXMF doesn't use Channel anyway per the 2026-05-04
investigation).
Three API migrations to keep the graft moving against vanilla
attermann/microReticulum @ 0.3.0:
(A.5) Identity persistence migrated to OS::set_loop_callback.
Was: Identity::set_persist_yield_callback(cb) // fork-only
Identity::should_persist_data() // fork-only
Now: RNS::Utilities::OS::set_loop_callback(cb) // upstream global
reticulum->should_persist_data() // already used
The fork's split between Identity-specific 5s fast-flush and
Reticulum-level 60s full-persist is unified upstream into a single
Reticulum::should_persist_data() entry point. The fast cadence is
folded into microStore's dirty-tracking. If we observe excessive
lost-known-destinations after crashes, revisit microStore's flush
cadence rather than re-adding the fork-only Identity API.
(A.6) Transport stats diagnostics disabled — vanilla upstream doesn't
expose the *_count() getter family the fork added. Two [TABLES]
diagnostic blocks in main.cpp now print a placeholder. Restore by
porting to upstream's get_path_table().size() and friends, or PR the
getters back to upstream Transport. Tracked in
pyxis_microReticulum_graft_spike_findings.md.
(A.7) BLE/SX1262 Interface stat methods are no longer virtual overrides.
Vanilla upstream Interface base class doesn't declare get_stats /
get_rssi / get_snr. Kept the methods as plain (non-virtual)
BLEInterface / SX1262Interface members; callers needing stats access
must hold the concrete type, not the base Interface*. Propose
upstream PR adding to base API if polymorphic access matters.
Also: setLogCallback -> set_log_callback (renamed in upstream commit
4d6f0b9 "Added dual-class PSRAM/TLSF allocator system").
Pyxis still doesn't build — next failures (4 distinct):
- OS::register_filesystem signature changed to microStore::FileSystem&.
Real microStore migration needed for UniversalFileSystem.
- LXMRouter::process_sync still missing despite vendored src-shim copy.
Include-order or shadowing — needs investigation.
- MEMORY_MONITOR_POLL macro not picked up despite -I src-shim/Instrumentation.
- Identity::should_persist_data appears to still be referenced via
LXMF or another vendored layer — would surface once the above land.
Configuration moves to make pyxis buildable against vanilla
attermann/microReticulum @ 0.3.0 instead of the fork:
- lib/libbz2/ — pulled from torlando-tech/microReticulum @ ca355e5
(the previous submodule pin). Used by LXMF compression. Will
eventually move into the standalone microLXMF library (Track B).
- src-shim/ — the 37 fork-only files NOT already vendored into
pyxis/lib/ as part of the original split. Includes:
- LXMF/* (11 files) — to extract as a standalone library (Track B)
- Cryptography/{BZ2,Ratchet}.{cpp,h} — used by LXMF + RNS 1.x compat
- Instrumentation/{BootProfiler,MemoryMonitor}.{cpp,h} — pyxis-specific
- Utilities/{Print,Stream}.{cpp,h} — pyxis logging supplements
- root: Buffer, BytesPool, ChannelData, Display+Graphics, FileStream,
FileSystem, MessageBase, ObjectPool, PSRAMAllocator, SegmentAccumulator
- platformio.ini lib_deps: drop the symlink:// to deps/microReticulum/lib/libbz2
(gone in upstream), add microStore as a github URL dep (vanilla
upstream needs it for Bytes.h's microStore/Codec.h include).
- platformio.ini build_flags: add -Isrc-shim plus its subdirectories
so the vendored ../X.h-style includes (rewritten to <X.h>) resolve
through pyxis's own paths first.
src-shim/* was bulk-rewritten so `#include "../X.h"` (relative to
the fork's src/) becomes `#include <X.h>` resolved via -I deps/microReticulum/src.
Pyxis still doesn't build at this point — the next layer of breakage
is API-level (not file-level): pyxis main.cpp calls fork-only
Identity/LXMRouter/Transport methods that vanilla upstream doesn't
have. See the followup commit's notes / branch description for the
full list.
- lib/lxst_audio/{packet,encoded}_ring_buffer.cpp use malloc/free without
including <cstdlib>. macOS leaks it via header transitivity but Linux
clang is stricter — real portability bug surfaced by the new pytest CI.
- microReticulum native17 tests link against system libbz2 via the fork's
pre:link_bz2.py script. Ubuntu runners need libbz2-dev installed.
The unprotected _clients.find() could race with
processPendingDisconnects() erasing from the map concurrently.
Mutex is released before the blocking getService() GATT call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously, a mutex timeout left characteristic caches empty but
still signalled success to callers, making all GATT ops silently
fail for the connection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Re-check hasActiveWriteOperations() after acquiring mutex in
processPendingDisconnects() to close race where write() registers
an op between the pre-mutex check and mutex acquisition
- Move cached char pointer writes inside connection-exists guard in
discoverServices() to prevent dangling pointers on handle reuse
- Add WARNING logs to both onConnect callbacks on mutex timeout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move getService()/getCharacteristic() out of mutex-held paths in
writeCharacteristic(), read(), enableNotifications() by caching all
three char pointers (RX, TX, Identity) during discoverServices()
- Replace 5-second spin-wait in processPendingDisconnects() with
non-blocking deferral: break if GATT ops in flight, retry next loop
- Add WARNING logs to all read-path helpers on mutex timeout
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move beginWriteOperation() before xSemaphoreGive(_conn_mutex) in
write(), writeCharacteristic(), read(), and enableNotifications()
so the active-op counter is incremented while the mutex is still
held. This closes the window where processPendingDisconnects()
could observe hasActiveWriteOperations()==false and delete the
client before the GATT caller has registered its operation.
- Add _conn_mutex around _connections/_clients insertions in both
server and client onConnect() callbacks, preventing concurrent
map insertions from corrupting the red-black tree.
- Protect updateConnectionMTU() with _conn_mutex.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
writeCharacteristic(), read(), and enableNotifications() resolve
characteristic pointers under _conn_mutex then call blocking GATT
ops after releasing it — same pattern as write(). Without the
active-operation guard, processPendingDisconnects() could delete
the client (and its child characteristics) during the GATT call.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Defer NimBLEDevice::deleteClient() in processPendingDisconnects()
until after releasing _conn_mutex and waiting for any active write
operations to complete. Prevents use-after-free when write() holds
a child NimBLERemoteCharacteristic* pointer across the mutex boundary.
- Add _conn_mutex protection to getConnectionCount(), isConnectedTo(),
and isDeviceConnected() which read _connections without synchronization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
send_outgoing() on loopTask (core 1) calls write() which reads
_connections, _clients, and _cached_rx_chars maps, while
processPendingDisconnects() on the BLE task (core 0) erases from
them — with no synchronization. This causes std::map red-black tree
corruption, manifesting as LoadProhibited crashes in map rotate/insert
operations (EXCVADDR=0x00000008).
Protect all map accesses in write(), writeCharacteristic(), read(),
enableNotifications(), getConnection(), getConnections(), and
processPendingDisconnects() with _conn_mutex. The mutex is released
before any blocking GATT operations (writeValue, readValue, subscribe)
to avoid holding it during 10-30s NimBLE timeouts.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Consolidate #ifndef/#ifdef into single #ifdef/#else/#endif block.
Add warning comment to generated header about static linkage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move BG_COLOR inline into #ifndef block to avoid unused variable
when HAS_SPLASH_IMAGE is defined. Make show_splash() private since
it's only called internally from init_hardware_only().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matches the guard already on notify() to prevent use-after-free
of _tx_char during a NimBLE host reset.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
BLE task is no longer subscribed to WDT, so these 23 calls were
silently returning ESP_ERR_NOT_FOUND. Removes dead code and the
now-unused esp_task_wdt.h include.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents double PSRAM allocation and LVGL driver re-registration
if init() were called more than once.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Display::init_hardware_only() and POWER_EN to right after serial
banner, before GPS/WiFi/SD/Reticulum init. Add 150ms delay after
POWER_EN HIGH so ST7789V power rail stabilizes before SPI commands
(without this, SWRESET is sent to an unpowered chip and silently lost).
Splash now visible for entire boot period (~18s) until LVGL takes over.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two root causes for frequent device reboots:
1. LVGL task (priority 2) starved loopTask (priority 1) on core 1.
During heavy screen rendering, loopTask couldn't run for 30+ seconds,
triggering the Task WDT. Fixed by lowering LVGL to priority 1 so
FreeRTOS round-robins both tasks fairly.
2. BLE task was registered with the 30s Task WDT, but blocking NimBLE
GATT operations (connect + service discovery + subscribe + read) can
legitimately take 30-60s total. Removed BLE task from WDT since
NimBLE has its own internal ~30s timeouts per GATT operation.
Also added ble_hs_synced() guards to write(), read(), notify(),
writeCharacteristic(), discoverServices(), and enableNotifications()
to prevent use-after-free on stale NimBLE client pointers during
host resets.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Display and LoRa were creating separate SPIClass(HSPI) instances which
claimed GPIO pins via the matrix, preventing SD card (on FSPI) from
accessing MISO after Display init. Now all three peripherals use the
global SPI (FSPI) instance, eliminating GPIO routing conflicts.
- Display: use &SPI instead of new SPIClass(HSPI)
- SX1262Interface: use &SPI instead of new SPIClass(HSPI)
- SDAccess: enable format_if_empty for unformatted cards
Verified on device: SD (128GB SDHC), display, and LoRa all coexist.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
SD card was unresponsive (MISO stuck 0xFF) because Display's HSPI
peripheral had already claimed the GPIO pins via the matrix, preventing
FSPI from routing MISO. Fix by initializing SD card BEFORE Display,
using the global SPI (FSPI) instance — matching LilyGo's reference code.
- Move SD card init before display init in boot sequence
- Use global SPI (FSPI) instead of Display's SPIClass(HSPI)
- Lower SPI frequency to 800kHz matching LilyGo example
- Drive all CS lines (display, LoRa, SD) high before SD init
- Add MISO=38 to Display's SPI.begin for post-init bus sharing
- Add Display::get_spi() accessor for future shared use
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The T-Deck Plus shares HSPI across the display (CS=12), LoRa (CS=9),
and SD card (CS=39). Previously SD logging was disabled because
SD.begin() reconfigured the SPI bus and blanked the display.
This introduces a FreeRTOS mutex created in main.cpp and injected into
Display, SX1262Interface, and a new SDAccess class so all three
peripherals serialize their SPI transactions safely.
- Add SDAccess class wrapping SD.begin() and file ops with mutex
- Add set_spi_mutex() to Display and SX1262Interface
- Wrap Display flush, fill, draw, and power ops in mutex
- Refactor SDLogger to use SDAccess mutex instead of owning SD.begin()
- Wire up mutex creation and injection order in setup()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
UIManager.cpp includes Tone.h, so tdeck_ui should declare this
dependency rather than relying on implicit global discovery.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
After a 574 connection failure, the NimBLE controller's scan state can
become corrupted (returning rc=530 / Invalid HCI Params) even after the
host re-syncs. This led to scan failure escalation and device reboots.
Key fixes:
- Add ble_gap_conn_cancel() to enterErrorRecovery() — stuck GAP master
connection operations were blocking all subsequent scans
- Add ble_hs_sched_reset(BLE_HS_ECONTROLLER) in error recovery to force
a full host-controller resynchronization after desync
- Proactively cancel stale GAP connections before scan start
- Reduce SCAN_FAIL_RECOVERY_THRESHOLD from 10 to 5 for faster recovery
- Enhanced scan failure logging with GAP state diagnostics
- Move ESP reset reason logging after WiFi init for UDP log visibility
- Suppress connection candidate log spam when at max connections
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
