[SENDT] send-pipeline timing and the microReticulum [PG] path-store
call-site counters now compile to no-ops unless explicitly enabled:
- DPYXIS_SEND_DIAG / -DRNS_PATHGET_DIAG added to env:tdeck base flags
- both removed by env:tdeck-release build_unflags (no-op in release)
So production tdeck/tdeck-release builds are merge-clean, while the
instrumented build stays one env/flag away for the path-request
spammer hunt. Bumps microReticulum pin to 921b3aa (same endpoint
hot-path read gate, counters gated behind RNS_PATHGET_DIAG).
Contract suite: 176/176.
Bumps microReticulum pin to e2c9d4d (diag/path-get-caller on cd0338e):
Transport::inbound() and path_request() no longer perform a full
microStore get() (flash write + read) for every inbound packet / path
request on endpoint-only nodes. The read is gated on the exact
conditions where destination_entry is consumed, and the local-destination
path-request answer uses the in-memory _destinations table, so the
device stays discoverable. The build still carries the temporary [PG]
counters for the live before/after capture; counters are stripped in a
follow-up before anything merges.
TEMPORARY DIAGNOSTIC PIN. ef07187 = cd0338e + per-call-site
_new_path_table.get() counters ([PG] summary every 30s on serial).
Purpose: identify which Transport call site drives the ~1.5s full
FileStore get() on the offline propagation node (6b9f6601...).
Revert this commit (back to cd0338e) after the capture.
Bumps microLXMF d7e05fd -> 82d2e54 (fork branch
fix/sync-path-wait-backoff): LXMRouter::process_sync() PR_PATH_REQUESTED
now re-polls Transport::has_path() at most once per PATH_REQUEST_WAIT
instead of every main-loop iteration. In the CBA microStore-backed
microReticulum fork has_path() is a FileStore exists() = a LittleFS
read, so the old code issued a path-table read every ~2s for the whole
60s sync window while the propagation node was unreachable. Measured on
the T-Deck: one destination fetched 293x in 600s (~every 2s), all on
the same 2MB partition as message storage, with [DISP] flush stalls
interleaved. Worst-case sync cycle cost drops from ~30 reads to ~3-4.
The periodic message sync was already 4h default with a user setting
(Settings 'Prop Sync Interval (hrs)', sync_int=14400) — unchanged.
Adds tests/build_scripts/test_propagation_sync_poll_contract.py:
pins the re-poll gate ordering, the PATH_REQUEST_WAIT window, the
untouched one-shot check in request_messages_from_propagation_node,
and enforces platformio.ini <-> audit-tool pin agreement.
Tracks microLXMF PR #11 head d7e05fd: archived-branch cache sync,
bulk-delete cache invalidation, and the uniform capped-content
contract for load_message_metadata(). No Pyxis code change — the
chat/list paths already rely on the capped preview and the full-view
accessor.
Uses the microLXMF in-process message-metadata cache (bumped pin
6bea23c -> 58a6eb0, branch feat/conversation-preview-cache):
- Reopening a conversation (and background page fill / paging) is now
O(1) in-memory once warmed instead of re-reading each message file
from SPI LittleFS (~230ms/read, ~2.3s per open measured on the T-Deck).
- The cache table is PSRAM-allocated on ESP32; a static .bss placement
starved internal DRAM and the LVGL task's 8 KiB stack allocation
failed at boot ("Failed to create LVGL task", hang at startup logo).
- Long-press full-message view now defers to the main loop: the LVGL
event handler only records the hash; tick_pending_full_message() does
load_message_content() (uncapped, no msgpack unpack) off the LVGL
task, then builds the modal. Fixes the capped (600-char) text that
the in-memory rows carry.
- A peer change cancels an in-flight background fill from the previous
conversation (it would otherwise prepend the old conversation's
rows into the new one).
Validated on the T-Deck (T-Deck Plus, 8MB PSRAM): warm opens
reads=3 sync=0-2ms total=15-35ms (was ~2.3s); cold first open per
conversation still pays ~0.6-1.2s disk while warming the cache; no
panics/watchdogs over a ~6-minute interaction session.
Host gates: microLXMF conformance 6/6 (incl. 32-assertion
test_message_metadata_cache), native reference test against pinned
58a6eb0, tdeck build SUCCESS.
On-device [PERF] capture decomposed the remaining ~1s tap latency:
each uncached conversation costs one SPI-LittleFS metadata read
(~230ms). Repeat taps still paid 2 perpetual fallbacks (empty-
content tails never got cached), and the first tap after every
boot paid all 9 (the in-memory repop was never committed to the
index).
- bump microLXMF pin c8d3156 -> 6bea23c (feat/conversation-
preview-cache): preview_valid index flag so 'cached empty'
differs from 'unpopulated'; bounded preview copy (the old
strncpy read past the non-terminated content Bytes); public
commit_index().
- refresh() re-pops empty-content tails as a valid cached
preview and arms a one-shot deferred index commit;
UIManager::update() drains it out-of-lock (flush_pending_
index_commit, between drops and mark-read) so the warmed
previews persist and the next cold boot reads them from the
index.
- fix the [PERF] skip-log total= wrap (printed p_t_diff - p_t0,
a uint32 underflow; total was already p_t_diff).
[PERF] instrumentation stays in this commit (temporary,
marked); it is removed before merge once the fix is validated
on-device.
refresh() previously opened + parsed each conversation's newest message
file (load_message_metadata) on every list refresh, which dominated
list-load time on SPI LittleFS. It now reads the per-conversation
last-message preview + timestamp from the store's in-memory index
(O(1), zero I/O) and falls back to load_message_metadata only when the
index has no cached preview — the first refresh after a firmware
upgrade or after a corrupt-tail drop — then writes the preview back
through so the fallback happens at most once per conversation per
firmware generation. The write-through is skipped when drops were
queued (the drained deletes move the tail, so a preview written for the
old tail would be stale for one refresh).
Host benchmark (x86 + POSIX fs, 24 msgs/conv): the per-conversation
store-load work drops from ~0.149ms (9 convs) / ~0.342ms (20 convs) to
~0.002ms / ~0.005ms, and the cold-boot path (store reconstructed from
disk) matches the warm path because the preview now persists in the
index.
Bumps the microLXMF pin to c8d3156 (feat/conversation-preview-cache)
in platformio.ini, the release audit, and the native reference test.
Adds tests/microlxmf/bench_conversation_list_load.cpp (baseline vs
index warm/cold) and the bench target in tests/microlxmf/CMakeLists.txt.
PR #10 merged to main; swap the temporary branch-head pin (59ca70a) for
the merge commit so the consumer tracks a stable main revision. Update
the release-audit allowlist in the same commit. Firmware source is
byte-identical to 59ca70a (the merge added only the CI workflow commit).
Greptile P2 (round 1): a post-rename parent-fsync failure during activation now reports that the record is committed and visible with only durability uncertain, instead of the generic 'activation failed' message; and the subprocess lock-holder helper repr-escapes the pyxis-map path so a path containing a quote still yields valid child Python.
esptool accepts exactly one operation per invocation; the combined
'erase_flash write_flash 0x0 ...' form is rejected with a usage error.
Document erase_flash and write_flash as separate commands in the README
and the merge tool docstring (verified against esptool 5.3.1).
Add tools/merge_pyxis_release.py, which assembles the four validated
release assets (bootloader, partition table, OTA selector, application)
into a single 8 MiB image at their fixed flash offsets on an erased
canvas, with per-image read-back and gap checks.
Wire it into release-firmware.yml so every tag build publishes
pyxis-<tag>-merged.bin as a release asset, and document the
provisioning semantics in the README: the merged image is a
first-install image (it erases NVS and LittleFS); data-preserving
updates keep using firmware.bin or the Columba .pyxis package.
Drives the firmware T: hooks to capture raw mic audio (UDP tap or the reliable
PSRAM/serial recorder), plays TTS references out the Mac speaker, and scores the
round-trip with envelope/STOI metrics and Whisper transcription. venv + recorded
WAVs are gitignored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
ArduinoOTA enables wireless firmware uploads (pio run -e tdeck-ota -t upload).
UDP log callback via RNS::setLogCallback sends all log lines plus Serial.printf
diagnostics to multicast group 239.0.99.99:9999 for untethered monitoring.
Includes safety guards: UDP suspended during WiFi transitions, reentrancy
protection, and WiFi status check before each send.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Torlando