Greptile follow-up on 67c1083: if xSemaphoreCreateMutex() failed,
QueueGuard silently locked nothing and the deferred queues still ran
unsynchronized. The constructor now logs the allocation failure and
every guarded site refuses mutation when _queue_mutex is null:
requesters stop enqueuing, flushers stop draining (nothing queued can
exist), and the one-shot preview commit is skipped. The screen keeps
rendering and chatting; only the deferred list mutations degrade.
Greptile P1 on the list-perf commit: the deferred-queue pair
(requester task -> main-loop flush) was unsynchronized. Producers
push from the LVGL task (click handlers, refresh() during navigation)
while flush_*() swaps the queues on the main loop without the LVGL
lock, so concurrent push/swap is a data race.
A single FreeRTOS mutex (QueueGuard, fail-closed) now guards all
four shared queues: _pending_mark_reads, _pending_drops,
_pending_name_writes, and the _index_commit_pending flag. Critical
sections are bounded vector operations only — no store I/O, no LVGL
lock, so acquisition cannot deadlock and the LVGL task is never
stalled on LittleFS. Lock order is LVGL-lock -> queue-mutex only;
queue sections never take the LVGL lock.
The mutex is created before the screen's LVGL_LOCK section and
deleted after it in the destructor.
On-device capture confirmed the fix (cold-boot tap: fallbacks=0,
gather=2ms, total=48ms vs 2086ms pre-fix; repeat taps no longer
perpetually fall back). All [PERF] timing markers and stage
variables are removed from refresh()/show()/render_route; the
branch is now clean of diagnostic code.
On-device [PERF] capture decomposed the remaining ~1s tap latency:
each uncached conversation costs one SPI-LittleFS metadata read
(~230ms). Repeat taps still paid 2 perpetual fallbacks (empty-
content tails never got cached), and the first tap after every
boot paid all 9 (the in-memory repop was never committed to the
index).
- bump microLXMF pin c8d3156 -> 6bea23c (feat/conversation-
preview-cache): preview_valid index flag so 'cached empty'
differs from 'unpopulated'; bounded preview copy (the old
strncpy read past the non-terminated content Bytes); public
commit_index().
- refresh() re-pops empty-content tails as a valid cached
preview and arms a one-shot deferred index commit;
UIManager::update() drains it out-of-lock (flush_pending_
index_commit, between drops and mark-read) so the warmed
previews persist and the next cold boot reads them from the
index.
- fix the [PERF] skip-log total= wrap (printed p_t_diff - p_t0,
a uint32 underflow; total was already p_t_diff).
[PERF] instrumentation stays in this commit (temporary,
marked); it is removed before merge once the fix is validated
on-device.
Temporary, removable instrumentation (marked [PERF] throughout):
- refresh(): gather / diff / rebuild stage ms + metadata-fallback count,
one [PERF] convlist line per call (skip vs build)
- show(): unhide + focus-group ms
- render_route(MESSAGES): whole route window incl. hide_all_screens
All INFO-level so they survive DEBUG-off and land in the serial capture.
refresh() unconditionally ran lv_obj_clean(_list) and recreated 5-7 LVGL
objects per row on every call - every navigation back to Messages, every
750ms coalesced inbound batch, and the periodic name-resolution sweep -
even when nothing had changed. With the store read now O(1) (index
preview cache), that widget churn was the remaining gap vs NomadNet /
Network / Maps, which build once and just unhide.
refresh() now gathers row data first (index preview + hash + unread, no
message-file I/O) and diffs it against the rendered rows; the rebuild
only happens when data actually changes. Rows keep focus-group
membership and the screen keeps its scroll position on the no-change
path. The per-refresh 'Found N conversations' log drops from INFO to
DEBUG (serial output under the LVGL lock stalls the render task).
get_conversations() ordering is deterministic (last_activity desc with
peer-hash tie-break), so the index-aligned comparison is stable.
refresh() previously opened + parsed each conversation's newest message
file (load_message_metadata) on every list refresh, which dominated
list-load time on SPI LittleFS. It now reads the per-conversation
last-message preview + timestamp from the store's in-memory index
(O(1), zero I/O) and falls back to load_message_metadata only when the
index has no cached preview — the first refresh after a firmware
upgrade or after a corrupt-tail drop — then writes the preview back
through so the fallback happens at most once per conversation per
firmware generation. The write-through is skipped when drops were
queued (the drained deletes move the tail, so a preview written for the
old tail would be stale for one refresh).
Host benchmark (x86 + POSIX fs, 24 msgs/conv): the per-conversation
store-load work drops from ~0.149ms (9 convs) / ~0.342ms (20 convs) to
~0.002ms / ~0.005ms, and the cold-boot path (store reconstructed from
disk) matches the warm path because the preview now persists in the
index.
Bumps the microLXMF pin to c8d3156 (feat/conversation-preview-cache)
in platformio.ini, the release audit, and the native reference test.
Adds tests/microlxmf/bench_conversation_list_load.cpp (baseline vs
index warm/cold) and the bench target in tests/microlxmf/CMakeLists.txt.
When a conversation's newest message is unreadable, walk the index
newest-to-oldest for the newest readable preview and queue the
unreadable messages for deletion (deferred out of the LVGL lock,
drained by UIManager::update()). The store's delete_message() commits
the index and updates last_message_hash, so the next refresh converges
to the same preview and the row never disappears over one bad message.
refresh() did heavy unnecessary work per conversation, on the LVGL
render task under the render lock:
- get_messages_for_conversation() copied the full 256-slot hash array
(8KB) just to read the newest hash (messages.back());
- load_message() on that hash read the payload file ~3x, JSON-parsed
it twice (including the large hex 'packed' blob), hex-decoded and
msgpack-unpacked the whole message — to extract a 30-char preview
and a timestamp;
- the unread badge was rendered from a hardwired 0 even though the
store maintains and persists unread_count.
Now:
- newest hash via the new O(1) MessageStore::get_last_message_hash
(index tail — always hot-tier, no I/O, no array copy);
- preview/timestamp via load_message_metadata (single open + filtered
parse, the fast path ChatScreen already uses for the same fields);
- unread badge from MessageStore::get_conversation_unread_count;
- badge cleared + mark-read on open (click) and when a message lands
in the currently-viewed chat, with the LittleFS index commit
deferred out of the LVGL lock (UIManager::update), matching the
existing deferred display-name write-through pattern.
Pins microLXMF 59ca70a (PR #10, temporary branch head) for the two new
accessors.
Repins microReticulum + microLXMF onto the upstream-0.4.1 graft and adapts
pyxis to the new src/microReticulum/ layout and 0.4.x APIs. The far-diverged
0.3.0 fork's Resource/Transport/Identity work is subsumed by upstream's
reimplementation; only the still-needed fixes ride on the pinned branches
(PKCS7/HMAC/X25519 crypto -- proven byte-identical to python RNS 1.3.1 --
Packet link-proof callback, Identity short-sig guard, and the bz2 layer +
decompress-on-receive in Resource::assemble()).
Consumer-side changes:
- platformio.ini: pin microReticulum @2f21fee (pyxis-fixes-on-0.4.1) and
microLXMF @33760d0 (chore/microreticulum-0.4.1-layout); bump microStore
ceea8f5 -> c5fb69d (0.4.x requires the new BasicFileStore::init API);
-std=gnu++11 -> gnu++17 (upstream requires C++17).
- Namespace all microReticulum includes (angle + quote) to <microReticulum/...>
for the relocated layout; shim-local Utilities/Stream.h|Print.h preserved.
- Interface::send_outgoing now returns bool: update TCP/BLE/SX1262/Auto
overrides with correct success/failure returns.
- SDArchiveFileSystem::init(bool reformatOnFail=true) to match new microStore.
- Static Transport::get_path_table() -> path_table(); instance getter unchanged.
- Remove duplicate shim Cryptography/BZ2 (microReticulum provides it now; keep
lib/libbz2 as the ESP32 bzlib provider).
- patch_littlefs_paths.py: normalize microStore's LittleFS adapter paths to a
leading "/" -- ESP32 Arduino LittleFS rejects "./"-prefixed paths, which
silently broke the path store (no peer paths learned, all messaging blocked).
Validated on T-Deck Plus: builds (RAM 27.5% / Flash 77.7%), boots stable
(no WDT/panic), and a full on-device LXMF e2e (DIRECT + OPPORTUNISTIC +
bz2-compressed-Resource receive) passes 5/5.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
refresh() called set_display_name() (a microStore/LittleFS write) while the
LVGL lock was held by UIManager::update(). On a cold-boot announce burst,
refresh() writes a name per newly-seen peer, serially stalling the LVGL render
task for the combined I/O time. Mirror the on_message_received fix: accumulate
the write-throughs in _pending_name_writes during refresh(), and flush them at
the top of UIManager::update() before it takes the LVGL lock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UWZuYkHBRqNb6BZHV8sTG5
Two related bugs in the conversation list:
1. Without GPS/NTP, Utilities::OS::time() returns uptime seconds — way
smaller than any real unix-epoch message timestamp. The
format_timestamp "diff < 0 → Future" branch then fired on every
row. Add a sane-epoch threshold (2024-01-01) below which we render
"?" instead, since "Future" is misleading when it just means
"we don't know what time it is."
2. Identity::recall_app_data is in-memory only and lost on reboot. The
conversation list always re-fell back to truncated hashes on cold
start. Wire the three-tier resolution flow: live announce →
MessageStore-persisted name → hash. When the live cache hits, write
through to the persisted side via MessageStore::set_display_name so
future cold boots get the name back immediately.
Pulls in microLXMF 5531a59 (MessageStore display-name cache impl).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TinyGPSPlus's `satellites` only updates from $GPGGA (sats USED in
the position fix). A module seeing the sky but not yet acquired
shows fix=0 — previously rendered as red "0", which read as
"GPS broken." It's actually "GPS healthy, waiting for lock."
Bind a TinyGPSCustom to $GPGSV field 3 (satellites in view) and
fall through to it when fix-sats is 0:
-- muted no GPS handle / no NMEA
? yellow NMEA flowing, no fix, no GSV count yet
?N yellow N satellites visible but not yet locked
N colored N satellites locked in fix
So a cold start now shows a meaningful number ticking up as the
module finds birds, then flips to a green N once it gets a fix.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The top status bar's LoRa indicator was hard-disabled (`if (false &&
_lora_interface)`) — every render printed "--" no matter what the
radio was doing. Re-enable it: SX1262Interface::get_rssi() is
non-virtual on the impl class, so we drop down to the InterfaceImpl*
via Interface::get() and static_cast to SX1262Interface.
Also adds a third tier to the GPS readout so "module connected but no
fix yet" is distinguishable from "no GPS hardware":
-- muted no GPS handle, or no NMEA bytes parsed
? yellow NMEA flowing but no $GPGGA sat-count yet
N colored satellite count valid
Before: a TinyGPSPlus that was happily streaming $GPGSV but hadn't
yet parsed a $GPGGA showed "--" indistinguishably from a missing
module. The "?" tier surfaces "alive, waiting for sky."
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
relocate shim to lib/; pyxis lib API renames
Compile-tier graft progress on top of 40e561f. Key changes:
- Re-vendor lib/microreticulum-shim/ (was src-shim/) from the *actual*
ca355e5 commit content rather than the stale "feat/t-deck HEAD"
/tmp clone the previous commit pulled from. Recovers process_sync()
on LXMRouter and MEMORY_MONITOR_POLL macro that were missing.
- Move src-shim/ → lib/microreticulum-shim/ + add library.json so
PlatformIO discovers the .cpp files and links them. Was previously
only on the include path; the .cpps weren't in the build.
(This unblocks the 30+ undefined-reference linker errors for LXMF
and Instrumentation symbols.)
- Drop -Isrc-shim/Utilities (/Cryptography/Instrumentation) from
build_flags — they were over-broad and put our Stream.h on the
GLOBAL header path, breaking Arduino's Wire.cpp which has
`class TwoWire: public Stream`. -Ilib/microreticulum-shim alone
resolves subdir lookups via <Cryptography/X.h>, <Utilities/Y.h>.
- UniversalFileSystem migrated to microStore::Adapters::SPIFFSFileSystem
(activated by -DUSTORE_USE_SPIFFS). Vanilla upstream microReticulum
@ 0.3.0 deleted RNS::FileSystem in favor of microStore. Pyxis's
lib/universal_filesystem/ is now dead code on this build path.
- pyxis lib API renames for the post-graft world:
SDLogger.cpp: RNS::setLogCallback -> RNS::set_log_callback
AnnounceListScreen: Transport::get_destination_table ->
Transport::get_path_table
Transport::DestinationEntry ->
RNS::Persistence::DestinationEntry
UIManager.cpp: _lxst_destination ctor explicit RNS::Type::NONE
(vanilla Destination has no default ctor)
Identity::mark_persistent calls disabled w/
restoration TODO
ConversationListScreen: Interface::get_rssi/get_stats calls
disabled (the methods are non-virtual
on BLEInterface/SX1262Interface post
de-virtualization in a0ff631)
Compile is clean against the fixed-cryptography submodule pin; current
failure layer is fork-only Type::Channel constants referenced by the
vendored shim's Buffer/ChannelData files. That's the next session's
problem — see pyxis_microReticulum_graft_spike_findings.md for the
plan options (most likely: remove Channel/Buffer/ChannelData/Ratchet
from the shim, since LXMF doesn't use Channel anyway per the 2026-05-04
investigation).
After boot, the conversation list called recall_app_data() once during
initial load. If announces hadn't arrived yet (or known destinations
hadn't been loaded with app_data), conversations showed raw hashes
permanently until the user navigated away and back.
Add a lazy name resolution check to update_status() (called every 3s):
if any conversations have unresolved names, try recall_app_data() again
and refresh the list when a display name becomes available.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Fix use-after-free crash on hangup: set _call_state=IDLE before deleting
_lxst_audio, preventing pump_call_tx() (runs without LVGL lock) from
accessing freed memory
- Replace single-slot _call_signal_pending with 8-element ring buffer queue
to prevent signal loss when CONNECTING+ESTABLISHED arrive in rapid succession
- Extract TX pump into pump_call_tx() called right after reticulum->loop()
for low-latency audio TX without LVGL lock dependency (was buried at step 10)
- Tune ES7210 mic gain to 21dB (was 15dB) to improve Codec2 input level
without ADC clipping that occurred at 24dB
- I2S capture: use APLL for accurate 8kHz clock, direct 8kHz sampling
(no more 16→8kHz decimation), DMA 16x64 for encode burst headroom
- Reduce Reticulum log verbosity to LOG_INFO (was LOG_TRACE)
- BLE: add ble_hs_sched_reset() tiered recovery before reboot on desync,
widen supervision timeout to 4.0s for WiFi coexistence
- Add UDP multicast log broadcasting and OTA flash support
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split T-Deck firmware from microReticulum examples/lxmf_tdeck/ into its
own repo. microReticulum is consumed as a git submodule dependency pinned
to feat/t-deck. All include paths updated from relative symlinks to bare
includes resolved via library build flags.
Both tdeck (NimBLE) and tdeck-bluedroid environments compile successfully.
Licensed under AGPLv3.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Torlando