The ThinkNode M9's microSD is soldered on, so the SDK's "drop it on the card"
route does not exist there and the Store can only fetch from its own host.
Three console commands write into the same /apps (or /lang) the Store uses:
fput /apps/<name> open (truncate); names [A-Za-z0-9._-]
fadd <off> <len> <sum> <base64> append a chunk, every field verified
fend close
Same physical-access trust level as the existing "rm"/"erase", narrower scope
(two directories). DataStore gains a root-aware mkdirRooted(); the CLI line
buffer grows from 80 to 200 bytes.
Why the chunks are self-checking and short: the UART interrupt is not
IRAM-resident, so while the loop is inside a flash-cache pause only the
128-byte hardware FIFO buffers console input and the middle of a longer line
is lost -- observed on the M9 as a 197-char line echoed back as 120. Offset,
decoded length and byte sum reject a damaged line; the host re-sends. The
host script also waits for "[BOOT] ui ready" because the CH34x bridge resets
the board whenever the port is opened and the console is not serviced before
the UI is up.
Verified on the M9: gpscompass.lua (19055 B) + .json pushed, sizes confirmed
by the device and by `ls /apps` after a reboot. M9 and V4-R8 envs compile.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
User report: the M9 "does not remember chats or sync them to the app" while
the V4-R8 does. The companion sync ring (MyMesh::history_ring + per-client
cursors) was RAM-only on every board — invisible on a USB-powered V4 that
rarely loses power, fatal on the M9 whose only off is the hard-cut slider.
Shared fix: the ring is now an append log (/synchist) plus client cursors
(/synccur) on DataStore::getHotDataFS(), restored in MyMesh::begin() and
flushed by persistSyncHistoryNow() on the paths that lose RAM (deep sleep,
power-off, explicit save). Record + bench-test recipe: M9_PORT.md Deferred #13.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The in-place patch replaces a whole-file rewrite with positioned writes through
fopen("r+") + fseek. That is a much less travelled path than the rewrite it
replaces, it behaves differently per filesystem (SPIFFS / FAT / LittleFS), and
the data at stake is the operator's contact list — worth more than the freeze
this fixes. I could not exercise it on real SPIFFS from the bench (the V4's USB
is the ROM serial/JTAG peripheral, so the companion protocol is unreachable and
the app's own console goes to UART0), so rather than ship it on the strength of
a code read, it now checks its own work: after patching a slab, read those bytes
back and compare. Any disagreement at all returns false and the caller falls
through to the full atomic rewrite, which is the known-good path.
A filesystem that accepts a positioned write and silently drops it is exactly
the failure that would otherwise corrupt a contact list while reporting success.
The test harness now simulates one and asserts the fast path refuses it.
Also adds the save cost to About > System info ("last save: in-place, 1 rec,
12 ms" vs "FULL rewrite, 431 rec, ..."), so which path ran is observable in the
field instead of only inferable. Confirmed rendering on a V4: 431/2000 contacts
(~63 KB), internal flash 338/3169 KB (10%) — which also corrects the earlier
assumption that these volumes run near-full; at 2000 contacts it projects to
roughly 60%. The mechanism is the size of each write, not the fullness.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Yoss101 reported the beta_62/63 fix helped but did not cure it: the freezes got
rarer AND longer. Rarer is the blob-delete queue working; longer is what was
left, and it was always the bigger half.
MAX_CONTACTS is 2000 and a contact record is 152 bytes, so saveContacts was
rewriting a 304 KB file on every change, with a same-sized .tmp alongside it for
the atomic swap. On a card-less V4 that is ~608 KB of peak churn on a 3.375 MB
SPIFFS volume that is already carrying one blob file per contact. SPIFFS GC cost
scales with how full the volume is, and GC suspends the flash cache, which stalls
BOTH cores — so the fuller the table, the longer the device is simply gone.
The table never actually changes in bulk: eviction replaces contacts[oldest] in
place (the array is unsorted and never shifts) and an advert refresh touches one
entry. So compare each record against what is already on disk and write back only
the ones that differ. An eviction now writes 152 bytes instead of 304 KB, an
unchanged table writes nothing, and there is no .tmp and no free-space spike.
Reads never trigger GC, so the compare scan is cheap.
Falls back to the full atomic rewrite whenever the mapping is not provably safe
(no live file, ragged size, or a table that shrank — there is no truncate here),
and never truncates or renames, so a fallback always leaves a valid list on disk.
Verified against the real function source with an in-memory FS harness: steady
state, eviction, growth, shrink, filtered anon slots, missing file, ragged file,
and 167 scattered changes all match a full rewrite byte-for-byte.
Also: the blob-delete queue silently ORPHANED a blob on overflow — nothing else
ever deletes it, so it leaked flash permanently on the one metric that drives GC
cost. Depth 8 -> 32, and overflows are now counted and surfaced.
New About > diagnostics block "Contact store": contact count, approximate on-disk
size, internal-flash used percentage, and orphaned blobs. The used percentage is
the number that predicts these freezes, so a reporter can photograph it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Core (core-v1.17.1, meshcomod 5406093): 1.17 reserved MAX_ANON_CONTACTS slots
at the head of contacts[], which broke two things in beta_60 —
- resetContacts() claimed those slots while the lazily-allocated PSRAM table was
still NULL, so the new bootstrapRTCfromContacts() NULL-deref'd at boot on any
device whose contact store loaded nothing: fresh install, erase-flash, or SD
not mounted yet (#249). Boards with saved contacts booted fine, which is how
it passed bench testing.
- getContactByIdx() stayed raw while getNumContacts() excludes the reserved
slots, so every pairing of the two — contact list, action sheets, phone-app
sync, getContactForSave — read empty slots and could not reach the newest 8
real contacts. A just-added contact was invisible and its action-sheet
operations resolved to a blank slot (#252).
Fork side:
- loadContacts() skips blank records, clearing the placeholder contacts beta_60
wrote into the contacts file (the next save drops them permanently).
- Chat threads follow a peer's rename. The thread is matched by key but its name
was never updated, while inbound messages are filed by sender name — so the
first message after a rename created a duplicate thread (#252).
- release.sh derives FIRMWARE_VERSION / build date / core version from the tag
and the pinned core, keeps the in-tree dev default in step, and ABORTS the
release if any staged image does not embed its own tag. beta_60 shipped
reporting v1.16.0-touch because that value was hand-maintained.
All 8 S3 envs build green on core-v1.17.1.
Reported-by: Pierre747, rustinmyeye, myshoeisonfire
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The p4StorageCall dispatcher committed in 099ccd8 had an inverted guard: it
skipped marshalling when already on core 0 -- written believing the main task
ran on core 1, but ESP_MAIN_TASK_AFFINITY_CPU0 puts the UI+mesh main task on
core 0, so every "hopped" write actually ran inline on the main task (proven by
a sector-level SDMMC trace: all card I/O on task 'main'). Fixed: the guard is
task-identity (p4OnStorageTask), all 11 write wrappers use it, and the storage
task runs at priority 3 on core 0 -- core 1 was tried and is pathological for
SD I/O (the SDMMC completion ISR lives on core 0; cross-core wakeups made a
contact save take ~20 s).
With the dispatcher REALLY engaging, the mechanism of the SD-store flash was
finally measured instead of theorized: a gap tracker in the DSI frame-restart
ISR recorded 35-39 ms inter-frame gaps (vs the 20 ms cadence) exactly
interleaved with SD sector transactions. The panel's frame DMA stops after
every frame and waits for an ISR to re-arm it; anything that masks interrupts
long enough (SD-transaction critical sections -- and on RISC-V a critical
section masks ALL levels, which is why an ESP_INTR_FLAG_LEVEL3 experiment
changed nothing) makes the restart late, and the panel displays the extended
blanking as the whole-screen flash. No error register ever latches because
nothing fails; frames are merely late. This also finally explains the tile
asymmetry: a tile is one large sequential transfer (one interrupt burst per
file), a FATFS store save is dozens of small transactions and metadata
round-trips -- dozens of chances per save to push a frame restart late.
Ship state stays: hot store on internal LittleFS (no SD transactions on the
hot path -- no late frames), SD for tiles/bulk. The promising future fix for
restoring the T-Deck-style SD store is making the panel autonomous: a CIRCULAR
single-FB DMA descriptor (dw_gdma_lli_set_next(item, item), is_last=false) so
the hardware loops frames with no ISR in the path at all -- prototyped against
the local IDF's esp_lcd_panel_dpi.c and reverted with the other diagnostics
(sdmmc sector trace, DSILATE gap tracker, LEVEL3); it needs its own verified
session and a durable patch mechanism before it can ship.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to 3bf56cd, driven by the (correct) challenge that "SD writes upset
the panel electrically" could not be the whole story when tile writes never
flash and other P4 firmwares use their cards freely. The completed on-device
matrix now separates two distinct mechanisms:
M1 An SD write EXECUTING on core 1 (where display.begin() allocated the DSI
frame-restart ISR, and where the Arduino loop task runs) masks that
core's interrupts inside the SDMMC/FATFS critical sections long enough
to drop a display frame. Proven: 80 contact saves to SD from core 1
flash every time; the identical 80 saves hopped to core 0 never flash.
NOT electrical -- the challenge was right.
M2 An SD write CONCURRENT with live UI rendering still flashes even from
core 0 (manual add/delete flows pump LVGL while the hopped write runs;
the write-open trace shows those runs performed only correct core-0
writes and flashed anyway). The same manual flow with the card mounted
but ZERO SD writes (internal LittleFS store) is clean.
Shipped design therefore stays: hot store on internal LittleFS (immune to both
mechanisms, verified across every flow), SD for tiles/bulk. Tiles' long-standing
immunity is consistent: brief core-0 writes vs the store's long ones.
New in this commit, kept for good:
- p4StorageCall (DataStore.cpp): a core-0 storage task; every hot write entry
(saveContacts/saveChannels/savePrefs/blobs, history segs, threads file) hops
onto it, blocking the caller -- semantics unchanged, execution core moved.
With the internal store this is still wanted hygiene: no storage write ever
runs on the render core again.
- WADA_SD_STORE_TEST build gate: puts the store back on the SD card for future
M2 investigation (known to still flash on manual flows; do not ship).
- TDP4_POKE_TRACE now also logs every write-open with its issuing core ([SDW]),
which is what pinned M2.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Contact-loss fix (microSD devices): atomic temp+swap saveContacts, unified robust
boot SD-mount ladder, and a storage-truth line in the Store-on-SD setting.
- Heltec V4-R8: route MISO for the shared FSPI bus so the microSD mounts; wake-from-sleep
freeze fixed (panel sleep goes through LovyanGFX's own bus, not the HSPI shim).
- Do Not Disturb (originally PR #156 by Tesso Codemonkey / @codemonkeybr): scheduled
sound-silencing window in Settings > Sound + a status-bar bell.
- Control-center redesign: round icon-only chips that spread to fill each row (T-Deck 2x4,
V4 portrait 2 rows), frosted-glass panel, real lock glyph + dynamic bell/bell-slash DND
chip + keyboard icon+word chip (new cc_icons_16 FontAwesome font), per-chip confirmation
toasts, press-grow clip fix, and a V4-portrait status-bar BLE/clock overlap fix.
- LilyGo T-Pager: map tile cache falls back to the microSD under the launcher (fixes the
"storage error / reflash the tiles partition" dead-end).
Co-Authored-By: Tesso Codemonkey <798026+codemonkeybr@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Full keyboard+encoder-driven board port: TCA8418 QWERTY matrix, rotary encoder
focus-nav, ST7796 display, ES8311 notification sound, both radio variants.
Hardware-verified by the author against a live mesh.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
# Conflicts:
# src/ui-touch/UITask.cpp
The P4's internal FFat 'locfd' has a broken FAT metadata layer: open/
read/write work but f_stat/exists()/size() return garbage - and WHICH
garbage shifts with the build. At -Og the exists()-gated identity and
NodePrefs loads worked by luck; the -Os switch flipped the lie and the
device booted with a fresh identity and default name, and profile
changes never survived a reboot (the loads failed, not the saves).
Chat history was unaffected because it already lives on SD_MMC.
- DataStore::useSdMmcStorage(): full-store adoption of the card
(/meshcomod root, identity store included), mirroring the T-Deck's
useSdStorage(). The old card-root contacts3/channels2/adv_blobs from
the secondary-FS era are renamed under /meshcomod.
- Tanmatsu boot: one-time rescue of identity + prefs off FFat by OPEN
and READ probing (never exists()/size() on that FS), then adopt the
card. FFat remains the no-card fallback only.
- DataStore::loadPrefs() probes candidates by opening and reading a
byte instead of exists() - truthful on the P4, identical elsewhere.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Boot TX power was left at a conservative 10dBm (Heltec V4's pattern) with
no way to raise it, unlike the T-Deck which boots straight at the chip's
real 22dBm ceiling. On hardware this was silently degrading outbound
range: adverts/DMs sent from the pager were unreliable while inbound
reception was fine, since the far node's own TX power was unaffected.
22dBm is confirmed as this chip's actual sub-GHz HP-PA ceiling via
RadioLib's LR1120::checkOutputPower() (LR1121 inherits it) and matches
trail-mate's own working config for this board.
Also widen the three RX-boosted-gain gates in MyMesh.cpp/DataStore.cpp
that only checked USE_SX1262/USE_SX1268 to include USE_LR1121 --
CustomLR1121Wrapper already implements setRxBoostedGainMode/
getRxBoostedGainMode, so the setting was silently a no-op for this radio.
Signed-off-by: Tesso M Costa <tesso.martins@gmail.com>
- History flush runs on the core-0 worker (snapshot + storage-busy gate +
bounded shutdown wait): kills the V4's multi-second ui:hist stalls and
the refuses-to-wake-after-message symptom (SPIFFS GC off the UI thread).
- BLE + Wi-Fi runtime enables share the boot co-init heap guard (50 KB
free + 20 KB block): refuse with a toast + revert the switch instead of
panicking (BLE) or silently not starting while claiming on (Wi-Fi).
- Internal DRAM: static footprint 95,365 -> 64,908 B (30.5 KB freed):
serial_interface -> PSRAM; 23 keyboard layout maps const'd -> flash;
nine rings/tables -> psAlloc PSRAM (UITask/TouchPrefsStore/MyMesh);
unused TinyUSB MSC + DFU-mode class drivers stubbed out of the link
(usb_unused_class_stubs.c) — CDC + runtime-DFU untouched, double-flash
verified over the stubbed stack.
- DataStore prefs writes are crash-safe (write .tmp, swap) with a
self-healing loader (.tmp recovery) and a truthful save result up to
the profile-name toast — fixes the boots-with-default-name-once report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- DataStore::saveContacts packs byte-identical 152B records into ~5KB chunk
writes (was ~12 tiny FS writes per contact: 2.6-3.0s at ~570 contacts on the
loop thread — the #82 sluggishness); falls back to per-field writes on alloc failure
- Contacts 60s age tick updates the Heard labels in place (was a ~1.3s full
teardown+rebuild); advert-driven rebuilds coalesced to one per 2.5s
- Spectrum: rolling waterfall row + live trace every quarter sweep (~250ms),
8-bin/8ms sweep chunks (~20% faster full sweep); trace Y range now tracks the
live min/max — instant expand, 3dB/sweep contract, 30dB minimum span
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Decoded a T-Deck coredump (Task WDT, loopTask stuck in spiffs_gc_clean under
DataStore::saveContacts): a contact save on a churned SPIFFS can trigger a
multi-second garbage-collection pass that starves core 0 and trips the task
watchdog, giving a reboot loop.
- Route contacts/channels to SD when a card is present, even on upgraded
devices whose data is still on SPIFFS (main.cpp mounts SD every boot; if not
fully adopting SD, store.setSecondaryFS(&SD)). DataStore::begin() migrates the
existing /contacts3 + /channels2 to the card once; identity/prefs stay put.
- Move the ref-counted WdtHeavyGuard into a shared header so the core
DataStore::saveContacts shares one count with the UI history/backup saves, and
wrap saveContacts with it (guaranteed fix for card-less devices).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First board lands. wadamesh builds the LVGL touch firmware as its own project,
consuming the MeshCore core via lib_deps @ git tag (ALLFATHER-BV/MeshCore
#v1.16.0-wada.0) — no vendored core in this repo. Output is byte-identical to the
in-tree meshcomod build (delta = embedded build-path strings only).
Contents: companion_radio app (MyMesh/main/DataStore), ui-touch LVGL UI,
variants/heltec_v4 board glue, boards/heltec_v4.json, lv_conf, the bundled
ed25519 lib, AsyncElegantOTA, partition table, and a flattened platformio.ini
for env heltec_v4_tft_companion_radio_usb_tcp_touch.
MeshCore-derived files (app glue) remain MIT; ui-touch is GPL — see NOTICE.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: Kaj Schittecat <kaj@schittecat.com>
