The quiescence gate I added keyed the "don't paint from a stale backend"
early return on may_swap alone, so mapNoteStorageChanged() dropped every
decoded tile and then returned without repainting whenever ANY tile fetch
was in flight -- including the common case where no backend swap was needed
or deferred at all.
A wedge-recovery remount or a reinsert while already on the card takes
neither branch, so the backend is correct and the map should repaint
immediately. Instead it went blank until the next unrelated pan/zoom/tab
change, and neither sdHealthTick retry covers that case: both require a
pending swap (s_tile_fs == &SD with the card gone, or s_tile_fs != &SD with
it present), so neither fires when nothing needed swapping.
Compute what the swap would actually change and skip the repaint only when
that change was wanted and had to wait.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The quiescence gate drops the decoded tiles and clears the dedup ring
unconditionally, which is right, but it also fell through to
renderMapTiles() with s_tile_fs still pointing at the old backend. On the
teardown side that means probing &SD for a card that is gone -- up to five
opens per tile, each paying a full sd_diskio timeout, on the loop task.
Return after the invalidation when the swap could not be applied.
sdHealthTick renders as soon as it lands it.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The CAP_SD branch no longer references s_map_style or s_tiles_from_sd, so
the OSM-vs-topo explanation above it described a guard that is not there.
Describe what the two branches actually do.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The deferred reinsert swap runs before the Pager removal handler, and
s_sd_mounted stays true until that handler wins quiescence. A card yanked
while the swap was still pending therefore got promoted to s_tile_fs =
&SD, costing a full freeMapTiles() + renderMapTiles() on the loop task,
before the same tick unmounted it and re-rendered again.
Harmless in outcome -- the read path short-circuits on the
s_sd_fail_note_ms that card-detect stamps -- but two wasted map passes on
the UI thread per removal.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The unconditional tileCacheRemove() on a bad JPEG header only covers the
generic cache path, which the microSD branch never reaches in the mode
this PR is about: with SD tiles on and OSM selected, /tiles/<z>/<x>/<y>.jpg
is read by the SD branch, returned as-is, and then fails in
decodeJpegScaledToRgb565(). That path only does ++n_missing, so unlike a
load miss it does not queue a re-fetch either -- a half-written tile sat
there until a manual "Reload visible tiles".
Apply the same 3-byte SOI check to that leg only. /maps/osm packs and PNG
packs dropped into /tiles keep their read-only contract: they cannot be
re-downloaded, so they are never removed.
Signed-off-by: Pixel Perfect <me@pixp.cc>
loadTileJpeg() and tileExistsAt() both qualify the microSD branch with
s_map_style == 0 because SD packs are OSM-only, but the status label and
mapTileSourceReady() did not. With microSD tiles on and topo selected the
empty-map notice told the user to install /maps/osm/z/x/y.png and
suppressed the "Downloading map tiles... ok/fail/http" diagnostics, while
the code was in fact fetching topo tiles into /tiles/topo on the card.
Pre-existing on T-Deck; the persistent Pager source toggle newly exposes
it there.
Signed-off-by: Pixel Perfect <me@pixp.cc>
fmSdDoFormat() armed the deferred f_mkfs with no quiescence check, so a
confirm tap while the tile worker held an open File (the whole of a
download, up to the 12 s cap) ran SD.end() + sdcard_init + f_mkfs
underneath it. The worker's next f.write()/f.close() then acted on a
de-registered pdrv and a freshly rewritten volume. The format-failure
path already comments that a worker File may be open; by then it had
been invalid for tens of seconds.
Gate the confirm on sdRuntimeLifecycleBusy() (the same predicate the
Pager factory reset uses) and re-check on the tick before f_mkfs fires,
since the two ticks that let the notice paint are also two ticks in which
a map repaint can queue fetches. Bounded at 15 s so a stuck consumer
cancels the format instead of hanging behind the "do not power off"
notice.
Also stop arming the countdown on M9 / V4-R8. Those builds offer the
format button but carry no executor for it (the deferred f_mkfs block is
HAS_TDECK_GT911-only), so a single tap left s_sd_format_pending set for
the rest of the boot and sdHealthTick returned early at its
s_sd_format_pending guard forever after — silently disabling wedge
detection, remount and the SD tile-backend swap.
Signed-off-by: Pixel Perfect <me@pixp.cc>
mapNoteStorageChanged() guarded only the adopt direction. The teardown
direction repointed s_tile_fs away from &SD with no check at all, even
though that is the side that strands an open SD File: the worker holds
one for the whole of a download (up to the 12 s cap) and dereferences
the global backend for every filesystem operation of the request.
Every current caller happens to gate, but the format-failure path checks
only tileFetchPendingLoad() and the format-success path checks nothing,
so the invariant was one caller away from being violated.
The adopt branch's early return was also too broad. It skipped the dedup
clear, freeMapTiles() and the re-render that are the whole point of the
call, so swapping cards while a fetch was in flight left the previous
card's tiles on screen with the dedup ring suppressing the re-fetch until
sdHealthTick's retry landed.
Compute may_swap once, apply it to both branches, and always fall through
to the invalidation.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The boot gate was widened from origin's
if (SPIFFS.exists(id) && !SD.exists(sdid))
to just SPIFFS.exists(id), and the mig_busy branch now demands
kSdMigrationComplete before it will adopt a card that already has an
identity. That marker is new in this branch, so no card written by earlier
firmware can carry it.
Earlier firmware also cleared sd_mig_busy whenever the copy RETURNED rather
than when it succeeded, and it copied the identity in SPIFFS walk order
instead of last. A device whose migration died after the identity landed
therefore adopted the card anyway and has been running full-SD ever since,
with sd_mig_busy stuck at 1. On upgrade that device now computes
committed=false and boots from its months-stale SPIFFS snapshot: old node
name, old settings, old contacts, history rebound to internal, and the
secondary-store path then copies the stale contacts to the card root.
Recoverable via the Settings banner, but a bad first impression of an OTA.
Grant a one-time amnesty when the card identity is byte-identical to the
internal one. That proves the card holds THIS profile and is the newer
store, so adopting it is strictly closer to the user's live data. The
recovery banner stays up so a reconciling Copy-to-SD can still fill anything
the interrupted copy missed. A card holding a different identity is
untouched by this and still fails closed.
The byte comparison is hoisted out of the Pager-only block into
sdIdentityMatchesInternal() and meshcomodSdProfileMatchesInternal() now
calls it, so the two copies cannot drift. That also tightens one edge:
a zero-length identity used to compare equal to another zero-length
identity and now counts as a mismatch, which is the safe direction.
Also correct a comment that claimed the owner-marker cleanup removes "only
our own incomplete marker" — that branch runs precisely when the digest does
NOT match.
Signed-off-by: Pixel Perfect <me@pixp.cc>
sdProfileTreeContainsFile() re-stat'ed every recursive child with
SD.exists() before opening it. The child came straight from
openNextFile(), so it provably exists; the check can only ever produce a
false negative, and false is the single verdict that lets a residual
foreign subtree read as empty and the card get claimed. Every other
uncertainty in this function already fails closed.
Keep the check at the root, where "/meshcomod is absent" genuinely means
there is nothing to conflict with, and let the open() fail-closed path
govern below it.
Signed-off-by: Pixel Perfect <me@pixp.cc>
The Store fetches a language from /apps/lang/<ver>/<code>.lang, not from the
mutable copy at the top of the directory -- so bumping langs.json to v8 without
creating lang/8/ would have offered every device an Update that 404s.
Also normalised the "ver" spacing in langs.json back to the file's own style.
The device parser skips whitespace after the colon so both forms work, but
there is no reason for the file to be inconsistent with itself.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The audit found three keys the Store work added or renamed since v7 -- Store,
the out-of-memory message and the Languages hint -- untranslated in all 13
languages. Filled, using each language's own word for the Use button so the
hint matches the control it points at, and bumped the files to v8. Every
language is back to full coverage at 856 keys.
New deploy/site/sdk.html: the wada.* reference. The beta ships a public API and
had no user-facing documentation for it, so nobody outside the repo could write
an app. Covers the lifecycle, all seven tables, the app format, side-loading and
how to publish, plus the two constraints that surprise people -- the mesh is
read-only in v1 and http_get is plain HTTP because TLS does not fit in the heap
left after Wi-Fi associates.
Site device list: the Attaky Core and the T-Display P4 are fully supported now
(Kaj's call), so their not-hardware-verified caveats are gone. Dropped the "one
of the two primary development boards" wording, which stopped being true once
the P4 joined the test loop. The terms section named only the T-Deck and Heltec
V4 as flashable when the flasher offers ten boards; it now points at the list on
the page instead of going stale again.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three things, all "the Store only ever ran on a 320 px T-Deck".
LINK FIX (supersedes 7e5eac4, which treated the symptom). LuaAppHost.cpp
forward-declared `class WiFiClient;`. That declares a BRAND NEW incomplete
class, which is only correct where WiFiClient really is a class of that name --
it is a #define to C6Client on the T-Display P4 and a `using WiFiClient =
NetworkClient` alias under arduino-esp32 3.x on the Tanmatsu. So the file
emitted luaNetWorkerService under a signature the caller never referenced and
BOTH P4 targets failed to link, with three different mangled names, while all 8
S3 envs built clean -- which is why it stayed hidden until the first P4 build.
The TU now refuses to name the type at all: UITask owns the networking and
passes the objects as opaque handles through luaStoreHttpGetOpaque, casting
them back on its side. This kills the whole class of bug rather than adding a
third per-board alias.
LAYOUT. Cards measure their text instead of assuming "title + two description
lines" -- lv_txt_get_size on both title and description, height floored so it
can never be shorter than the icon chip or the action button. The title wraps
instead of truncating and the description sits below its ACTUAL height, which
is what was overlapping. Measuring beats reading geometry back, which returns 0
before layout. This fixes every board for translations longer than the English.
LV_LABEL_LONG_DOT only ellipsizes when the HEIGHT is bounded; with just a width
it silently wraps, which hit the card title, the language rows, the sideloaded
rows and the tab labels. Tab cells are now sized in proportion to their text,
not even thirds: an even third on the P4 (284 px logical, scaled fonts) is
~90 px and "Languages" does not fit that, while "Apps" left half its cell
empty. The bar height follows the font, floored at 30 so S3 stays identical.
FEEDBACK. The pressed button's caption becomes a spinner for the whole
download, not a 1.2 s toast, so a slow link no longer reads as a dead tap. It
is repainted from the in-flight id after a rebuild, since a card scan finishing
mid-download would otherwise restore a plain Get button. The caption is hidden
rather than deleted -- this runs inside the button's own event.
Drawer tile is "Store", short enough for every panel.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The T-Display P4 failed to link with an undefined reference to
luaNetWorkerService(C6Client&, HTTPClient&).
On that board the C6 runs ESP-AT, so UITask.cpp does #define WiFiClient
C6Client to route sockets through AT-over-SDIO. LuaAppHost.cpp forward-declares
WiFiClient without the alias, so it emitted the definition as
_Z19luaNetWorkerServiceR10WiFiClientR10HTTPClient while the caller referenced
the C6Client& overload -- the symbol was present, just under the wrong type.
Confirmed by reading both object files.
Mirroring the alias fixes it. This TU only forwards the reference to
luaStoreHttpGet, so an incomplete type is sufficient. Every S3 board built
fine throughout, which is why it went unnoticed until the first P4 build of
the Lua work.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Three T-Decks report "Save update bin to SD" dying with "SD write failed",
usually at 1% and never past 7%, while map tiles and settings save fine.
f.write() passes the buffer down to SD/SPI, which DMAs from it. PSRAM is not
reliably usable as a DMA source, so the write commits fewer bytes than asked
and the transfer fails at an offset that moves around with the network chunk
sizes. sdFwWorkerRun allocated its copy buffer MALLOC_CAP_SPIRAM with the
comment "PSRAM preferred", which is exactly backwards for this.
The three-way comparison makes it clear: the tile cache copies through a 1 KB
*stack* buffer and works, the .lang installer used PSRAM and failed the same
way (fixed earlier via langWriteAll's internal bounce buffer), and this used
PSRAM and fails. It has been broken since the feature landed in beta_36 --
it never worked, which matches nobody reporting it as a regression.
Buffers moved to MALLOC_CAP_DMA in all three remaining places that write to a
card: the firmware download, fmCopyFile (the Files app copy, any file to SD)
and crashDumpExport (coredump to SD, broken in the same way).
Also in the download loop: retry a short write before declaring failure, and
yield to IDLE each chunk. A ~3 MB download over a fast link never yielded, so
it could starve the task watchdog -- the trap the tile fetcher already
documents.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The RAM investigation and the language-download debugging left Serial probes
across three files: the BOOTMEM/MESHMEM/MEMPROBE macros and their 48 call
sites, plus the [STORE]/[APPCAT]/[LANGDL]/[LANGSEL] traces. They have served
their purpose and were flooding the companion USB-CDC, so they all go. Two
locals in the .lang installer (wr, wr3) existed only to feed a trace and go
with it; the control-write repair path that actually rebuilds /lang stays.
Separately, the map tile-cache help text read "256\xC3\x97256": the 256 after
the multiplication sign is all hex digits, so the compiler folded it into the
escape and emitted one out-of-range byte instead of "256x256". Breaking the
literal after the escape terminates it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per Kaj: keep the Store page (Settings > Language opens it) but drop what a
board without the Store cannot do.
- Store tile restored everywhere; the page is the language picker.
- No Apps tab where CAP_BUILTIN_LUA_APPS: the tab bar is Built-in +
Languages, sized for two, and opening never lands on the absent tab.
- Long-pressing a seeded app no longer offers Remove: there is no file to
delete, and the built-in would just reappear on the next scan.
- Languages tab lists all 13 translations with Use where CAP_BUILTIN_LANGS
(they ship in the image), skipping the installed-files and
available-to-download sections and the /lang hint that does not apply.
T-Deck unchanged at 75.5%; V4 85.7%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PSRAM move earlier tonight turned s_lua_cat/s_lua_inst/s_langcat/s_langinst
into pointers, but four size expressions still read
'sizeof(s_lua_inst)/sizeof(s_lua_inst[0])' — 4/60 = 0 on a pointer. Every list
was silently capped at zero entries, so installed apps stopped listing on ALL
boards and the seeded built-ins could never be added. Replaced with named
constants (kLuaCatMax/kLuaInstMax/kLangCatMax/kLangInstMax), which the
allocator now uses too, so the cap and the allocation can no longer disagree.
Also: the Store tile is gone on boards that ship their apps in the image
(CAP_BUILTIN_LUA_APPS) — a deliberate product decision rather than a broken
button, per Kaj: the V4 is not a Store board.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kaj's idea, and better than restoring the deleted native pages: the Lua host
already had the mechanism. luaAppLaunchFile() is file-first with an embedded
fallback — it just never had anything seeded into it.
gen-lua-builtin.py generates src/ui-touch/lua_builtin.h from out/firmware/apps/,
the same .lua the Store serves, as C++ raw strings (Snake, RF Monitor, Airtime;
10 KB). luaStoreScanInstalled() now wraps the filesystem scan and seeds any
built-in the scan did not find — after it, so a downloaded copy keeps its own
entry and version, and seeding still happens when there is no filesystem at all.
Tiles, the drawer and the Store's Installed list all work through the existing
paths; launching passes the embedded source, and a downloaded file still wins.
CAP_BUILTIN_LUA_APPS is ON for the V4 only. Flash: V4 85.5% -> 85.8%,
T-Deck 75.5% and Pager 78.1% unchanged. No RAM cost.
So the V4 now has the apps and all 13 languages out of the box; the only thing
it loses without a working Store is browsing for new ones.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gen-lang-builtin.py generates src/ui-touch/i18n_builtin.h from the SAME
deploy/apps/lang/*.lang files the Store serves, so the downloaded and the
compiled-in translations can never drift — translators still edit one place.
TR() consults the loaded file overlay first, then the baked table, then
English, so a downloaded language still wins on boards that can fetch one.
CAP_BUILTIN_LANGS is ON for the V4 (its net worker is fragile at ~95%
internal RAM, so the Store is not dependable there and it would otherwise be
stuck on English) and OFF elsewhere. Cost is FLASH only, no RAM:
V4 71.7% -> 85.5%
T-Deck 75.5% (unchanged, gate off)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured on the V4, bracketing each step of the post-mesh init block:
wifiConfigBegin() 172 bytes
serial_interface.begin(Serial, TCP, WS) 0 bytes
WiFi.mode()/WiFi.begin() -> esp_wifi_init 36,432 bytes
So gating the TCP and WebSocket servers (or MQTT, which already returns
early when disabled) frees nothing: listening sockets are cheap and the
cost only arrives per connected client. Wi-Fi is ~100 KB of the 289 KB
heap (36 KB init + ~64 KB associate), it is DMA memory that cannot live in
PSRAM, and it is not divisible by feature.
Conclusion for the V4: no application-level lever of size remains. The
options are restoring the native apps/languages so the board does not need
the network for them, rebuilding the Arduino libs with a custom sdkconfig
(Wi-Fi buffer counts + IRAM shadow), or accepting 95%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured per builder: tabview+tab bar 8.5 KB, settings 4.5, chat detail 4.4,
map 2.9, contacts 2.7, home 1.2, chats 0.2. So deferred tab construction is
worth ~14.5 KB but needs four separate lazy builders, not one.
Cumulative effect of this session at the same boot point:
boot int free 87444 -> 104816
post-UI-build 60692 -> 78056 (+17.4 KB)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured on the V4: post-spiffs 167496 -> 171316, pre-iface 121432 ->
126836. Verified on the T-Deck (8 MB PSRAM, the working reference) that
pushing ordinary allocations to PSRAM breaks nothing: boots, chats and the
store all fine, so the threshold is safe for every board.
Session total on the V4: ~12 KB internal reclaimed (6.7 KB statics + 5.4 KB
threshold).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SDK ships CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=4096, so every allocation
under 4 KB is forced into internal DRAM even with PSRAM free.
heap_caps_malloc_extmem_enable(1024) changes that at runtime; anything that
needs internal/DMA memory asks by capability and is unaffected.
Measured on the V4: post-spiffs 167496 -> 170292 free. Real but modest,
because most internal use is NOT generic malloc — Wi-Fi DMA (capability
tagged), 171 KB of static .bss/.data/IRAM-shadow, and task stacks.
Session total on the V4: ~9.5 KB internal reclaimed (6.7 KB statics to
PSRAM + 2.8 KB here), plus the worker-stack fix that made the store, map
tiles and the update check work there at all.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Attribution on the V4, per step, measured on hardware:
SPIFFS mount 4.4 KB
storage selection 7.5 KB
store.begin() 0
the_mesh.begin() 52 BYTES (contacts load is 369 KB, all PSRAM)
post-mesh init block 38.6 KB <- Wi-Fi/TCP/MQTT/companion interface
UI construction 38 KB
Wi-Fi associate ~55 KB
So MAX_CONTACTS and the mesh core are both exonerated: contacts already live
in PSRAM and MyMesh::begin() itself costs 52 bytes. The weight is Wi-Fi plus
the interface block that follows the mesh, and the UI.
Also checked and already optimal: LVGL's heap and the LVGL draw buffer are
on PSRAM, CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP=y, and Arduino defaults
useStaticBuffers to false (static TX 0 / static RX 4 / dynamic 32), so the
obvious Wi-Fi buffer lever is already pulled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Kaj