trail-mate/docs/engineering/linux-tmap-creation.md

On-device TMAP creation

CM0 and Cardputer Zero expose Extensions → Make TMAP directly in the bottom action bar, without opening Menu. It also opens directly with T while on Extensions; the button displays T Make TMAP and H Help lists the shortcut. Text editors keep literal T input. The shared LVGL wizard uses five pages: global administrative area, map content, save name, review, and progress/result. This follows Center's area-first authoring flow. Administrative search is the default; Coordinates switches to explicit WGS84 bounds and back. Results show country and administrative context to distinguish namesakes. Only actual boundary extents are accepted; point-only geocoding results are not turned into invented administrative boundaries.

Enter a place name and press Enter or activate Search. Tab moves focus and Enter activates the selected control. Map name, filename and output defaults are prefilled after selecting an area. Advanced contains the stable package key, revision, output budget and optional local PBF input. Balanced adjusts zoom for the chosen area; Overview, Walking, Detailed and custom zoom remain available. Read-only panels scroll with arrow keys. Closing the page leaves a job running; Cancel stops it. Reopening shows the job, and a completed job offers View map and New map. Before the first source bytes arrive, the job shows the elapsed wait for the map server; it then reports received KiB and later producer stages.

Sources and outputs

The normal flow downloads OSM data for the selected enclosing rectangle into its private job workspace. The installed osmium-tool converts that XML stream to PBF before the existing Center producer runs. A user does not need to obtain or copy a PBF first. Downloading, conversion, production, verification and installation all run away from the UI thread and can be cancelled. Network or producer errors remain visible and do not publish an incomplete map.

Global administrative lookup uses the Photon API. Searches are explicit, serialized and spaced by at least a second, with a bounded cache of successful queries. A new search cancels obsolete work; an older response cannot replace the latest results. TRAIL_MATE_AREA_SEARCH_URL can select another compatible deployment without rebuilding. Map acquisition uses an Overpass interpreter, currently defaulting to the Private.coffee public instance; TRAIL_MATE_MAP_SOURCE_URL can select another interpreter. The original default returned timeouts and HTTP 502 during device validation. Attribution is shown in Review. Public services can be unavailable or throttle requests; the UI exposes retry and coordinate entry instead of treating a failure as a result.

Online jobs on the small device are limited to 2500 square kilometres and a 64 MiB source response. Larger selections must be reduced or use an appropriate local PBF through Advanced. The default map output budget is 128 MiB and is independent of the source response limit. The estimate also checks working space. Bounds use WGS84 and must not cross the antimeridian.

Optional local .osm.pbf files are scanned beneath map-sources or maps/sources in the configured storage root. TRAIL_MATE_TMAP_SOURCE_ROOT remains available for a development source directory. Advanced lists these inputs alongside Online download; local-file scans remain asynchronous.

OSM, terrain and satellite layers use Center's existing PBF rendering and raster sources. Terrain and satellite may fetch source data over the system network. Their source cache belongs to the private job workspace and is deleted with it after success, failure or cancellation. POI, search and independent labels are retained in the authoritative OSM package, including when only another raster layer is requested. The current wizard produces Center's display/search TMAP profile; the separate experimental navigation/region-production CLI and contour production are not new wizard modes.

Completed files use the existing maps/tmap/osm, terrain and satellite layout. Referenced font resources are installed before their maps. The normal map catalog's existing directory/file watcher discovers newly published maps. Existing map filenames are refused; select another name instead of silently replacing a map. Keep a package key stable for updates to the same region and choose its revision explicitly.

Search for a small city or district, select the correctly disambiguated result, choose OSM and Balanced, then review and create. A manual example uses West 102.70, South 25.00, East 102.71, North 25.01; those numbers are optional input, not a default inferred location. After success, return to Map → Map Layer and select the produced base layer, move to the covered region and use a zoom in its produced range. The completed job also offers View map, which opens the existing map renderer at the generated area's centre and zoom, with Back returning to the result. No manual output copy or separate import is required.

Only the original PBF and final map resources need to remain after a normal job. Final output includes the .tmap files and any referenced resources under trailmate/packs/fonts; deleting those fonts can break map labels. Plans, temporary indexes, raster downloads and staging output belong to the private workspace and are removed after success, failure or cancellation. Center's geometry and font-summary JSON reports are not installed as map payloads. The app installation/runtime, its regular settings and unrelated map caches are separate from creator intermediates.

Interrupted jobs are recovered automatically on service startup. Producer work and publication staging both live beneath the configured storage root's private trailmate/.tmap-jobs/job-XXXXXX directories. The namespace has a checked ownership record and a process lease. Recovery only removes job directories in that marked namespace, traverses relative to directory descriptors without following symlinks, and never scans input PBF or final map directories for deletion. It preserves unknown siblings and refuses unmarked nonempty namespaces. A small .owner control file remains; it is not a retained rendering cache.

The producer inherits the lease. If the app dies while its producer is still alive, a newly started service waits and retries recovery every two seconds until that process releases the lease. After power restoration the lease is released by the OS and abandoned work is recovered on startup. Files already published as final maps/fonts remain intact; recovery does not roll back published outputs. Legacy unmarked cache folders or generic .creating files from older versions are not blindly deleted because their ownership cannot be proved. This version no longer creates generic staging files beside final maps.

Estimates count uncompressed RGB565 tiles and conservative working space. Indexes, font resources, geometry complexity and actual source processing affect final size and duration; deduplication can reduce pixels. The producer enforces output and free-space budgets again before processing. A failed/cancelled job discards its private workspace and rolls back only files created by that job.

Ownership and reuse

The application does not fork the TMAP writer, OSM importer, renderer or search index format. LinuxTmapCreation invokes the existing Center TmapTool in a separate process, using a captured PBF descriptor, a private JSON plan, copied progress and fixed build/verify commands. Center Core owns production and format verification; the existing independent TMAP SDK remains the device reader.

The portable tmap_creation::Service seam contains plans/status only. Composition registers the Linux worker for CM0's tmap_creation=true profile and Cardputer's startup. Unsupported ESP32/nRF targets have no registered producer, hidden entry and no producer thread/runtime dependency. Other Linux compositions must opt in. All filesystem scans, rendering, copying and child-process control run off the UI thread. One job owns its state, and storage publication uses descriptor-rooted I/O, an exclusive storage lease and no-replace publication.

Producer packaging

Publish the existing Center project for the target; do not copy its source into the firmware repository:

./tools/publish_tmap_producer.ps1 -CenterRoot ../trail-mate-center `
  -OutputDirectory ./build/tmap-producer/linux-arm -Runtime linux-arm

Use linux-arm for CM0, linux-arm64 for the Cardputer package builder, and linux-x64 for native integration tests. The self-contained output includes its runtime, Skia assets, Center license and a producer/hash manifest. CMake's TRAIL_MATE_TMAP_PRODUCER_DIR installs it under /usr/lib/trailmate/tmap-producer and checks its architecture. CM0 and Cardputer package metadata include their distribution's ICU dependency when bundling this producer, and the Cardputer package depends on osmium-tool for online source conversion. No .NET SDK or Center checkout is required on the device. An explicit TRAIL_MATE_TMAP_PRODUCER executable override is available for development.

The Debian Lintian overrides are restricted to SkiaSharp's bundled raster codecs and the published SQLite native asset's debug symbols. These upstream runtime binaries are preserved so their hashes continue to match the producer manifest.

The Cardputer build_cardputer_zero_deb.sh helper also accepts TRAIL_MATE_TMAP_PRODUCER_DIR and transfers the published bundle to its existing ARM64 Docker builder. A developer build without a bundle reports an unavailable producer; it cannot claim a working production creator.

Validation

Native contract cases cover invalid plans, budget refusal, single-job ownership, producer failure, cancellation, workspace cleanup and preservation of an existing destination. A small real OSM PBF is processed by the published Center producer; the resulting map is independently discovered/read by the application's TMAP SDK and its POI is found through the normal map-search runtime. Both 1232×568 and 320×170 wizard tests cover every step, form preservation, background continuation, cancellation and unsupported-service behavior. The online fixture also exercises namesakes, invalid and point-only search results, caching, obsolete searches, source download/conversion, HTTP failures, response limits and cancellation. Its successful output is read and searched through the ordinary device SDK. Release/device results are recorded in the CM0 acceptance audit after those checks run.