trail-mate/docs/engineering/linux-hostlink-track-management.md

Linux HostLink track management

The Linux tracker owns recording and local-file lifecycle on the UI loop. HostLink owns its management requests on a separate worker. Neither side may borrow the other's mutable state. Board profiles select hardware; this protocol adapter takes the configured storage root and contains no board or node indices.

Catalog

HostlinkTrackCatalog implements formal operation 80 through the existing management dispatcher. Its optional worker-owned coordinator controls capability advertisement; callers without that coordinator retain their existing subset. The production web readDeviceTrackCatalogPage helper consumes its responses.

The request is [null | [media, position]]. The response is [media, nextPosition|null, [name, format, bytes, recording]|null], with formal format IDs KML=0, GPX=1, CSV=2 and BIN=3. A returned row always has a continuation; an additional page terminates the listing. Names are valid UTF-8, at most 127 bytes, without controls or path separators. Hidden files, active.bin, unknown extensions, directories, symlinks and multiply linked files are excluded.

Reads use descriptor-relative no-follow opens under the configured tracks directory. The existing recorder holds an exclusive file lock throughout its lifetime. An independent nonblocking lock attempt observes that ownership without reading the UI's globals or interrupting the recorder. Each request visits up to four new directory entries and returns at most one row. Continuations use the formal 32-byte position units and 128 MiB cursor range. The shared scanner keeps its existing 192-entry default for upload journals; the catalog explicitly uses the larger protocol range.

The opaque media token binds pagination to the HostLink session, storage root and directory identity/timestamps. A changed directory, replaced root or different session invalidates the old cursor. The catalog is live metadata, not an immutable track-content snapshot. No private identity is required or generated by listing.

Snapshot lifecycle

Operations 81 and 86-88 now bind to HostlinkTrackSnapshots. The coordinator uses the existing shared selection decoder, bounded builder and session state machine. No second worker, UI-thread copy or whole-file payload buffer is added. The worker advances up to 64 copy slices per visit, each at most 320 bytes. Session close, TCP disconnect, idle expiry and application stop retire unsealed work. Another session cannot cancel or release an active owner's result.

The source is opened relative to the configured root with no-follow semantics and kept locked until finish or abort. This is the same file lock honored by recording and removal. Inode, size and timestamp checks detect replacement and ordinary writes. Timestamp checks alone are insufficient: an independent host probe found 196 of 200 successive same-size writes preserved both timestamps. Before publication, the native adapter therefore re-reads the complete source in bounded 4096-byte verification slices and compares its SHA-256 with the copied payload. Sealing remains pending until verification finishes. The regression waits for that terminal result rather than requiring immediate timestamp-based failure on the first scheduler visit.

The platform-neutral TSN1 container codec now resides in core HostLink. Existing ESP entry points forward to it with their original signatures. Linux adds its own file/hash/lease adapter, so it does not depend on ESP hardware or storage classes. Header and exact payload length travel in one immutable container; flush and no-replace publication precede a Ready response. A failed copy never publishes a mixed snapshot. On a root/media replacement, cleanup cannot delete files on the newly mounted medium; an unsealed old-medium candidate may remain for later maintenance.

The shared quota allows at most eight private snapshot files and twice the maximum container size, including unknown/orphaned entries. It never silently deletes old sealed results to admit another build. Reads resolve only a validated snapshot ID, check the header and optional expected hash, and return at most 320 payload bytes. The formal client independently checks the complete SHA-256. Sealed data survives source changes and reconnects; explicit release validates the container before removing it and is idempotent for an absent result.

The focused gate passed 20/20, including the original ESP file/service fixtures, formal catalog and import clients, eight native lifecycle/failure scenarios, and upload/identity/tracker regressions. The complete native suite then passed 246/246, and the compact suite passed 19/19. Same-size rewrite rejection passed 100 consecutive native runs. A final admission check revalidates the catalog media token after obtaining the source lock, closing the preflight-to-open race. The final affected gate passed 21/21, including a test-only linker interposition that changes the directory at that exact lock boundary. Native, compact and ARM affected targets were rebuilt; the whole suite was not repeated for this guard.

On the actual CM0, all nine native scenarios passed: commit, live recorder lease, same-size mutation, cancellation, disconnect, media replacement, idle expiry, quota and lock-boundary admission. An independent Python TCP client exercised catalog/build/read/release for all four format IDs, fresh-session reconnects, source mutation after sealing, SHA-256 validation, wrong-hash rejection and idempotent release. The same wire test passed against the exact installed /usr/bin/trailmate-linux, with generated temporary roots and dummy SDL/radio- disabled profiles. Its source payloads are transport fixtures; production CSV and binary recording conversion is separately covered by the formal Node test.

The initial snapshot-delivery deb SHA-256 was ac73601189dee173e4f51e8a7f7a8844e503469cbd6a1f210b0f9255347fc585; its installed executable was 6d7cacf277bd0df6ec292bd06519c7f09a4e8d8e714d43e80853c60bef38fe08. CPack ran once. The first lintian process failed with a host Perl internal error; a clean, deterministic Perl-environment retry checked the same unchanged deb and passed. Installation, dpkg -V, original-logo comparison, isolated startup/ timed exit and installed formal-application wire checks passed. The previous 4364b859... package is retained as trailmate-cm0-before-track.deb. No current user recording or identity was changed, and no screenshots or reboot were used.

Remaining contracts

The current CM0 deb includes the catalog and formal-client snapshot import chain. Local preparation/signing 82/96/99 is also delivered; see local track signing. Directory list/discovery 89/90 is delivered through the native directory service. Publication, history and remote-read families require further native ownership/admission review. Existing signed package installation 100-101 is already delivered and must be preserved. Catalog support alone is not full track-management parity. ESP stack hygiene and Reticulum scope guards passed.

The focused web fixture covers real HostLink framing and production catalog decoding, all four formats, UTF-8 names, pagination beyond 192 files, recorder ownership from a separate process, stale cursors, invalid requests, directory symlink rejection and malformed UTF-8 filenames. Its identities and files live under temporary roots; it does not touch device recordings or physical radio.

The focused gate passed 5/5: formal catalog client, existing HostLink client, author restore, radio restore and management dispatch. The first catalog run failed because its temporary fixture configured all roots to one directory while the test wrote under an extra sd subdirectory. Correcting the test path exposed the intended files; the passing run additionally covers 204 listed records. This is native host evidence, not ARM or installed-package acceptance.

The scanner's two callers were manually audited because the new symbols are not in the GitNexus index. After parameterizing the scan ceiling, the existing upload limit remains the default. Tracker storage, production tracker decoding, upload maintenance and upload lifecycle regressions passed 4/4. The accumulated whole-worktree GitNexus check reports CRITICAL (213 files, 849 symbols, 27 flows), not the scope of this catalog alone. No commit was made.