Files
..

Native track directories

directory_service.py runs the production trailmate.track.v1 service with an explicit existing Reticulum configuration. Device author signatures remain on the original manifests; the service's transport identity is separate.

python -m tools.tracks.directory_service --state /private/tracks --rns-config /private/reticulum --replica-seed <track-discovery-hash>

Use at most three 32-hex track discovery hashes. Do not use LXMF delivery or Geocaching discovery hashes. Signed announcements and an extended capability exchange still bind the discovered directory's public key and epoch. A seed requests discovery; it grants no trust to content. The handler accepts signed path responses so a new service can find directories that started earlier.

Global country/region and administrative selection

data/country-regions.json adapts the 242 country/region boundaries already shipped with the Geocaching page. It adds no province/city/county database. The directory serves these as one selectable level through its existing region operations; compatible operator catalogs may provide additional levels. This dataset does not claim complete worldwide first-level administration.

data/world-regions.json adds one administrative selection level using pinned Natural Earth Admin 1 data: 251 country/region parents and 4,596 administrative units. It retains the existing 242 country boundaries and adds nine parents needed by the administrative source. The file is 7,963,790 bytes, within the existing 8 MiB reader limit. Use it with the same directory protocol:

python -m tools.tracks.directory_service --state /private/tracks --rns-config /private/reticulum --regions-file tools/tracks/data/world-regions.json --regions-namespace world

The global hierarchy has only country/region and one administrative level. It adds no city/county database, country-specific hierarchy or firmware data. Native names are provided for the 11 Web locales where the source supplies them; missing names use the region's base name. The first-level polygons use topology-preserving 0.025-degree simplification and five-decimal coordinates. This is approximate discovery geometry, not a current legal boundary service. The source may classify departments, territories and unnamed areas as Admin 1; its administrative units can differ from current national subdivisions. Map-extent search remains available near simplified boundaries.

Natural Earth is public domain. Source documentation: Admin 1 and terms. The generator pins revision ca96624a56bd078437bca8184e78163e5039ad19 and verifies both input SHA-256 values. Download the two GeoJSON inputs from that revision's geojson/ directory and run with build-time Shapely 2.1.2 and the existing directory Python dependencies installed:

python -m tools.tracks.build_world_regions --countries-source ne_10m_admin_0_countries.geojson --admin-source ne_10m_admin_1_states_provinces.geojson --output tools/tracks/data/world-regions.json

Generation uses the existing country catalog and Node locale registry. Runtime loading requires neither Shapely nor Node. Regenerating administrative data changes the catalog version, so connected browsers must refresh their selected directory catalog. Directory identity, published routes and author keys remain in the existing service state.

python -m tools.tracks.directory_service --state /private/tracks --rns-config /private/reticulum --regions-file tools/tracks/data/country-regions.json --regions-namespace world

The source and public-domain terms are documented in site/geocaching/data/README.md. Country labels are generated for all 11 Web locales using Node's Intl.DisplayNames. Generalized boundaries have the same coastal/small-island limits as the source; map-extent search remains available. The polar region is divided into bounded polygon pieces without omitting it.

To regenerate, install build-time Shapely 2.1.2 and run:

python -m tools.tracks.build_country_regions --output tools/tracks/data/country-regions.json

Node and Shapely are needed only for generation. The running directory uses the existing JSON catalog reader; neither the dataset nor a new dependency is sent to firmware or allocated in device RAM.

TRAIL_MATE_TRACK_REPLICA_SEEDS supplies comma-separated defaults when no --replica-seed argument is passed. The shared NAS Compose template maps TRACK_REPLICA_SEEDS from .env, including with the administrative-catalogue override. Service identity, content, upload offsets and peer progress persist in the track data volume. No browser login, device RF mode or USB transport is used for directory-to-directory replication.

Replication runs automatically for compatible discovered peers. The loop queues at most 32 announcements, stores at most 32 peers, and allows one active replica request. Pages contain at most 16 references; each file request carries at most 1024 bytes. A round has a 256 KiB wire budget, then yields. Completed rounds poll for changes every 300 seconds; failures back off 60/300/1800/21600 seconds. These are explicit bounds, not measured NAS throughput or capacity guarantees.

The receiver freezes an inventory high watermark, obtains exact original manifests and verifies every inventory binding. Missing parents can be fetched with a 64-ancestor bound. It copies the exact KML and signed attachment index, then required resources and available optional resources. File hashes and package bindings are checked before version import. A page checkpoint advances only after every referenced version and its required files have committed. Duplicate versions must still pass physical content checks; missing files are refilled before progress is claimed. Valid signature forks propagate as evidence, without declaring either branch the current winner.

Before each request, exact request bytes and ID are persisted. File offsets commit first, so a crash between a chunk write and peer-state update can retry the original range. Replica-only temporary-file expiry extends on newly written bytes; repeated old ranges do not renew it. Ordinary browser upload lifetime is unchanged. Peer epoch changes reset the checkpoint and cancel only that peer's unfinished local temporary upload, preserving immutable files and versions.

By default, automatic file copies wait for a reported next-hop bitrate of at least 100,000 bit/s. Unknown paths are requested and rechecked separately from known slow paths, including after restart. The bitrate is Reticulum's interface estimate (for example a TCP guess), not a bandwidth measurement; shared instances may expose the local shared interface rather than the complete underlying path. Keep the NAS transport on the intended IP interfaces. The operator may explicitly enable slow/unknown paths with --allow-low-bandwidth-replication; this can create large background traffic and is not enabled by the Compose template.

TRAIL_MATE_REPLICATION_DEBUG=1 adds public request IDs/operations to logs for diagnostics. It does not print author private keys, file bodies or upload tokens. Normal logs report discovery, completed pages and failures.

Local verification

python -m unittest tools.tracks.test_replication_inventory tools.tracks.test_replication
python -m unittest tools.tracks.test_storage_recovery
python -m unittest tools.tracks.test_storage_network
python -m unittest tools.tracks.test_replication_network

The network test requires website dependencies, npm run build:tracks from site/, and a Playwright browser. It runs two real native directories over loopback, interrupts a partial file copy, restarts the replica with its persistent state, and stops the source. Only after the source TCP listener is confirmed offline does a fresh browser read the archived v2 package, original author-signed history, exact KML, index and PNG from the replica. v1 and v2 histories and a v2 author fork are copied. The built track page then reads both original signatures, selects a historical version, verifies its required package and saves an editable copy. After disconnect and reload, the test independently checks the persistent draft and original author proof. The production history reader also verifies the selected archive's full signature path (3→2→1) and the built page opens an exact historical version. The selected fork branch's path is checked separately; this does not claim the other branch or globally unique currentness. History reads use metadata operation 2 only and stop at a 64-version budget with an explicit incomplete state when needed. No USB, RF or public publication is involved. Windows testing owns the actual interpreter PID instead of a venv redirector so stopping a source really removes its listener.

This local test does not verify public deployment, Linux container permissions, maximum file/package throughput, many-peer fairness, power-loss behavior, or global fork resolution. Retention remains bounded by disk/file/ version admission; a directory can refuse additional copies. Data should not be considered globally replicated merely because one node issued a publish receipt.

Valid requests encountering SQLite or file I/O failures return an authenticated 503 storage unavailable with the original request ID, without a success receipt or private diagnostic text. Background replication waits 30 seconds after a storage exception; cleanup uses an independent 60-second guard. Durable requests, file offsets and checkpoints remain the recovery authority. The storage tests use actual SQLite readonly/locking/full/failed-commit conditions, controlled transport and clocks, plus an injected fsync error; they do not prove physical power-loss recovery. Where database registration cannot be established, uncertain temporary files are retained rather than deleted speculatively.

The storage-network test runs a real local directory, gateway and browser against an isolated SQLite connection set to readonly, then restores writes through a test-only marker. It verifies authenticated 503 classification, capabilities while writes fail, no empty upload accumulation, fixed-ID upload recovery and exact seal/read without changing directory identity or epoch. The fault seam is not present in the production service. Browser publication UI tests separately verify retained unconfirmed tasks and the same signed content/request ID across storage failure, reload, lost receipt and successful confirmation in all eleven locales; their USB/network boundaries are controlled.