HopReach RF coverage rollout and recovery

Status: implemented and release-gated for production rollout.

Evidence date: 2026-08-02

Canonical model: the public UK Mesh HopReach tag v0.1.32-ukmesh.3 at commit 0230702be70a2729c5acc5640401f56ab9d65fd4, based directly on upstream v0.1.32 commit 61efac0b4678f55496fe08f53eda0c79eb18655b. The tagged tree is vendored at third_party/hopreach.

Release invariants

Publish the exact integration release revision before exposing any HopReach-derived image. rf-coverage/SOURCE-OFFER.md, the public fork/tag, the vendored license, and both recorded revisions must be reachable without operator access.
Use digest-pinned backend, HopReach, and app images built by the signed release workflow. Confirm the backend/app revision label is the integration release commit and the HopReach revision label is 0230702be70a2729c5acc5640401f56ab9d65fd4.
Do not enable calibrated variants. The production profile deliberately has calibration.enabled: false; evidence validation is a separate rollout.
Do not reduce range, node count, terrain zoom, supersampling, or RF fidelity to meet a runtime target. scripts/benchmark-hopreach.sh is the release gate and rf-coverage/BENCHMARKS.md records the accepted local measurement.
Keep node_coverage, old RF images, and the prior signed release for one release as rollback material. The live app and calculator must never read them.

Runtime shape

The backend mounts a private /hopreach compatibility router. It supplies only UK-visible positioned repeaters and genuinely observed node_links; it does not read predicted node_coverage. Nodes are paginated at 500, bulk calibration is bounded at 5,000 public keys and 250,000 returned rows, and a per-node fallback remains available. Short bounded caches and in-flight coalescing protect cold-start pagination and concurrent evidence loads. The router rejects public or forwarded traffic and is not exposed by Nginx.

HopReach uses the immutable uk-operational-v1.geojson boundary, containing Great Britain, Northern Ireland, the Isle of Man, Jersey, and Guernsey. It publishes into the dedicated rf_coverage_data volume:

/data/output/meta.json
/data/output/progress.json
/data/output/tiles/standard/{row}-{column}.png
/data/output/tiles/precision/{row}-{column}.png

Standard is computed first at 2,000 pixels and terrain zoom 11. Precision may start only after Standard is live, the free-disk gate passes, and then uses 6,000 pixels, terrain zoom 13, and 2x supersampling. Each 1,024-pixel publication tile computes against every transmitter within the unchanged link-budget maximum range. Tiles and checkpoints are atomically replaced.

The calculator owns a singleton lock, nightly schedule, persistent DEM cache, input signature, and resumable checkpoint. During recomputation the app serves the previous last-known-good set plus newly completed current tiles. A failed or restarted run therefore cannot blank the layer.

Preflight

From a clean reviewed revision:

docker compose config --quiet
docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp \
  -v "$PWD:/work" -w /work/third_party/hopreach \
  golang:1.25.7-bookworm@sha256:564e366a28ad1d70f460a2b97d1d299a562f08707eb0ecb24b659e5bd6c108e1 go test ./...
docker run --rm --user "$(id -u):$(id -g)" -e HOME=/tmp \
  -v "$PWD:/work" -w /work \
  golang:1.25.7-bookworm@sha256:564e366a28ad1d70f460a2b97d1d299a562f08707eb0ecb24b659e5bd6c108e1 \
  /work/scripts/benchmark-hopreach.sh

Also require backend tests/typecheck/build, frontend unit tests/build, and the desktop/mobile Playwright RF journey. The compatibility load test is the test named handles UK pagination and coalesces concurrent calibration loads; it uses 4,600 rows, ten pages, 32 concurrent requests, a cold pass, and verified cache hits.

Before production, inspect capacity without deleting anything:

docker volume inspect meshcore-analytics_rf_coverage_data
docker system df
docker compose config --images

The default calculator limit is four CPUs, GOMAXPROCS=4, and 8 GiB memory. Precision additionally requires the configured 8 GiB free inside /data.

Deployment order

Verify the release revision and corresponding source are public, then verify signatures, attestations, image digests, and revision labels using docs/runbook-release-rollback.md.
Deploy migrations if the release has any, then deploy backend. From the calculator network namespace, confirm GET /hopreach/healthz, node pagination, and one bounded bulk-link request. Confirm the same paths are unavailable through the public app origin.
Deploy hopreach. Watch its logs and /data/output/progress.json; do not wait for the whole UK before proceeding. Confirm a valid meta.json and at least one atomically readable Standard PNG exist.
Deploy app-ukmesh built with VITE_RF_COVERAGE_ENABLED=true. The Nginx contract exposes only metadata, progress, and numeric Standard/Precision PNG paths from the read-only shared volume.
Allow Precision to start only after metadata reports Standard live and the resource gate passes. A denied Precision tier is visible as a tier failure; it must not remove Standard.

Do not start a viewshed-worker; that service no longer exists in Compose. The separate link-worker remains required because it writes observed node_links used as calibration evidence.

Live verification

On desktop and mobile, verify:

the first Standard tile appears progressively and new sessions enable RF coverage automatically;
a manual visibility choice persists, Standard/Precision selection appears only when available, and the orange-to-green legend/model details are shown;
stage, backend, percent, ETA, and any last-known-good failure state update;
coverage remains nearest-neighbour and below roads, place labels, nodes, and interactive layers while pan/zoom remains responsive;
browser network logs contain only /rf-coverage/meta.json, /rf-coverage/progress.json, and valid tile paths, with no request to /api/coverage or /api/coverage/planned;
docker compose ps contains hopreach and link-worker, but no viewshed coverage worker.

After one Standard run, restart only hopreach. The same incomplete run ID must resume, completed tiles must remain readable, and the next checkpoint must advance without recomputing completed tiles. Repeat once during Precision.

Failure and rollback

If HopReach fails, leave the app and shared volume in place: metadata exposes the failure and the UI keeps last-known-good tiles. Preserve logs, meta.json, progress.json, and the checkpoint before restarting the single calculator. Do not delete the DEM cache or output volume as a recovery step.

If the native UI is defective, roll back only app-ukmesh to its previous verified digest; HopReach can continue safely in the background. If the calculator or compatibility contract is defective, stop hopreach, roll back backend and app together, and retain its volume for diagnosis. Returning to the old viewshed product requires rolling back the complete prior signed release; never mix its worker/API/UI with the new release.

At the end of the one-release rollback window, removal of node_coverage, old images, legacy queue code, or caches needs a separate reviewed change with a fresh backup and exact target inventory.

7.4 KiB Raw Blame History