ukmesh/plan.md

MeshCore Analytics: codebase audit and implementation plan

Status: implemented; final runtime evidence is bound by the signed deployment receipt described below

Audit date: 2026-07-29

Audited revision: b9b2ecdbfaa9688a79a62ccee38d6b406f14deaa

Repository: https://github.com/gadgethd/ukmesh.git

Scope: all 366 Git-tracked files; no project source was changed during the audit

Implementation outcome

The original read-only audit and its acceptance criteria remain below as the release contract. The implementation completed every phase without enabling irreversible packet retention/deletion: those policies intentionally remain disabled behind the restore, inventory, dry-run, and explicit operator-approval gates required by Sections 4, 4.4, and 7.3.

Phase	Implemented evidence
0	Clean CI installs, pinned builds, worker image tests, deterministic Mosquitto bootstrap, migration checksum/lock tests, internal metrics, performance budgets, and signed rollback/restore foundations.
1	docs/security-closure.md maps every finding to the shared visibility, network, webhook, proxy, telemetry, queue, container, and coordinate controls plus their regression tests.
2	One registered map route, complete cursor delivery, lease-fenced analysis runs, coordinated shutdown, shared bounded input/error handling, and CSRF-protected secure operator sessions.
3	Explicit terrain outcomes, bounded versioned Redis state machines, database publication fences, recoverable ML checkpoints/leadership, and RF golden/latency/RSS tests in the exact worker images.
4	Immutable additive migrations, coalesced packet writes, resumable stats rollups with shadow comparison, privacy-versioned durable chart snapshots, gated lifecycle tooling, cache policy enforcement, and accurate public/operator health DTOs.
5	Privacy-safe inferred nodes and packet arcs, authoritative epoch-scoped stores, corrected Feed connectivity/path state, and repaired repeater/detail/Stats behavior.
6	Bounded visibility-aware polling/caches, terrain and service-worker lifecycle controls, throttled topology rendering, accessible UI primitives, route-wide axe/keyboard/mobile tests, component extraction, and enforced bundle/CSS budgets.
7	Low-cardinality metrics, Alloy/Prometheus/Alertmanager validation, encrypted isolated restore drills, signed immutable release/rollback controls, digest/hash dependency pins, SBOM/scan/signing workflow, and fail-closed single-chunk maintenance.
8	Owner alert delivery history/test actions, observer review states, audited queue/model/planned-publication operations, actionable scoped watchlists, wildcard 404/shared route metadata, runtime feature kill switches, complete generated OpenAPI coverage, and corrected architecture/runbooks.

The full pre-deployment matrix is rerun from clean lockfile installs. At the implemented source state it comprises 231 backend unit tests plus isolated PostgreSQL/Redis integration fixtures, 35 frontend unit tests, 45 Playwright desktop/mobile/accessibility tests with retries disabled, 21 RF/viewshed worker tests, four ML worker tests, operational recovery drills, configuration validators, and dependency policy checks.

The authoritative production result is the newest verified /home/ben/meshcore-releases/local-deploy-*.json receipt and detached signature. It binds the complete worktree fingerprint to exact image IDs, schema/migration state, the signed backup and rollback evidence, live privacy and proxy probes, monitoring state, performance measurements, and the production soak. A failed required check means this status is not satisfied and requires a corrected rebuild or the recorded compatible rollback.

1. Executive summary

MeshCore Analytics is a single-host, containerized analytics platform for MeshCore radio networks. Authenticated MQTT observers forward radio envelopes; the Node/TypeScript backend validates, decodes, deduplicates, classifies, privacy-filters, persists, and broadcasts them. PostgreSQL/TimescaleDB is the authoritative store, Redis carries queues and realtime fanout, Python workers perform terrain/RF/link calculations, and a React/MapLibre application provides the live dashboard, public UK pages, health/status views, and an authenticated owner portal. Nginx, Anubis, and Cloudflare form the intended public edge; Prometheus, Grafana, Loki, and the currently deployed Promtail collector form the operations stack; Phase 7 replaces the now-EOL collector.

The project has a strong functional base: the backend type-checks and all 164 backend tests pass; the frontend type-checks, has a build/E2E pipeline, and its two unit tests pass; configuration is generally coherent; privacy and queue abstractions already exist in several paths. The principal risk is not a lack of features but inconsistent application of those abstractions. Similar routes, queues, caches, proxy locations, and worker states often implement subtly different rules. That has produced several user-visible defects, seven validated security findings, recoverability gaps, and avoidable database/Redis/browser work.

No P0/critical issue was established in this static audit. The recommended sequence is:

1. Make CI and regression fixtures capable of proving the intended contracts.
2. Close security/privacy and host-boundary gaps.
3. Repair correctness, lifecycle, queue, and worker recovery.
4. Reduce database, RF, Redis, network, and browser work.
5. Complete broken product features and accessibility.
6. Add observability, backup/restore, immutable release controls, and accurate documentation.

The plan deliberately uses additive migrations, dual-read/dual-protocol transitions, feature flags for risky UI/compute changes, and rollback to known image versions. It does not rely on destructive down-migrations.

2. Intended system and data flow

MeshCore radios
    |
    v
authenticated observers -> Mosquitto MQTT
                              |
                              v
                    backend MQTT ingestion
              validate -> decode -> dedupe -> privacy/spam
                    |                    |
                    v                    v
          TimescaleDB/PostgreSQL     Redis queues/fanout
                    |                    |
                    |             Node + Python workers
                    |          path/RF/viewshed/link/health
                    |                    |
                    +---------+----------+
                              |
                      REST + WebSocket API
                              |
                Cloudflare -> Anubis -> Nginx
                              |
               React dashboard and public UK sites

Important intended properties inferred from code and documentation:

• Network scope is currently ambiguous. Runtime and the dedicated test hostname support anonymous ukmesh and test network selection, while the security invariant supplied for this audit says public REST/WS should exclude test/development identities. The UK Feed also has a separate content-channel value named test; it is not the same dimension as network scope. An ADR must resolve both dimensions before scoping code changes. all is not an anonymous public scope under either interpretation.
• Names containing the private marker are an opt-out boundary. It must apply to direct rows, derived links/paths, caches, exports, WebSocket events, and historical views.
• Owner access is based on current grants from the separate owner-auth store, not merely on possession of a stale cookie.
• Raw packet history is currently retained indefinitely. This is an operating choice, not a safe default for an indefinitely growing public service.
• Planned coverage is an optional, resource-limited capability; ordinary link processing is expected to work in a fresh default deployment.
• Public health is meant to report service state, while operator detail and raw diagnostics belong behind a local/operator boundary.

3. Audit method and verified baseline

The audit partitioned the 366 tracked files into 194 backend files, 102 frontend files, and 70 deployment/worker/documentation files. Every tracked path was inspected. Large lockfiles, generated dashboard JSON, geospatial coordinates, and the two font binaries received targeted or machine-assisted semantic review. Ignored dependencies, build output, runtime secrets, terrain tiles, live databases, Redis state, Cloudflare configuration, and deployment hosts were outside the repository inventory.

Check	Result at audited revision
Backend npm test	164/164 passed
Backend npm run typecheck	Passed
Backend production dependency audit	0 vulnerabilities
Frontend npm test	2/2 passed; coverage is far too small
Frontend npx tsc --noEmit	Passed
Frontend dependency audit	3 high advisory entries; PostCSS is reachable only at build time and the React Router advisory is RSC-only, but CI still fails and dependencies should be upgraded
Python parsing	All 8 tracked Python files parsed
Shell syntax	All tracked shell scripts passed bash -n
JSON parsing	Passed
Docker Compose render	Passed with a complete placeholder environment
Security scan	Complete inventory; 7 reportable findings: 3 medium, 4 low

The security report and sealed machine-readable evidence are under:

/tmp/codex-security-scans/meshcore-analytics/b9b2ecdbfaa9688a79a62ccee38d6b406f14deaa_20260729T110834Z/

The report is supplemental; this plan is self-contained because temporary audit artifacts may not be retained.

4. Priorities and sequencing rules

Priority	Meaning	Release expectation
P0	Active critical compromise, unrecoverable loss, or universal outage	Stop release and remediate immediately
P1	Validated security boundary, data correctness, permanent wedge/loss, broken primary feature, or broken deployment gate	Address before broad feature work
P2	Material performance, resilience, accessibility, observability, or maintainability risk	Schedule in the next engineering cycle
P3	Product polish, documentation drift, or lower-risk cleanup	Bundle after foundations are stable

Rules for the implementing agent:

• Write a failing regression test before each defect fix.
• Keep public visibility and network scoping in shared query/DTO helpers, not in each caller.
• Give every queue and long-running job a capacity, lease, retry, deadline, terminal state, cleanup operation, and metrics.
• Give every cache a scope key, TTL, entry/byte bound, invalidation rule, and observable eviction count.
• Use expand/migrate/contract database changes. Never renumber an applied migration; the duplicate 016 names need a recorded compatibility mapping.
• Deploy consumers/readers before producers/writers when changing protocols.
• Do not enable packet deletion until a restore drill and feature-window audit prove that rollups preserve advertised behavior.
• Treat privacy regression, data loss, and cross-scope state as automatic rollback conditions.

5. Decisions required before coding

Record these in a short ADR or docs/architecture.md before the dependent phase starts.

Decision	Recommended starting point	Affected work
Network test and Feed channel test visibility	Resolve these as two independent dimensions. Recommended safe contract: production host defaults/locks to ukmesh; the dedicated test host may expose network test if approved; neither host permits anonymous all. Decide separately whether the Feed content-channel test is public.	SEC-16, SEC-09, BE-12, FE-04
Raw packet retention	Choose a finite hot window after measuring product history requirements; compress cold data first, then retain aggregates longer than raw rows.	BE-13, OPS-03
Owner webhook destinations	Prefer a dedicated egress proxy plus public-address validation; decide whether arbitrary public hosts are allowed or an allowlist is acceptable.	SEC-01, BE-17
Operator authentication and transport	Short-lived HttpOnly Secure, SameSite=Strict session after local token login, served through local HTTPS or an SSH tunnel to HTTPS/localhost; retain bearer auth for automation for one release. Never weaken Secure merely to support private-IP HTTP.	BE-03
Mosquitto ACL reload authority	Dedicated internal reconciler/helper with atomic ACL file and broker-local reload; no Docker socket in the public backend.	SEC-02
Telemetry trust	Anonymous browser telemetry is always untrusted diagnostics, even with a build token or nonce. Authoritative health comes only from backend/synthetic signals; sampling, deduplication, and quotas protect diagnostic storage.	SEC-06, BE-14
Planned/live RF service level	Benchmark first; then set job deadline, queue count/byte cap, retry budget, and acceptable path-loss/geometry tolerance.	SEC-03, WK-09, WK-10
Backup objectives	Initial proposal: daily encrypted off-host backup with RPO 24h/RTO 4h, followed by WAL/PITR targeting RPO 15m if justified.	OPS-03
Public planned-node data	Prefer owner/operator-only records plus a separate explicitly publishable DTO. Decide whether public submissions are a future feature.	BE-15
Anonymous coordinate policy	Recommended: exact private-node, private-observer, and private-path coordinates must not enter an anonymous browser response or WebSocket event. Server DTOs redact first; client masking is defense in depth only.	SEC-13, FE-02, FE-03
/nodes/map delivery contract	Choose a complete, bounded contract before deleting the duplicate route: viewport/bbox tiles or deterministic cursor iteration with explicit row/byte budgets and frontend completion semantics. Silent truncation is not acceptable.	BE-01, BE-12
Embedded channel key	Confirm it is the documented public MeshCore channel material. If it is secret, rotate and perform history cleanup; do not merely move it to an environment variable.	SEC-12

6. Phase 0 — establish regression and release guardrails

Goal: make subsequent changes measurable and prevent known broken states from shipping again. This phase may run in parallel with the small edge proxy fix in Phase 1.

0.1 Repair dependency and test gates

Files:

• .github/workflows/ci.yml
• .github/workflows/frontend-mobile.yml
• frontend/package.json, frontend/package-lock.json
• backend/package.json
• new viewshed-worker/tests/**
• new ml-path-learner/tests/**

Implementation:

1. Upgrade PostCSS and React Router/React Router DOM to advisory-fixed compatible versions, regenerate the lockfile with npm install, run the complete frontend suite, and record why the old advisories were not runtime-exploitable in this client-only Vite build. Do not add a permanent audit exception when a compatible patch exists.
2. Add npm test to the frontend CI job before build/E2E.
3. Run viewshed/link/RF imports and pytest inside the built worker image so the test environment uses the exact matching system GDAL/osgeo stack. Test the ML worker in its image or a locked equivalent environment. Cover worker.py, link_queue_v3.py, backfill_profiles.py, RF modules, and the ML worker. Land known-red regression cases with their fixing slice, or mark them explicitly non-blocking and linked to their issue until that slice; Phase 0 itself must remain green.
4. Build every Dockerfile and relevant Compose profile in CI, then start a unique, empty-volume Compose project and wait for health/readiness.
5. Keep secret scanning and add generated SBOM/image scanning later in Phase 7; do not run non-reproducible network audits inside production image builds.

0.2 Make clean bootstrap deterministic

Files:

• .github/workflows/ci.yml
• .env.example
• README.md
• mosquitto/mosquitto.conf
• new scripts/bootstrap-mosquitto.sh
• docs/operations.md, docs/contributing.md

Implementation:

1. Supply every required CI variable, including ANUBIS_ED25519_PRIVATE_KEY_HEX in nightly load.
2. Create an idempotent bootstrap script that generates a backend Mosquitto password and least-privilege ACL before broker startup. Use restrictive permissions, back up existing files, and refuse destructive overwrite.
3. Update Quick Start so required secrets and bootstrap order match Compose.
4. Add a clean-checkout smoke job that proves no developer .env, ignored credential file, or existing Docker volume is required.

0.3 Capture contracts and performance baselines

Add an audit fixture or ADR containing:

• API/WS/export/cache visibility matrix that independently varies network (ukmesh, test, all) and Feed content channel (public, test), plus host, observer scope, owner scope, and private identities.
• Current response schemas for /nodes/map, packet detail, link history, planned nodes, health, and owner endpoints.
• Approved /nodes/map completeness strategy and numeric row/byte/request budgets, including the frontend behavior needed to finish a viewport or cursor sequence without showing a silently incomplete map.
• EXPLAIN (ANALYZE, BUFFERS) plus p50/p95 and row counts for map, summary, charts, coverage, topology, link history, and path resolution on a sanitized representative dataset.
• Ingest throughput, packet-batch latency, Redis queue depth/bytes, viewshed job duration/RSS, frontend bundle size, map initialization count, browser long tasks, and cache memory.
• Numeric Phase 6 release budgets: browser/terrain memory, service-worker storage, request/path concurrency, polling interval/deadline, topology node count/frame time, bundle/chunk size, and CSS duplication target.

Tests and acceptance:

• A clean checkout builds every image and reaches backend, broker, link worker, frontends, and monitoring health.
• Backend tests/typecheck/build, frontend tests/typecheck/build/Playwright, all Python tests/imports, Compose render, shell checks, and dependency policy pass in CI.
• The harness can express duplicate-route, private-scope, queue-invariant, transient-RF, and cross-network-store regressions. Known-red cases become blocking only in the slice that ships their fix.
• Baselines are versioned and later phases declare a maximum tolerated regression (default: no more than 10% without explicit review).

0.4 Land minimum production safety foundations

This subsection must complete before any new production migration or queue protocol ships.

Files:

• backend/src/db/migrations.ts
• migration ledger schema and tests
• minimal backup/restore tooling and docs/operations.md
• backend/src/metrics.ts, internal metrics route, and core queue/lifecycle instrumentation
• release metadata workflow

Implementation:

1. Add a PostgreSQL advisory migration lock, immutable filename/checksum ledger, duplicate-prefix detection, and an explicit audited path for non-transactional operations such as CREATE INDEX CONCURRENTLY.
2. Pin the TimescaleDB image and extension version, take a current production snapshot, and demonstrate a minimal restore path before the first new migration. Full scheduling, encryption, PITR, and restore automation remain in Phase 7.
3. Record immutable image digests for the current and last known-good release. Signing/provenance can mature in Phase 7; early rollback instructions must say last known-good digest, not assume signatures already exist.
4. Serve the metrics registry on an internal-only endpoint and add the minimum metrics needed by Phases 1–3: queue count/bytes/age/rejections, webhook outcomes, lifecycle drain, worker heartbeat, lease loss, and job outcomes. Essential saturation/error alerts must ship with the queue protocol; richer exporters and dashboards remain in Phase 7.

Acceptance:

• Concurrent migration runners serialize and checksum drift fails closed.
• A pinned-version snapshot restores into an empty environment far enough to run migrations and application readiness.
• The prior compatible image digest is recorded and retrievable.
• Internal metrics scrape works, public access fails, and core labels pass a low-cardinality test.

Rollout/rollback:

• This phase changes CI, tests, bootstrap, and additive safety foundations only. Bootstrap must never replace existing credentials without explicit operator approval.
• Dependency upgrades are one isolated PR with a lockfile-only rollback path.

7. Phase 1 — contain security and privacy boundaries

Goal: close all seven validated security findings and the adjacent defense-in-depth gaps before enabling more public or owner features.

1.1 Owner webhook SSRF (SEC-01, P1)

Files:

• backend/src/api/routes/owner.ts
• backend/src/owner/alertRules.ts
• new backend/src/security/outboundWebhook.ts
• backend/src/workers/health.ts
• a uniquely numbered additive owner-alert-delivery migration allocated after the Phase 0 migration ledger is active
• new unit/integration tests beside those modules

Implementation:

1. Parse and length-bound the URL; reject credentials, fragments, non-HTTPS schemes, disallowed ports, literal loopback/private/link-local/multicast/ reserved addresses, and ambiguous IPv4/IPv6 encodings.
2. Resolve A and AAAA records immediately before delivery and reject if any result is non-public. Disable redirects or validate every hop with the same policy. Prevent DNS rebinding by connecting through the validated address while preserving TLS hostname verification, or use the approved egress proxy.
3. Revalidate at send time, not only at rule creation. Bound concurrent deliveries, keep the existing deadline, cancel/drain bodies, and cap response bytes.
4. Persist a bounded delivery/outbox record with rule ID, attempt, timestamps, sanitized destination host, outcome, and next retry. Add exponential backoff, idempotency, and a test-delivery operation.

Acceptance:

• Table tests reject every non-public range, IPv4-mapped IPv6, userinfo, alternate encoding, disallowed port, private DNS answer, public-to-private redirect, and DNS-rebinding fixture without issuing the internal request.
• A permitted public webhook still receives the documented payload. If webhook authenticity is a product requirement, define a separate HMAC header, secret storage/rotation, canonical serialization, and replay-window contract before calling the payload signed.
• Delivery concurrency, retry count, age, and history retention remain bounded.
• Egress policy independently denies internal networks.

Rollout/rollback:

• Ship schema and disabled delivery history first, then the new sender.
• If delivery breaks, disable webhook delivery and retain queued events; never roll back to unrestricted fetching.

1.2 Remove backend Docker host authority (SEC-02, P1)

Files:

• docker-compose.yml
• Dockerfile.backend
• backend/src/owner/ownerAclReconciler.ts
• relevant MQTT ACL manager/tests
• mosquitto/mosquitto.conf
• new broker-local reload helper under docker/

Implementation:

1. Write ACL output atomically to a dedicated shared volume, validate it, retain a last-known-good copy, and request reload through a narrow authenticated broker-local helper.
2. Land a dual reload mode, verify production readback, then remove /var/run/docker.sock from the backend.
3. Run the backend with a numeric non-root user, no-new-privileges, dropped capabilities, a read-only root filesystem, and explicit writable mounts.
4. Add policy-as-code that rejects Docker socket mounts on request-serving services.

Acceptance:

• Owner grant/revocation updates the ACL and reloads Mosquitto under concurrent changes; invalid ACL output leaves the live ACL untouched.
• Backend cannot list containers, create workloads, mount host paths, or signal unrelated processes.
• Backend health and packet ingest pass under non-root/read-only restrictions.

Rollout/rollback:

• Deploy helper and dual mode before removing socket access. Keep the old mode for one release only, guarded by an operator-only flag.
• Roll back to the last known-good compatible backend image digest if ACL reload health fails; do not expose a general host control plane to restore functionality indefinitely.

1.3 Bound live viewshed admission (SEC-03, P1)

Files:

• backend/src/queue/publisher.ts
• backend/src/features.ts
• backend/src/index.ts
• viewshed-worker/worker.py
• shared queue tests in both languages
• backend/src/health/status.ts

Implementation:

1. Make the existing SADD plus LPUSH admission atomic and enforce a tactical global job-count and payload-byte cap plus worker-heartbeat check while preserving the current consumer payload. Rejecting admission must remove any pending marker created by the same operation.
2. Coalesce repeated coordinate updates to the latest state for one node.
3. Export admission, rejection, count, bytes, oldest age, and pending/list-divergence metrics and add essential saturation alerts.
4. Do not invent a second lease/retry/DLQ protocol here. The single versioned recoverable state machine, dedicated/no-eviction Redis decision, and producer/consumer key migration are implemented once in Phase 3.2.

Acceptance:

• Atomic concurrency/property tests cannot exceed the exact job or byte cap.
• Load above worker drain rate keeps Redis memory and core REST/WS/ingest within the Phase 0 budget.
• Rejected and coalesced jobs leave the existing pending set/list consistent.

Rollout/rollback:

• This is a producer-side compatible admission patch. Deploy with the feature disabled, validate counters against the existing list/set, then enable. Phase 3 performs the only versioned queue-key migration.

1.4 Fix public website WebSocket identity (SEC-04, P1)

Files:

• nginx.website.conf
• nginx.app.conf
• anubis/botPolicy.yaml
• backend/src/http/trustedProxy.ts
• backend/src/ws/server.ts
• frontend/src/hooks/useWebSocket.ts
• end-to-end proxy tests

Implementation:

1. Replace overlapping website WebSocket locations with one canonical exact /ws configuration (or one shared include) that overwrites both X-Real-IP and X-Forwarded-For with $mesh_client_ip.
2. Keep backend trusted peers explicit; caller headers from an untrusted peer must never influence identity.
3. After the identity fix, stage-test whether Anubis can require a solved challenge instead of unconditionally allowing all /ws handshakes.

Acceptance:

• At least 100 distinct test client IPs can connect concurrently.
• One IP exceeding 20 connections or the handshake budget does not affect a second IP.
• /ws and /ws/ cannot select different forwarding contracts.
• Spoofed forwarding headers are overwritten.

Rollout/rollback:

• Deploy the Nginx identity fix independently first. Treat any cross-user quota sharing as rollback-worthy. Change Anubis policy only after browser reconnect tests; its policy can roll back separately.

1.5 Scope public link history (SEC-05, P1)

Files:

• backend/src/api/routes/productFeatures.ts
• backend/src/http/requestScope.ts
• backend/src/api/utils/networkFilters.ts
• new repository/query helper and route tests
• frontend/src/components/Map/LinkQualitySparkline.tsx

Implementation:

1. Resolve the explicit public network from host/header/query under the agreed contract.
2. Join both endpoint nodes; require same requested scope and public visibility. Apply the private marker before reading derived radio/link rows.
3. Return indistinguishable 404 responses for hidden, cross-scope, missing, and mixed pairs. Include scope in frontend request/cache keys.
4. Prefer a database view that defines a public visible link once for history, map, topology, export, cache warmup, and WebSocket use.

Acceptance:

• Fixtures cover visible, private, cross-network, missing, reversed, and mixed pairs under every host/network exposure approved by the ADR.
• A:B and B:A enforce identical authorization and cache behavior.
• No link-derived response reveals the existence or activity of a private node.

1.6 Make health telemetry trustworthy (SEC-06, P1)

Files:

• backend/src/api/routes/telemetry.ts
• backend/src/health/status.ts
• backend/src/api/bootstrap/limiters.ts
• frontend error reporter/bootstrap HTML
• migration for bounded aggregation/retention if needed

Implementation:

1. Separate untrusted diagnostic volume from authoritative health.
2. Remove frontend-error counts from authoritative warning/critical health. Backend checks, worker heartbeats, synthetic probes, and other server-owned evidence are the authoritative sources.
3. Apply a dedicated lower rate/storage quota, sampling, normalized fingerprint deduplication, and per-source contribution cap to diagnostic reports. A build token or same-origin nonce may reduce casual noise but must not be treated as user identity or authenticity.
4. Retain raw details briefly and aggregates longer. Show diagnostic trends only in the operator view with an explicit untrusted label; public status must not derive severity from them.

Acceptance:

• One anonymous or repeated source cannot set warning/critical health.
• Replayed and duplicate reports remain within storage/rate bounds.
• Backend/synthetic failures still trigger deterministic health alerts without relying on browser telemetry.

1.7 Pin external health-check build input (SEC-07, P1)

Files:

• Dockerfile.mesh-health-check
• docker-compose.yml
• .env.example
• CI release workflow and operations docs

Implementation:

1. Require a full reviewed upstream commit and verify fetched HEAD exactly matches it; use npm ci.
2. Prefer building the external project in its own trusted pipeline and referencing an immutable image digest. Phase 7 adds signing, SBOM, and provenance; the immediate finding closes when build input is bound to a reviewed immutable commit and output digest.
3. Fail Compose/release validation when the pin is absent, mutable, or unapproved.

Acceptance:

• Two builds from one pin consume the same declared source and lockfile.
• A branch/tag or mismatched commit fails before executable content is copied.
• Release inventory records repository revision, source pin, and resulting image digest; Phase 7 extends this record with SBOM, scan, signature, and provenance.

1.8 Adjacent hardening, retained as non-vulnerabilities

These did not survive attack-path calibration but should be fixed with the nearest work:

• backend/src/api/utils/localOnly.ts: accept forwarding metadata only from a trusted socket peer; require operator auth for test diagnostics. Add a direct remote peer plus private spoofed-header 403 test.
• backend/src/stats/statsRepository.ts: scope prefix candidates by network for correctness under whichever network ADR is approved.
• scripts/generate-observer-key.ts: create parent directory 0700, file 0600, refuse unsafe symlinks/existing paths, and avoid printing private material by default. Add scripts/package-lock.json.
• README.md: make all public HTTP Cloudflare routes target Anubis sidecars; direct MQTT routing remains intentional.
• Immediately fail closed or narrow anonymous /planned-nodes: remove owner_pubkey and free-form notes, add an explicit publish/visibility predicate and pagination, and return no records until the public DTO decision is approved. Phase 8 can productize a reviewed public view.
• Planned coverage: retain existing admission controls but add a per-job execution budget and load test in Phase 3.

1.9 Enforce one anonymous coordinate boundary (SEC-13, P1 hardening)

Files and sinks:

• server DTOs/queries that feed packet detail, Stats decoded paths, topology, coverage/LOS, observers, planned nodes, and map layers
• frontend/src/components/Map/DeckGLOverlay.tsx
• frontend/src/pages/ukmesh/PacketDetailPanel.tsx
• frontend/src/pages/StatsPage.tsx
• frontend/src/pages/TopologyPage.tsx
• frontend/src/components/Map/MapLibreMap.tsx
• shared client privacy/masking utility and E2E fixtures

Implementation:

1. Implement the decision-table recommendation: exact private-node, private-observer, private-relay/path, raw-advertisement, LOS, topology, and planned-node coordinates do not enter anonymous REST/WS payloads.
2. Create privacy-safe server DTOs/views and apply them before caching, exporting, fanout, and response serialization. Add a defense-in-depth client mask before any GeoJSON, Deck.gl picking object, raw packet view, table, or tooltip.
3. Inventory and test every secondary map and textual coordinate sink, not only the primary map. Keep heatmap disabled until all fixtures pass.

Acceptance:

• Public browser/network traces contain no prohibited exact coordinate or recoverable raw advert for every listed sink.
• Server tests fail before a prohibited coordinate reaches cache or fanout; client tests prove a malformed/legacy payload is still masked.
• Owner/operator views retain authorized exact data through separate DTOs and cannot be cached under anonymous keys.

Phase 1 completion gate:

• All seven security regression suites pass.
• Public privacy matrix passes for REST, WS, exports, derived paths/links, caches, both network-scope outcomes in the ADR, and the independent Feed channel dimension.
• No Internet-facing service mounts the Docker socket or runs with unnecessary host authority.
• Security report findings have been revalidated against the implementation and closed with evidence.

8. Phase 2 — backend correctness, lifecycle, and operator usability

2.1 Remove duplicate route and unify map/stat semantics

Files:

• backend/src/api/routes/nodes.ts
• backend/src/stats/statsRepository.ts
• backend/src/api/routes.ts
• new route-registration and repository integration tests

Implementation:

1. Delete the first GET /nodes/map handler at the audited nodes.ts:84-105; retain one limited implementation.
2. Define one public map predicate/DTO for roles, coordinate validity, freshness, visibility, network, observer, and allowed fields. Reuse the same predicate for map counts and relevant statistics.
3. Implement the Phase 0 ADR's complete bounded delivery contract. If it is bbox/viewport based, update the frontend for pan/zoom loading and overlap deduplication; if cursor based, fetch through completion with a snapshot token. Never silently truncate the initial map.
4. Add a route registry assertion that fails for duplicate method/path pairs and large-result tests for the approved row/byte/request budgets.

Acceptance:

• Exactly one handler exists and it is protected by nodesLimiter.
• The client can prove when its current viewport/snapshot is complete and presents explicit loading/error state for an incomplete sequence.
• Map rows and map summary counts agree on the same fixture.
• Private, stale, invalid-coordinate, wrong-role, and wrong-scope nodes are consistently included or excluded according to the ADR.

2.2 Lease and atomically complete analysis runs

Files:

• a uniquely numbered additive analysis_leases migration allocated after the Phase 0 migration ledger is active
• backend/src/analysis/runState.ts
• backend/src/analysis/boundedRun.ts
• backend/src/workers/path-history.ts
• backend/src/path-learning/rebuild.ts
• backend/src/spam/analyzer.ts

Implementation:

1. Add lease owner/token, expiry, heartbeat, attempt, and terminal reason.
2. Claim, heartbeat, publish, and finish with compare-and-swap on the lease token. Make completion and active-run clearing one transaction.
3. Reclaim expired runs as failed/retryable; impose a work deadline and pass an abort signal through database, resolver, and worker-thread operations.
4. Compare privacy/model generation before publishing canonical results.
5. After draining legacy workers, run an audited one-shot transaction that marks orphaned legacy active_run_id rows failed and clears them before lease enforcement. Report every affected workload and require operator approval.

Acceptance:

• Killing a worker mid-run permits exactly one reclaim after TTL.
• Two workers cannot publish the same scope/generation.
• A failure at every transaction boundary cannot leave a permanent active_run_id.
• Work stops after lease loss or deadline and reports the correct terminal state.

Rollout/rollback:

• Deploy additive columns, then lease-aware readers/consumers, drain legacy workers, perform/read back the orphan cleanup, and only then require leases. Retain old columns for one release.

2.3 Graceful process shutdown

Files:

• backend/src/index.ts
• backend/src/mqtt/client.ts
• backend/src/db/packetBatch.ts
• backend/src/ws/server.ts
• backend/src/queue/publisher.ts
• backend/src/path-beta/resolvePool.ts
• database/owner-auth pools and every timer-owning worker

Implementation order:

1. Add a lifecycle coordinator and idempotent close() contract.
2. On SIGTERM/SIGINT, mark readiness draining; stop HTTP/MQTT/job admission and timers; stop MQTT reconnect; close HTTP keep-alive and WS with code 1001.
3. Flush accepted packet batches, finish or release leases, then close worker threads, Redis, and PostgreSQL pools.
4. Enforce a configurable deadline and emit outstanding-resource diagnostics before forced exit. Any unhandled rejection or uncaught exception must itself initiate the same idempotent drain and exit non-zero; continuing in an uncertain state is not allowed.

Acceptance:

• Define application acceptance explicitly. Every packet handed to a persistence batch before admission closes must either be stored or produce a recorded failure before exit; every admitted durable job must be completed or recoverable. Do not equate MQTT broker acknowledgment with database persistence.
• Readiness fails before listeners stop, repeat signals are harmless, and exit occurs within 30 seconds.
• No timer, socket, worker thread, Redis connection, or pool keeps the process alive after successful drain.

2.4 Central error and input handling

Files:

• all backend/src/api/routes/*.ts, starting with owner.ts, misc.ts, nodes.ts, and radio.ts
• new shared async handler/error middleware and schema/parser utilities

Implementation:

• Wrap Express 4 async handlers so rejected promises always produce a structured response and request ID.
• Centralize bounded integer/hour/cursor/hash/coordinate/observer/network parsing; reject negative, NaN, infinite, oversized, and ambiguous values with 400.
• Add upstream connect/total timeouts, body-size caps, allowed content types, and circuit behavior to radio proxying.
• Keep internal error detail in logs, not public payloads.

Acceptance:

• Injected database failures in every owner alert handler return 500 promptly without hanging or unhandled rejection.
• Invalid parameters never reach SQL and table-driven boundary tests cover every shared parser.
• Slow, oversized, redirecting, and invalid-content upstream radio responses fail within the configured budget.

2.5 Browser-usable operator site

Files:

• backend/src/backend-site/routes.ts
• backend/src/backend-site/template.html
• new backend/src/security/operatorSession.ts
• operator site tests and operations docs

Implementation:

• Add local-only token login that exchanges the operator token for a short-lived HttpOnly, Secure, SameSite=Strict session; keep bearer authentication for automation for one release.
• Document and test the supported secure transport: local HTTPS or an SSH tunnel to the HTTPS/localhost endpoint. Ordinary private-IP HTTP is not a supported reason to remove the Secure cookie flag.
• Add CSRF protection, logout, expiry, audit logging, and a strict CSP.
• Cache/aggregate the expensive dashboard payload and replace two-second polling with a visibility-aware slower interval or server events.

Acceptance:

• A normal browser using the documented local HTTPS/SSH path can log in and load all operator API calls without an extension or token in JavaScript/local storage.
• Remote/non-trusted peers cannot initiate a session.
• Expired/logout/CSRF cases fail closed; automated bearer clients remain compatible during transition.

Phase 2 rollback:

• Schema changes remain additive. Keep old header auth and compatible job reads for one release. Small route and lifecycle fixes should be independently revertible; never roll back privacy predicates or restore an unrecoverable run protocol.

9. Phase 3 — make workers and queues recoverable

Goal: ordinary link/RF work succeeds on a fresh deployment, transient failures remain retryable, and no terminal job leaks capacity or commits after losing its lease.

3.1 Centralize terrain acquisition and result classification

Files:

• viewshed-worker/rf/terrain.py
• viewshed-worker/worker.py
• viewshed-worker/rf/loss.py
• viewshed-worker/backfill_profiles.py
• new viewshed-worker/tests/**
• docker-compose.yml

Implementation:

1. Create one ensure_tiles_for_link()/terrain context boundary used by physical links, planned links, diagnostics, ordinary coverage, and backfills. A fresh default link-worker must be able to download its required bounded tiles without relying on the profile-gated viewshed worker.
2. Replace None/empty geometry overloading with explicit outcomes: Computed, RetryableTerrainError, PermanentOutOfScope, InvalidJob, and Cancelled.
3. Never store current-model empty coverage for a timeout, failed download, corrupt tile, GDAL error, clipping failure, database error, or deadline. Persist a separate permanent reason for valid out-of-bounds/unsupported cases.
4. Replace positional compute_path_loss() tuples with a named result object shared by worker and backfill, or retire the broken backfill if it is no longer required.
5. Bound URL, redirect, tile size, decompressed size, time, file path, and cache eviction behavior. Use atomic downloads and validate before promotion.
6. Inventory existing current-model empty coverage rows. Use available provenance/timestamps/job logs to classify legitimate permanent empty results; mark ambiguous/transient rows stale and requeue or delete their calculated marker before new writers are enabled. Report counts and sample every destructive classification.

Acceptance:

• An empty SRTM volume downloads a fixture tile and completes an ordinary link.
• Transient HTTP, timeout, corrupt, GDAL, and database failures retry and leave no “already calculated” marker.
• An out-of-UK job terminates once with a durable permanent reason.
• Backfill fixture stores the expected profile rather than raising on tuple unpacking.
• Planned jobs cannot become ready with a transient empty placeholder.
• Legacy transient/ambiguous empty rows are no longer treated as calculated; legitimate permanent empty outcomes remain explicitly classified.

Rollout/rollback:

• Keep result/status schema additive. Dual-read old null/empty state, but write only explicit new outcomes after rollout.
• A feature flag may select the legacy calculator for output comparison; it must not re-enable transient-empty success.

3.2 Repair link-v3 and ordinary viewshed queue state

Files:

• viewshed-worker/link_queue_v3.py
• backend/src/queue/linkQueueV3.ts
• backend/src/queue/publisher.ts
• viewshed-worker/worker.py
• queue health/status and operator tooling
• property/state-machine tests in TypeScript and Python

Implementation:

1. Define one versioned state model: queued, leased, retry-wait, succeeded, dead, expired, and purged.
2. Make admit, claim, renew, ACK, NACK, requeue, dead-letter, and purge atomic. Active capacity and the DLQ have separate count/byte accounting. Final NACK atomically removes active capacity and moves the payload/dedupe/attempt state into the bounded DLQ; requeue reverses that move under admission checks; purge alone deletes retained payload and dedupe state.
3. Give dead jobs a count/byte cap, retention, and bounded operator requeue/purge command. Define what happens when the DLQ itself is full.
4. Make LeaseRenewer reconnect/back off on Redis errors and communicate definitive lease loss to the main job. Redis notification alone cannot fence a PostgreSQL commit: add a unique job/idempotency key and commit marker, take the scoped PostgreSQL advisory lock, and recheck lease token plus model/privacy generation inside the output transaction before commit.
5. Add an invariant repair command that scans the bounded keyspace, reports discrepancies, and can repair counters under an operator lock.
6. Put durable queue keys on a dedicated Redis instance or configure verified noeviction; never allow generic cache eviction to delete list/hash/lease state. Disposable caches must use separate memory/recovery policy.

Acceptance:

• Randomized state-machine tests always make recorded count/bytes equal actual queued/in-flight active state and separate DLQ state.
• Final NACK restores active capacity while retaining a requeueable DLQ payload; purge deletes the DLQ payload/dedupe and decrements DLQ accounting.
• Worker death produces one recoverable retry; lease loss prevents duplicate committed output through the database transaction fence and unique job key.
• Redis interruption during renewal does not silently continue ownership.
• Depth, bytes, oldest age, lease losses, retries, dead jobs, and repair deltas are observable.

Rollout/rollback:

• Deploy consumers that understand old and new keys first, freeze producers, drain/snapshot old active work, reconcile counters, then switch producers. Retain old keys read-only for the maximum retry/DLQ window. Essential queue alerts must be live before producer cutover.

3.3 Repair ML learner semantics before enabling it

Files:

• ml-path-learner/worker.py
• additive migration for cursor/lease state
• new ml-path-learner/tests/**
• docker-compose.yml

Implementation:

• Replace timestamp-only checkpointing with a stable tuple such as (observed_at, packet_id) and deterministic ordering.
• Use bounded transactions/savepoints. Do not advance the checkpoint beyond failed work, and compute inserted counts only after commit.
• Sample/filter in SQL before loading training data.
• Add a single-leader lease, heartbeat, run deadline, graceful shutdown, model/data version, and publication compare-and-swap.

Acceptance:

• More than 5,000 rows sharing one timestamp are processed exactly once across batches and restarts.
• Injected row failure preserves already committed batches and does not skip failed/later rows.
• Two replicas produce one active run and a crashed leader is reclaimed.
• Memory use is bounded by configured sample/batch size.

Rollout/rollback:

• Keep the service disabled in Compose until all tests and a shadow-data run pass. Additive state may remain unused on rollback.

3.4 Optimize RF computation against a correctness budget

Files:

• viewshed-worker/worker.py
• viewshed-worker/rf/loss.py
• benchmark/golden fixtures under viewshed-worker/tests/
• docker-compose.yml

Implementation:

1. Instrument tile acquisition, VRT construction, radial calculation, polygonization, peer lookup, per-link calculation, DB write, duration, and peak RSS.
2. Replace repeated full-prefix loss calculations along each of 360 rays with incremental/vectorized state.
3. Reuse one tile/VRT context per job and spatially index eligible peers before selecting the bounded nearest/relevant set.
4. Add a configurable hard job deadline, cancellation checks inside loops, and model-version separation for the new algorithm.

Acceptance:

• Golden fixtures stay inside an agreed path-loss and geometry tolerance.
• Representative p50/p95 duration and peak RSS meet the reviewed Phase 0 SLO. A 2x p95 improvement is a stretch objective, not a release gate if the approved SLO is already met.
• Deadline produces retry/dead status, never empty success.
• No job exceeds its container memory/CPU budget and cancellation latency is bounded.

10. Phase 4 — database, ingest, API, and cache efficiency

4.1 Make migrations safe and attributable

Files:

• backend/src/db/migrations.ts
• backend/src/db/migrations/README.md
• historical migrations 016_private_prefixes.sql, 016_stale_mqtt_observer_cleanup.sql, and 018_api_performance_indexes.sql for inventory only; keep their bytes unchanged
• new bounded backfill tools under backend/src/tools/

Implementation:

1. Extend/read back the Phase 0 advisory lock and immutable migration ledger. Inventory every environment and record a compatibility mapping for the two existing 016 filenames; enforce unique numeric prefixes for new files.
2. Never edit, rename, or reformat an applied historical migration after checksums exist. For an environment where 016/018 is still pending, use an approved runbook: either execute the original byte-identical file in a backed-up maintenance window, or explicitly record a reviewed supersession and apply a new schema-only migration plus bounded backfill. Never silently skip it.
3. Keep new schema migrations additive, short, and protected by lock/statement timeouts. Support audited non-transactional operations such as CREATE INDEX CONCURRENTLY without wrapping them in an incompatible transaction.
4. Implement future/replacement historical rewrites as resumable bounded tools with checkpoint, dry-run, batch size, pause, progress, verification, and idempotent restart. The old 016/018 files remain byte-identical.

Acceptance:

• Two migration runners cannot execute concurrently.
• Altered applied migration content fails before execution.
• Every environment has an explicit 016/018 status or supersession record; no checksum is rewritten to hide history.
• A production-sized backfill can stop/restart without long locks, skipped rows, duplicate results, or statement_timeout=0.
• Large indexes use the tested concurrent path and survive interruption/retry.
• CI tests an upgrade from the previous release snapshot.

4.2 Consolidate ingest writes

Files:

• backend/src/mqtt/client.ts
• backend/src/db/packetBatch.ts
• backend/src/db/index.ts
• schema/indexes supporting observer and network rollups

Implementation:

1. Make the packet batch transaction maintain all observer fields now written by the separate MQTT upsertNode; remove the redundant per-packet upsert.
2. Batch/coalesce node network sightings and rollup deltas. Preserve raw packet persistence as the primary accepted-work boundary.
3. If derived updates move asynchronously, add an idempotent outbox/repair job, lag metric, and replay cursor; do not silently drop them.
4. Review hot-row/index contention using real query plans and reduce unnecessary updates when values have not changed.

Acceptance:

• One observer update occurs per batch rather than two writes per packet.
• MQTT-to-DB-to-WS integration fixtures produce identical packet, node-online, observer, privacy, network-sighting, and rollup results.
• Sustained-load throughput improves and DB CPU/WAL per packet falls from the baseline without increasing application-accepted persistence failures.

4.3 Replace full-table statistics with aggregates

Files:

• backend/src/stats/statsRepository.ts
• backend/src/api/bootstrap/caches.ts
• new additive stats aggregate migration
• new resumable aggregate backfill tool
• stats route/repository/load tests

Implementation:

1. Collapse the 17 chart scans into grouped queries and appropriate hourly/daily Timescale continuous aggregates or maintained rollup tables.
2. Create aggregates with no initial data; backfill one bounded time slice per transaction with a checkpoint.
3. Add indexes aligned to final map, privacy, network, observer, time, and link-history predicates. Validate with EXPLAIN (ANALYZE, BUFFERS).
4. Add STATS_AGGREGATE_READS_ENABLED; shadow-query old and new paths, compare totals/percentiles across scopes, then switch.
5. Replace long cold-cache recomputation with snapshot versioning and stale-while-revalidate that has single-flight refresh and explicit failure behavior.

Acceptance:

• Shadow outputs match within documented exact/tolerance rules.
• Stats p95 and DB CPU meet the target set from Phase 0; no endpoint launches an unbounded number of full raw scans.
• Cache refresh cannot stampede and stale data remains clearly timestamped.
• A backfill can resume and does not block ingest.

Rollout/rollback:

• Schema is expand-only and legacy queries remain available for one release. Disable the read flag to roll back; do not delete new aggregates immediately.

4.4 Introduce an explicit data lifecycle

Files:

• additive lifecycle migration after approval
• docs/db-lifecycle.md
• README.md
• backup/restore tooling from Phase 7
• affected stats/path/export queries

Implementation:

1. Inventory every advertised lookback and dependency on raw packets.
2. Enable Timescale compression for cold chunks first and measure CPU/disk.
3. After restore drills and aggregate cutover, enable retention table by table: raw packets, frontend errors, operational snapshots, link/job history, dead jobs, logs, and derived models each need separate policy.
4. Preserve longer-lived privacy-safe aggregates/archive if product requires historical trends. Define deletion/export handling for owner/private data.

Acceptance:

• A dry run reports exact rows/chunks and every feature that would lose history.
• Backup freshness and successful restore are hard gates before deletion.
• All documented history windows work using retained data/aggregates.
• Disk/WAL/chunk growth and retention failures alert.

Rollback:

• Compression and retention flags ship disabled. Once data is deleted, rollback requires tested restore, so enable only after explicit operator approval.

4.5 Bound backend caches and external work

Files:

• backend/src/path-beta/resolveCache.ts
• backend/src/cache/boundedTtlMap.ts
• owner/spam/API cache modules
• backend/src/api/routes/radio.ts
• cache metrics and tests

Implementation:

• Replace plain maps with BoundedTtlMap or an extended byte-aware variant.
• Key entries by privacy generation, network, observer, and owner/session where relevant; invalidate without full-map scans.
• Define maximum entries/bytes, TTL, negative-cache policy, single-flight behavior, and eviction for every cache.
• Bound outbound concurrency, time, redirects, and response bytes.

Acceptance:

• A reviewed registry/static check covers process-lifetime data caches and fails when one lacks a policy; it explicitly excludes short-lived algorithmic, request-local, and in-flight coordination maps.
• Soak/load tests keep heap under its budget and network switches or privacy generation changes cannot reuse stale cross-scope values.

4.6 Correct health resource semantics

Files:

• backend/src/health/status.ts
• docker-compose.yml
• public/operator health DTOs

Implementation:

• Measure the actual database, Redis, terrain, and log volumes rather than container /.
• Split public coarse health/readiness from authenticated operator detail.
• Keep process CPU/memory labels accurate to the container/cgroup being measured and expose unknown rather than misleading host/server values.

Acceptance:

• Filling a fixture data volume drives the correct warning while unrelated root filesystem usage does not.
• Public responses disclose no unnecessary platform detail; operator responses retain actionable evidence.

11. Phase 5 — repair frontend correctness and state contracts

Goal: make headline features work, make snapshots authoritative, prevent cross-network stale state, and remove races and misleading UI.

5.1 Spam Watch map lifecycle

Files:

• frontend/src/pages/SpamTransparencyPage.tsx
• frontend/src/pages/spam-page.css
• frontend/test/e2e/public.spec.ts

Implementation:

• Render the map container unconditionally.
• Create one MapLibre instance independently of incident data, wait for load, and update one source as loading/empty/error/data states change.
• Clean up the instance and pending fetches on navigation.

Acceptance:

• Delayed, empty, failed, and refreshed zone responses all produce the correct overlay and never prevent map initialization.
• Only one map exists and navigation removes it.

5.2 Complete inferred nodes and packet arcs

Files:

• frontend/src/hooks/useNodes.ts
• frontend/src/components/Map/geojsonBuilders.ts
• frontend/src/components/Map/MapLibreMap.tsx
• frontend/src/components/Map/LiveOverlayController.tsx
• frontend/src/components/Map/DeckGLOverlay.tsx
• frontend/src/config/features.ts
• new fail-closed runtime feature configuration loaded before map startup
• tests and README

Implementation:

1. Build inferred-node features from the actual inferred collection and active-ID set, with an explicit is_inferred property and distinct style.
2. Populate bounded, expiring arcs only when both privacy-safe endpoints can be resolved. If product does not want arcs, remove the dead control and claims instead of keeping a no-op feature.
3. Apply the shared privacy mask before features reach MapLibre/Deck.gl and cap collection size/age.
4. Implement and verify the layers in this phase while disabled by default. Phase 8 performs the product go/no-go and production enablement; it does not reimplement them.

Acceptance:

• Fixture inferred nodes appear, update, and disappear.
• One packet creates the expected arc and it expires.
• Private endpoints never enter GeoJSON, picking data, cache, or tooltip.

Rollout:

• Keep inferred nodes, arcs, and heatmap independently feature-flagged through same-origin runtime configuration that is loaded before map startup and defaults false on failure. Current VITE_* build-time flags alone are not an immediate kill switch. Test on the dedicated test hostname first; an operator must be able to disable the layer without rebuilding the image and clients must receive the change inside the documented cache/config TTL.

5.3 Make realtime stores authoritative and scoped

Files:

• frontend/src/hooks/useLinkState.ts
• frontend/src/hooks/useNodes.ts
• frontend/src/hooks/useCoverage.ts
• frontend/src/hooks/useAppMessageHandler.ts
• frontend/src/hooks/useDashboardStats.ts
• frontend/src/App.tsx
• new store tests

Implementation:

• Treat every initial snapshot, including [], as authoritative.
• Delete a link when itm_viable becomes false; define unknown/null behavior.
• Add reset(scope) and epoch/version actions and call them before network/observer changes. Ignore late messages from older epochs.
• Canonicalize IDs at store boundaries.
• Pass actual packet batch size to dashboard statistics.

Acceptance:

• Empty snapshot clears prior links; true -> false -> true transitions are exact.
• Network/observer switch cannot display prior nodes, links, coverage, arcs, sparklines, counts, or pending fetch results.
• A batch of N packets increments observed count by N.
• Out-of-order snapshot/live fixtures converge deterministically.

5.4 Fix Feed state, path lifecycle, and connectivity

Files:

• frontend/src/pages/ukmesh/UKFeedPage.tsx
• frontend/src/hooks/useWebSocket.ts
• public/mobile E2E tests

Implementation:

• Do not validate/reset the stored IATA region until asynchronous options have finished loading.
• Model packet path as idle | unavailable | settling | loading | ready | error; a packet with no hashes is unavailable, not forever loading.
• Use actual WebSocket ready state plus last-message age for status. Distinguish connected-and-quiet, reconnecting, offline-with-cache, and live.
• Abort old scope requests and stop eager background resolution for every recent packet; resolve selected/near-viewport items under the numeric concurrency cap approved in Phase 0.

Acceptance:

• Stored region survives reload.
• No-hash packet shows a terminal “no path available” state.
• Quiet connection remains connected and cached disconnected data is not labelled live.
• Old network requests cannot overwrite current feed/path state.

5.5 Fix repeater, packet-detail, Stats, and count defects

Files:

• frontend/src/pages/ukmesh/UKRepeaterSearchPage.tsx
• frontend/src/pages/ukmesh/PacketDetailPanel.tsx
• frontend/src/pages/StatsPage.tsx
• frontend/src/hooks/usePacketDetailData.ts
• frontend/src/hooks/useAppMessageHandler.ts
• frontend/src/hooks/useDashboardStats.ts

Implementation:

• Request advert_count, elevation_m, and every displayed repeater field. Use null checks so longitude 0 is valid. Abort or sequence A-to-B detail requests.
• Remove the second propagation unit suffix.
• Check every Stats response status; add error, empty, retry, and last-updated states. Supply a glyph-enabled style or remove the unsupported text layer.
• Cache global radio statistics once per scope rather than under packet keys; skip path calls when no hashes exist.

Acceptance:

• Rapid A→B selection can never show A under B; longitude zero renders.
• Propagation units appear exactly once.
• Failed/empty Stats requests show useful retryable feedback rather than blank content.
• Packet detail makes no unnecessary global-radio or empty-path requests.

Phase 5 rollback:

• Preserve API shapes and tolerate old local-storage values during the release.
• Ship broken-feature repairs in small PRs with feature flags for new layers; retain the security property during rollback. If an epoch/reset change is faulty, disable the affected UI, clear scoped in-memory/persisted state, and roll forward or restore the prior privacy-safe reader; do not force faulty state code to remain live or restore a cross-scope reader.

12. Phase 6 — frontend performance, accessibility, and architecture

6.1 Standardize polling, requests, and caches

Add:

• frontend/src/utils/scopedCache.ts: TTL/LRU with entry and estimated-byte bounds, scope invalidation, single-flight, and metrics/debug snapshot.
• frontend/src/hooks/useVisibilityPoll.ts: one request at a time, AbortController, timeout, page-visibility gating, jitter, backoff, and immediate refresh on return.
• checked, abort-aware response helpers in frontend/src/utils/api.ts.

Migrate:

• frontend/src/components/Map/MapLibreMap.tsx
• frontend/src/hooks/usePacketDetailData.ts
• frontend/src/pages/ukmesh/UKFeedPage.tsx
• frontend/src/pages/ukmesh/UKRepeaterSearchPage.tsx
• frontend/src/pages/OwnerPortalPage.tsx
• frontend/src/components/LiveStatsSection.tsx
• frontend/src/pages/StatusPage.tsx
• frontend/src/pages/ukmesh/UKCompanionPage.tsx
• frontend/src/components/Map/LinkQualitySparkline.tsx

Implementation:

• Key all cache and request state by network, observer, owner session, privacy generation, and relevant query parameters.
• Bound node links, packet detail, repeater detail, owner last-hop, Feed paths, and every other module cache; clear owner data on logout.
• Refresh observer health when scope changes.
• Replace perpetual two-second planned coverage polling with bounded, visibility-aware job polling and a terminal timeout/error action.
• Keep the owner MapLibre instance stable and update its sources rather than rebuilding every ten seconds.
• Include network/observer in sparkline requests and aggregate/cap concurrent charts rather than mounting up to eight independent unscoped calls.

Acceptance:

• No polling occurs while hidden; returning triggers one refresh.
• A slow request cannot overlap its successor or overwrite a newer scope.
• Every cache has documented scope, TTL, count/byte bound, and invalidation.
• Owner map construction count remains one across repeated live polls.
• Feed path concurrency, polling deadlines, and cache memory never exceed the numeric Phase 0 budgets.

6.2 Bound browser terrain, service-worker, topology, and list work

Files:

• frontend/src/utils/terrainSampler.ts
• frontend/public/sw.js
• frontend/src/main.tsx
• frontend/src/pages/TopologyPage.tsx
• frontend/src/pages/SpamTransparencyPage.tsx
• frontend/src/pages/ukmesh/UKFeedPage.tsx

Implementation:

• Terrain sampler: add in-flight tile deduplication, true LRU, abort support, the numeric Phase 0 memory budget, and ImageBitmap.close() on eviction.
• Service worker: use a new cache namespace, the Phase 0 storage quota, amortized pruning, and metadata rather than cache.keys() on every tile write. Do not force reload loops on controller/update changes.
• Topology: keep D3 positions in the rendering layer or throttle immutable React snapshots to a visible-frame budget rather than copying every node on every simulation tick.
• Use measured/dynamic virtualization for expandable rows, or render a bounded non-virtualized detail region outside the fixed-height list.

Acceptance:

• A 30-minute map/terrain soak remains under the numeric Phase 0 ceiling, closes evicted bitmaps, and issues one fetch per concurrent tile key.
• Service-worker tests cover warm and cold offline navigation, an update arriving during an in-progress interaction, explicit deferred activation, rollback to the previous worker/cache, and cleanup of the retained namespace after the rollback window; no reload loop occurs and pruning is amortized.
• Representative topology stays inside the Phase 0 node-count/frame-time budget with no per-tick full React state copy.
• Expanded Feed/Spam rows never overlap or disappear.

Rollout/rollback:

• Bump the service-worker cache namespace and retain the previous namespace for one rollback window. Activation must be explicit/deferred while the user has unsaved or in-progress state. Release browser-cache changes separately from map logic and remove the old namespace only after rollback expiry.

6.3 Build shared accessible primitives

Add:

• frontend/src/components/ui/Dialog.tsx
• frontend/src/components/ui/Tabs.tsx
• frontend/src/components/ui/Combobox.tsx
• frontend/src/hooks/useReducedMotion.ts

Prefer a maintained, proven accessible primitive library that fits the bundle and styling constraints. If custom Dialog/Combobox primitives are retained, budget explicit cross-browser, IME/composition, nested-dialog, screen-reader, and focus-restoration testing rather than relying on ARIA attributes alone.

Migrate:

• dialogs in DisclaimerModal.tsx, StatsPage.tsx, and UKFeedPage.tsx
• tabs in NodePopupContent.tsx and StatsPage.tsx
• search in NodeSearch.tsx and UKRepeaterSearchPage.tsx
• interactive rows in PacketFeed.tsx and UKFeedPage.tsx
• range inputs in FilterPanel.tsx and MobileControls.tsx
• animations in StatsPanel.tsx, globals.css, map-app.css, spam-page.css, and feed-page.css

Acceptance:

• Dialogs trap focus, close with Escape, restore focus, and have labelled semantics.
• Tabs implement arrow/Home/End movement and linked tabpanels.
• Search exposes labelled combobox/listbox/options and active descendant.
• Rows work with Enter and Space; all controls have programmatic labels.
• Reduced-motion preference stops nonessential infinite and timer-driven animation.
• Every interactive element has a visible :focus-visible indicator. Touch controls target 44 CSS pixels where layout permits and never fall below the approved WCAG target-size/spacing rule.
• Charts and map-only summaries have an accessible text/table equivalent for their important values and trends.
• Axe reports no serious/critical issue on each public route and representative dashboard state.
• Keyboard plus NVDA/Firefox and VoiceOver/Safari smoke tests cover dialogs, combobox IME input, tabs, map controls, and nested overlays.
• A keyboard-only user can search/select a node and read its details without needing the map canvas.

6.4 Decompose large components and CSS

Targets:

• MapLibreMap.tsx (map lifecycle, source controller, polling, interaction, popup)
• OwnerPortalPage.tsx (session, live data, map, sections)
• UKFeedPage.tsx (controller, filters, list, path resolver, dialogs)
• StatsPage.tsx (data, charts, decoded-path map, dialogs)
• ukmesh/PacketDetailPanel.tsx (data presentation, path map, packet decoder)
• backend/src/path-beta/resolver.ts, backend/src/db/index.ts, backend/src/mqtt/client.ts, and backend/src/stats/statsRepository.ts

CSS:

• Keep tokens, reset, and site-wide typography global.
• Co-locate page/component styles or use CSS modules.
• Move dashboard selectors out of globals.css.
• Remove the 71 duplicate selectors before relying on a different import order.

Acceptance:

• Agree a soft 500–700 line ceiling for page/controller files; deviations need a written reason.
• Extracted controllers/hooks have focused tests and stable public interfaces.
• Duplicate-selector count meets the numeric Phase 0 target and no feature relies on load-order overrides.
• Desktop/mobile screenshots and the numeric bundle/chunk budgets do not regress.

13. Phase 7 — observability, data safety, and release hardening

7.1 Expand and operationalize metrics

Files:

• backend/src/metrics.ts
• backend/src/index.ts
• internal metrics route/middleware
• Node and Python queue/worker modules
• logging/prometheus.yml

Implementation:

1. Verify and extend the Phase 0 internal-only metrics endpoint. Do not publish raw operational labels on the public origin.
2. Actually increment MQTT message/outcome and DB-pool metrics. Add route latency/status, packet batch, WS admission, webhook delivery, cache, analysis lease, queue count/bytes/age/retry/dead, SRTM, RF phase, worker heartbeat, graceful shutdown, and privacy-filter metrics.
3. Keep labels low-cardinality; never use node ID, owner, webhook URL, packet hash, or exception message as a label.

Acceptance:

• Prometheus scrape returns valid data while public external access is denied.
• Each critical queue/dependency/worker has outcome, saturation, latency, and freshness signals.
• A label-cardinality test and scrape-size budget pass.

7.2 Complete monitoring and alerting

Files:

• logging/promtail.yaml
• new Grafana Alloy configuration and migration tests
• logging/prometheus.yml
• logging/loki.yaml
• logging/grafana/**
• docker-compose.yml
• new logging/alertmanager.yml
• new logging/rules/*.yml
• docs/operations.md

Implementation:

• Replace Promtail, which is end of life as of 2026-03-02, with Grafana Alloy or another maintained collector. Use the official Alloy migration path and persist positions/checkpoints in a named volume.
• Collect bounded log files or use a supported logging driver. Do not mount the raw Docker socket into a collector; a read-only filesystem mount does not make Docker API methods read-only. If container metadata is essential, use a separately authenticated GET-only proxy.
• Add Postgres/Timescale, Redis, Mosquitto, container/disk, and worker exporters or equivalent collectors.
• Add Alertmanager and rules for readiness, ingest silence, queue age/capacity, dead jobs, lease loss, worker heartbeat, DB pool/WAL/disk, backup age, telemetry integrity, and synthetic failures.
• Add health checks to monitoring services.
• Make provisioned dashboards non-editable or define an explicit export-to-Git workflow. Remove/disable the stale ML dashboard until the worker is enabled.
• Put runbook links and ownership in alert annotations.

Acceptance:

• Alloy/collector restart does not replay an already-ingested sentinel line; labels and Loki queries remain compatible through the cutover.
• Synthetic firing and recovery reach a test receiver.
• Killing each worker or filling a bounded test queue produces the named alert inside its documented grace period.
• All rules, dashboards, and Compose health checks validate in CI.

7.3 Backups, restore drills, and retention gates

Files:

• new encrypted off-host PostgreSQL backup/restore tooling or pgBackRest config
• Compose service/config if self-hosted
• docs/db-lifecycle.md
• docs/operations.md
• README.md

Implementation:

• Back up the analytics and owner-auth databases, required Mosquitto credential/ACL state, and critical configuration. Keep secrets encrypted and access-separated.
• Classify Redis state explicitly. Durable ready/in-flight/retry/dead jobs, dedupe/counters, lease/commit markers, and planned-job coordination must either be persisted/restored consistently or deterministically rebuilt from PostgreSQL under a documented fenced recovery procedure. Name disposable caches separately and prove their loss is safe.
• Automate restore into an empty isolated environment, migrate it, sample row/checksum integrity, verify owner lookup, and run readiness/smoke tests.
• Alert on backup age/failure. Add WAL/PITR only after credentials/storage and RPO are approved.
• Make every retention activation require a fresh successful restore receipt.

Acceptance:

• The agreed initial RPO/RTO is demonstrated, not merely documented.
• Restore to an empty environment passes application smoke tests.
• A missing/stale backup blocks destructive retention or maintenance.

7.4 Immutable, reversible deployment

Files:

• scripts/replace-container.sh
• all Dockerfiles
• docker-compose.yml
• release workflow
• operations docs

Implementation:

1. Publish one immutable signed image digest per service/release.
2. Replace force-recreate/no-deps deployment with: preflight, backup gate, migration job, deploy digest, readiness wait, smoke checks, metric check, and automatic prior-digest rollback only when the prior image has passed a compatibility test against the post-migration schema.
3. Record deployed revision/digest/schema version. Keep the previous compatible image for at least one release window.
4. Treat migrations as forward-compatible expansion; rollback deploys old code against the compatible newer schema, never a destructive down-migration.

Acceptance:

• A deliberately failed readiness/smoke check automatically restores the prior signed digest only when schema compatibility is proven. Otherwise rollout stops before traffic and requires the documented manual recovery path.
• Release status identifies exact source, image, migration, and configuration.
• A partial service update cannot skip required dependency/migration checks.

7.5 Supply-chain and container policy

Files:

• all Dockerfiles
• docker-compose.yml
• .github/dependabot.yml
• CI/release workflows
• Python requirements/locks
• scripts/package.json plus new lockfile

Implementation:

• Pin production/base images by digest with automated Docker update PRs.
• Add pip update automation and hash-locked Python dependencies.
• Generate SBOMs, scan images, sign/prove release artifacts, and define an explicit waiver process. No unwaived critical finding may release.
• Use non-root users, read-only roots, cap_drop: [ALL], no-new-privileges, bounded tmpfs, and explicit writable data volumes where compatible.
• Pin Actions to immutable commits if repository policy requires it.

Acceptance:

• Rebuilds consume only pinned source, images, and dependencies.
• Every service passes smoke tests under hardened permissions.
• Release inventory contains digest, SBOM, scan, signature, and provenance.
• Dependabot/update automation covers npm, Actions, Docker, and Python.

7.6 Replace dangerous maintenance behavior

Files:

• vacuum-compressed-chunks.sh
• docs/db-lifecycle.md
• docs/operations.md

Implementation:

• Remove unsupervised blanket VACUUM FULL. Require dry-run output, explicit database/chunk selection, lock/statement timeout, disk-space preflight, backup freshness, one-chunk approval, progress/checkpoint, and DBA review.
• Prefer ordinary vacuum/reindex or online maintenance based on measured bloat.

Acceptance:

• Default invocation is read-only and prints exact intended work.
• Insufficient disk, stale backup, lock contention, or timeout aborts safely.
• A staged fixture proves resumability and no long unbounded lock.

14. Phase 8 — product completion and documentation

These features should follow the security/reliability foundations so they reuse durable events, scoped stores, bounded jobs, and real metrics.

8.1 Owner alert experience

Backend:

• durable alert events/outbox, per-channel attempts, idempotency, bounded retry, delivery history, test delivery, last success/error, pause reason, and audit.

Frontend:

• owner portal list/detail for rules and delivery history, “send test,” clear validation, retry status, and accessible controls.

Acceptance:

• Every triggered alert is durably traceable to zero or one successful delivery per idempotency key.
• Owner sees only granted-node events and sanitized destination information.
• Retry/dead-letter behavior is operator-visible and bounded.

8.2 Observer registration workflow

Files:

• existing registration API/schema
• a new operator queue/service and operator UI
• owner/operator audit records and runbook

Implementation:

• Give submissions explicit pending, approved, rejected, expired, and provisioned states; validate/normalize PII; define retention and access.
• Add operator review, duplicate detection, decision reason, audit trail, and notification/provisioning handoff.

Acceptance:

• A submission cannot vanish into an unconsumed table.
• PII is operator-only, retention-bounded, and absent from public logs/metrics.

8.3 Job and model operations

• Add an operator view/API for analysis runs, live/planned viewshed, link-v3, ML runs, queue capacity, lease owner/expiry, attempts, oldest age, dead-letter reason, requeue, purge, and repair.
• Add path-confidence calibration trends by model version/network and compare predictions with later observations.

Acceptance:

• Operators can diagnose and recover a stuck/dead job without direct Redis/SQL mutation.
• Every action is authorized, audited, idempotent, and protected from bulk accidents.

8.4 Complete public discovery features

• Decide the public planned-node contract. Prefer owner/operator records and a separately publishable, privacy-reviewed DTO with pagination and expiry.
• Make Watchlist entries actionable: restore filter/search/selection, navigate to the saved item, and expose the panel on mobile.
• Add a wildcard 404 route.
• Use one route metadata source for runtime routing, canonical tags, sitemap, and build-time SEO.
• Enable and productize the already implemented arcs/inferred layers only after Phase 5 privacy/correctness tests and the runtime kill switch pass.

Acceptance:

• Planned records never expose owner public keys or free-form private notes unless an explicit reviewed publish action exists.
• Watchlist restore is deterministic across desktop/mobile and network scope.
• Invalid URLs render a useful 404; every public route has consistent canonical metadata and sitemap coverage.

8.5 API and architecture contracts

Files:

• docs/openapi.yaml
• request/response schemas
• route/repository/service modules
• docs/architecture.md, README.md, docs/frontend-map.md, docs/operations.md

Implementation:

• Generate or validate OpenAPI from the same schemas used at runtime; the current document covers only a small fraction of routes and advertises behavior that runtime rejects.
• Move SQL out of route modules, prioritizing nodes, planned coverage, topology, RF validation, owner, and product features.
• Delete confirmed dead backend/src/api/caches.ts.
• Correct nonexistent component references, route/feature claims, service topology, bootstrap, migrations, monitoring, Anubis ingress, retention, and runbooks.

Acceptance:

• CI fails when a route lacks or contradicts its OpenAPI contract.
• Example requests pass contract tests for public, test, owner, operator, and error cases.
• Documentation can bootstrap, deploy, roll back, restore, recover queues, manage SRTM, and respond to alerts without undocumented shell/SQL knowledge.

15. Cross-phase test and release matrix

Run the smallest relevant suite during development and the full matrix before each phase is considered complete.

Layer	Required gate
Backend fast	cd backend && npm ci && npm run typecheck && npm test && npm run build
Frontend fast	cd frontend && npm ci && npm test && npx tsc --noEmit && npm run build
Frontend browser	All Playwright desktop/public/dashboard/mobile projects plus axe and privacy fixtures
Python	Build worker images; run pytest, imports, and CLI smoke tests inside the exact GDAL/viewshed and ML image environments
Database	Pinned Timescale version; current backup/restore preflight; fresh and previous-release migrate; concurrent runner; concurrent-index retry; backfill resume; query-plan budgets
Redis/queues	Randomized state-machine/invariant tests, crash/lease-loss/restart, capacity/byte/load tests
MQTT E2E	Broker credential bootstrap, radio envelope to DB/rollup/WS, private/test/network fixtures, graceful termination
Proxy E2E	Cloudflare-header fixture through Anubis/Nginx to REST/WS, spoof resistance, per-client quotas
Containers	Build every Dockerfile/profile; non-root/read-only smoke; health checks; no Docker socket
Dependencies	npm audit policy, Python/image scan, SBOM, signature/provenance
Operations	Alert firing/recovery, persisted Alloy/collector position, PostgreSQL plus durable-Redis recovery, empty-environment restore, schema-compatible failed-deploy rollback
Performance	Compare p50/p95, DB CPU/WAL, heap/RSS, queue bytes/age, browser memory/long tasks, and bundle size with Phase 0

Release order for each substantial protocol/schema change:

1. Land tests and observability.
2. Deploy additive schema or compatible consumers.
3. Backfill/shadow/dual-read and compare.
4. Deploy producers/writers behind a flag.
5. Enable on the test hostname or one worker.
6. Expand gradually while watching correctness, privacy, queue, and SLO gates.
7. Keep the previous known-good digest that has passed compatibility against the new schema, plus the old read path, for one release window.
8. Contract/remove old state only after readback, restore, and rollback windows pass.

16. Complete issue register

This register is the audit backlog. Line numbers refer to the audited revision and will move as implementation proceeds.

Security and privacy

ID	Pri	Finding and evidence	Planned resolution
SEC-01	P1	Owner webhook accepts only an HTTPS prefix and later calls unrestricted fetch (backend/src/api/routes/owner.ts:233-257, backend/src/owner/alertRules.ts:39-53). Blind SSRF is reachable by an authenticated owner.	Phase 1.1
SEC-02	P1	Public backend mounts /var/run/docker.sock read-write and runs without a non-root USER (docker-compose.yml:205-211, Dockerfile.backend:9-19). A backend compromise can become host control.	Phase 1.2
SEC-03	P1	Radio-influenced distinct node IDs can grow the live viewshed Redis set/list without a global count or byte cap (backend/src/index.ts:104-111, backend/src/queue/publisher.ts:54-83,125-131).	Phase 1.3
SEC-04	P1	Website exact /ws omits forwarded client identity, collapsing all clients into the proxy's quota (nginx.website.conf:68-76, backend/src/http/trustedProxy.ts:25-35, backend/src/ws/server.ts:432-448).	Phase 1.4
SEC-05	P1	Anonymous link history queries global pair records without network/private-node scope (backend/src/api/routes/productFeatures.ts:70-94).	Phase 1.5
SEC-06	P1	Anonymous frontend error rows directly set health warning/critical at 25/100, inside one IP's 120/minute quota (backend/src/api/routes/telemetry.ts:16-45, backend/src/health/status.ts:341-346,513-519, backend/src/index.ts:150-157).	Phase 1.6
SEC-07	P1	Health-check image defaults to fetching and executing upstream main (docker-compose.yml:535-542, Dockerfile.mesh-health-check:3-24).	Phase 1.7
SEC-08	P2	requireLocalOnly can accept caller-supplied private forwarded values if backend ingress changes, although default Compose binds loopback (backend/src/api/utils/localOnly.ts:24-49, docker-compose.yml:203-204).	Phase 1.8
SEC-09	P2	Scoped decoded-path statistics resolve prefixes against all positioned repeaters, not the requested network (backend/src/stats/statsRepository.ts:320-345,403-432). Under current runtime exposure this is correctness rather than a validated disclosure; the ADR may strengthen the boundary.	Phases 1.8/4.3
SEC-10	P2	Observer key generator writes a private seed/PEM without explicit 0600 (scripts/generate-observer-key.ts:48-64). Effective exposure depends on umask and directory.	Phase 1.8
SEC-11	P2	README Cloudflare examples target origins while Compose says public HTTP must target Anubis (README.md:208-220, docker-compose.yml:637-640).	Phases 1.8/8.5
SEC-12	P2	A literal MeshCore channel key exists in a tracked backfill script (backend/backfill-transport-codes.mjs:32-40). It appears to be intentional public channel material but needs owner classification.	Decision gate / Phase 8.5
SEC-13	P2	Heatmap/packet-detail path and advertisement coordinates use multiple rendering paths; candidates were suppressed because APIs/privacy masking provide counterevidence, not because frontend sinks are intrinsically safe.	Phases 1.9/5
SEC-14	P2	Planned coverage performs expensive radial/peer work but has rate/global caps, feature gating, and worker isolation; no security outage was demonstrated (backend/src/api/routes/plannedCoverage.ts:114-140, viewshed-worker/worker.py:504-539,879-938).	Phases 3.4/6.1
SEC-15	P2	Frontend lock resolves vulnerable advisory versions of PostCSS and React Router; audited attack modes were not applicable, but the high audit gate fails.	Phase 0.1
SEC-16	P1	Runtime/test-host behavior and the supplied security invariant disagree about anonymous network test; UK Feed also has a separate content-channel test value. Encoding either interpretation before an ADR risks privacy or feature breakage.	Decision gate / Phases 0.3/1

Backend, API, database, and lifecycle

ID	Pri	Finding and evidence	Planned resolution
BE-01	P1	GET /nodes/map is registered twice; the first unbounded handler shadows the intended limited implementation (backend/src/api/routes/nodes.ts:84-105,317-345).	Phase 2.1
BE-02	P1	Analysis active_run_id has no expiry/heartbeat and completion is split across autocommit updates (backend/src/analysis/runState.ts:35-45,92-127). Crashes can wedge a workload.	Phase 2.2
BE-03	P1	Operator HTML requires bearer/custom headers but browser fetches send neither (backend/src/backend-site/routes.ts:87-117,132-137, template.html:325-346).	Phase 2.5
BE-04	P1	No coordinated SIGTERM/SIGINT drain exists; packet batches and MQTT flush can remain in flight (backend/src/index.ts:199-230, backend/src/db/packetBatch.ts:46-74,214-230, backend/src/mqtt/client.ts:526-531).	Phase 2.3
BE-05	P1	Migrations perform whole-table rewrites despite policy; duplicate 016 exists; runner lacks advisory lock/checksum (016_private_prefixes.sql:1,150-251, 018_api_performance_indexes.sql:15-22, backend/src/db/migrations.ts:12-58).	Phases 0.4/4.1
BE-06	P1	Owner alert async handlers lack Express 4 error boundaries and may hang/reject outside middleware (backend/src/api/routes/owner.ts:219-270).	Phase 2.4
BE-07	P2	Metrics are registered but /metrics is absent; MQTT and DB metrics are not populated (backend/src/metrics.ts:9-45, logging/prometheus.yml:5-9).	Phases 0.4/7.1
BE-08	P2	Stats charts launch 17 scans and cold recompute has taken about 80 seconds/DB CPU saturation (backend/src/stats/statsRepository.ts:117-125, backend/src/api/bootstrap/caches.ts:5-10).	Phase 4.3
BE-09	P2	MQTT and packet batch redundantly update observer rows, followed by separate rollup/sighting writes (backend/src/mqtt/client.ts:832-838, backend/src/db/packetBatch.ts:177-191, backend/src/db/index.ts:518-575,644-655).	Phase 4.2
BE-10	P2	Negative/NaN limits can reach SQL; radio proxy lacks timeout/body bound (backend/src/api/routes/misc.ts:39-49, nodes.ts:648-675, radio.ts:6-40).	Phase 2.4
BE-11	P2	Resolve/sticky caches are unbounded maps and invalidation scans the map (backend/src/path-beta/resolveCache.ts:11-37,48-86) despite an available bounded cache.	Phase 4.5
BE-12	P2	Map and stats disagree on roles/freshness, so headline totals can conflict (nodes.ts:84-101, statsRepository.ts:716-732).	Phases 0.3/2.1
BE-13	P2	Raw packets and derived state are retained indefinitely; schema deliberately removes retention policies (backend/src/db/schema/base.sql:1-2,64-101, README.md:316-320).	Phase 4.4
BE-14	P2	Health disk check observes container root rather than DB/Redis/terrain volumes; public “server” resource labels can mislead.	Phase 4.6
BE-15	P2	Anonymous /planned-nodes exposes all proposed records, owner public keys, coordinates, names, and notes without lifecycle/pagination (backend/src/api/routes/misc.ts:113-123). Product intent is unclear.	Immediate containment in Phase 1.8; product decision in Phase 8.4
BE-16	P2	Observer registration PII enters storage without an implemented operator workflow, status lifecycle, notification, or retention process.	Phase 8.2
BE-17	P2	Owner alerts have no durable event/outbox/history/test delivery despite product claims; sequential delivery can make polling slow.	Phases 1.1/8.1
BE-18	P2	Database numeric environment parsing lacks consistent minimum/maximum validation, permitting invalid pool/time settings.	Phase 2.4
BE-19	P2	Unhandled rejection logging continues the process in uncertain state; no fatal/drain policy exists.	Phase 2.3
BE-20	P3	SQL remains embedded in many routes; several core modules are very large; backend/src/api/caches.ts is an unused unbounded predecessor.	Phases 6.4/8.5
BE-21	P3	OpenAPI covers only a small subset of roughly 59 routes and disagrees about public all scope.	Phase 8.5
BE-22	P3	Schema comments/documentation disagree with runtime node roles and service layering.	Phase 8.5

Python workers, queues, CI, and operations

ID	Pri	Finding and evidence	Planned resolution
WK-01	P1	Fresh default link worker cannot acquire terrain; only the profile-gated viewshed worker downloads tiles (viewshed-worker/worker.py:1007-1010, rf/terrain.py:151-158, docker-compose.yml:373-440).	Phase 3.1
WK-02	P1	Transient terrain/GDAL failures are stored as current-version empty success and suppressed from retry; planned jobs can also become ready empty (worker.py:618-663,771-782,1489-1525,1588-1603).	Phase 3.1
WK-03	P1	backfill_profiles.py unpacks three values while compute_path_loss returns two, so its update path skips every valid row (backfill_profiles.py:55-76, rf/loss.py:21-38).	Phases 0.1/3.1
WK-04	P1	Dead link jobs retain counters/payload/dedupe and permanently consume queue capacity (viewshed-worker/link_queue_v3.py:35-68,99-139).	Phase 3.2
WK-05	P1	Redis error terminates LeaseRenewer; work may continue and commit after ownership is lost (link_queue_v3.py:271-299).	Phase 3.2
WK-06	P1	ML timestamp-only cursor plus 5,000 limit permanently skips rows sharing the boundary timestamp (ml-path-learner/worker.py:238-255).	Phase 3.3
WK-07	P1	ML insert rollback can erase earlier writes while processing advances the checkpoint (ml-path-learner/worker.py:359-394).	Phase 3.3
WK-08	P2	ML loads the full gold table before sampling and lacks leader lease/deadline (ml-path-learner/worker.py:408-446). It is disabled today.	Phase 3.3
WK-09	P2	RF radial loop recomputes every growing prefix across 360 rays, roughly 180 million prefix elements at maximum settings (viewshed-worker/worker.py:504-539).	Phase 3.4
WK-10	P2	Planned coverage evaluates up to 60 peer terrain paths serially (worker.py:879-938).	Phase 3.4
CI-01	P1	Nightly Compose omits required Anubis key and does not create ignored Mosquitto password/ACL files, so a clean job cannot start (.github/workflows/ci.yml:91-125, docker-compose.yml:648,670,692).	Phase 0.2
CI-02	P1	Python CI is syntax-only, omits queue/backfill modules, installs no dependencies, and runs no tests/import/image smoke; GDAL tests need the exact worker image.	Phase 0.1
CI-03	P2	Normal CI builds no containers; nightly covers only a subset, allowing frontends, workers, Nginx, profiles, and health source integration to rot.	Phase 0.1
CI-04	P2	Main CI omits the frontend unit suite.	Phase 0.1
OPS-01	P1	scripts/replace-container.sh force-recreates with --no-deps and has no migration, readiness, smoke, immutable record, or rollback.	Phase 7.4
OPS-02	P1	vacuum-compressed-chunks.sh runs blocking VACUUM FULL over every compressed chunk without safe preflights or resumability.	Phase 7.6
OPS-03	P1	No general backup, PITR, scheduled restore test, or capacity/retention gate exists.	Minimum preflight in Phase 0.4; full system in Phase 7.3
OPS-04	P2	Promtail positions live under ephemeral /tmp, causing replay after restart (logging/promtail.yaml:5-9, docker-compose.yml:750-760), and Promtail is now EOL.	Migrate to Alloy in Phase 7.2
OPS-05	P2	Prometheus scrapes only backend (currently 404), with no DB/Redis/MQTT/worker/container/disk exporters, Alertmanager, or rules.	Minimum in Phase 0.4; expansion in Phases 7.1/7.2
OPS-06	P2	Monitoring services lack health checks; provisioned dashboards are editable and the ML dashboard targets a disabled service.	Phase 7.2
OPS-07	P2	Base/service images use mutable tags; Python has no hash lock; Dependabot omits Docker/pip; scripts lack a lock; no SBOM/signing/provenance.	Phases 1.7/7.5
OPS-08	P2	Runtime containers generally lack explicit non-root user, read-only filesystem, capability drops, and no-new-privileges.	Phases 1.2/7.5
OPS-09	P2	Quick Start omits required secrets and has impossible Mosquitto ordering; architecture/topology and Cloudflare instructions drift from Compose.	Phases 0.2/8.5
OPS-10	P2	Operations docs lack release/rollback, backup/restore, dead-letter, SRTM cache, capacity, alert delivery, and disaster-recovery runbooks.	Phases 7/8.5

Frontend correctness, performance, accessibility, and maintainability

ID	Pri	Finding and evidence	Planned resolution
FE-01	P1	Spam Watch map ref does not exist when its one-shot effect runs, so the main map never initializes (SpamTransparencyPage.tsx:434-477,527-531).	Phase 5.1
FE-02	P1	Packet arcs are never populated and overlay receives arcs={[]} despite product claims (useNodes.ts:83-97, LiveOverlayController.tsx:177-183).	Phase 5.2
FE-03	P1	Inferred nodes are polled but ignored by MapLibre/GeoJSON, which hard-codes is_inferred: false (App.tsx:286-331, MapLibreMap.tsx:99-123, geojsonBuilders.ts:94-158).	Phase 5.2
FE-04	P1	Link updates never delete nonviable links and empty snapshots do not clear old network state (useLinkState.ts:76-137).	Phase 5.3
FE-05	P1	Repeater details omit displayed fields, reject longitude zero, and allow A→B request races (UKRepeaterSearchPage.tsx:183-190,236-255,379-409).	Phase 5.5
FE-06	P1	Stored Feed region resets to all before asynchronous options load (UKFeedPage.tsx:395-401,485-510,546-551).	Phase 5.4
FE-07	P1	Packets without path hashes are skipped by resolution but rendered as permanently loading (UKFeedPage.tsx:635-638,716-718,869-878,918-925).	Phase 5.4
FE-08	P1	Propagation formatter includes units and JSX appends another s (PacketDetailPanel.tsx:470-479,561-565).	Phase 5.5
FE-09	P1	Observed packet count increments once per animation-frame batch, not per packet (useAppMessageHandler.ts:181-184, useDashboardStats.ts:36-40).	Phases 5.3/5.5
FE-10	P1	Stats ignores response status and renders failures blank; its raster style lacks glyphs for a text layer (StatsPage.tsx:234-247,303-315,333-367,429-442).	Phase 5.5
FE-11	P1	Feed ignores actual WebSocket ready state and infers connectivity only from packet age (UKFeedPage.tsx:452-453,685-687).	Phase 5.4
FE-12	P2	Owner map is destroyed/rebuilt on each ten-second data refresh (OwnerPortalPage.tsx:683-841,982-1015).	Phase 6.1
FE-13	P2	Owner/session/live/last-hop polling overlaps, continues off-tab, and lacks consistent timeout/cancellation (OwnerPortalPage.tsx:890-923,982-1052).	Phase 6.1
FE-14	P2	Planned coverage polls every two seconds indefinitely without visibility gating/backoff (MapLibreMap.tsx:329-357).	Phase 6.1
FE-15	P2	Observer health is fetched once, unscoped, and not refreshed on network switch (MapLibreMap.tsx:775-819).	Phase 6.1
FE-16	P2	Node link, packet, repeater, owner last-hop, and Feed path module caches are unbounded and incompletely scoped.	Phase 6.1
FE-17	P2	Feed starts background path resolution for every recent path packet rather than on user/viewport demand (UKFeedPage.tsx:711-750).	Phases 5.4/6.1
FE-18	P2	Packet detail caches global radio stats under packet keys and calls path API without hashes (usePacketDetailData.ts:31-55,87-109).	Phase 5.5
FE-19	P2	Terrain sampler may retain about 200 MB decoded data, has no in-flight dedupe, FIFO eviction, and never closes bitmaps (terrainSampler.ts:7-12,39-58).	Phase 6.2
FE-20	P2	Service worker prunes with cache.keys() on every write, allows 8,000/~120 MB tiles, and update handling can force reload (public/sw.js:1-6,25-52, main.tsx:36-56).	Phase 6.2
FE-21	P2	Topology copies every simulation node into React state on every D3 tick (TopologyPage.tsx:90-122).	Phase 6.2
FE-22	P2	Spam/Feed fixed-height virtualization conflicts with expanded detail rows.	Phase 6.2
FE-23	P2	Link sparklines omit network/observer scope and up to eight independent requests mount per popup (LinkQualitySparkline.tsx:14-21, NodePopupContent.tsx:182-201).	Phase 6.1
FE-24	P2	Dialogs lack focus trap/restore and consistent Escape handling (DisclaimerModal.tsx:8-42, StatsPage.tsx:936-975, UKFeedPage.tsx:983-1031).	Phase 6.3
FE-25	P2	Tabs lack full tab/tabpanel and keyboard semantics (NodePopupContent.tsx:62-73, StatsPage.tsx:444-457).	Phase 6.3
FE-26	P2	Packet rows emulate buttons incompletely; Feed articles are not keyboard focusable (PacketFeed.tsx:49-76, UKFeedPage.tsx:855-868).	Phase 6.3
FE-27	P2	Node/repeater searches lack combobox/listbox labels, active descendant, and arrows (NodeSearch.tsx:114-140, UKRepeaterSearchPage.tsx:269-310).	Phase 6.3
FE-28	P2	Range input visible labels are not programmatically associated (FilterPanel.tsx:153-166, MobileControls.tsx:88-101).	Phase 6.3
FE-29	P2	Reduced-motion handling still permits pulse/timer/page animation (globals.css:262-265, StatsPanel.tsx:14-37).	Phase 6.3
FE-30	P2	Watchlist can only display/remove and is absent from mobile controls (WatchlistPanel.tsx:21-28, MobileControls.tsx:37-109).	Phase 8.4
FE-31	P2	MapLibre, Owner, Feed, Stats, and packet-detail components range from ~845 to 1,652 lines and mix data, polling, rendering, and map lifecycle.	Phase 6.4
FE-32	P2	Eleven CSS files total 7,761 lines with 71 duplicate selectors and load-order risk.	Phase 6.4
FE-33	P3	No wildcard/404 route exists (frontend/src/main.tsx:69-89).	Phase 8.4
FE-34	P3	Build-time SEO/sitemap routes and runtime SEO routes drift (vite-seo.ts:5-54,139-234, config/seo.ts:8-85).	Phase 8.4
FE-35	P3	Architecture docs reference nonexistent NodeDetailDrawer.tsx; README route/feature claims drift.	Phase 8.5

17. Suggested pull-request slices and dependencies

Keep each slice independently testable and reversible. Parallel work is safe only where the dependency column permits it.

Order	Slice	Depends on
1	CI dependency patches, frontend unit gate, worker-image test skeleton, clean Mosquitto bootstrap	none
2	Migration lock/checksum ledger, pinned Timescale snapshot/restore preflight, known-good digests, internal metrics endpoint	slice 1
3	Website /ws identity forwarding and proxy E2E test	slice 1 test harness only
4	Network/channel ADR, coordinate DTO boundary, link-history scope, planned-node containment, trusted local-only check	slice 2
5	Webhook egress policy and delivery/outbox schema	slice 2 plus allowlist/egress decision
6	Broker-local ACL helper, backend non-root, Docker socket removal	slice 1 clean-stack CI
7	Duplicate map route plus chosen completeness contract, shared parsers/error boundary, operator session	slice 4 visibility/map contract
8	Graceful lifecycle coordinator	queue/batch fixtures plus core metrics
9	Analysis leases, legacy orphan cleanup, additive job-state migration	slice 2 migration/backup foundation
10	Terrain acquisition/result types, legacy empty-row repair, backfill fix	worker-image tests
11	Tactical live-viewshed cap, then one queue-v3/viewshed protocol with bounded DLQ and database fencing	slices 2 and 10 plus essential alerts
12	Ingest write consolidation and historical-migration/backfill runbook	database integration fixtures
13	Stats aggregates/shadow reads and cache bounds	visibility predicate plus verified restore
14	Frontend broken-feature regressions, runtime kill switch, and fixes	scoped privacy-safe API fixtures
15	Scoped stores, polling/cache utilities, browser resource bounds	slice 14
16	Accessibility primitives and component/CSS decomposition	stable behavior from slices 14–15
17	Metrics/exporter expansion, Alloy migration, alerts, dashboards	stable queue/job semantics
18	Signed/pinned images, schema-compatible immutable deploy, automated backup/restore, safe maintenance	clean build plus core metrics
19	Retention/compression activation	aggregate cutover plus successful restore drill
20	Owner alert UI, observer workflow, job admin, watchlist/SEO/API docs	foundational slices complete

Where migration number conflicts arise, select the next unused number at implementation time and preserve the semantic filename. Never create a second new migration with a number already present.

18. Definition of done

The improvement program is complete when:

• All P1 items are fixed and backed by regression tests; all P2 items are fixed or have an explicit owner/date/accepted-risk record.
• The seven security findings are revalidated as closed, public privacy contracts pass across REST/WS/export/cache/derived data, and no public backend has host Docker authority.
• A fresh checkout can build and start the documented stack without hidden local files.
• Accepted packets and jobs have deterministic acknowledgement, drain, lease, retry, terminal, cleanup, and recovery behavior.
• Database migrations are locked/checksummed and contain no unbounded historical rewrite; backfills are resumable and observable.
• Map/stats queries and RF/browser workloads meet baselined budgets without correctness or privacy drift.
• Frontend primary flows work across desktop/mobile, network switching cannot leak stale state, and representative routes pass keyboard and axe checks.
• Metrics, alerts, runbooks, backup restore, and failed-deployment rollback are demonstrated in CI or a controlled staging drill.
• Releases use pinned inputs and signed immutable images with SBOM/provenance.
• OpenAPI and operator/user documentation match the implemented runtime.

For handoff, the implementing agent should reference these issue IDs in commits and PRs, update this file with status and measured before/after results, and attach the exact test/rollout/rollback evidence for each completed slice.