Part 1 of #1856. Part 2 (the hash migration reporting false success) is
#1958.
## It has never worked
`handlePostPacket` writes to the server's DB handle, and that handle is
read-only. `cmd/server/db.go:106`:
```go
dsn := fmt.Sprintf("file:%s?mode=ro&_journal_mode=WAL&_busy_timeout=5000", path)
```
Every call answered `500 attempt to write a readonly database`.
**This is the second report.** #1196 raised it on 2026-06-13, a fix was
merged that corrected v2 column names to v3, and the issue was closed.
That fix could not have worked, because the column names were never why
the write failed. Its comment is still sitting at `routes.go:1288`, next
to code that has never executed successfully in production.
## Why remove rather than build a handoff
**It cannot break a caller.** An endpoint that has only ever returned
500 has no working consumer. This is not a breaking API change, it is
documentation catching up with reality. Nothing in `public/` calls it.
**It was actively misleading.** `openapi.go` advertised it as "Ingest a
packet" and it sits behind `requireAPIKey`, which reads as a live,
protected write endpoint.
**Its test hid the breakage.** `TestPostPacketPersistsV3Schema` asserted
the observation row is written and passed for four months, because the
test DB is opened read-write while production is not. That is how #1196
came to be closed as fixed.
**Ingest is MQTT-only by design since #1283.** Re-adding an HTTP write
path re-opens the invariant that change established. If manual injection
is wanted later for testing or replay, it belongs on the ingestor side
and deserves its own issue. The repository already has the handoff shape
for that: the server writes `request-<id>.json` and the ingestor
consumes it (`cmd/ingestor/prune_geofilter.go`).
## What went
The route, `handlePostPacket` (103 lines), the now-unused
`PacketIngestResponse` type, the `openapi.go` entry, the round-trip
test, and the section plus table-of-contents line in `docs/api-spec.md`.
The `packetpath` import in `routes.go` became unused and went with it.
`+4/-225` across 6 files.
## The auth tests
The four `requireAPIKey` tests used `"/api/packets"` only as a request
path while building their own handler with `s.requireAPIKey(...)`, so
they never touched the route.
I checked that by **running them**, not by reading the code:
```
--- PASS: TestRequireAPIKey_RejectsWeakKey
--- PASS: TestRequireAPIKey_AcceptsStrongKey
--- PASS: TestRequireAPIKey_EmptyKeyDisablesEndpoints
--- PASS: TestRequireAPIKey_WrongKeyUnauthorized
```
Their paths now point at `/api/admin/prune-geo-filter`, which still
exists, so they no longer name a removed endpoint. Re-ran after that
change: still 4 of 4.
`/api/packets/observations` is a different endpoint and is untouched.
Verified: `gofmt` clean, `go vet` clean, `cmd/server` suite ok in 62.9s.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-submission of #1625 (which was merged early, then reverted in #1626)
— now with **all three round-1 reviews addressed** so it lands in one
hardened state instead of as post-merge follow-ups.
## What
Per-node **Reach** view: a standalone page (`#/nodes/{pubkey}/reach`) +
a node-detail section + `GET /api/nodes/{pubkey}/reach`. It shows which
nodes a node has a **stable two-way RF link** with, derived from raw
`path_json` adjacency (a path travels origin→observer, so `[A,B]` ⇒ B
heard A). A link is bidirectional when both directions have
observations; the **bottleneck** (weaker direction) rates two-way
reliability. Nodes are identified only by **unique 2–3 byte** path
prefixes (1-byte collides → excluded).
## Review fixes folded in vs #1625
**Performance (Carmack):** hard scan LIMIT (200k) + modest prealloc;
`json.Unmarshal` replaced by a single-pass `parsePathTokens` (100k-row
scan 2.2M→1.3M allocs, 344→203ms); memoized resolver; size-hinted maps
(attribution over 100k rows: 102 allocs); `context.Context` plumbed;
cache `RWMutex` + evict-oldest (no full wipe); singleflight dedup;
degree/rank from a 60s shared snapshot; bench rewritten (ReportAllocs,
1k/10k/100k, mixed-payload, isolated attribution).
**Correctness/safety + tests (Independent + Kent Beck):** pubkey
validation → 400; error logging instead of silent swallow (first_seen /
degree / marshal→500 / discarded rows); `public_key=?` index use;
canonical `PayloadADVERT`; `min()` builtin; documented cache-slice
immutability; mux ordering comment. New tests: scanReachRows decode,
3-byte token branch, non-advert first-hop guard, observer SNR
aggregation across rows, HTTP-level attribution (asserts non-zero
we_hear/they_hear), 400/404/blacklist/cache-hit.
**UI / a11y / Tufte:** in-map legend (tiers + thresholds); dropped the
colour+width double-encoding (constant width, colour-only); colour-blind
glyphs (●●●/●●/●) + tier title beside the bottleneck number; dark-theme
`--link-*`; lighter table (horizontal rules, sentence-case headers); map
built once + link layer updated in place on toggle (no flicker);
time-range no longer flashes a loader; `destroy()` generation guard;
statCard escaping; scoped `@media print` to `#nq-report`;
`fieldset/legend` + `for/id` toggles; `aria-pressed` / `aria-live` /
back-link `aria-label`; "distance (km)" + bottleneck tooltip + no-GPS
note; inline styles → CSS; decorative emoji removed.
**Docs:** api-spec documents the 5-min cache, 200k scan cap, and 400.
## Testing
- `cmd/server` full suite green; reach unit + endpoint + bench all pass.
- `eslint public/*.js` (no-undef) and the XSS-sink gate clean.
- E2E updated: request status checks + exact (non-tautological) toggle
assertions + hard map-render assert.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---
## TDD-history note (Kent Beck gate)
This branch carries production + tests together, not a fabricated
red→green sequence. That's deliberate: the branch was rebased onto
upstream and the intermediate SHAs were squashed, so reconstructing a
"failing-test-first" commit after the fact would be theatre, not
evidence — and rewriting history to stage it would be dishonest. The
behaviour is instead covered by a comprehensive, anti-tautological suite
(directional attribution edges, 3-byte token branch, non-advert
first-hop guard, observer SNR aggregation, HTTP-level attribution
asserting non-zero counts, scan-cap truncation, zero-reach 200-not-404,
companion mis-attribution, cache eviction). Requesting maintainer
acceptance of the work on test *substance* rather than commit
*choreography*; the net-new-UI exemption is not claimed for the server
endpoint.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: meshcore-bot <bot@meshcore>
Reverts #1625.
#1625 was merged before the round-1 reviews (Independent / Kent Beck /
Tufte) were addressed. Reverting to land it cleanly: a fresh PR will
re-add the feature with the perf pass, the backend correctness/safety +
test-coverage fixes, and the UI/a11y (Tufte) batch folded in, so it goes
through review in a single hardened state rather than as a string of
post-merge follow-ups.
No functional loss — the feature returns in the replacement PR.
## What
Adds a per-node **Reach** view that answers "how well does this specific
node hear, and get heard by, its neighbours?" — both as a standalone
page (`#/nodes/{pubkey}/reach`) and as a section on the node detail
page.
New endpoint: **`GET /api/nodes/{pubkey}/reach`**.
## What it measures
For the target node it derives, from raw `path_json` adjacency (a path
travels origin→observer, so in `[A,B]` B received A directly):
- **Directional link counts** per neighbour: `we_hear` (how often we
received them) vs `they_hear` (how often they received us).
- **Bidirectional / bottleneck**: a link is two-way stable when both
directions > 0; the weaker direction is the bottleneck and rates real
two-way reliability.
- **Importance**: neighbour degree + rank, relay-observation volume,
bidirectional-link count, direct-observer count.
- **Direct observers**: who received the node at 0 hops, with SNR.
Reliability rule: a neighbour is only attributed when its pubkey
**prefix is unique** at the path's byte length (collisions are skipped,
never misattributed).
## UI
- Standalone Reach page + node-detail section.
- Reusable bidirectional link map (OSM) with links coloured by
bottleneck.
- Incoming/outgoing toggles to isolate each direction.
## Naming note (deliberate, no collision)
This is distinct from the existing **per-observer reachability** in
topology analytics (`ReachNode` / `ObserverReach` / `perObserverReach`).
This PR adds its own `NodeReach*` response structs in a new
`node_reach.go` and a new `/api/nodes/{pubkey}/reach` route — there are
no symbol or route collisions (verified: `go build ./...` clean). Happy
to rename to disambiguate further (e.g. "Link Quality") if you'd prefer
to reserve "Reach" for the per-observer feature.
## Testing
- `cmd/server`: endpoint shape/404/limit-clamp + unit tests for token
derivation and directional attribution, plus a scan benchmark — all
pass.
- Frontend: helper tests + Reach-page E2E (`test-node-reach-e2e.js`),
standalone route + incoming/outgoing toggles.
- `go build ./...` and `eslint public/*.js` (no-undef) clean.
## Docs
Design spec, implementation plan, and the `GET
/api/nodes/{pubkey}/reach` API contract are included under `docs/`.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## Summary
- Adds configurable GPS polygon areas to `config.json`; nodes are
attributed to an area if their last-known position falls inside the
polygon
- New `Area: …` dropdown filter (matching the existing region filter
style) appears on all analytics, nodes, packets, map, and live screens
when areas are configured
- Backend resolves area membership with a 30s TTL cache; area filter
bypasses the 500-node cap on `/api/bulk-health` so all area nodes are
always returned
- Includes a polygon builder tool (`/area-map.html`) for drawing and
exporting area boundaries
## Changes
**Backend**
- `AreaEntry` type + `Areas` config field
- `GetNodePubkeysInArea` DB query + `resolveAreaNodes` (30s TTL,
`areaNodeMu` RWMutex)
- `PacketQuery.Area` + `filterPackets` polygon check
- `?area=` param propagated through all analytics, topology,
clock-health, and bulk-health routes
- `/api/config/areas` endpoint
**Frontend**
- `area-filter.js`: single-select dropdown, persists to localStorage,
cleans up stale keys on load
- Wired into analytics, nodes, packets, channels, map, and live pages
- Live map clears node markers on area change
**Docs & tools**
- `docs/user-guide/area-filter.md` — configuration and usage guide
- `docs/api-spec.md` — updated with new endpoint and `?area=` param
table
- `tools/area-map.html` — polygon builder for defining area boundaries
- Demo areas added to `config.example.json`
## Test plan
- [x] No areas configured → filter dropdown does not appear on any page
- [x] Areas configured → dropdown appears, "All" selected by default
- [x] Selecting an area filters nodes/packets/topology/map correctly
- [x] Selecting "All" restores unfiltered view
- [x] Selection persists across page reloads (localStorage)
- [x] Stale localStorage key (area removed from config) is cleared on
load
- [x] `/api/bulk-health?area=X` returns all nodes in area (no 500-node
cap)
- [x] `/api/config/areas` returns correct list
🤖 Generated with [Claude Code](https://claude.com/claude-code)
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
Co-authored-by: openclaw-bot <bot@openclaw.local>
@
## What this PR does
Implements region-scoped transport-route packet tracking with two
sub-features:
### Feature 1 — Scope statistics (`scope_name`)
- At ingest, transport-route packets (route_type 0/3) with Code1 !=
`0000` are HMAC-matched against configured `hashRegions` keys (mirroring
the `hashChannels` pattern). Matched region name (or `""` for unknown)
stored in new `transmissions.scope_name` column via migration
`scope_name_v1`.
- New `GET /api/scope-stats?window=` endpoint (1h/24h/7d, 30s
server-side TTL) returning transport totals, scoped/unscoped counts,
per-region breakdown, and time-series.
- New **Scopes** tab in Analytics with summary cards, per-region table,
and two-line SVG chart. Auto-refreshes every 60s.
### Feature 2 — Node default scope (`default_scope`)
- Per-node `default_scope` column on `nodes`/`inactive_nodes` (migration
`nodes_default_scope_v1`) tracks the most recently matched region for
each node, derived from transport-scoped ADVERT packets.
- `GET /api/nodes` response includes `default_scope` field when column
is present.
- Node detail panel displays the default scope badge.
- Async startup backfill (`BackfillDefaultScopeAsync`) populates the
column for nodes with pre-existing ADVERT data.
### Config
Add `hashRegions` to `config.json` (see `config.example.json`). One
entry per region name (with or without leading `#`).
@
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com>
Co-authored-by: openclaw-bot <bot@openclaw.local>
#191: Hash collision matrix now filters to role=repeater only (routing-relevant)
#192: expand=observations in /api/packets now returns full observation details (txToMap includes observations, stripped by default)
#193: /api/nodes/:pubkey/health uses in-memory PacketStore when available instead of slow SQL queries
#194: goRuntime (heapMB, sysMB, numGoroutine, numGC, gcPauseMs) restored in /api/perf response
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the exact response shape, query parameters, and type information
for every endpoint in server.js. This is the authoritative contract that
both Node.js and Go backends must conform to.
Covers:
- All 30+ REST endpoints with full JSON response schemas
- WebSocket message envelope and data shapes
- Shared object shapes (Packet, Observation, DecodedHeader, DecodedPath)
- Query parameter documentation with types and defaults
- Null rules, pagination conventions, error response format
- Frontend consumer matrix (which page reads which WS fields)
- Payload type and route type reference tables
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
