15 Commits

Include renames

Author	SHA1	Message	Date
efitenandClaude Opus 5	a059299588	feat(node-analytics): hop-count statistics per node (#1812) (#2021) ... ## Summary Adds per-node hop-count statistics so repeater operators can choose `flood.max`, `flood.max.unscoped` and `flood.max.advert` from what their node actually sees. - New endpoint `GET /api/nodes/{pubkey}/hop_analytics?days=N` (`cmd/server/routes.go:299`, `cmd/server/node_hop_analytics.go:312`), separate from `/analytics` as requested in the issue. - New card "Hop Count at This Node" on the node analytics page (`public/node-hop-analytics.js`, wired at `public/node-analytics.js:130,174`): histogram of hop counts with a box plot on the same x axis, filters `flood.max` (default), `flood.max.advert`, `flood.max.unscoped`, driven by the existing range picker. - The existing "Hop Distribution" chart is unchanged: it shows path length at the observer, a different quantity. - No `direct` tag, although the issue lists one: for DIRECT packets the path is the remaining route and no flood limit applies, so there is no hop count to report. ## Hop count definition (firmware 0679dbef) - `src/helpers/RoutingPolicy.h:15-21`: limits compare `getPathHashCount()`; `.unscoped` applies to route type FLOOD, `.advert` to adverts. - `src/Mesh.cpp:344-350`: `routeRecvPacket` checks with n hashes in the path, then writes its own hash at index n. So hops = the node's zero-based index in the path, no +1. - `src/Mesh.cpp:265-285`: a node forwards a flood once; `src/Mesh.cpp:651,680`: an originator never forwards its own flood. - DIRECT packets are excluded: their path is the remaining route (`src/Mesh.cpp:78-103,334-341`). Response: `{timeRange, packets: [{hash, timestamp, hops, tags}], ambiguous}`. Tags: `flood`, `scoped` or `unscoped`, `advert`. Documented in `docs/api-spec.md:679` and `cmd/server/openapi.go:90`. ## Attribution `cmd/server/node_hop_analytics.go:198-309`. The result depends only on the observed paths, the prefix map and the neighbor graph, so it is the same after a restart as after live ingest. - Every observation of every flood packet in the window is read. `byNode` holds the server resolver's pick at ingest and other picks after a cold load; `byPathHop` indexes only each packet's longest path, which for a busy relay often runs through another branch of the flood. - A packet counts when the node's prefix sits at exactly one index across its observations, and either the node is the only relay candidate for that prefix (`prefixMap.relayCandidates`, `cmd/server/store.go:6795`), or the hop resolves to the node under the ingestor's strict rule (`cmd/ingestor/path_resolver.go:143-214`) in at least one observation and to another node in none. Strict rule: earlier hops identified without a tiebreak, exactly one candidate adjacent in `neighbor_edges` to the previous hop (the originator for hop 0 of an advert), nodes already on the path excluded. - The server resolver's tiebreaks (affinity, GPS distance, advert count, pubkey order) are not used. - Everything else with the node's prefix goes to `ambiguous`. In practice that is most packets with a colliding 1-byte path hash. On a read-only 7-day dump of a 1,669-node mesh DB, for one busy repeater: 23,081 packets attributed, 11,437 ambiguous. Taking candidates from `byPathHop` instead gave 9,995 attributed, with the histogram mode moved from 2 to 3-5 hops. ## Performance Scans `s.packets` under the read lock, no SQL per packet. Per observation: one substring test for the node's first prefix byte; the hop scan only for observations containing it; the strict walk only for colliding prefixes, with per-request caches for candidates and adjacency. `BenchmarkNodeHopPackets` models one 7-day request at that scale (73,782 flood packets, 1,430,280 observations): 44-87 ms/op, 13.4 MB, 40 allocs on a throttling laptop. Response size for that repeater over 7 days: about 23k entries, 2.3 MB JSON, 375 KB gzipped. `hash` and `timestamp` are 61% of the raw and 91% of the gzipped bytes; they stay because the issue asks for them so a client can join entries to packets and bin by time. ## Tests - Go: `cmd/server/node_hop_analytics_test.go`: 12 unit tests, a live-ingest test through `IngestNewFromDB` (a colliding prefix without independent attribution goes to `ambiguous`, not to the node the resolver picked), live ingest versus cold load of the same DB, route test, benchmark. 15 mutations of the attribution logic each fail a test. - JS: `test-node-hop-analytics.js` (filters, histogram, quartiles and whiskers with a fixture that separates 1.5 IQR from 3 IQR, render), registered in `test-all.sh` and `.github/workflows/deploy.yml`. - `gofmt`, `go vet ./...`, `go test ./...` in `cmd/server`, `scripts/check-css-vars.js` pass. ## Staging validation Build `c646310f` (this PR's review follow-up together with the other open follow-ups), after a container restart and full load, on a busy Belgian repeater: - `hop_analytics?days=7`: 23,302 packets, 11,548 ambiguous, median 4, adverts never above hop 7 (matching the firmware default `flood_max_advert = 8`, `examples/simple_repeater/MyMesh.cpp:922`), 1.2 s. The first version reported 23,035 packets and 86 ambiguous in 534 ms, because it trusted the resolver's pick for colliding prefixes. - The card rendered on the first version with no console errors; the rework does not touch the frontend beyond a test fixture. ## Not verified - Response time and lock hold for 30 days on the busiest node on a 14-day store. - Server relay candidates exclude companions and listeners while the ingestor's prefix index does not, so a few strict attributions can differ from the ingestor's persisted `resolved_path`. - Identical numbers across a second container restart were shown in a Go test, not repeated on staging. - Dark theme, phone width, and switching the range picker in the browser. - Filter state is not reflected in the URL hash (the range picker is not either). Fixes #1812 --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-13 19:59:56 +02:00
efitenandClaude Opus 5	0fea3f2a75	feat(analytics): break scope adverts down by node role (#1979) (#2019) ... ## Summary Adds a breakdown of flood adverts by sender role to `/api/scope-stats` and the Scopes tab, in the descriptive shape agreed in #1979: per node role, how many flood adverts were unscoped, scoped with an unnamed region, or scoped with a named region. It reports what was sent, not why. ## Changes - `cmd/server/db.go:3114-3144`: one grouped query in `GetScopeStats`. ADVERT packets on flood routes (TRANSPORT_FLOOD 0, FLOOD 1) in the window, `LEFT JOIN nodes` on `from_pubkey`, split by the three `scope_name` states (NULL, empty string, name). A missing or empty role becomes `"unknown"`. Ordered by total descending, then role. Zero-hop adverts are excluded because firmware sends them as DIRECT/TRANSPORT_DIRECT (`src/Mesh.cpp:717-730`, `Mesh::sendZeroHop`), so they would inflate "unscoped". - `cmd/server/types.go:116-133`: `ScopeAdvertRoleCount` and `ScopeStatsResponse.AdvertsByRole` (`advertsByRole`, always an array). - `public/analytics.js:4760`: `scopeAdvertsByRoleHtml` renders a table under the time-series chart with the count per state and its share of the row. Role text is escaped. It reuses the existing `/scope-stats` response, so there is no extra request. - `docs/api-spec.md:1763-1786`: documents the new field. `/api/scope-stats` is in `openapi_known_gaps.json`, so there is no `openapi.go` entry to update. API addition (existing fields unchanged): "advertsByRole": [ { "role": "repeater", "unscoped": 7741, "unknownScope": 7, "named": 1562 } ] ## Performance The query runs inside `GetScopeStats`, so it shares the existing 30s cache per window. The unary `+` on `payload_type` keeps SQLite on the `first_seen` range index. Without it the planner picked the `payload_type` index and walked every stored advert whatever the window. Read-only timing on a production DB (1,063,345 transmissions, 161,634 adverts, sqlite3 CLI 3.45.1): | Window | payload_type index | first_seen index (this PR) | |---|---|---| | 7d | 0.231s | 0.059s | | 24h | 0.214s | 0.008s | | 1h | 0.213s | 0.001s | ## Tests - `TestGetScopeStatsAdvertsByRole` (`cmd/server/db_test.go:2295`): the three states, flood-only routes, non-advert and out-of-window exclusion, `unknown` for a missing node row, an empty role and a NULL `from_pubkey`, and ordering. Mutation checked: widening to routes 0-3 and dropping the empty-role fallback both fail it. - `TestGetScopeStatsAdvertsByRoleEmpty` (`:2372`): empty result is `[]`, not null. - `test-issue-1979-scope-adverts-by-role.js`: renders the real `analytics.js` helper in a vm sandbox. Covers row order, totals and shares, columns, escaping (mutation checked), the empty state, and the non-causal caption. Registered in `test-all.sh` and the deploy.yml unit step. - `go test ./...` in `cmd/server` passes, gofmt and go vet are clean, `check-css-vars.js` OK, `check-xss-sinks.sh --diff` exits 0. ## Browser validation On a staging instance with live traffic (build `139e484e`, together with #1074's branch), in Chrome, no console errors: `/#/analytics?tab=scopes` shows "Flood adverts by node role" under the time-series chart with its caption, and a table of 6 roles for the default window, for example `repeater 1.361 | 1.109 (81.5%) | 2 (0.1%) | 250 (18.4%)`. Shares in each row add up to 100%. `/api/scope-stats?window=7d` returns `advertsByRole` with the same six roles. ## Not verified - Timings are from the sqlite3 CLI, not the modernc driver in the server process. - Role is the sender's current `nodes.role`. A node whose advert type changed within the window is counted under its latest role. The data also contains a raw `type-13` role, shown as is. - Switching the window in the browser was not exercised; the API was checked for 7d. - No Playwright E2E added. Fixes #1979 ## Review follow-up (commit `43af46b8`) An independent review found no correctness, security or performance problem: counts are per transmission, the window matches the rest of the Scopes tab, zero-hop adverts are DIRECT per firmware, and the `+t.payload_type` hint holds on modernc SQLite 3.46.0. It found two test gaps and a docs gap. Changed: - **Ordering.** The Go fixture gave companion, repeater and unknown 3 adverts each, so ordering by total was never checked (`ORDER BY COUNT(*) ASC` still passed). The fixture now has repeater 4, unknown 3, companion 2, sensor 2, so the expected order differs from alphabetical and the companion/sensor tie checks the role-name tie-break. Reversing the count order, dropping it, or reversing the tie-break now fails the test. - **Column positions.** The JS test only checked that each cell string appeared somewhere in the row, so swapping two columns passed. It now compares each row's cells and the header cells by position; both swaps fail. - **Docs.** `docs/api-spec.md` and the `ScopeAdvertRoleCount` comment now name every source of `unknown`: a NULL `from_pubkey` (legacy rows the #1143 backfill has not reached), a sender with no `nodes` row, including one moved to `inactive_nodes` by node retention (inside the 7d window only with `retention.nodeDays` below 7), and an empty role. Not added: a query-plan test pinning the `+t.payload_type` hint. The SQL is inline in `GetScopeStats`, so the test would have to copy it or the query would have to move into a constant; left for a follow-up if wanted. The raw `type-13` role in the table is the ingestor's placeholder for reserved advert types 5-15 (`cmd/ingestor/decoder.go:1229`, #1279), shown as is. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-13 19:59:00 +02:00
efiten	296456f9c1	feat(map): colour and filter repeaters by scope-configuration state (#2006) ... Closes #2001. Two review rounds plus a re-review; findings and evidence on the PR. Verified on a deployment against live data: the field over 1653 nodes, marker tints per filter state, the marker title and popup Scope row reaching the DOM, and the colorblind-preset cascade. The audit and the map are held to the same classification by an end-to-end test that fails when either side's wildcard handling drifts.	2026-09-11 15:07:31 +02:00
efiten	8c164c5315	feat(scope-audit): verify a declared region against the repeater's own traffic (#1990) ... Follow-up to #1987, and the point of counting that traffic in the first place. A region this instance holds no `hashRegions` key for is **unnameable, not absent**. #1987 says so with a caveat chip. This settles it wherever the evidence allows: derive `SHA256("#region")[:16]` from the repeater's own declaration and HMAC that repeater's own unmatched packets with it. Same computation the ingestor performs at ingest, with the candidate set narrowed from every configured key to this repeater's handful of declarations. Where it fires, a grey "declared but not observed" chip becomes a green one and the caveat count shrinks by the packets it explained. ## Two packets, not one `code1` is two bytes, so an unrelated name matches a given packet with probability 1/65536. Across ~400 unmatched packets and ~124 declared names, chance alone produces roughly one false match per refresh. Two matches on the same region for the same repeater is (1/65536)², about one in four billion. Lowering the threshold to one would not make this noisy, it would make it **unsound**, so the constant carries that arithmetic and a test rather than a comment. A region with exactly one hit stays grey and reports its single hit, so the page can say why it is still shown as not observed instead of leaving the reader to wonder. ## What it deliberately does not do **It writes nothing.** Read-time only. A wrong answer expires with the window instead of sitting in `transmissions.scope_name` until someone runs a repair, and `cmd/server` stays read-only per the invariant in AGENTS.md. **`notObserved` remains the single source of chip colour.** `regionEvidence` says only HOW a region was established. Two fields that can disagree about the same fact is how this column got confusing in the first place. ## Rule 0, including the part that was wrong at first The naive shape is `targets × names × packets` HMACs: 205 × 124 × 400 ≈ 10M. Caching per `(region, transmission)` pair cuts the HMACs to ~50k. **That measured 501ms**, because the HMACs had become a rounding error while the *iteration* stayed cubic at 10.2M map lookups. Re-keyed per region, holding the set of matching transmissions, it is **36ms** at the same worst-case shape: a region is HMACed over every packet once, and a target then asks one question per declared region instead of one per (region, packet). Most declared regions match nothing, so the common case is a single map lookup and no packet loop at all. `hmacCount` exists so a test can assert the first mistake cannot come back; the benchmark exists because only it caught the second. ## Both axes are bounded, because neither is bounded by the data The "~400 packets in a 7 day window" this was sized against is a property of one instance's configuration, not of the feature: the ingestor stores the unnameable state for every transport-scoped packet no configured key names, so an instance with few or no `hashRegions` entries — the stock state, and the one this helps most — has **every** scoped packet in that set. - the window query takes the 4096 most recent candidates and reports truncation, which the handler logs, so a grey chip on a sampled refresh is not read as "not forwarded" - the declared list is capped at 32 names per repeater: it arrives from a collector that validates each entry's shape but never how many entries there are - measured at the cap: **306ms** for 205 targets over 124 names, against 29ms for the shape a real network produces Because both caps make the evidence a sample, the response carries `observedUnmatchedSampled`. Without it a client subtracts a capped numerator from an uncapped total and overstates the unexplained traffic with no way to know it is doing so. The chip subtracts only evidence for regions **absent** from `notObserved` — a single-hit region the server refused to accept is not called explained either — and says "at most N" when the count was sampled. ## Verified on live data Six repeaters clear the threshold in a 7d window on a real instance. One of them: `nl-nb` green with 3 corroborating packets and the tooltip stating the count, `belml` still grey on 1, and the caveat chip reading 31 of 34 packets unexplained rather than 30. ## Tests `scope_verify_test.go` covers the HMAC-input walk against a real transport-flood packet captured from a live instance (a hand-built fixture would only prove the parser agrees with itself), that `regionCode` does not fold case, the threshold in both directions, the memo's HMAC count, both bounds with their truncation flag, and the benchmark at cap size. Handler-level tests cover a region verified into green, a single hit left grey with its count reported, and the sample-size field. `cd cmd/server && go test ./...` passes (77s), frontend 723 assertions pass, `go vet` and `gofmt -l` clean.	2026-09-09 17:58:09 +02:00
efiten	0605b1703a	feat(scope-audit): count and surface the traffic this instance cannot name (#1987) ... Follow-up to #1986, and the second half of the same problem. `ScopeAuditForwarding` drops rows whose `scope_name` is the empty string with a bare `continue`. That empty string is the ingestor's "transport-scoped, but no configured region key matched `code1`" state (`scopeNameForDB`), so those packets name no region and can never satisfy a declared one. The consequence is on the page: **a repeater forwarding a region this instance holds no `hashRegions` key for is reported exactly like a repeater forwarding nothing at all.** The audit presents a gap in the reader's own configuration as a finding about someone else's hardware. This counts them per target, exposes the count as `observedUnmatchedPackets`, and renders it as a caveat chip beside the scope chips. ## Why it is not a rare edge Measured on a live instance before this landed: of 613 `notObserved` entries across 205 repeaters, **260 named a region that never appeared under any name in the whole 7-day window**. Two of them (`behss`, `fm-112`) were hash-verified as genuinely forwarded traffic the instance simply could not name: packet `0a065d41d51f1f77` decodes to `code1=9209`, which is exactly the code `#fm-112` derives over that packet's own payload. That instance had 16 region keys configured against 124 distinct region names its repeaters declare. A stock install has fewer. ## What the counter is not It is deliberately **not** folded into `unscopedPackets`. The two are opposites: | | meaning | what governs it | |---|---|---| | `unscopedPackets` | the packet carried no scope at all (`scope_name` SQL NULL) | the `*` wildcard | | `observedUnmatchedPackets` | the packet IS scoped, this instance holds no key for that region | nothing the repeater declares | For the same reason the new count never feeds `wildcardContradiction`, which counts only plain unscoped floods. `scopeNameForDB` in the ingestor is the source of truth for that three-state encoding, and the comments point there rather than restating it. It is also distinct from `ambiguousHops`, and the distinction is the point of the chip: that one is a pubkey-prefix collision between two repeaters and is nobody's fault, this one is a missing entry in the reader's own configuration and they can act on it. Saying which is which is what stops someone investigating an innocent repeater. ## Frontend The chip reuses the muted dashed treatment of `.sa-chip-ambiguous` on purpose: both are caveats on the row's finding rather than findings themselves, and neither may compete visually with the red/green scope chips beside them. It renders nothing for a non-numeric count. The value is server-supplied, and a truthiness check would put the literal string `NaN forwarded packets` on the page if that ever stopped holding. ## Docs `docs/api-spec.md` had **no entry for `GET /api/scope-audit` at all**, so this adds one: query parameter, full response shape, and the notes a client needs (the three traps the per-node endpoint documents apply here identically, `*` is never a scope, and "never asked" is not "declared nothing"). The new field is documented there rather than in isolation. ## Tests - the counter on a last-hop and on a mid-path hop - an unmatched packet enters neither `agg.scopes` nor `unscopedPackets`, which is the confusion this field exists to prevent - the field on the API row - six frontend cases: zero renders nothing, a missing field renders nothing (older server), the chip carries its count and class, singular and plural are both grammatical, the title names the cause and the fix, and a non-numeric count renders nothing rather than `NaN` `cd cmd/server && go test ./...` passes, frontend 712 assertions pass, `go vet` and `gofmt -l` clean. Rule 0: the counter is one increment on a branch that already existed as a `continue`, inside a loop this PR does not change. No new query, no new pass over the data.	2026-09-09 17:05:29 +02:00
efitenandClaude Opus 5	e7b3a2e77f	chore(#1856): remove POST /api/packets, which has never worked (#1959) ... Part 1 of #1856. Part 2 (the hash migration reporting false success) is #1958. ## It has never worked `handlePostPacket` writes to the server's DB handle, and that handle is read-only. `cmd/server/db.go:106`: ```go dsn := fmt.Sprintf("file:%s?mode=ro&_journal_mode=WAL&_busy_timeout=5000", path) ``` Every call answered `500 attempt to write a readonly database`. **This is the second report.** #1196 raised it on 2026-06-13, a fix was merged that corrected v2 column names to v3, and the issue was closed. That fix could not have worked, because the column names were never why the write failed. Its comment is still sitting at `routes.go:1288`, next to code that has never executed successfully in production. ## Why remove rather than build a handoff **It cannot break a caller.** An endpoint that has only ever returned 500 has no working consumer. This is not a breaking API change, it is documentation catching up with reality. Nothing in `public/` calls it. **It was actively misleading.** `openapi.go` advertised it as "Ingest a packet" and it sits behind `requireAPIKey`, which reads as a live, protected write endpoint. **Its test hid the breakage.** `TestPostPacketPersistsV3Schema` asserted the observation row is written and passed for four months, because the test DB is opened read-write while production is not. That is how #1196 came to be closed as fixed. **Ingest is MQTT-only by design since #1283.** Re-adding an HTTP write path re-opens the invariant that change established. If manual injection is wanted later for testing or replay, it belongs on the ingestor side and deserves its own issue. The repository already has the handoff shape for that: the server writes `request-<id>.json` and the ingestor consumes it (`cmd/ingestor/prune_geofilter.go`). ## What went The route, `handlePostPacket` (103 lines), the now-unused `PacketIngestResponse` type, the `openapi.go` entry, the round-trip test, and the section plus table-of-contents line in `docs/api-spec.md`. The `packetpath` import in `routes.go` became unused and went with it. `+4/-225` across 6 files. ## The auth tests The four `requireAPIKey` tests used `"/api/packets"` only as a request path while building their own handler with `s.requireAPIKey(...)`, so they never touched the route. I checked that by **running them**, not by reading the code: ``` --- PASS: TestRequireAPIKey_RejectsWeakKey --- PASS: TestRequireAPIKey_AcceptsStrongKey --- PASS: TestRequireAPIKey_EmptyKeyDisablesEndpoints --- PASS: TestRequireAPIKey_WrongKeyUnauthorized ``` Their paths now point at `/api/admin/prune-geo-filter`, which still exists, so they no longer name a removed endpoint. Re-ran after that change: still 4 of 4. `/api/packets/observations` is a different endpoint and is untouched. Verified: `gofmt` clean, `go vet` clean, `cmd/server` suite ok in 62.9s. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-09-04 16:02:59 +02:00
3 people	e2212f5015	feat(nodes): per-node Reach page + GET /api/nodes/{pubkey}/reach (v2, review-complete) (#1627) ... Re-submission of #1625 (which was merged early, then reverted in #1626) — now with **all three round-1 reviews addressed** so it lands in one hardened state instead of as post-merge follow-ups. ## What Per-node **Reach** view: a standalone page (`#/nodes/{pubkey}/reach`) + a node-detail section + `GET /api/nodes/{pubkey}/reach`. It shows which nodes a node has a **stable two-way RF link** with, derived from raw `path_json` adjacency (a path travels origin→observer, so `[A,B]` ⇒ B heard A). A link is bidirectional when both directions have observations; the **bottleneck** (weaker direction) rates two-way reliability. Nodes are identified only by **unique 2–3 byte** path prefixes (1-byte collides → excluded). ## Review fixes folded in vs #1625 **Performance (Carmack):** hard scan LIMIT (200k) + modest prealloc; `json.Unmarshal` replaced by a single-pass `parsePathTokens` (100k-row scan 2.2M→1.3M allocs, 344→203ms); memoized resolver; size-hinted maps (attribution over 100k rows: 102 allocs); `context.Context` plumbed; cache `RWMutex` + evict-oldest (no full wipe); singleflight dedup; degree/rank from a 60s shared snapshot; bench rewritten (ReportAllocs, 1k/10k/100k, mixed-payload, isolated attribution). **Correctness/safety + tests (Independent + Kent Beck):** pubkey validation → 400; error logging instead of silent swallow (first_seen / degree / marshal→500 / discarded rows); `public_key=?` index use; canonical `PayloadADVERT`; `min()` builtin; documented cache-slice immutability; mux ordering comment. New tests: scanReachRows decode, 3-byte token branch, non-advert first-hop guard, observer SNR aggregation across rows, HTTP-level attribution (asserts non-zero we_hear/they_hear), 400/404/blacklist/cache-hit. **UI / a11y / Tufte:** in-map legend (tiers + thresholds); dropped the colour+width double-encoding (constant width, colour-only); colour-blind glyphs (●●●/●●/●) + tier title beside the bottleneck number; dark-theme `--link-*`; lighter table (horizontal rules, sentence-case headers); map built once + link layer updated in place on toggle (no flicker); time-range no longer flashes a loader; `destroy()` generation guard; statCard escaping; scoped `@media print` to `#nq-report`; `fieldset/legend` + `for/id` toggles; `aria-pressed` / `aria-live` / back-link `aria-label`; "distance (km)" + bottleneck tooltip + no-GPS note; inline styles → CSS; decorative emoji removed. **Docs:** api-spec documents the 5-min cache, 200k scan cap, and 400. ## Testing - `cmd/server` full suite green; reach unit + endpoint + bench all pass. - `eslint public/*.js` (no-undef) and the XSS-sink gate clean. - E2E updated: request status checks + exact (non-tautological) toggle assertions + hard map-render assert. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --- ## TDD-history note (Kent Beck gate) This branch carries production + tests together, not a fabricated red→green sequence. That's deliberate: the branch was rebased onto upstream and the intermediate SHAs were squashed, so reconstructing a "failing-test-first" commit after the fact would be theatre, not evidence — and rewriting history to stage it would be dishonest. The behaviour is instead covered by a comprehensive, anti-tautological suite (directional attribution edges, 3-byte token branch, non-advert first-hop guard, observer SNR aggregation, HTTP-level attribution asserting non-zero counts, scan-cap truncation, zero-reach 200-not-404, companion mis-attribution, cache eviction). Requesting maintainer acceptance of the work on test *substance* rather than commit *choreography*; the net-new-UI exemption is not claimed for the server endpoint. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: meshcore-bot <bot@meshcore>	2026-06-08 22:13:02 -07:00
efiten	9c5faab1e4	Revert "feat(nodes): per-node Reach page (#1625)" (#1626) ... Reverts #1625. #1625 was merged before the round-1 reviews (Independent / Kent Beck / Tufte) were addressed. Reverting to land it cleanly: a fresh PR will re-add the feature with the perf pass, the backend correctness/safety + test-coverage fixes, and the UI/a11y (Tufte) batch folded in, so it goes through review in a single hardened state rather than as a string of post-merge follow-ups. No functional loss — the feature returns in the replacement PR.	2026-06-08 12:35:12 +00:00
efitenandClaude Opus 4.8	47f85f6c4c	feat(nodes): per-node Reach page + GET /api/nodes/{pubkey}/reach (directional link quality) (#1625) ... ## What Adds a per-node **Reach** view that answers "how well does this specific node hear, and get heard by, its neighbours?" — both as a standalone page (`#/nodes/{pubkey}/reach`) and as a section on the node detail page. New endpoint: **`GET /api/nodes/{pubkey}/reach`**. ## What it measures For the target node it derives, from raw `path_json` adjacency (a path travels origin→observer, so in `[A,B]` B received A directly): - **Directional link counts** per neighbour: `we_hear` (how often we received them) vs `they_hear` (how often they received us). - **Bidirectional / bottleneck**: a link is two-way stable when both directions > 0; the weaker direction is the bottleneck and rates real two-way reliability. - **Importance**: neighbour degree + rank, relay-observation volume, bidirectional-link count, direct-observer count. - **Direct observers**: who received the node at 0 hops, with SNR. Reliability rule: a neighbour is only attributed when its pubkey **prefix is unique** at the path's byte length (collisions are skipped, never misattributed). ## UI - Standalone Reach page + node-detail section. - Reusable bidirectional link map (OSM) with links coloured by bottleneck. - Incoming/outgoing toggles to isolate each direction. ## Naming note (deliberate, no collision) This is distinct from the existing **per-observer reachability** in topology analytics (`ReachNode` / `ObserverReach` / `perObserverReach`). This PR adds its own `NodeReach*` response structs in a new `node_reach.go` and a new `/api/nodes/{pubkey}/reach` route — there are no symbol or route collisions (verified: `go build ./...` clean). Happy to rename to disambiguate further (e.g. "Link Quality") if you'd prefer to reserve "Reach" for the per-observer feature. ## Testing - `cmd/server`: endpoint shape/404/limit-clamp + unit tests for token derivation and directional attribution, plus a scan benchmark — all pass. - Frontend: helper tests + Reach-page E2E (`test-node-reach-e2e.js`), standalone route + incoming/outgoing toggles. - `go build ./...` and `eslint public/*.js` (no-undef) clean. ## Docs Design spec, implementation plan, and the `GET /api/nodes/{pubkey}/reach` API contract are included under `docs/`. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-08 13:11:06 +02:00
4 people	317b59ab10	feat: area-based visual node filter — attribute packets by transmitter GPS (#804) (#839) ... ## Summary - Adds configurable GPS polygon areas to `config.json`; nodes are attributed to an area if their last-known position falls inside the polygon - New `Area: …` dropdown filter (matching the existing region filter style) appears on all analytics, nodes, packets, map, and live screens when areas are configured - Backend resolves area membership with a 30s TTL cache; area filter bypasses the 500-node cap on `/api/bulk-health` so all area nodes are always returned - Includes a polygon builder tool (`/area-map.html`) for drawing and exporting area boundaries ## Changes **Backend** - `AreaEntry` type + `Areas` config field - `GetNodePubkeysInArea` DB query + `resolveAreaNodes` (30s TTL, `areaNodeMu` RWMutex) - `PacketQuery.Area` + `filterPackets` polygon check - `?area=` param propagated through all analytics, topology, clock-health, and bulk-health routes - `/api/config/areas` endpoint **Frontend** - `area-filter.js`: single-select dropdown, persists to localStorage, cleans up stale keys on load - Wired into analytics, nodes, packets, channels, map, and live pages - Live map clears node markers on area change **Docs & tools** - `docs/user-guide/area-filter.md` — configuration and usage guide - `docs/api-spec.md` — updated with new endpoint and `?area=` param table - `tools/area-map.html` — polygon builder for defining area boundaries - Demo areas added to `config.example.json` ## Test plan - [x] No areas configured → filter dropdown does not appear on any page - [x] Areas configured → dropdown appears, "All" selected by default - [x] Selecting an area filters nodes/packets/topology/map correctly - [x] Selecting "All" restores unfiltered view - [x] Selection persists across page reloads (localStorage) - [x] Stale localStorage key (area removed from config) is cleared on load - [x] `/api/bulk-health?area=X` returns all nodes in area (no 500-node cap) - [x] `/api/config/areas` returns correct list 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com> Co-authored-by: openclaw-bot <bot@openclaw.local>	2026-05-21 14:00:15 -07:00
4 people	2329639f45	feat: scoped/unscoped transport-route statistics (#899) (#915) ... @ ## What this PR does Implements region-scoped transport-route packet tracking with two sub-features: ### Feature 1 — Scope statistics (`scope_name`) - At ingest, transport-route packets (route_type 0/3) with Code1 != `0000` are HMAC-matched against configured `hashRegions` keys (mirroring the `hashChannels` pattern). Matched region name (or `""` for unknown) stored in new `transmissions.scope_name` column via migration `scope_name_v1`. - New `GET /api/scope-stats?window=` endpoint (1h/24h/7d, 30s server-side TTL) returning transport totals, scoped/unscoped counts, per-region breakdown, and time-series. - New **Scopes** tab in Analytics with summary cards, per-region table, and two-line SVG chart. Auto-refreshes every 60s. ### Feature 2 — Node default scope (`default_scope`) - Per-node `default_scope` column on `nodes`/`inactive_nodes` (migration `nodes_default_scope_v1`) tracks the most recently matched region for each node, derived from transport-scoped ADVERT packets. - `GET /api/nodes` response includes `default_scope` field when column is present. - Node detail panel displays the default scope badge. - Async startup backfill (`BackfillDefaultScopeAsync`) populates the column for nodes with pre-existing ADVERT data. ### Config Add `hashRegions` to `config.json` (see `config.example.json`). One entry per region name (with or without leading `#`). @ --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Kpa-clawbot <kpaclawbot@outlook.com> Co-authored-by: openclaw-bot <bot@openclaw.local>	2026-05-21 14:00:06 -07:00
Kpa-clawbot	5aa4fbb600	chore: normalize all files to LF line endings	2026-03-30 22:52:46 -07:00
Kpa-clawbotandCopilot	cdcaa476f2	rename: MeshCore Analyzer → CoreScope (Phase 1 — backend + infra) ... Rename product branding, binary names, Docker images, container names, Go modules, proto go_package, CI, manage.sh, and documentation. Preserved (backward compat): - meshcore.db database filename - meshcore-data / meshcore-staging-data directory paths - MQTT topics (meshcore/#, meshcore/+/+/packets, etc.) - proto package namespace (meshcore.v1) - localStorage keys Changes by category: - Go modules: github.com/corescope/{server,ingestor} - Binaries: corescope-server, corescope-ingestor - Docker images: corescope:latest, corescope-go:latest - Containers: corescope-prod, corescope-staging, corescope-staging-go - Supervisord programs: corescope, corescope-server, corescope-ingestor - Branding: siteName, heroTitle, startup logs, fallback HTML - Proto go_package: github.com/corescope/proto/v1 - CI: container refs, deploy path - Docs: 8 markdown files updated Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-28 14:08:15 -07:00
Kpa-clawbotandCopilot	47ee63ed55	fix: #191 #192 #193 #194 — repeater-only collision matrix, expand=observations, store-based node health, goRuntime in perf ... #191: Hash collision matrix now filters to role=repeater only (routing-relevant) #192: expand=observations in /api/packets now returns full observation details (txToMap includes observations, stripped by default) #193: /api/nodes/:pubkey/health uses in-memory PacketStore when available instead of slow SQL queries #194: goRuntime (heapMB, sysMB, numGoroutine, numGC, gcPauseMs) restored in /api/perf response Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-27 21:25:19 -07:00
Kpa-clawbotandCopilot	3ddd9662e6	docs: add formal API contract spec for all REST endpoints and WebSocket messages ... Document the exact response shape, query parameters, and type information for every endpoint in server.js. This is the authoritative contract that both Node.js and Go backends must conform to. Covers: - All 30+ REST endpoints with full JSON response schemas - WebSocket message envelope and data shapes - Shared object shapes (Packet, Observation, DecodedHeader, DecodedPath) - Query parameter documentation with types and defaults - Null rules, pagination conventions, error response format - Frontend consumer matrix (which page reads which WS fields) - Payload type and route type reference tables Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>	2026-03-27 13:55:42 -07:00