mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-11 21:37:36 +00:00
Adam
6d56f55168
Red commit:cd546b23(fails on assertions, e.g. `region "SAT": got [pk_aus pk_edge pk_far], want [pk_edge pk_far]`) Red commit (fewest-hops fallback):a66a7f36(fails on assertions, e.g. `region "AUS": got [pk_cap pk_gulf pk_near pk_tie], want [pk_cap]`) Red commits (review round 1):d4154ee1(`region_placement` missing),862a4510(`zero: RegionFallbackMaxHops() = 3, want 0`),16dcd8dd(`pk_mixed:{fewest_hops [AUS] hops=0}`, want `[SAT] hops=2`),95f2aca4(`pk_late not placed after the startup load finished`) Fixes #1879 ## What this does `/api/nodes?region=` put a node in every region where any observer heard any of its adverts. Adverts flood, so a node showed up in every region its adverts got relayed into, not just where it actually is. Now, as agreed in the issue: - **Nodes heard zero-hop are placed by their home regions.** Same rule as #804: a zero-hop DIRECT advert can't have been relayed, so the observer that heard it is in RF range. This is the tally `computeNodeHomeRegions()` already does, pulled out into `nodeZeroHopRegionCounts()` so analytics and the filter share one definition. - **It's a set, not a winner.** A node heard zero-hop in two regions is in both. So per-region counts don't add up to the node total anymore, on purpose. - **Everything else goes by fewest hops.** A node never heard zero-hop is in the region(s) whose observers heard it in the fewest hops, and in no region when even that is more than `regionFallback.maxHops` (config, default 3); it still shows unfiltered. The old fallback (every region that heard it at all) turned out to be most of the leak: see the numbers at the bottom. Hearings with no `path_json` (legacy NULL rows) are unknown, not zero hops, and are skipped. - **Only nodes the last recompute didn't cover** (first advert newer than it, or no hearing with a path) still use the old observer check. - **`region_placement` on every node row** (list and detail) says which rule placed it, so an operator can tell the four outcomes apart: ```json {"basis": "zero_hop", "regions": ["AUS"]} {"basis": "fewest_hops", "regions": ["HOU", "SAT"], "hops": 2} {"basis": "none", "hops": 5} {"basis": "observer_scan"} ``` It's a named struct (`NodeRegionPlacement`), documented as its own OpenAPI component. It replaces the earlier `home_regions` field. ## How - `node_home_regions.go`: a recomputer in the `bridge_recomputer.go` shape (sync prewarm, ticker on `AnalyticsDefaultRecomputeInterval()`, atomic snapshot, panics swallowed), started from `main.go` next to the bridge one. It also recomputes once when `StartupLoadDone()` closes, like `recomputeWhenLoaded` does for the analytics recomputers, because the prewarm only sees the first chunk. - Each recompute builds one snapshot, pubkey → `NodeRegionPlacement`, from `nodeZeroHopRegionCounts` plus `nodeFewestHopsByRegion`. Hop counts come from `path_json` without decoding (`pathHopCount`: hop hashes hold no commas). - `RegionNodePubkeys` loads the snapshot (atomic, no lock), skips placed nodes in the observer scan, then adds the ones whose regions intersect the request. The 30 s region cache is unchanged. - `computeNodeHomeRegions()` keeps its plurality output for analytics; only the counting moved. The #804 / #1025 tests are unchanged and pass. - No new `map[string]interface{}`: the node rows set `region_placement` to the struct, the OpenAPI component uses a small typed `openAPISchema`, and the old `enrichNodeHomeRegions` helper is gone. Touched files are at the upstream count. ## Performance - Request path: one atomic load plus map lookups. `RegionNodePubkeys` adds one pass over the snapshot per cache miss, so O(nodes), well under the existing advert scan it sits next to. `BenchmarkRegionNodePubkeys` stays ~19 ms. - Recompute, off the request path, once per interval under `s.mu.RLock`: `BenchmarkRecomputeNodeHomeRegionSets` times the whole function at 220k adverts × 8 observations over 0-7 hop paths (one in eight zero-hop direct) plus 1M channel messages: **~54 ms/op** on an Apple M5 (Docker, linux/arm64), of which the fewest-hops pass (`BenchmarkNodeFewestHopsByRegion`) is ~40 ms. - That's after two small speedups in3dc3cb7d: `nodeZeroHopRegionCounts` walks `byPayloadType[ADVERT]` instead of every packet (same counts), and the fewest-hops pass looks up a node's map once per advert instead of once per observation. Before them the whole recompute was ~72 ms on the same machine. ## Tests - `TestRegionNodePubkeysUsesZeroHopHomeRegions`: a node whose relayed advert reached SAT stays out of SAT; the boundary node is in both; the never-zero-hop node is placed by the fallback. - `TestBoundaryNodeStaysInEachHomeRegion`: the two-region case on its own, so a later refactor can't quietly collapse it back to one winner. - `TestNodeRegionPlacementsZeroHop`: the snapshot for zero-hop and fallback nodes. - `TestHandleNodesRegionPlacement`: `/api/nodes` returns `zero_hop`, `none` and `observer_scan` placements for seeded nodes. - `TestRegionFallbackUsesFewestHops`: fewest hops wins, ties keep both, exactly the cap is placed, past it is in no region (and the observer scan doesn't re-add it), a closer later advert counts. - `TestRegionFallbackMaxHopsConfig`: the config accessor (unset or negative keeps 3, 0 allowed, parsed from JSON) and the recompute honoring it. - `TestRegionFallbackIgnoresUnknownPaths`: an empty `path_json` doesn't count as zero hops; `[]` still does. - `TestRegionFilterObserverScanForNodesNewerThanRecompute`: the observer-scan path for a node newer than the snapshot. - `TestRegionFilterListingEveryRegionOmitsUnplacedNodes`: naming every region explicitly still leaves out a node in no region. - `TestNodeRegionsRecomputeWhenStartupLoadDone`: placements cover a later chunk once the startup load is done. - `cd cmd/server && go test ./...` and `cd cmd/ingestor && go test ./...` pass (`CGO_ENABLED=1`). `gofmt` and `go vet` are clean. No UI change; the API change is covered by the handler test above. ## Config - `regionFallback.maxHops` (default 3), documented in `config.example.json`. - Per AGENTS.md rule 8, this should be exposed in the customizer in a later milestone. Not done here. ## Not in this PR - **Transport-direct zero-hop adverts aren't counted as zero-hop** (pre-existing, `nodeZeroHopRegionCounts`): for `ROUTE_TYPE_TRANSPORT_DIRECT`, `RawHex[2:4]` is the first transport-code byte, not `path_len`. The fallback still places such nodes (their `path_json` is `[]`), but as `fewest_hops`, not `zero_hop`. I'll send that as its own PR since it changes #804 analytics too. - **Memory window.** The answer only covers adverts still in the packet store, so quiet nodes fall back. That's the long-tail point from the thread. A `node_regions` table kept by the ingestor (the `UpdateNodeDefaultScope` pattern) would remove the window; happy to do that as a follow-up if you want it. - **One-off long-range hearings.** A single zero-hop hearing during a ducting event still adds that region. On our deployment we run the same rule with a 20% share threshold per region (ignore a region holding under a fifth of a node's direct hearings), which drops those. Easy to add here if you'd like it. - **Flood adverts heard with an empty path** are direct evidence too, and much more common than zero-hop DIRECT adverts. Left out of the zero-hop rule to keep one definition with #804; the fallback places them at 0 hops. - **Moving #804 / #1025 analytics to the set version**, as you suggested, after this lands. **Fallback numbers.** On our network only ~27% of nodes are ever heard directly, so the fallback decides most of the filter. With "every region that heard it", our San Antonio filter showed 241 nodes whose GPS puts them outside SAT against 149 inside, and Austin picked up Louisiana and Gulf Coast nodes heard 3-10 hops out. We've been running the fewest-hops/cap-3 rule in production since 2026-10-14; nodes with GPS outside the selected region went SAT 241 → 7, DFW 76 → 7, HOU 47 → 2, AUS 226 → 48. About 240 nodes are in no region (they still show unfiltered). For context, on our network (Texas, ~1,500 nodes) the same idea with the broader "heard directly" rule took San Antonio from 543 nodes to 482. The 61 removed were all Austin-area nodes whose adverts get relayed south, and only 3 nodes statewide are heard directly in two regions.