mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-11-11 11:57:57 +00:00
CoderNemesisandClaude Opus 5.5
e55c95db24
## Summary On meshes with 1-byte path hashes, most hop prefixes are shared by several repeaters. The packet detail pane currently resolves each hop on its own and stores the result by prefix. Two things go wrong as a result: - A prefix that appears twice in one path resolves to the same node both times. - Nothing checks that the chosen hops form a path a packet could actually take. This PR adds `HopResolver.inferPath()`, which infers the most likely node behind every hop of **one observation's path**, scoring the path as a whole and anchoring it on the observer that heard it. The packet detail pane and the map route view now use it, so both show the same nodes for the selected observation (`?obs=`). ## The problem, on a real packet Packet `57da37d1636f895a`, observation `1388126`: a 46-hop flood with 1-byte hashes. The server's `resolved_path` resolved 1 hop of 46, so the client resolver did the rest. Eight prefixes appear twice in the path. | | Current resolver | `inferPath()` | |---|---|---| | Hops longer than 120 km | 13 (21 longer than 60 km, two over 950 km) | 0 | | Distance from the last hop to the observer | 59 km | 0.1 km | | Hops reported as unlisted (no listed node fits) | 0, every hop guessed | 13 | | Agreement on the shared path prefix across the packet's 23 observations | — | 97.8% | The current resolver picks badly for three reasons: 1. **Results are keyed by prefix.** Hop 0 and hop 45 share a prefix, so both show the node near the start of the path, 59 km from the observer. 2. **Picks are greedy.** Each hop looks only at its nearest resolved neighbours, so nothing penalises a 1,100 km jump. 3. **The detail pane read the wrong `resolved_path`.** It used the packet row's, not the selected observation's, and when the lengths matched it never fell back for the hops that were `null`. ## Firmware basis The model relies on two facts from `firmware/src/Mesh.cpp`: - **The last hop is the one the observer heard.** `Mesh::routeRecvPacket` appends the relaying node's own hash to the end of a flood path. - **A node appears at most once in a flood path.** A node forwards a given packet once: `_tables->wasSeen()` / `markSeen()`. ## How the model works Each hop is either one of the **located** repeaters or room servers sharing its prefix, or **unlisted**: a node missing from the node list, or one without GPS. With 1-byte prefixes, unlisted is common, and naming it beats drawing a 300 km hop. | Term | Cost | |---|---| | A hop between two located nodes | distance / 50 km; past 100 km per hop, the cost rises 4× faster | | A hop with a neighbour-graph link | the distance cost × 0.5 | | An unlisted hop | 1.6, flat (about one 80 km hop) | | The last hop to the observer | the same distance cost; falls back to the observer's IATA region centre (with 50 km slack) when the observer reports no GPS or 0,0 | | The first hop to the sender | the same distance cost, when the sender's position is known (e.g. an ADVERT) | - **Search:** dynamic programming over (hop, candidate) finds the cheapest sequence. When the best path uses a node twice, the node is banned from whichever position costs less to give up. - **Confidence:** each hop's confidence comes from the min-marginal costs of its alternatives, put through a softmax. A node the path already uses at another hop is not counted as an alternative. - **Server hint:** the observation's own `resolved_path` restricts a hop to the named node, but that hop can still be reported unlisted if the node doesn't fit. The tuning values live in `PATH_MODEL` in `hop-resolver.js`, with a TODO to expose them in the customizer (AGENTS rule 8). ## What changes in the UI **Packet detail pane** (`packets.js`): - The Path row, the cross-observer aggregate row and the byte breakdown render hops **by position**, from the inferred entries. - The detail pane no longer writes into the list's shared per-prefix cache (`hopNameCache`), so it can't change what the packets list shows. **Hop display** (`hop-display.js`): | Hop state | Display | |---|---| | Confident (≥ 70%) | The node's name; the tooltip says `NN% likely` and how many nodes share the prefix | | A guess (< 70%) | Dashed underline, plus a candidate badge when there are alternatives | | Unlisted | The hex prefix in muted text; the badge lists the rejected candidates | | The only known node, without GPS | The name, with `(no GPS)` | The candidate popover lists each candidate with its likelihood, or `hop N` when the path uses that node elsewhere. The "Path symbols" legend gains an entry for unlisted hops. **Map route view** (`map.js`, `route-view.js`): - The `#/map?packet=…&obs=…` deep link infers the chosen observation's hops with the same shared helper, so the map and the detail pane agree. - Unlisted hops render as `unlisted` placeholders rather than as the nearest node that happens to share the prefix. - When inferred hops are available, the `/api/resolve-hops` request is skipped. **Shared helpers** (`packet-helpers.js`): - `inferObservationPath(obs, observerPos, originPos)` - `ensureHopAffinity()`: loads the neighbour graph once per page, cached by `api()` for 5 minutes. - `routeHopFromInferred(hop, entry, role)` **Cleanup:** removed dead hop resolution from the "View route on map" button handler. It computed hops, sender and recipient, then navigated by URL and discarded them. **Unchanged:** the packets list, live view, node pages and `HopResolver.resolve()`. ## Performance AGENTS rule 0 asks for proof of performance claims; the figures follow. - **Complexity:** O(hops × 8 × candidates²). Hops more than 8 positions apart aren't compared directly; a longer run of unlisted hops costs a flat amount, tracked with a running minimum. Hop-to-hop costs are computed once and reused by every pass. - **Where it runs:** once per detail render (plus once for the aggregate row) and once per map deep link. Nothing runs in the list, live feed or WebSocket paths. - **Measured** (Node 22, warm JIT): | Case | Time | |---|---| | 64 hops × 16 candidates, synthetic worst case | 9–14 ms median | | Real 39–48 hop paths, 1,153 nodes (23 observations) | 1–12 ms each | - **Enforced:** `test-hop-path-inference.js` asserts a median under 40 ms (target 20 ms) for the 64 × 16 case. It times the module outside Node's `vm` sandbox, because the sandbox makes global lookups about 10× slower than a browser. - **Network:** one extra bulk request per page life (`/api/analytics/neighbor-graph`, cached server-side, TTL 5 min). No per-hop or per-packet requests. ## Tests **New:** - `tests/unit/test-hop-path-inference.js` (37 assertions). The model itself: - a repeated prefix resolves per position, and no node appears twice; - the observer anchors the last hop, with the IATA fallback when the observer has no position or reports 0,0; - a far-away lone candidate is reported unlisted; - a neighbour-graph link breaks a tie; - the server hint and the origin anchor are applied; - a node without GPS is handled; - output shape, and the performance gate. - `tests/unit/test-hop-path-detail.js` (32 assertions). Rendering and wiring: - inferred-hop rendering and the popover; - positional `renderPath`, including with the 1-byte hop filter on; - `renderDetail` infers the shown observation from its own `resolved_path` and leaves the list cache alone; - the neighbour graph is fetched once; - the shared map helpers, and source checks on the map wiring. - `tests/e2e/test-hop-path-inference-e2e.js` (6 steps, shard 1). Runs on fixture packet `57224a4a01c9c682`, observation 147, a 22-hop path that repeats one prefix three times: - every hop is inferred; - the repeats resolve per position; - the popover shows likelihoods; - the map's route list matches the detail pane hop for hop. **Updated, because they pinned the old behaviour:** - `test-issue-1418-deeplink-hops-channels.js`: it tested a copy of the old map priority logic. It now tests the real `inferObservationPath()`, including `resolved_path` as a hint. - `test-issue-2098-last-hop-region.js`: the detail pane no longer writes the prefix cache. The test now reads the region-anchored last hop from the rendered Path row. - `test-issue-2097-hop-ambiguity-badge.js`: the source check allows the new `entries` option while still asserting that the detail pane never uses the summarised form. - `.eslintrc.json`: declares the three new shared helpers as globals. **Results on this branch:** - `sh test-all.sh`: all standalone frontend suites pass. - `npx eslint public/*.js`: 0 errors. All 85 warnings are pre-existing; none come from code this PR adds. - The "Run Playwright E2E tests (fail-fast)" step was run locally with its script taken verbatim from `deploy.yml` (`bash -eo pipefail`, fixture prepared as in CI): - shard 1: 27 of 27 passed; - shard 2: 47 of 47 passed; - shard 3: 50 of 50 passed. Note for reviewers: `test-issue-1692-packets-init-parallel-e2e.js` only passes while the freshened fixture's newest packets are less than 15 minutes old. The test sets the time window in `localStorage` after the page has already read it. CI meets that condition because shards start right after freshening. Unmodified `master` fails the same way when run late. **Browser validation:** the local server ran on the fixture DB. Checked in the browser: - the Path row, the candidate popover and the byte breakdown; - the map route view, which matched the detail pane hop for hop with unlisted placeholders; - the console showed no errors. ## Known limitations and follow-ups - **TRACE packets are not modelled yet.** A TRACE's path is its *planned* route. The header path byte counts the hops actually forwarded (repeaters append SNR, not their hash, `Mesh.cpp`), and a trace may revisit a node on its way back (see the comment in `Packet::calculatePacketHash`). A follow-up will add a `heardAfter` / `allowRepeats` option, so a trace is anchored after the hops it actually completed and may repeat nodes. - **Customizer (AGENTS rule 8):** the `PATH_MODEL` tuning values are hardcoded for now. The unlisted cost, typical and maximum hop length, and the confidence threshold should become customizer settings. - **Server-side `resolved_path`:** the ingestor writes a hop as resolved whenever its prefix is unique in the node table, even when that node is hundreds of km from the neighbouring hops. `inferPath()` treats these as hints and can overrule them, but the stored data is still misleading. - **The first hop without a sender position is often a close call.** On encrypted channel messages, hop 0 is constrained only by what follows it, so 55/45-style splits between a long chain of known nodes and "unlisted" are expected. The badge and the percentage show this rather than hide it. ## Screenshots This is what a normal corescope instance would show for a 1 byte path. <img width="318" height="338" alt="image" src="https://github.com/user-attachments/assets/4056bc27-0b37-4107-9b25-31ac0f7864ec" /> <img width="430" height="601" alt="image" src="https://github.com/user-attachments/assets/01da5ee8-babf-4e41-a9ab-836b5d640549" /> Now the same packet but with the logic to infer each hope along the path different nodes are chosen as they're ranked and selected based on proximity to the next hop, <img width="436" height="646" alt="image" src="https://github.com/user-attachments/assets/bcca73b8-daf0-4031-8ea1-8c96d716c8b3" /> <img width="436" height="646" alt="image" src="https://github.com/user-attachments/assets/fc926d9e-158c-48ce-bee6-93086afb0775" /> The higher the confidence level then there's no need to show the exclamation mark to show that multiple nodes share the same prefix but it does continue to share that Examples of it currently running can be found at, https://corescope.baymc.org - modified instance https://analyzer.comchan.net - unmodified instance https://analyzer-stage.comchan.net - modified instance --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>