dandri/meshcore-analyzer
1
0 Fork 0
mirror of https://github.com/Kpa-clawbot/meshcore-analyzer.git synced 2026-11-11 18:37:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
Files
dbcdca9dc55cdef1af9ed322ee0c6244c759ed8b
meshcore-analyzer/scripts/non-unit-tests.json
T
efitenandClaude Opus 5.5 dc98886d92 feat(nodes): name and show repeaters known only from the name they reported (#2238) (#2239) 
Part of #2238. Path hops through such a repeater (`/api/nodes`,
`/api/resolve-hops`, the frontend hop resolver) still show a bare
prefix; that part of the issue stays open.

## Summary

A repeater that never sends an advert can still tell a nearby companion
its name. CoreDrive RX v1.20.0 asks for it (`ANON_REQ_TYPE_OWNER`) and
publishes the answer as `NODE_NAME` on `meshcore/client/{pk}/names`.
This PR stores those answers, resolves names with them, and shows these
repeaters on the map, in the nodes list and in search, marked as
self-reported.

### Ingestor: store the names
- `367ecde1` adds a new `names` subtopic.
  - It is gated on `clientNodeNames.enabled`, default off.
- Answers go into a new table, `client_node_names`: one row per
observation, `UNIQUE(target, rx_pubkey, observed_at)`.
- **Validation.**
  - `target` must be 64 hex.
- The name is trimmed and rejected when it is empty, over 31 runes
(firmware `node_name[32]`, `src/helpers/CommonCLI.h:29`), invalid UTF-8,
or contains a control character.
- **Retention.** `retention.clientNodeNamesDays` (0 = keep).
`config.example.json` sets 30, like the other client tables: a companion
only asks a repeater for its name when CoreScope has none (`nameMissing`
in CoreDrive's `src/names.js`), so pruning is also how a renamed
repeater gets its new name.
- **Compatibility.** An ingestor without this change drops `/names`
silently, so CoreDrive v1.20.0 works against either version.

### Server: resolve by the reported name
- `11d97b36` makes `lookupPrefixes` count candidates in both `nodes` and
`client_node_names`. `/api/nodes/resolve` and `batchResolveHeardKeys`
(the coverage tooltips) both use it.
- **Which name wins.** The advert name, otherwise the newest reported
name by `observed_at`. A prefix that matches different pubkeys across
the two tables stays ambiguous.
- **Only repeaters heard directly.** A reported name counts only once
its target has at least one `discover` or `advert` reception in
`client_receptions`, keyed by the full pubkey, or by its 8-byte prefix
when no other pubkey in `nodes`, `inactive_nodes` or `client_node_names`
starts with that prefix (`clientNameHeard`). A name for a pubkey nobody
heard adds no candidate, so it cannot make a real node's prefix
ambiguous. Without the uniqueness check, a made-up pubkey that shares a
heard repeater's first 8 bytes borrowed that repeater's receptions and
passed (fixed in `4c569f91`).
- **Filters.** The blacklist and the hidden name prefixes apply to both
sources.
- **Missing table.** A database without the table resolves from `nodes`
alone, and the server checks for the table again until the ingestor has
created it.
- **Response shape.** `/api/nodes/resolve` keeps `{prefix, pubkey?,
name?, ambiguous}`.

### Server: estimated position
- `98c76957` adds `GET /api/nodes/reported`, documented in `openapi.go`.
It lists every repeater in `client_node_names` that never sent an advert
(no row in `nodes` or `inactive_nodes`) and was heard directly at least
once (see above). A node with an advert but no position stays out: it is
already in `/api/nodes` under its advert name.
- **Gated** like the other endpoints on `client_receptions`: 404 when
`clientRxCoverage.enabled` is off, because the counts and the estimate
come from companion GPS fixes.
- **Which receptions count.** Only direct ones:
- `discover` and `advert` rows, keyed by the full pubkey, or by its
8-byte prefix when no other known pubkey shares it;
  - `rxlog` rows whose 2-3 byte hash resolves to this repeater alone.

  `geo` rows are left out.
- **How the estimate is made.**
- Receptions within 100 m form one spot, with the best SNR of that spot.
  - An estimate needs at least 3 spots, at least 200 m apart.
- The position is the SNR-weighted centroid (`10^(snr/20)`). The radius
is the distance to the farthest spot, at least 500 m.
- The response is cached for 5 minutes, and the compute time is logged.

### Map
- `5fcde554` adds `public/map-estimated.js`, which draws each estimate
on its own layer:
  - a hollow, dashed "≈" marker and a dashed uncertainty circle;
- a popup with an "Estimated position" badge, the method, the caveat
that one-sided driving skews the estimate, and a link to the coverage
map;
- a Display toggle, "Estimated positions": on by default, kept in
localStorage, and also hidden by the repeater role filter and in route
view.

### Nodes page and search
- `12e6b345` adds a "Self-reported repeaters" section under the nodes
table, on the All and Repeaters tabs, filtered by the search box.
- **`#/nodes/<pubkey>`.** Where `/api/nodes/<pubkey>` answers 404, a
self-reported detail view replaces "Node not found":
  - who reported the name;
  - the receptions;
  - the estimate, or why there is none;
  - links to the map and the coverage map.

  An unknown key still shows "Node not found".
- **Header search** adds matching self-reported repeaters, tagged
"self-reported".
- **Unchanged on purpose.** `/api/nodes` and `/api/nodes/search` stay as
they are. These repeaters have no packets, so the packet filters, the
live view and the channel sender lookup must not match them.

### Docs
- `03580437` corrects the `/regions` section of
`docs/client-rx-coverage.md`: it named `clientRxCoverage.enabled` as the
switch, but the code uses `clientRegions.enabled`.
- The new subtopic, endpoint and config keys are documented in the same
file.

## Trust

Any companion the broker lets publish can report a name for any
repeater, and the newest report wins. This is the same model as
`/regions`. `clientRxCoverage.requireLinkedCompanion` limits it to
linked companions.

A report for a repeater nobody heard directly is stored but not used: it
is not listed and adds no prefix candidate. A companion can still report
a wrong name for a repeater that was heard. The same companion can also
publish a fake `discover` reception for a made-up pubkey, so the "heard
directly" check filters junk, not a malicious companion;
`requireLinkedCompanion` is the control for that (its gate runs before
the `/names` handler too).

## Performance

- **`lookupPrefixes`** adds one term per key on `client_node_names`.
- It matches the prefix as a range, `target >= ? AND target < ?` (upper
bound prefix + `"g"`), not with `LIKE`: `LIKE` on this case-sensitive
column cannot use the index. `EXPLAIN QUERY PLAN`: `SEARCH c USING INDEX
idx_cnn_target (target>? AND target<?)`, and the reception check is
`SEARCH r USING INDEX idx_client_recept_heard_geo (heard_key=?)`.
- It is still one query per chunk of 200 keys: 400 compound terms, under
SQLite's default limit of 500.
  - `TestLookupPrefixesManyKeys` covers 250 keys over two chunks.
- **`/api/nodes/reported`** makes three kinds of query:
  - one grouped query on `client_node_names`;
  - one `lookupPrefixes` call for the short hashes;
- one `IN` query per 400 keys on `client_receptions`, served by
`idx_client_recept_heard_geo`.

The spread check is O(spots²) per repeater, with at most one spot per
100 m visited. The result is cached for 5 minutes; concurrent misses
wait for one computation, and the response is written after the lock is
released.
- **The map does not wait for it.** It draws its markers first and adds
the estimates when the list arrives. The map and the nodes page share
one cached fetch (`ReportedNodes.load()`).
- **Not benchmarked.** On the instance where this runs,
`client_node_names` holds one row today. I expect at most tens of such
repeaters per instance, but have not measured that.

## Notes

- **`map[string]interface{}`.** `cmd/ingestor/client_names.go` has 2
occurrences: the handler's `msg` parameter and its `gps` field. That is
the decoded MQTT message, passed in the same shape as
`handleClientPacket` and `handleClientRegions` already take it. No API
response or internal structure uses it.
- **Thresholds are hardcoded:** 100 m, 3 spots / 200 m, 500 m, 5 min
TTL. They should move to the customizer later (rule 8).
- **Map filters.** The estimates follow the "Estimated positions" toggle
and the repeater role filter, not the area, region or last-heard
filters. The toggle's tooltip says so.
- **Name characters.** Names are rejected for control characters but not
for Unicode format characters (bidi overrides, zero-width joiners),
because emoji sequences in node names use the zero-width joiner. All
output goes through `escapeHtml`.
- **Not done:** the home-page search box (`home.js`, `/api/nodes/search`
with a Claim button) does not list these repeaters. Claiming a node
without adverts would have nothing to show.

## Tests

On this branch (based on `550829af`):
- `go vet ./...` and `go test -count=1 ./...` pass in `cmd/server` and
`cmd/ingestor`.
- **New Go tests:**
  - handler and validation cases;
  - idempotent redelivery;
  - newest `observed_at` wins;
  - prune;
  - the gate on and off, without falling through to the observer path;
- client-only node, nameless `nodes` row, advert name wins, ambiguity
across the two sources;
  - hidden and blacklisted nodes;
  - absent-then-created table;
  - spot merging, threshold, weighted centroid and radius floor;
- `discover` / `rxlog` / `geo` handling, an ambiguous path hash, an
advert position and a missing table;
- an advert without a position, a never-heard name, a name heard only in
`rxlog` paths, and the `clientRxCoverage` gate (404). These fail on
`12e6b345` and pass at head.
- a made-up pubkey sharing a heard repeater's 8-byte prefix, and two
reported targets sharing one (`4c569f91`). Both fail without that
commit.
- **New frontend unit suites:** `tests/unit/test-map-estimated.js` and
`tests/unit/test-reported-nodes.js` pass. Both load the real
`escapeHtml` from `app.js`.
- **`test-all.sh`** passes, all suites (with Python available for the
XSS gate self-tests). `scripts/check-xss-sinks.sh --file` passes on the
four changed frontend files.
- **E2E**, against a static server with a mocked API:
- `test-map-estimated-e2e.js`: 6/6, including the repeater role filter
and route view hiding the estimates. Mutating either hide makes its step
fail.
  - `test-reported-nodes-e2e.js`: 6/6.
  - `test-issue-1150-404-state-e2e.js`: 5/5.
- **Real data, API only.** On the fork's staging, `/api/nodes/reported`
lists one repeater: 5 spots, 17 receptions, an estimate with a radius of
2,305 m. It has no `nodes` row and 24 `discover` receptions, so it stays
listed under the stricter rules. Not re-checked after `4c569f91`.
- **Not done:** a check of that repeater in a browser.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-21 15:17:58 +02:00

149 lines
7.6 KiB
JSON
Raw Blame History

{
"browser": {
"reason": "Requires Playwright and Chromium; most suites also require a running local fixture server via BASE_URL. Existing E2E workflow steps retain their separate setup and selection. Never point these tests at production.",
"files": [
"tests/e2e/test-1110-live-filter.js",
"tests/e2e/test-a11y-axe-1668.js",
"tests/e2e/test-analytics-fluid-charts.js",
"tests/e2e/test-audio-lab-1297-e2e.js",
"tests/e2e/test-audio-live-1297-e2e.js",
"tests/e2e/test-bottom-nav-1061-e2e.js",
"tests/e2e/test-channel-color-picker-e2e.js",
"tests/e2e/test-channel-decrypt-e2e.js",
"tests/e2e/test-channel-fluid-e2e.js",
"tests/e2e/test-channel-issue-1087-e2e.js",
"tests/e2e/test-channel-issue-1111-e2e.js",
"tests/e2e/test-channel-modal-e2e.js",
"tests/e2e/test-channel-qr-e2e.js",
"tests/e2e/test-channels-add-modal-e2e.js",
"tests/e2e/test-channels-list-render-e2e.js",
"tests/e2e/test-channels-selection-flow-e2e.js",
"tests/e2e/test-channels-share-color-e2e.js",
"tests/e2e/test-channels-ws-batch-e2e.js",
"tests/e2e/test-channels-ws-race-1498-e2e.js",
"tests/e2e/test-charts-fluid-1058-e2e.js",
"tests/e2e/test-customize-branding-e2e.js",
"tests/e2e/test-customize-display-e2e.js",
"tests/e2e/test-customize-export-e2e.js",
"tests/e2e/test-customize-theme-e2e.js",
"tests/e2e/test-drag-manager-e2e.js",
"tests/e2e/test-e2e-1267-mobile-vcr.js",
"tests/e2e/test-e2e-playwright.js",
"tests/e2e/test-filter-ux-e2e.js",
"tests/e2e/test-gesture-hints-1065-e2e.js",
"tests/e2e/test-gestures-1062-e2e.js",
"tests/e2e/test-gestures-1185-scroll-discriminator-e2e.js",
"tests/e2e/test-health-status-e2e.js",
"tests/e2e/test-home-coverage-e2e.js",
"tests/e2e/test-hop-path-inference-e2e.js",
"tests/e2e/test-issue-1109-hamburger-dropdown-visible-e2e.js",
"tests/e2e/test-issue-1122-packets-filter-ux-e2e.js",
"tests/e2e/test-issue-1128-multi-viewport-e2e.js",
"tests/e2e/test-issue-1128-packets-layout-e2e.js",
"tests/e2e/test-issue-1136-live-region-e2e.js",
"tests/e2e/test-issue-1146-path-link-contrast-e2e.js",
"tests/e2e/test-issue-1147-section-order-e2e.js",
"tests/e2e/test-issue-1150-404-state-e2e.js",
"tests/e2e/test-issue-1151-orphan-separators-e2e.js",
"tests/e2e/test-issue-1204-live-panel-structure-e2e.js",
"tests/e2e/test-issue-1205-live-controls-anchor-e2e.js",
"tests/e2e/test-issue-1206-resize-observer-leak-e2e.js",
"tests/e2e/test-issue-1206-vcr-overlap-e2e.js",
"tests/e2e/test-issue-1224-channels-mobile-ux-e2e.js",
"tests/e2e/test-issue-1234-live-chrome-pass2-e2e.js",
"tests/e2e/test-issue-1236-map-mobile-e2e.js",
"tests/e2e/test-issue-1244-live-vcr-row-hints-e2e.js",
"tests/e2e/test-issue-1273-qr-overlay-height-e2e.js",
"tests/e2e/test-issue-1274-legend-coverage-e2e.js",
"tests/e2e/test-issue-1279-legend-p2-e2e.js",
"tests/e2e/test-issue-1281-location-row-e2e.js",
"tests/e2e/test-issue-1306-collisions-terminology-e2e.js",
"tests/e2e/test-issue-1329-map-controls-accordion-e2e.js",
"tests/e2e/test-issue-1367-channels-chat-app-e2e.js",
"tests/e2e/test-issue-1374-route-map-a11y-e2e.js",
"tests/e2e/test-issue-1400-nav-vertical-clip.js",
"tests/e2e/test-issue-1413-nav-overlap-e2e.js",
"tests/e2e/test-issue-1486-collapse-reopens-detail-e2e.js",
"tests/e2e/test-issue-1487-byop-modal-layout-e2e.js",
"tests/e2e/test-issue-1510-live-nav-pin-e2e.js",
"tests/e2e/test-issue-1567-corner-clears-drag-e2e.js",
"tests/e2e/test-issue-1599-replay-freeze-e2e.js",
"tests/e2e/test-issue-1630-reach-mobile-e2e.js",
"tests/e2e/test-issue-1268-rf-params-e2e.js",
"tests/e2e/test-issue-1639-observers-sort-e2e.js",
"tests/e2e/test-issue-748-node-analytics-coverage-e2e.js",
"tests/e2e/test-issue-2218-node-analytics-stale-e2e.js",
"tests/e2e/test-issue-661-airtime-e2e.js",
"tests/e2e/test-issue-1640-compare-discovery-e2e.js",
"tests/e2e/test-issue-1648-m1-icons-e2e.js",
"tests/e2e/test-issue-1648-m2-icons-e2e.js",
"tests/e2e/test-issue-1648-m3-icons-e2e.js",
"tests/e2e/test-issue-1648-m4-icons-e2e.js",
"tests/e2e/test-issue-1648-m5-icons-e2e.js",
"tests/e2e/test-issue-1657-analytics-channels-group-sprites-e2e.js",
"tests/e2e/test-issue-1692-packets-init-parallel-e2e.js",
"tests/e2e/test-issue-1705-subpath-contrast-e2e.js",
"tests/e2e/test-issue-1758-ng-filter-rerenders-e2e.js",
"tests/e2e/test-issue-1799-label-vocab-e2e.js",
"tests/e2e/test-issue-1833-legend-toggle-clickable-e2e.js",
"tests/e2e/test-issue-1997-distance-building-e2e.js",
"tests/e2e/test-live-dedup.js",
"tests/e2e/test-live-fullscreen-1572-e2e.js",
"tests/e2e/test-live-layout-1178-1179-e2e.js",
"tests/e2e/test-live-mql-leak-1180-e2e.js",
"tests/e2e/test-live-multibyte-only-e2e.js",
"tests/e2e/test-logo-default-sage-teal-e2e.js",
"tests/e2e/test-logo-pulse-1173-e2e.js",
"tests/e2e/test-logo-rebrand-e2e.js",
"tests/e2e/test-logo-theme-e2e.js",
"tests/e2e/test-map-estimated-e2e.js",
"tests/e2e/test-reported-nodes-e2e.js",
"tests/e2e/test-map-modal-fluid-e2e.js",
"tests/e2e/test-map-nodes-pagination-e2e.js",
"tests/e2e/test-nav-drawer-1064-e2e.js",
"tests/e2e/test-nav-fluid-1055-e2e.js",
"tests/e2e/test-nav-more-floor-1139-e2e.js",
"tests/e2e/test-nav-priority-1102-e2e.js",
"tests/e2e/test-nav-priority-1311-e2e.js",
"tests/e2e/test-nav-priority-1391-e2e.js",
"tests/e2e/test-neighbor-map-btn-clip-e2e.js",
"tests/e2e/test-node-reach-coverage-e2e.js",
"tests/e2e/test-node-reach-e2e.js",
"tests/e2e/test-nodes-export-e2e.js",
"tests/e2e/test-observer-iata-1188-e2e.js",
"tests/e2e/test-packets-compact-columns-e2e.js",
"tests/e2e/test-packets-node-filter-e2e.js",
"tests/e2e/test-packets-scope-column.js",
"tests/e2e/test-path-inspector-coverage-e2e.js",
"tests/e2e/test-rx-coverage-mobile-nav-e2e.js",
"tests/e2e/test-rx-coverage-gaps-e2e.js",
"tests/e2e/test-rx-coverage-noise-e2e.js",
"tests/e2e/test-rx-coverage-viewport-e2e.js",
"tests/e2e/test-show-neighbors.js",
"tests/e2e/test-slideover-1056-e2e.js",
"tests/e2e/test-slideover-1168-munger-e2e.js",
"tests/e2e/test-table-fluid-e2e.js",
"tests/e2e/test-theme-url-param-2236-e2e.js",
"tests/e2e/test-touch-gestures-coverage-e2e.js",
"tests/e2e/test-touch-targets.js",
"tests/e2e/test-user-management-e2e.js"
]
},
"domIntegration": {
"reason": "Requires jsdom. The JS unit gate runs with no node_modules at all (its job has no npm install), so a suite needing a third-party module cannot live in test-all.sh. It runs in the E2E job, which does npm ci, and is invoked by its own deploy.yml line.",
"files": [
"tests/e2e/test-region-quick-picks.js",
"tests/e2e/test-table-sort.js"
]
},
"playwrightRunner": {
"reason": "Requires the @playwright/test runner (not plain node), its configuration, Chromium, and a local server. @playwright/test is not currently declared in package.json; these suites are inventoried but not part of the standalone unit gate.",
"files": [
"tests/e2e/test-issue-1522-trace-url-sync-e2e.js",
"tests/e2e/test-marker-outline-weight.js",
"tests/e2e/test-path-inspector-e2e.js",
"tests/e2e/test-pr-1490-live-map-gpu-animations-e2e.js"
]
}
}
Reference in new issue View Git Blame Copy Permalink