Files
meshcore-analyzer/docs/client-rx-coverage.md
T
efiten 22fe929da2 feat: opt-in mobile client-RX coverage (crowdsourced RF reach) + /api/nodes/resolve (#1728)
Implements #1727.

## What this adds

**Mobile client-RX coverage** — an opt-in, crowdsourced RF-coverage
feature. A roaming MeshCore **companion** radio (driven by the
open-source [corescope-rx](https://github.com/efiten/corescope-rx) PWA,
GPLv3) reports which nodes it heard directly, tagged with the phone's
GPS and the packet's SNR/RSSI. CoreScope ingests these into a new
`client_receptions` table and renders per-node **hex coverage** on the
Reach page, plus a standalone **Coverage dashboard** (`#/rx-coverage`)
with a top-mobile-observers leaderboard.

Also includes **`GET /api/nodes/resolve?prefix=<hex>`** — a read-only
node-name lookup by pubkey prefix (`{name, pubkey, ambiguous}`), used by
the companion app for friendly names.

## Opt-in — default OFF (zero impact on existing deployments)

The whole feature is gated behind one config flag, **disabled by
default**:

```jsonc
"clientRxCoverage": { "enabled": false }
```

When disabled (the default): the ingestor writes **no**
`client_receptions`; the three coverage endpoints return a clean
**404**; the UI hides the Coverage nav link, the `#/rx-coverage` route,
and the Reach-page toggle. `/api/nodes/resolve` is always available (not
coverage-specific).

## How it works

```
companion ──BLE 0x88 (snr+rssi+raw)──▶ corescope-rx PWA ──▶ MQTT meshcore/client/{pubkey}/packets
                                                                      │
                                          ingestor (gated) ──▶ client_receptions (GPS + SNR + heard-key)
                                                                      │
              server: pure-Go hex grid ──▶ GeoJSON ──▶ Reach hex overlay + Coverage dashboard
```

- **Direct-only capture:** records only what the companion heard itself
and directly — a 0-hop advert's pubkey, or `path[last]` (last forwarder)
for FLOOD routes; ≥2-byte path-hash required. Upstream hops discarded.
- **No new deps:** hexbins are a pure-Go pointy-top grid over Web
Mercator (`cmd/server/hexgrid.go`) computed at query time
(`CGO_ENABLED=0` / `modernc.org/sqlite` friendly); frontend uses the
existing Leaflet.
- **Trust:** companion pubkey = identity; an EMQX ACL binds each client
to publish only to its own `meshcore/client/{pubkey}/packets` topic.
Payload contract in `docs/client-rx-coverage.md`.

## How to enable / try it

1. In `config.json`, set `"clientRxCoverage": { "enabled": true }` and
restart server + ingestor.
2. Point an EMQX (or any broker) listener so a client can publish to
`meshcore/client/<pubkey>/packets`; the ingestor already subscribes
under `meshcore/#`.
3. Run the [corescope-rx](https://github.com/efiten/corescope-rx) PWA on
an Android phone paired (BLE) to a MeshCore companion — it captures
heard nodes + GPS and publishes.
4. View results: per-node Reach page → toggle **coverage**, or the
**Coverage** dashboard at `#/rx-coverage`.

## What's where

- **Ingestor:** `cmd/ingestor/client_reception.go` (ingest), `db.go`
(`client_receptions` + `client_observers` schema), `main.go` (gated
dispatch), `config.go` (flag).
- **Server:** `cmd/server/rx_coverage.go` + `rx_dashboard.go`
(endpoints, self-guard 404 when off), `hexgrid.go` (pure-Go grid),
`node_resolve.go` (resolve), `routes.go` / `types.go` / `config.go`
(wiring + flag + `/api/config/client` field).
- **Frontend:** `public/rx-coverage.js` (dashboard),
`node-reach-coverage.js` + `.css` (overlay), `node-reach.js` (Reach
toggle, flag-gated), `roles.js` (reads the flag, hides nav when off).
- **Docs:** `docs/client-rx-coverage.md`.

## Testing

- Go: `cd cmd/server && go test ./...` and `cd cmd/ingestor && go test
./...` — green, including new gate tests (`coverage_gate_test.go` in
both: off → no rows / 404, on → works) and the rx-coverage / resolve /
hexgrid suites.
- JS: `node test-coverage-gate.js`, `node test-node-reach-coverage.js`
(wired into CI). The Playwright `test-node-reach-coverage-e2e.js` is
wired into the e2e job and **skips when `clientRxCoverage` is
disabled**, so it's safe under the default-off config.

## Notes for reviewers

- The four new routes are registered in
`cmd/server/openapi_known_gaps.json` (the existing OpenAPI-completeness
ratchet), matching how other not-yet-spec'd routes are tracked. Happy to
write full OpenAPI spec entries instead if you prefer.
- Commits are split per layer (ingestor / server endpoints / resolve /
frontend / CI) for review.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Erwin Fiten <e.fiten@opteco.be>
2026-06-19 11:37:16 -07:00

12 KiB
Raw Blame History

Client RX Coverage

Crowdsourced RF coverage from mobile clients: a phone connects over BLE to a MeshCore companion radio, captures which nodes the companion hears (with SNR/RSSI), tags each reception with the phone's GPS position, and publishes it to MQTT. CoreScope ingests these into client_receptions and renders per-node H3-style hex coverage on the Reach page.

Companion app — where to get it

The mobile capture side is corescope-rx — an open-source (GPL-3.0) Android PWA. Operators who enable coverage point their users at it: it connects over BLE to a MeshCore companion radio, captures directly-heard nodes + the phone's GPS, and publishes the payload defined below. It's self-hostable and generic — a runtime config.json aims it at your own MQTT broker + CoreScope instance (see its README).

Enabling coverage (operators)

Coverage is off by default. To turn it on:

  1. In CoreScope's config.json, set "clientRxCoverage": { "enabled": true } and restart the server and ingestor. This is a single flag read by both processes — the ingestor and server each parse the same config.json, so you set clientRxCoverage.enabled once and it gates both the ingest write path and the read endpoints. There is no separate per-process flag.
  2. Required: an ACL-capable broker. Bind meshcore/client/{PUBLIC_KEY}/packets so each client may publish only under its own pubkey (e.g. an EMQX ACL keyed on the connected client's identity). This is the trust boundary, not an optimization — see Trust. The ingestor already subscribes under meshcore/#.
  3. Optionally set retention.clientRxDays to bound the coverage tables (see Storage).
  4. Point your users at corescope-rx and they start contributing. Results show on each node's Reach page (coverage toggle) and the #/rx-coverage dashboard. Warn them first that their contribution is world-readable and a per-observer view can reconstruct their movements — see Privacy.

The rest of this document is the MQTT payload contract the companion app implements.

Companion BLE source (verified against firmware)

The mobile app's RX data comes from the companion's PUSH_CODE_LOG_RX_DATA (0x88) BLE frame: [0x88][snr×4 int8][rssi int8][raw packet bytes]. This is emitted for every received packet (promiscuous, incl. overheard flood traffic), not just messages addressed to the device:

  • src/Dispatcher.cpp:198 calls logRxRaw(getLastSNR(), getLastRSSI(), raw, len) in checkRecv() unconditionally — NOT behind #if MESH_PACKET_LOGGING. So it works on stock firmware.
  • examples/companion_radio/MyMesh.cpp:283 overrides it to write the 0x88 frame whenever the app is connected over BLE (_serial->isConnected()).

So per received packet the app gets SNR + RSSI + the raw bytes. It decodes the raw packet (standard MeshCore format) to derive the directly-heard node (path[last] or 0-hop advert pubkey) and pairs it with the phone's GPS. The bare advert push (PUSH_CODE_ADVERT 0x80) carries only a pubkey (no SNR/ RSSI/path) and is NOT used — 0x88 already covers adverts (the raw advert is in its payload).

Caveats: 0x88 is only sent while the app is BLE-connected; packets larger than MAX_FRAME_SIZE are skipped; the firmware doc labels 0x88 "can be ignored" (messaging-app view) — for coverage it is the primary frame. GPS is always the phone's, never the companion's.

MQTT topic & payload

Topic: meshcore/client/{PUBLIC_KEY}/packets{PUBLIC_KEY} is the companion's pubkey. The broker (EMQX) should ACL-restrict each client to publish only under its own pubkey, which is how "a connected companion may only inject under the keys that apply" is enforced.

Payload — meshcoretomqtt-compatible packet, plus a gps object:

{
  "origin": "<companion name>",
  "origin_id": "<companion pubkey hex>",
  "timestamp": "2026-06-09T12:00:00Z",
  "type": "PACKET",
  "direction": "rx",
  "raw": "<packet hex>",
  "SNR": -7,
  "RSSI": -92,
  "gps": { "lat": 51.05, "lon": 3.72, "acc_m": 8 }
}
  • The discriminator is the gps object. A packet without gps is dropped (coverage needs a position).
  • raw is decoded server-side to derive the directly-heard node and the path; hash/path fields are not required.
  • Subscription: the ingestor's default subscription (meshcore/#) already covers this topic. Sources configured with an explicit topic list must add meshcore/client/+/packets.

Capture HARD RULE — only what was heard directly

The app and ingestor record only the node the companion physically received, never upstream relayers:

  • FLOOD packet with a path (≥1 hop) → record path[len-1] (the last forwarder = the immediate RF transmitter). Confirmed against firmware Mesh.cpp (routeRecvPacket appends the forwarder's hash to the END of the path) and CoreScope's neighbor_builder.go:226-228.
  • DIRECT packet with a pathNOT attributable, discarded. Direct forwarders consume the next hop from the FRONT (Mesh.cpp removeSelfFromPath), so path[len-1] is the route's destination-side end, NOT the node we heard. Attributing it credits the SNR to the wrong (often far-away) node. Only FLOOD routes (0,1) are recorded from a path.
  • Packet with no path (0 hops) and an advert → record the advertiser's full pubkey.
  • direction must be rx. 1-byte (2 hex char) prefixes are excluded (collision-prone, like Reach).
  • The RSSI/SNR belong to the directly-received transmission, so they attach to the recorded node.
  • The rest of the path is discarded for coverage.

Storage — client_receptions (ingestor-owned)

A roaming companion is a mobile observer with a moving position, so it gets its own table (not observations, which assumes a fixed observer location). Per the #1283 read/write invariant, the table and all writes live in cmd/ingestor/.

client_receptions(
  id, rx_pubkey, heard_key, heard_keylen, rssi, snr,
  lat, lon, pos_acc_m, rx_at, ingested_at, src,
  UNIQUE(rx_pubkey, heard_key, rx_at))   -- idempotent re-ingest

heard_keylen is 32 for a full pubkey (0-hop advert) or 2/3 for a multibyte prefix. src is advert or rxlog. No hex cell is stored — binning is computed server-side from lat/lon.

Indexes: a composite (heard_key, heard_keylen, lat, lon) and a (lat, lon) index back the coverage queries; the per-node query matches a sargable heard_key IN (pubkey, prefix6, prefix4) list so the composite is used instead of a table scan (see the benchmark in cmd/ingestor).

Retention: the table grows on every submission, so set retention.clientRxDays (ingestor) to delete rows older than N days (and stale client_observers); 0 disables it. Without it the table is unbounded.

Read API — coverage GeoJSON

GET /api/nodes/{pubkey}/rx-coverage?bbox={minLat,minLon,maxLat,maxLon}&z={zoom}

Returns a GeoJSON FeatureCollection of hexagons covering where clients heard the node, aggregated server-side (read-only). Each feature:

{ "type": "Feature",
  "geometry": { "type": "Polygon", "coordinates": [[[lon,lat], ...]] },
  "properties": { "cell": "9:123:-45", "count": 7, "best_snr": -6, "has_sig": true,
                  "nodes": [{ "prefix": "aabbcc", "name": "Alice", "snr": -6, "count": 3 }],
                  "nodes_truncated": false } }
  • Hex binning is a pure-Go pointy-top grid over Web Mercator (cmd/server/hexgrid.go). We do not use uber/h3-go because it is CGO and the project builds with CGO_ENABLED=0. Latitude is only defined within ±85.05° (Web Mercator limit) and is clamped to that range.
  • z (Leaflet zoom) selects the hex resolution (zoom-adaptive). Raw points never leave the server (privacy: contributors' tracks are not exposed).
  • best_snr / has_sig drive the colour: green→orange by best SNR, grey when no signal metric.
  • Features are sorted by cell for a deterministic (cacheable) payload.
  • Bounds: the per-cell nodes list is capped (with nodes_truncated), and the collection is capped at a fixed feature count — when exceeded, the densest cells are kept and the top-level truncated flag is set. The per-node endpoint also returns mobile_receptions and mobile_clients totals (node-wide, independent of the bbox).

Frontend

Shown only in the Reach view (#/nodes/{pubkey}/reach), as a toggleable hex layer drawn on the existing Leaflet map (public/node-reach-coverage.js), deep-linked via ?coverage=1. No new frontend dependencies. Colours come from CSS variables in public/node-reach.css (--nq-cov-strong|mid|weak|grey).

Trust

Identity = the companion pubkey (rx_pubkey), taken from the {PUBLIC_KEY} topic segment.

The feature requires an ACL-capable broker. The reported GPS position is the contributor's own claim, so the only thing anchoring a reception to a real identity is the broker ACL binding meshcore/client/{PUBLIC_KEY}/packets to the client that holds that key. Without such an ACL, the topic — and therefore the GPS and the heard-node attribution — is spoofable: anyone who can publish to the broker could inject coverage under any pubkey. Do not enable this feature on an open/no-ACL broker if you trust the resulting map.

Server/ingestor-side defense-in-depth (these reduce blast radius but do not replace the ACL):

  • The ingestor rejects any topic pubkey that is not lowercase hex before writing, and never falls back to a payload-supplied id (cmd/ingestor/client_reception.go, #2/#10).
  • A blacklisted operator cannot contribute via the client topic (the blacklist is enforced before the coverage write, #1).
  • The frontend HTML-escapes the pubkey it renders, so a junk pubkey can't inject markup (#14).
  • /api/nodes/resolve and coverage tooltips never reveal blacklisted or hidden-prefix node identities (#15).

Privacy — contributor location is public

⚠️ Enabling coverage publishes contributors' GPS-tagged receptions, and the per-observer view can reconstruct a contributor's movements. The hex map is read without authentication. The leaderboard exposes each companion's pubkey, and clicking one filters the map to that single companion (/api/rx-coverage?rx=<pubkey>); at high zoom over the retention window this is effectively a public movement trail (home / work / commute) of whoever carries that companion. A pseudonymous companion name does not mitigate this — the locations themselves are identifying (overnight clustering = home), and all of one contributor's points are linked by the pubkey.

This is an accepted tradeoff of the feature, not a bug: fine resolution is what makes the aggregate coverage map useful, the feature is opt-in and OFF by default, and contributors choose to run the companion. But the consent must be informed:

  • Operators: tell your users, before they contribute, that their coverage (including a per-observer view of their own track) is world-readable for as long as retention.clientRxDays keeps it.
  • Contributors: do not contribute from a device you carry on your person if a public record of where you have been is a concern. Use a dedicated/stationary node, or accept that the trail is public.

Operators who want to harden this further can lower retention.clientRxDays, run the dashboard behind their own auth/proxy, or (future hardening) coarsen stored coordinates / apply a k-anonymity threshold to the per-observer view.

Optional future hardening: have the companion sign a broker-issued token (the firmware exposes on-device signing) — not required for the MVP, tracked as a follow-up.

Configurable values (future customizer)

Hardcoded initially, tracked for the customizer per AGENTS.md rule 8: hex resolution per zoom (zoomToHexRes), colour SNR thresholds (coverageColorVar), and any rx_at max-age validation.