dandri/meshcore-analyzer
1
0
Fork 0
mirror of https://github.com/Kpa-clawbot/meshcore-analyzer.git synced 2026-10-13 14:37:22 +00:00
Code Issues Packages Projects Releases Wiki Activity
3,025 Commits 84 Branches 39 Tags
4e91c5cf0bb2e35e12a074580f4d993c32cb6f00
Commit Graph
3 Commits
Author SHA1 Message Date
efitenandClaude Opus 5.5 f3004d977e feat(rx-coverage): show where a companion drove and received nothing (#2172) 
The Mobile RX coverage page (`#/rx-coverage`) shows only cells where a
companion received something. Where a companion drove and received
nothing is just as useful for operators, and the data is already there:
the RF sample track (`client_rf_samples`, #1905/#1906) and the
receptions (`client_receptions`) use the same hex grid (`hexCellAt`,
`zoomToHexRes`). This PR adds those cells as a hatched layer under the
reception cells.

## Changes in this PR

**Server**
- `GET /api/rx-coverage?...&gaps=1` adds a `gaps` member to the
FeatureCollection: cells with moving (non-stationary) track samples and
no reception in the same `days` window and `bbox`, each with `samples`.
`gaps_truncated` is set when more than 5000 cells qualify (the
most-sampled are kept, same pattern as `coverageFeatureCap` and
`rfNoiseFeatureCap`).
- Gaps are only returned when `clientRfSamples` is on and there is no
`node` filter: with `?node=` the question would be "this node was not
heard", which these cells do not answer. `?rx=` limits both the track
and the receptions to that companion.
- Without `gaps=1` the payload is unchanged: the `gaps` member is
omitted.
- New code lives in `cmd/server/rx_coverage_gaps.go` (pure
`aggregateGaps` plus `queryTrackPoints`); the handler change in
`rx_dashboard.go` is 14 lines.
- `/api/rx-coverage` is now documented in `openapi.go` and removed from
`openapi_known_gaps.json` (#1670).

**Frontend** (`public/rx-coverage.js`, `public/node-reach-coverage.css`)
- A "Nothing received" toggle on the signal layer, on by default, only
rendered with `MC_CLIENT_RF_SAMPLES`. Hidden on the noise layer.
- Gap cells use a 45-degree hatch (an SVG pattern in a dedicated
renderer, so they sit under the reception cells). The hatch is denser
for cells driven through more often (1 / 2-3 / 4+ samples). Tooltip:
sample count. Legend entry: "driven, nothing received".
- Off state is deep-linkable as `gaps=0`.
- New token `--nq-cov-gap` for light and dark. Plain grey
(`--nq-cov-grey`) already means "received, no SNR", so the hatch keeps
the two apart.

## Threshold: one sample

Measured on a live network (analyzer.on8ar.eu, Belgium, September to
October 2026), with cells of about 250 m: a cell with track samples and
no reception in one half of a month, driven again in the other half. How
often did it receive something the second time?

| Minimum samples | Cells "nothing received" | Driven again | Received
something later |
|---|---|---|---|
| 1 | 3971 | 1263 | 29% |
| 2 | 1565 | 592 | 32% |
| 3 | 619 | 243 | 35% |
| 5 | 142 | 63 | 40% |

A higher threshold hides cells without making the rest more reliable, so
the threshold is 1 and the wording is "nothing received in this period",
never "no coverage". Using every decoded packet as reception evidence
(including 1-byte path hashes) gave 25% to 32%, so the capture rule for
1-byte hashes is not what drives this. The median cell has 2 samples.

## Verification

- Go: 7 new tests in `rx_coverage_gaps_test.go` (gap/no gap, stationary
excluded, single sample, cap keeps the most-sampled, endpoint, `?rx=`
filter, no `gaps` member when not asked / with `?node=` / with RF
samples off). The endpoint tests fail without the handler change. Full
`cmd/server` suite green locally, `go vet` and `gofmt` clean.
- E2E: `tests/e2e/test-rx-coverage-gaps-e2e.js` (4 steps: no toggle with
RF samples off, on by default with hatched cells under the reception
cells, toggle off with `gaps=0` and deep link, noise layer hides it). 2
steps fail without the frontend change. Existing
`test-rx-coverage-noise-e2e.js` and `test-rx-coverage-viewport-e2e.js`
still pass. Listed in `scripts/non-unit-tests.json`.
- Frontend unit suite (`test-all.sh`) green.
- Real data on a staging instance (7 days, bbox 50.8,4.9,51.3,5.8,
z=12): 478 reception cells and 379 gap cells, no cell in both, reception
features identical with and without `gaps=1`, 0.48 s to 0.52 s per
request both ways.
- Browser: checked on staging with real data (light theme), and light,
dark and 390 px wide with mocked data.

## Not in this PR

- A per-node variant on the Reach page ("this repeater transmitted while
a companion was nearby, and was not heard"). That needs the transmit
moments from `observations` and is a heavier query.
- `deploy.yml` does not get a line for the new E2E file, like the other
entries that are only in `non-unit-tests.json` (#2037).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 17:39:33 +02:00
efitenClaude Opus 4.8Erwin Fiten
 		22fe929da2 feat: opt-in mobile client-RX coverage (crowdsourced RF reach) + /api/nodes/resolve (#1728) 
Implements #1727.

## What this adds

**Mobile client-RX coverage** — an opt-in, crowdsourced RF-coverage
feature. A roaming MeshCore **companion** radio (driven by the
open-source [corescope-rx](https://github.com/efiten/corescope-rx) PWA,
GPLv3) reports which nodes it heard directly, tagged with the phone's
GPS and the packet's SNR/RSSI. CoreScope ingests these into a new
`client_receptions` table and renders per-node **hex coverage** on the
Reach page, plus a standalone **Coverage dashboard** (`#/rx-coverage`)
with a top-mobile-observers leaderboard.

Also includes **`GET /api/nodes/resolve?prefix=<hex>`** — a read-only
node-name lookup by pubkey prefix (`{name, pubkey, ambiguous}`), used by
the companion app for friendly names.

## Opt-in — default OFF (zero impact on existing deployments)

The whole feature is gated behind one config flag, **disabled by
default**:

```jsonc
"clientRxCoverage": { "enabled": false }
```

When disabled (the default): the ingestor writes **no**
`client_receptions`; the three coverage endpoints return a clean
**404**; the UI hides the Coverage nav link, the `#/rx-coverage` route,
and the Reach-page toggle. `/api/nodes/resolve` is always available (not
coverage-specific).

## How it works

```
companion ──BLE 0x88 (snr+rssi+raw)──▶ corescope-rx PWA ──▶ MQTT meshcore/client/{pubkey}/packets
                                                                      │
                                          ingestor (gated) ──▶ client_receptions (GPS + SNR + heard-key)
                                                                      │
              server: pure-Go hex grid ──▶ GeoJSON ──▶ Reach hex overlay + Coverage dashboard
```

- **Direct-only capture:** records only what the companion heard itself
and directly — a 0-hop advert's pubkey, or `path[last]` (last forwarder)
for FLOOD routes; ≥2-byte path-hash required. Upstream hops discarded.
- **No new deps:** hexbins are a pure-Go pointy-top grid over Web
Mercator (`cmd/server/hexgrid.go`) computed at query time
(`CGO_ENABLED=0` / `modernc.org/sqlite` friendly); frontend uses the
existing Leaflet.
- **Trust:** companion pubkey = identity; an EMQX ACL binds each client
to publish only to its own `meshcore/client/{pubkey}/packets` topic.
Payload contract in `docs/client-rx-coverage.md`.

## How to enable / try it

1. In `config.json`, set `"clientRxCoverage": { "enabled": true }` and
restart server + ingestor.
2. Point an EMQX (or any broker) listener so a client can publish to
`meshcore/client/<pubkey>/packets`; the ingestor already subscribes
under `meshcore/#`.
3. Run the [corescope-rx](https://github.com/efiten/corescope-rx) PWA on
an Android phone paired (BLE) to a MeshCore companion — it captures
heard nodes + GPS and publishes.
4. View results: per-node Reach page → toggle **coverage**, or the
**Coverage** dashboard at `#/rx-coverage`.

## What's where

- **Ingestor:** `cmd/ingestor/client_reception.go` (ingest), `db.go`
(`client_receptions` + `client_observers` schema), `main.go` (gated
dispatch), `config.go` (flag).
- **Server:** `cmd/server/rx_coverage.go` + `rx_dashboard.go`
(endpoints, self-guard 404 when off), `hexgrid.go` (pure-Go grid),
`node_resolve.go` (resolve), `routes.go` / `types.go` / `config.go`
(wiring + flag + `/api/config/client` field).
- **Frontend:** `public/rx-coverage.js` (dashboard),
`node-reach-coverage.js` + `.css` (overlay), `node-reach.js` (Reach
toggle, flag-gated), `roles.js` (reads the flag, hides nav when off).
- **Docs:** `docs/client-rx-coverage.md`.

## Testing

- Go: `cd cmd/server && go test ./...` and `cd cmd/ingestor && go test
./...` — green, including new gate tests (`coverage_gate_test.go` in
both: off → no rows / 404, on → works) and the rx-coverage / resolve /
hexgrid suites.
- JS: `node test-coverage-gate.js`, `node test-node-reach-coverage.js`
(wired into CI). The Playwright `test-node-reach-coverage-e2e.js` is
wired into the e2e job and **skips when `clientRxCoverage` is
disabled**, so it's safe under the default-off config.

## Notes for reviewers

- The four new routes are registered in
`cmd/server/openapi_known_gaps.json` (the existing OpenAPI-completeness
ratchet), matching how other not-yet-spec'd routes are tracked. Happy to
write full OpenAPI spec entries instead if you prefer.
- Commits are split per layer (ingestor / server endpoints / resolve /
frontend / CI) for review.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Erwin Fiten <e.fiten@opteco.be>
 2026-06-19 11:37:16 -07:00
Kpa-clawbotandclawbot 653d47e03c test(openapi): add CI completeness gate for /api routes (Phase 1 of #1670) (#1678) 
## Summary

Partial fix for #1670 — **Phase 1 only** (CI completeness gate). Phase 2
(backfilling the 18 currently-undocumented routes into `openapi.go`) is
deferred to a separate issue per the triage on #1670 and is explicitly
out of scope here.

## What this adds

- `cmd/server/openapi_completeness_test.go` — AST-walks every
non-`_test.go` file in `cmd/server/`, finds string-literal first args to
`*.HandleFunc(...)` calls beginning with `/api/`, and diffs against the
paths declared in `routeDescriptions()` in `cmd/server/openapi.go`.
- `cmd/server/openapi_known_gaps.json` — seeded allowlist of the **18**
`/api/` routes currently registered via `HandleFunc` but not yet
documented in `openapi.go`.

## Ratchet pattern

From this branch forward, `TestOpenAPICompleteness` fails when:

1. A new `HandleFunc("/api/...")` is added without a matching entry in
`openapi.go` **or** the allowlist (regression gate — the main goal of
Phase 1).
2. A route in the allowlist is *also* documented in `openapi.go` — the
allowlist must shrink as Phase 2 backfills land, never go stale.

The two-commit history (red → green) demonstrates the gate works:

- **Red commit**: adds only the test. Fails on master with the 18
missing routes listed.
- **Green commit**: adds the allowlist seeded with that exact 18-route
set. Test passes at the current baseline.

## Local verification

- `go test ./cmd/server/ -run TestOpenAPICompleteness -v` → PASS at
baseline (`44/62 covered; 18 in allowlist; 18 gaps remain`).
- Ratchet validation: temporarily inserted
`r.HandleFunc("/api/ratchet-test-route", ...)` into `routes.go` → test
FAILED with that exact route name; reverted → test PASSES again.

## Files changed

- `cmd/server/openapi_completeness_test.go` (+203 / new)
- `cmd/server/openapi_known_gaps.json` (+24 / new)

## Preflight

`bash ~/.openclaw/skills/pr-preflight/scripts/run-all.sh origin/master`
→ all hard gates pass; no warnings.

## Out of scope

- Backfilling the 18 allowlisted routes into `openapi.go` (Phase 2 —
tracked separately).
- Schema validation of the spec against OpenAPI 3.0 (Phase 3 per the
issue).
- PR template checkbox update (Phase 2 follow-up).

Issue #1670 stays open for Phase 2.

---------

Co-authored-by: clawbot <bot@corescope.local>
 2026-06-12 01:52:12 -07:00