Files
meshcore-analyzer/cmd/server/rf_params_cache_test.go
T
efitenandClaude Opus 5.5 cab4f71c64 feat(nodes): LoRa radio settings per node from the MeshCore map (#1268) (#2209)
Fixes #1268.

Frequency, spreading factor, bandwidth and coding rate are per-device
settings that never travel in a packet. The public MeshCore map is the
only source. This PR adds them to the node detail views, opt-in, and
answers the open policy questions from the triage comments.

## Policy decisions

| Question (triage) | Answer in this PR |
|---|---|
| Opt-out for self-hosted instances | Opt-**in**: `rfParams.nodesUrl`
empty (default) = nothing fetched, `/api/rf-params` returns `enabled:
false`. Same model as `knownChannelsUrl`. |
| 33 MB every 6 h, thundering herd | Random 0-60 s delay before the
first fetch, then every `refreshHours` (default 6). The map sends gzip
(17 MB on the wire). It sends no ETag or Last-Modified, so conditional
requests are not possible. |
| Size cap | 128 MB on the decoded body (it is 51.5 MB today). Exceeding
it fails the fetch instead of parsing a truncated body. |
| Field shapes at the parse boundary | 64-hex key; 100-1100 MHz; SF
5-12; bandwidth from the LoRa set (7.8 to 500 kHz); CR 5-8. Invalid rows
are dropped. A body that is not a well-formed array fails the fetch. |
| Keep stale cache on anomaly | Any failure keeps the previous snapshot;
a failed presets fetch keeps the previous presets. `fetched_at` is
returned so an outage is visible rather than hidden. |
| Freshness | Only adverts within `maxAgeDays` (default 7) are kept at
fetch time and re-checked at serve time. |
| TLS pinning | Not done: Go's certificate validation plus the field
validation above. |

## What changes

- `cmd/server/rf_params_cache.go`: stream decoder, validation, cap
reader, preset parser (presets with identical values are merged, titles
joined), background refresh. In memory only, no DB writes, so the
read-only server invariant holds.
- `GET /api/rf-params`: `{enabled, fetched_at, source, presets, nodes}`;
`nodes` holds only nodes this instance knows, keyed by lower-case
pubkey. `/api/nodes` is unchanged, so its payload does not grow for
every consumer and the 500-row paging is unaffected.
- Node detail (side panel and full page): `LoRa` row with the four
values, the preset name when all four match one, and the age of the map
advert. Shown only when the feature is on and the server has fetched the
map at least once.
- No column in the Nodes table, on purpose: on our network such a column
showed one preset title in 814 of 1,366 rows and nothing in 443. Showing
only the nodes that differ from their region is the follow-up PR.
- `config.example.json` documents the block with the two URLs to use.

## Cost

- Measured on the real map payload (51.5 MB, 66,336 nodes): parse 0.52
s, 15 MB allocated in total, 26,815 nodes kept (advert within 7 days).
Runs in a background goroutine every 6 h.
- The endpoint walks the cached node list once with map lookups:
O(nodes).
- Frontend: one bulk fetch of `/api/rf-params` when a node detail opens
(client TTL 5 min), not one per node.

## Tests

- Go (`rf_params_cache_test.go`): parse with every rejection reason,
malformed bodies (not an array, truncated, wrong type, empty), preset
merge and invalid presets, cap reader (one byte over refused, exactly at
the cap accepted), fail-soft against an `httptest` upstream (503, HTML
body, presets-only failure), config defaults, route (disabled shape,
only known nodes, entry aged past max age dropped at serve time). Full
`cmd/server` suite passes locally (skipping
`TestSaveGeoFilterPreservesFileMode`, a Windows-only failure on clean
master).
- Frontend unit: `tests/unit/test-issue-1268-rf-params.js` (7 assertions
on the real file in `vm`), in `test-all.sh`; the full suite passes.
- E2E: `tests/e2e/test-issue-1268-rf-params-e2e.js` (registered in
`deploy.yml` and `scripts/non-unit-tests.json`). It checks the real
disabled state on the fixture server (no detail row), then serves
`/api/rf-params` through `page.route`: no table column, full detail page
with preset, side panel with raw values, and the hint for a node that is
not on the map. 4/4 locally, plus the existing node table and node
detail E2E files.

## Validation

On a staging instance with `rfParams` set to the map and preset URLs:

- Log: `[rf-params] 26821 nodes within max age, 20 presets`, about 6 s
after start.
- `/api/rf-params` returned 1,115 of our nodes: 988 on
869.618/SF8/BW62.5/CR8, 111 on the same with CR5, 15 on 433.65 MHz, 1 on
SF7.
- In the browser the detail row shows in the side panel and on the full
page, and the Nodes table keeps its original columns.

## Not done

- Highlighting nodes whose settings differ from their region's majority
(a badge in the Nodes list plus a filter) is the follow-up (separate
PR). In that follow-up a CR-only difference stays informational until it
is confirmed whether MeshCore radios use the explicit LoRa header (then
CR travels in the header and a mismatch still decodes).
- `maxAgeDays` is config-only; the customizer is a later step.
- No data is written to the DB, so the RF parameters are gone after a
restart until the next fetch (at most 60 s plus the fetch time).

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 09:26:45 +02:00

324 lines
11 KiB
Go

package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
const (
rfPK1 = "aa00000000000000000000000000000000000000000000000000000000000001"
rfPK2 = "aa00000000000000000000000000000000000000000000000000000000000002"
rfPK3 = "aa00000000000000000000000000000000000000000000000000000000000003"
)
var rfNow = time.Date(2026, 10, 14, 12, 0, 0, 0, time.UTC)
func rfNode(pk string, daysAgo int, params string) string {
return rfNodeAt(pk, rfNow.Add(-time.Duration(daysAgo)*24*time.Hour), params)
}
func rfNodeAt(pk string, at time.Time, params string) string {
ts := at.Format("2006-01-02T15:04:05.000Z")
p := "null"
if params != "" {
p = params
}
return fmt.Sprintf(`{"public_key":%q,"type":2,"adv_name":"x","last_advert":%q,"params":%s}`, pk, ts, p)
}
const rfEUNarrow = `{"freq":869.618,"bw":62.5,"cr":8,"sf":8}`
func TestParseRfNodes(t *testing.T) {
body := "[" + strings.Join([]string{
rfNode(strings.ToUpper(rfPK1), 1, rfEUNarrow), // kept, key lower-cased
rfNode(rfPK2, 30, rfEUNarrow), // older than max age
rfNode(rfPK3, 1, ""), // no params
rfNode("zz", 1, rfEUNarrow), // not a 64-hex key
rfNode(rfPK3[:63]+"4", 1, `{"freq":869.618,"bw":62.5,"cr":8,"sf":13}`), // SF out of range
rfNode(rfPK3[:63]+"5", 1, `{"freq":869.618,"bw":63,"cr":8,"sf":8}`), // not a LoRa bandwidth
rfNode(rfPK3[:63]+"6", 1, `{"freq":5000,"bw":62.5,"cr":8,"sf":8}`), // frequency out of range
rfNode(rfPK3[:63]+"7", 1, `{"freq":869.618,"bw":62.5,"cr":4,"sf":8}`), // CR out of range
rfNode(rfPK3[:63]+"8", 2, `{"freq":433.65,"bw":125,"cr":5,"sf":12}`), // kept
rfNodeAt(rfPK3[:63]+"9", rfNow.Add(2*24*time.Hour), rfEUNarrow), // dated in the future
rfNodeAt(rfPK3[:63]+"a", rfNow.AddDate(70, 0, 0), rfEUNarrow), // dated 2096
rfNodeAt(rfPK3[:63]+"b", rfNow.Add(5*time.Minute), rfEUNarrow), // kept, within clock skew
}, ",") + "]"
got, err := parseRfNodes(strings.NewReader(body), rfNow, 7*24*time.Hour)
if err != nil {
t.Fatal(err)
}
if len(got) != 3 {
t.Fatalf("kept %d nodes, want 3: %v", len(got), got)
}
p, ok := got[rfPK1]
if !ok || p.Freq != 869.618 || p.SF != 8 || p.BW != 62.5 || p.CR != 8 {
t.Errorf("rfPK1 = %+v, %v", p, ok)
}
if p.LastAdvert != "2026-10-13T12:00:00Z" {
t.Errorf("last_advert = %q", p.LastAdvert)
}
if _, ok := got[rfPK3[:63]+"8"]; !ok {
t.Error("433 MHz / BW125 / SF12 node should be kept")
}
if _, ok := got[rfPK3[:63]+"b"]; !ok {
t.Error("advert 5 min ahead (clock skew) should be kept")
}
}
func TestRfAdvertFresh(t *testing.T) {
maxAge := 7 * 24 * time.Hour
for _, c := range []struct {
name string
at time.Time
want bool
}{
{"now", rfNow, true},
{"at max age", rfNow.Add(-maxAge), true},
{"past max age", rfNow.Add(-maxAge - time.Second), false},
{"at the skew tolerance", rfNow.Add(rfAdvertFutureSkew), true},
{"past the skew tolerance", rfNow.Add(rfAdvertFutureSkew + time.Second), false},
{"dated 2096", time.Date(2096, 1, 1, 0, 0, 0, 0, time.UTC), false},
} {
if got := rfAdvertFresh(c.at, rfNow, maxAge); got != c.want {
t.Errorf("%s: got %v, want %v", c.name, got, c.want)
}
}
}
func TestParseRfNodesSkipsBadEntry(t *testing.T) {
body := "[" + strings.Join([]string{
rfNode(rfPK1, 1, `{"freq":869.618,"bw":62.5,"cr":8,"sf":"8"}`), // sf as a string
`{"public_key":42,"params":{}}`, // key as a number
rfNode(rfPK2, 1, rfEUNarrow),
}, ",") + "]"
got, err := parseRfNodes(strings.NewReader(body), rfNow, 7*24*time.Hour)
if err != nil {
t.Fatalf("one bad entry failed the whole decode: %v", err)
}
if _, ok := got[rfPK2]; !ok || len(got) != 1 {
t.Errorf("got %v, want only %s", got, rfPK2)
}
}
func TestRfNextWait(t *testing.T) {
refresh := 6 * time.Hour
for _, c := range []struct {
name string
hasData bool
failures int
want time.Duration
}{
{"data: regular refresh", true, 0, refresh},
{"data after failures: still the regular refresh", true, 3, refresh},
{"no data, first failure", false, 1, time.Minute},
{"no data, second failure", false, 2, 2 * time.Minute},
{"no data, fifth failure", false, 5, 16 * time.Minute},
{"no data, capped", false, 6, rfRetryMax},
{"no data, many failures stay capped", false, 60, rfRetryMax},
} {
if got := rfNextWait(c.hasData, c.failures, refresh); got != c.want {
t.Errorf("%s: got %v, want %v", c.name, got, c.want)
}
}
if got := rfNextWait(false, 10, 10*time.Minute); got != 10*time.Minute {
t.Errorf("retry wait must not exceed the refresh interval: got %v", got)
}
}
func TestParseRfNodesRejectsMalformed(t *testing.T) {
for name, body := range map[string]string{
"not an array": `{"nodes":[]}`,
"truncated": "[" + rfNode(rfPK1, 1, rfEUNarrow),
"empty payload": ``,
} {
if _, err := parseRfNodes(strings.NewReader(body), rfNow, 7*24*time.Hour); err == nil {
t.Errorf("%s: want an error", name)
}
}
}
func TestParseRfPresets(t *testing.T) {
body := `{"config":{"suggested_radio_settings":{"entries":[
{"title":"EU/UK (Narrow)","frequency":"869.618","spreading_factor":"8","bandwidth":"62.5","coding_rate":"8"},
{"title":"Switzerland","frequency":"869.618","spreading_factor":"8","bandwidth":"62.5","coding_rate":"8"},
{"title":"USA/Canada (Recommended)","frequency":"910.525","spreading_factor":"7","bandwidth":"62.5","coding_rate":"5"},
{"title":"Broken","frequency":"abc","spreading_factor":"8","bandwidth":"62.5","coding_rate":"8"},
{"title":"","frequency":"869.618","spreading_factor":"9","bandwidth":"62.5","coding_rate":"8"}
]}}}`
got, err := parseRfPresets(strings.NewReader(body))
if err != nil {
t.Fatal(err)
}
want := []RfPreset{
{Title: "EU/UK (Narrow) / Switzerland", Freq: 869.618, SF: 8, BW: 62.5, CR: 8},
{Title: "USA/Canada (Recommended)", Freq: 910.525, SF: 7, BW: 62.5, CR: 5},
}
if fmt.Sprint(got) != fmt.Sprint(want) {
t.Errorf("presets = %+v\nwant %+v", got, want)
}
}
func TestCapReaderRefusesOversize(t *testing.T) {
body := "[" + rfNode(rfPK1, 1, rfEUNarrow) + "]"
_, err := parseRfNodes(newCapReader(strings.NewReader(body), int64(len(body)-1)), rfNow, 7*24*time.Hour)
if !errors.Is(err, errRfTooLarge) {
t.Errorf("err = %v, want errRfTooLarge", err)
}
if _, err := parseRfNodes(newCapReader(strings.NewReader(body), int64(len(body))), rfNow, 7*24*time.Hour); err != nil {
t.Errorf("body exactly at the cap: %v", err)
}
}
func rfUpstream(t *testing.T, nodes, presets *string, status *int) *httptest.Server {
t.Helper()
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if *status != 200 {
w.WriteHeader(*status)
return
}
if r.URL.Path == "/presets" {
fmt.Fprint(w, *presets)
return
}
fmt.Fprint(w, *nodes)
}))
t.Cleanup(srv.Close)
return srv
}
func TestRfParamsCacheFailSoft(t *testing.T) {
nodes := "[" + rfNode(rfPK1, 1, rfEUNarrow) + "]"
presets := `{"config":{"suggested_radio_settings":{"entries":[{"title":"EU/UK (Narrow)","frequency":"869.618","spreading_factor":"8","bandwidth":"62.5","coding_rate":"8"}]}}}`
status := 200
up := rfUpstream(t, &nodes, &presets, &status)
c := newRfParamsCache(&RfParamsConfig{NodesURL: up.URL + "/nodes", PresetsURL: up.URL + "/presets"})
c.now = func() time.Time { return rfNow }
if err := c.fetchOnce(context.Background()); err != nil {
t.Fatal(err)
}
snap := c.load()
if snap == nil || len(snap.Nodes) != 1 || len(snap.Presets) != 1 || !snap.FetchedAt.Equal(rfNow) {
t.Fatalf("first fetch snapshot = %+v", snap)
}
// Upstream down: the previous snapshot stays.
status = 503
if err := c.fetchOnce(context.Background()); err == nil {
t.Error("want an error on 503")
}
if c.load() != snap {
t.Error("a failed fetch must keep the previous snapshot")
}
// Garbage body: same.
status = 200
nodes = `<html>maintenance</html>`
if err := c.fetchOnce(context.Background()); err == nil {
t.Error("want an error on a non-JSON body")
}
if c.load() != snap {
t.Error("a parse failure must keep the previous snapshot")
}
// Presets failing alone keeps the old presets but takes the new nodes.
nodes = "[" + rfNode(rfPK1, 1, rfEUNarrow) + "," + rfNode(rfPK2, 1, rfEUNarrow) + "]"
presets = `not json`
if err := c.fetchOnce(context.Background()); err == nil || !strings.Contains(err.Error(), "presets") {
t.Errorf("want a presets error, got %v", err)
}
if s := c.load(); len(s.Nodes) != 2 || len(s.Presets) != 1 {
t.Errorf("after presets failure: %d nodes, %d presets", len(s.Nodes), len(s.Presets))
}
}
func TestRfParamsConfigDefaults(t *testing.T) {
var nilCfg *RfParamsConfig
if nilCfg.enabled() {
t.Error("nil config must be disabled")
}
if (&RfParamsConfig{}).enabled() {
t.Error("empty nodesUrl must be disabled")
}
c := &RfParamsConfig{NodesURL: "x"}
if c.refresh() != 6*time.Hour || c.maxAge() != 7*24*time.Hour {
t.Errorf("defaults: refresh %v, maxAge %v", c.refresh(), c.maxAge())
}
c = &RfParamsConfig{NodesURL: "x", RefreshHours: 12, MaxAgeDays: 3}
if c.refresh() != 12*time.Hour || c.maxAge() != 3*24*time.Hour {
t.Errorf("configured: refresh %v, maxAge %v", c.refresh(), c.maxAge())
}
}
func TestRfParamsRoute(t *testing.T) {
srv, router := setupTestServer(t)
get := func() RfParamsResponse {
w := httptest.NewRecorder()
router.ServeHTTP(w, httptest.NewRequest("GET", "/api/rf-params", nil))
if w.Code != 200 {
t.Fatalf("status %d: %s", w.Code, w.Body.String())
}
var resp RfParamsResponse
if err := json.Unmarshal(w.Body.Bytes(), &resp); err != nil {
t.Fatal(err)
}
return resp
}
// Disabled: well-formed and empty.
resp := get()
if resp.Enabled || resp.FetchedAt != nil || resp.Nodes == nil || resp.Presets == nil {
t.Errorf("disabled response = %+v", resp)
}
// Enabled: only nodes this instance knows, and only within max age.
nodes, _ := srv.store.getCachedNodesAndPM()
if len(nodes) == 0 {
t.Fatal("test server has no nodes")
}
known := strings.ToLower(nodes[0].PublicKey)
c := newRfParamsCache(&RfParamsConfig{NodesURL: "http://unused", MaxAgeDays: 7})
c.ptr.Store(&rfParamsSnapshot{
FetchedAt: time.Now().UTC(),
Source: "http://unused",
Nodes: map[string]RfParams{
known: {Freq: 869.618, SF: 8, BW: 62.5, CR: 8, LastAdvert: time.Now().UTC().Format(time.RFC3339)},
rfPK3: {Freq: 869.618, SF: 8, BW: 62.5, CR: 8, LastAdvert: time.Now().UTC().Format(time.RFC3339)},
},
Presets: []RfPreset{{Title: "EU/UK (Narrow)", Freq: 869.618, SF: 8, BW: 62.5, CR: 8}},
})
srv.rfParams = c
resp = get()
if !resp.Enabled || resp.FetchedAt == nil || len(resp.Presets) != 1 {
t.Errorf("enabled response = %+v", resp)
}
if _, ok := resp.Nodes[known]; !ok || len(resp.Nodes) != 1 {
t.Errorf("nodes = %v, want only the known node %s", resp.Nodes, known)
}
// A snapshot entry that aged past maxAgeDays since the fetch is dropped at serve time.
snap := c.load()
old := snap.Nodes[known]
old.LastAdvert = time.Now().UTC().Add(-8 * 24 * time.Hour).Format(time.RFC3339)
snap.Nodes[known] = old
if resp = get(); len(resp.Nodes) != 0 {
t.Errorf("aged entry still served: %v", resp.Nodes)
}
// An entry dated in the future is not served as a fresh advert.
future := snap.Nodes[known]
future.LastAdvert = time.Now().UTC().Add(48 * time.Hour).Format(time.RFC3339)
snap.Nodes[known] = future
if resp = get(); len(resp.Nodes) != 0 {
t.Errorf("future-dated entry served: %v", resp.Nodes)
}
}