Files
meshcore-analyzer/internal/dbschema/observer_radios_test.go
T
TeTeHackoandClaude Opus 5.5 77a5f14925 feat(observers): radio setting per observation from the observer's radio history (#1869) (#2292)
Part of #1869.

## Problem

`observers.radio` keeps only the **last** `MHz,kHz,SF,CR` an observer
reported. Once an observer is retuned (say from SF7 to the EU narrow SF8
setting to listen to a neighbouring mesh), every packet it ever heard
reads as heard on the new setting, and there is no way to tell an SF7
packet from an SF8 one.

## What this does

As #1869 proposes, the packet data model is untouched: the setting of an
observation is resolved from its observer and time.

**Ingestor** (all schema in `dbschema.Apply`)

- `observer_radios`: one row per stretch of time an observer, and each
radio of a multi-radio node, stayed on one setting. Every live status
extends the current row and a change starts a new one; retained replays
never extend a row. On the first run it is seeded with every observer's
current radio, which is what the UI already showed for the past.
- Settings are stored canonical through `dbschema.CanonicalRadio`, so
`869.4320068,62.5,7,5` and `869.432,62.50,7,5` are one setting.
- `observation_radios` (`observation_id → rx_radio_id`): multi-radio
openHop nodes tag each packet with `rx_radio_id` and list `radios: [{id,
radio}]` in their status, so their observations resolve per radio.
- The table is sparse: single-radio nodes never send the field, so
`observations` and every query that scans it stay untouched.
  - The row id comes from the observation upsert via `RETURNING id`.
- Rows are pruned with their transmissions, and a purged observer takes
its history along.

**Server** (read-only)

- No `AssertReady` change: an absent table reads as empty and is probed
again later, the same rule as `advert_route_evidence`.
- The history is held in memory. The live poller checks it on every tick
(two primary-key lookups), so a retune or a new `rx_radio_id` is known
before the observation is broadcast. A full reload runs every 10 min.
- `radio` is added to observations, packets, grouped rows (taken from
the header observation) and the live feed.
- `GET /api/packets?radio=MHz,kHz,SF,CR` filters by setting.
- Combined with `observer=`, it means *that* observer heard it on that
setting.
- A radio query is served from memory instead of falling back to SQL,
which would return the window unfiltered.
- `GET /api/radios` (named struct) and `radio_history` on `GET
/api/observers/{id}`.

**Frontend**

- Packet filter fields `radio`, `freq`, `bw`, `sf`, `cr`, e.g. `sf == 8`
or `freq > 869.5 && bw == 62.5`.
- A Radio row in the packet detail and a Radio column in its
observations table (only when the server reports one).
- A radio history table on the observer page once there is more than one
stretch.
- `RfParams.radioText` formats the setting. It escapes all five
characters and returns non-numeric input as it came, escaped.

## Performance

`txMatchesRadio` runs once per observation inside `filterPackets`.

- Each observation costs one map lookup by observer id.
- Observers with a single stretch (nearly all) answer from that lookup
directly.
- Retuned observers add a binary search over their few stretches.
- Multi-radio observers add one lookup by observation id.
- The snapshot is loaded once per filter call.

`BenchmarkTxMatchesRadio`, 30K packets / 150K observations (20
observers, 2 of them retuned):

```
BenchmarkTxMatchesRadio-24    602    2056924 ns/op    0 B/op    0 allocs/op
```

Memory is one entry per stretch plus one per multi-radio observation;
nothing is added per observation otherwise.

## Tests

- **dbschema:** seed runs once, canonical spellings, the tables exist
without `observers.radio`.
- **ingestor:**
  - stretches extend and split; canonical spellings do not split
- two radios of one node hearing one transmission over two paths each
keep their own radio
  - a retained status does not write history
  - prune and purge
  - `rx_radio_id` reaches `PacketData`
- **server:**
- resolution before, between and after stretches, and with an unknown
radio id
  - `observer` + `radio`, and grouped rows carry `radio`
  - incremental refresh, `/api/radios`, `radio_history`, missing tables
  - the benchmark above
- **frontend:** the new filter fields in
`tests/unit/test-packet-filter.js`, and XSS cases through the real
`RfParams.radioText` and the history renderer.
- **E2E:** `tests/e2e/test-radio-setting-1869-e2e.js` checks the grouped
rows' radio, `sf ==` on the packets page, the detail Radio row and the
observer history. It is added to `scripts/non-unit-tests.json` and to
shard 2.

`go test -race` in `cmd/server`, `cmd/ingestor` and `internal/dbschema`
passes, along with `sh test-all.sh`, `eslint@8` (0 errors) and the XSS
diff gate. The new E2E test and `test-e2e-playwright.js` pass against
the fixture.

## Browser validation

This runs on a live instance with six observers, one of them retuned to
869.618/62.5/SF8/CR8:

- The packet detail shows `Radio: 869.432 MHz · SF7 · BW 62.5 kHz · CR
4/5`.
- `sf == 7` shows 106 of 106 grouped rows and `sf == 8` shows 0 of 107;
the SF8 observer has not heard anything yet.
- `/api/radios` lists both settings.
- No page errors.

## Notes

- **The customizer (rule 8)** gets nothing: the only constants are
server-side refresh intervals.
- **Attribution timing:** packets heard between a retune and the
observer's next status message (minutes) are attributed to the previous
setting, because a stretch starts when the status is handled. This is
documented at `RecordObserverRadios`.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-26 21:08:40 +01:00

88 lines
2.9 KiB
Go

package dbschema
import (
"testing"
)
// #1869: the first Apply seeds observer_radios with every observer's
// current radio, once; later Applies leave the history alone.
func TestEnsureObserverRadiosSeedsOnce(t *testing.T) {
db := minimalDB(t)
defer db.Close()
for _, s := range []string{
`ALTER TABLE observers ADD COLUMN radio TEXT`,
`ALTER TABLE observers ADD COLUMN first_seen TEXT`,
`ALTER TABLE observers ADD COLUMN last_seen TEXT`,
`INSERT INTO observers (id, name, radio, first_seen, last_seen) VALUES
('A', 'a', ' 869.4320068,62.50,7,5', '2026-08-01T00:00:00Z', '2026-10-25T09:00:00Z'),
('B', 'b', NULL, '2026-08-01T00:00:00Z', '2026-10-25T09:00:00Z'),
('C', 'c', ' ', '2026-08-01T00:00:00Z', '2026-10-25T09:00:00Z')`,
} {
if _, err := db.Exec(s); err != nil {
t.Fatal(err)
}
}
if err := ensureObserverRadios(db, nil); err != nil {
t.Fatal(err)
}
var obs, radio, first, last string
if err := db.QueryRow(`SELECT observer_id, radio, first_seen, last_seen FROM observer_radios`).
Scan(&obs, &radio, &first, &last); err != nil {
t.Fatal(err)
}
if obs != "A" || radio != "869.432,62.5,7,5" || first != "2026-08-01T00:00:00Z" || last != "2026-10-25T09:00:00Z" {
t.Errorf("seed row = %s %s %s %s", obs, radio, first, last)
}
// Retuned since: a second Apply must not add a row that claims the new
// setting covered the whole past.
if _, err := db.Exec(`UPDATE observers SET radio = '869.618,62.5,8,8'`); err != nil {
t.Fatal(err)
}
if err := ensureObserverRadios(db, func(string, ...interface{}) {}); err != nil {
t.Fatal(err)
}
var n int
if err := db.QueryRow(`SELECT COUNT(*) FROM observer_radios`).Scan(&n); err != nil {
t.Fatal(err)
}
if n != 1 {
t.Errorf("rows after second Apply = %d, want 1", n)
}
}
// A fixture without observers.radio still gets the tables.
func TestEnsureObserverRadiosWithoutRadioColumn(t *testing.T) {
db := minimalDB(t)
defer db.Close()
if err := Apply(db, nil); err != nil {
t.Fatal(err)
}
for _, table := range []string{"observer_radios", "observation_radios"} {
var name string
if err := db.QueryRow(`SELECT name FROM sqlite_master WHERE type='table' AND name=?`, table).Scan(&name); err != nil {
t.Errorf("table %s missing: %v", table, err)
}
}
}
func TestCanonicalRadio(t *testing.T) {
for in, want := range map[string]string{
"869.4320068,62.5,7,5": "869.432,62.5,7,5",
" 869.618,62.50,8,8 ": "869.618,62.5,8,8",
"910.5250244,250,11,5": "910.525,250,11,5",
"869.618, 62.5, 8, 8": "869.618,62.5,8,8",
"433.175000,125.0,9,6": "433.175,125,9,6",
"SX1262": "SX1262",
"869.432,62.5,seven,5": "869.432,62.5,seven,5",
"869.432,,7,5": "869.432,,7,5",
"NaN,62.5,7,5": "NaN,62.5,7,5",
"Inf,62.5,7,5": "Inf,62.5,7,5",
"": "",
} {
if got := CanonicalRadio(in); got != want {
t.Errorf("CanonicalRadio(%q) = %q, want %q", in, got, want)
}
}
}