Files
meshcore-analyzer/cmd/server/packet_scope_name_test.go
T
efitenandClaude Opus 5 376c3e9f4a fix(packets): surface the transport region scope — detail pane row and a sortable Scope column (#1894)
## Summary

`transmissions.scope_name` (#899) reached the database but never reached
the UI. Two problems, one dead feature and one missing surface.

## 1. The detail pane's Scope row was dead

`public/packets.js:3279` has rendered a **Scope** row since #899, gated
on `pkt.scope_name != null`. It never fires in practice.

`/api/packets` and `/api/packets/{id}` are served from the in-memory
`PacketStore`. The store reads `scope_name` out of SQLite fine
(`store.go:888`, `chunked_load.go:551` → `StoreTx.ScopeName`), but
`txToMap()` did not put it in the JSON. Only packets old enough to have
been evicted from the store — and thus served by the SQLite fallback in
`db.go`, which does emit it — could ever show a scope.

Verified against a live instance before the fix:

```
GET /api/packets/552e9687f1525537 → packet keys:
['_parsedPath','decoded_json','direction','first_seen','hash','id',
 'observation_count','observations','observer_iata','observer_id',
 'observer_name','path_json','payload_type','raw_hex','route_type','rssi','snr','timestamp']
```

No `scope_name`.

### The NULL / "" distinction

`StoreTx.ScopeName` was typed `string`, which collapses the two states
the frontend distinguishes:

| DB value | Meaning | UI |
|---|---|---|
| `NULL` | not transport-scoped | row hidden |
| `""` | transport-scoped, region matched no configured key | muted
"unknown scope" |
| `"#be"` | matched region | the region name |

`route_type` is **not** a usable proxy for that distinction: the
ingestor writes NULL for a transport route whose `transport_code_1` is
`0000` (`cmd/ingestor/db.go:1576` — `IsTransportScoped = route_type IN
(0,3) AND Code1 ≠ "0000"`). So the field is now `*string`, with
`nullStrPtr` preserving what `nullStrVal` collapsed.

The two internal consumers (`TransportedScopes` #1751,
`relayEntry.scope`) only care about non-empty named scopes and are
unchanged in behaviour.

## 2. New: a Scope column on the packets table

The scope was only reachable one packet at a time by opening the detail
pane. It now has its own sortable column between Type and Observer,
visible by default.

The default view is **Group by Hash**, served by mappers that did not
carry `scope_name` at all — so the column would have been empty in
exactly the view most people look at. Both grouped paths now select and
emit it: `groupedTxsToPage` in the store, and the dedicated grouped
query in the DB fallback (v3 and legacy shapes).

Rendering lives in `scopeCellHtml` (`public/app.js`, next to
`transportBadge`) and is used on all three row-render sites — group
header, expanded children, flat rows — so the column and the detail pane
cannot drift apart.

**Sorting** pins the empties last in both directions, as the nodes table
already does for `default_scope`. Only ~8% of packets carry a scope, so
an ascending sort would otherwise bury every scoped row under a wall of
dashes.

**Filtering**: `packet-filter.js` gains a `scope` field, so the cell is
click-to-filter like Type and Observer, and `scope == "#be"` works in
the filter bar.

**Column prefs**: a `packets-known-cols` companion key. The
`packets-visible-cols` array alone cannot distinguish "this column did
not exist when you saved" from "you unchecked it", so any new column
arrives silently hidden for every returning visitor. Keys absent from
`known-cols` get the default treatment; keys the visitor actually hid
stay hidden — there is a test for that second half specifically.

## Tests

Each watched fail first.

**Go** (`cmd/server/packet_scope_name_test.go`)
- `txToMap` unit tests for all three states, including a JSON round-trip
so a typed nil `*string` cannot pass as `null`
- end-to-end through `/api/packets/{hash}`
- `groupedTxsToPage` unit + end-to-end through
`/api/packets?groupByHash=true`, across **both** the store-backed and
DB-fallback paths
- `transported_scopes_1751_test.go`: the "no scope" guard now covers
both non-values (nil and a pointer to `""`)

**Frontend**
- `test-frontend-helpers.js`: `scopeCellHtml` three states + escaping
- `test-packet-filter.js`: `scope` matching, case-insensitivity, and
`FIELDS` registration
- `test-packets-scope-column.js` (new Playwright e2e): header position,
default visibility, one cell per row, em dash on non-transport rows,
empties-last sorting, the Columns toggle, and the prefs backfill

## Verification

Deployed and checked against a live instance:

```
/api/packets?groupByHash=true&limit=500 → scope_name present on 500/500,
                                          59 with a matched region, 1 unknown-scope
test-packets-scope-column.js            → 7 passed, 0 failed
cd cmd/server && go test ./...          → ok
```

Two pre-existing failures, unrelated and equally red on an unmodified
checkout: `test-e2e-playwright.js` "Customizer open does not overwrite
server home config" and `test-observer-iata-1188-e2e.js` (timeout on
`[data-loaded="true"]`).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-02 18:34:59 +02:00

231 lines
8.0 KiB
Go

package main
import (
"encoding/json"
"net/http/httptest"
"testing"
"time"
"github.com/gorilla/mux"
)
// The packet detail panel (public/packets.js) renders a "Scope" row gated on
// `pkt.scope_name != null`, distinguishing three states that transmissions.scope_name
// encodes: SQL NULL (not transport-scoped, row hidden), "" (transport-scoped but the
// region did not match a configured key → "unknown scope") and "#name" (matched
// region). txToMap is the shape /api/packets and /api/packets/{id} serve from the
// in-memory store, so it must carry all three states through.
func TestTxToMapScopeNameMatchedRegion(t *testing.T) {
scope := "#belgium"
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: &scope})
if m["scope_name"] != "#belgium" {
t.Errorf("scope_name = %#v, want %q", m["scope_name"], "#belgium")
}
}
func TestTxToMapScopeNameUnknownScope(t *testing.T) {
scope := ""
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: &scope})
v, ok := m["scope_name"]
if !ok {
t.Fatal("scope_name key missing for a transport-scoped packet with an unmatched region")
}
if v != "" {
t.Errorf("scope_name = %#v, want %q (frontend renders this as 'unknown scope')", v, "")
}
}
func TestTxToMapScopeNameNotTransportScoped(t *testing.T) {
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: nil})
if m["scope_name"] != nil {
t.Errorf("scope_name = %#v, want nil for a non-transport-scoped packet", m["scope_name"])
}
// A typed nil *string in the map would marshal as null but compare non-nil in
// Go; assert the JSON the browser actually receives.
b, err := json.Marshal(m)
if err != nil {
t.Fatalf("marshal: %v", err)
}
var decoded map[string]interface{}
if err := json.Unmarshal(b, &decoded); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if decoded["scope_name"] != nil {
t.Errorf("JSON scope_name = %#v, want null", decoded["scope_name"])
}
}
// TestPacketDetailExposesScopeName is the end-to-end guard: the packet-detail
// endpoint is served from the in-memory store, so scope_name must survive the
// SQL scan (nullStrPtr) and the map conversion (txToMap) with all three states
// intact. Before this test, txToMap dropped the field entirely and the Scope row
// only ever rendered for packets old enough to fall through to the DB.
func TestPacketDetailExposesScopeName(t *testing.T) {
db := setupTestDB(t)
if _, err := db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil {
t.Fatalf("add scope_name column: %v", err)
}
db.hasScopeName = true
now := time.Now().UTC().Format(time.RFC3339)
// route_type 1 = FLOOD (never transport-scoped → NULL); 0 = TRANSPORT_FLOOD.
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
VALUES ('AABB', 'aaaaaaaaaaaaaaa1', ?, 1, 4)`, now); err != nil {
t.Fatalf("insert unscoped: %v", err)
}
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
VALUES ('AABB', 'aaaaaaaaaaaaaaa2', ?, 0, 4, '')`, now); err != nil {
t.Fatalf("insert unknown-scope: %v", err)
}
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
VALUES ('AABB', 'aaaaaaaaaaaaaaa3', ?, 0, 4, '#belgium')`, now); err != nil {
t.Fatalf("insert matched-scope: %v", err)
}
srv := NewServer(db, &Config{Port: 3000}, NewHub())
store := NewPacketStore(db, nil)
if err := store.Load(); err != nil {
t.Fatalf("store.Load: %v", err)
}
if !store.WaitIndexesReady(5 * time.Second) {
t.Fatal("background indexes never became ready")
}
srv.store = store
router := mux.NewRouter()
srv.RegisterRoutes(router)
cases := []struct {
name string
hash string
want interface{}
}{
{"not transport-scoped", "aaaaaaaaaaaaaaa1", nil},
{"transport-scoped, region unmatched", "aaaaaaaaaaaaaaa2", ""},
{"transport-scoped, region matched", "aaaaaaaaaaaaaaa3", "#belgium"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
if store.GetPacketByHash(tc.hash) == nil {
t.Fatalf("precondition: %s not in store (would hit the DB fallback)", tc.hash)
}
req := httptest.NewRequest("GET", "/api/packets/"+tc.hash, nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
}
var body map[string]interface{}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("unmarshal: %v", err)
}
pkt, ok := body["packet"].(map[string]interface{})
if !ok {
t.Fatal("expected packet object")
}
if got := pkt["scope_name"]; got != tc.want {
t.Errorf("scope_name = %#v, want %#v", got, tc.want)
}
})
}
}
// --- grouped view ---
//
// The Packets tab defaults to "Group by Hash", which is served by a separate
// mapper (groupedTxsToPage in the store, a dedicated query in the DB fallback).
// The Scope column reads scope_name off those rows, so both paths must carry it.
func TestGroupedTxsToPageCarriesScopeName(t *testing.T) {
matched := "#belgium"
unmatched := ""
txs := []*StoreTx{
{ID: 1, Hash: "aa", ScopeName: &matched},
{ID: 2, Hash: "bb", ScopeName: &unmatched},
{ID: 3, Hash: "cc", ScopeName: nil},
}
res := groupedTxsToPage(txs, len(txs), 0, len(txs))
want := []interface{}{"#belgium", "", nil}
for i, w := range want {
if got := res.Packets[i]["scope_name"]; got != w {
t.Errorf("packet %d: scope_name = %#v, want %#v", i, got, w)
}
}
}
func TestGroupedPacketsEndpointExposesScopeName(t *testing.T) {
db := setupTestDB(t)
if _, err := db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil {
t.Fatalf("add scope_name column: %v", err)
}
db.hasScopeName = true
if _, err := db.conn.Exec(`DELETE FROM transmissions`); err != nil {
t.Fatalf("clear transmissions: %v", err)
}
now := time.Now().UTC().Format(time.RFC3339)
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
VALUES ('AABB', 'bbbbbbbbbbbbbbb1', ?, 1, 4)`, now); err != nil {
t.Fatalf("insert unscoped: %v", err)
}
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
VALUES ('AABB', 'bbbbbbbbbbbbbbb2', ?, 0, 4, '')`, now); err != nil {
t.Fatalf("insert unknown-scope: %v", err)
}
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
VALUES ('AABB', 'bbbbbbbbbbbbbbb3', ?, 0, 4, '#belgium')`, now); err != nil {
t.Fatalf("insert matched-scope: %v", err)
}
want := map[string]interface{}{
"bbbbbbbbbbbbbbb1": nil,
"bbbbbbbbbbbbbbb2": "",
"bbbbbbbbbbbbbbb3": "#belgium",
}
// Both the store-backed path and the DB-only fallback must agree.
for _, withStore := range []bool{true, false} {
name := "store"
if !withStore {
name = "db"
}
t.Run(name, func(t *testing.T) {
srv := NewServer(db, &Config{Port: 3000}, NewHub())
if withStore {
store := NewPacketStore(db, nil)
if err := store.Load(); err != nil {
t.Fatalf("store.Load: %v", err)
}
if !store.WaitIndexesReady(5 * time.Second) {
t.Fatal("background indexes never became ready")
}
srv.store = store
}
router := mux.NewRouter()
srv.RegisterRoutes(router)
req := httptest.NewRequest("GET", "/api/packets?groupByHash=true&limit=50", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != 200 {
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
}
var body struct {
Packets []map[string]interface{} `json:"packets"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("unmarshal: %v", err)
}
if len(body.Packets) != 3 {
t.Fatalf("expected 3 grouped packets, got %d", len(body.Packets))
}
for _, p := range body.Packets {
h, _ := p["hash"].(string)
if got := p["scope_name"]; got != want[h] {
t.Errorf("%s: scope_name = %#v, want %#v", h, got, want[h])
}
}
})
}
}