mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-11 16:45:35 +00:00
## Summary `transmissions.scope_name` (#899) reached the database but never reached the UI. Two problems, one dead feature and one missing surface. ## 1. The detail pane's Scope row was dead `public/packets.js:3279` has rendered a **Scope** row since #899, gated on `pkt.scope_name != null`. It never fires in practice. `/api/packets` and `/api/packets/{id}` are served from the in-memory `PacketStore`. The store reads `scope_name` out of SQLite fine (`store.go:888`, `chunked_load.go:551` → `StoreTx.ScopeName`), but `txToMap()` did not put it in the JSON. Only packets old enough to have been evicted from the store — and thus served by the SQLite fallback in `db.go`, which does emit it — could ever show a scope. Verified against a live instance before the fix: ``` GET /api/packets/552e9687f1525537 → packet keys: ['_parsedPath','decoded_json','direction','first_seen','hash','id', 'observation_count','observations','observer_iata','observer_id', 'observer_name','path_json','payload_type','raw_hex','route_type','rssi','snr','timestamp'] ``` No `scope_name`. ### The NULL / "" distinction `StoreTx.ScopeName` was typed `string`, which collapses the two states the frontend distinguishes: | DB value | Meaning | UI | |---|---|---| | `NULL` | not transport-scoped | row hidden | | `""` | transport-scoped, region matched no configured key | muted "unknown scope" | | `"#be"` | matched region | the region name | `route_type` is **not** a usable proxy for that distinction: the ingestor writes NULL for a transport route whose `transport_code_1` is `0000` (`cmd/ingestor/db.go:1576` — `IsTransportScoped = route_type IN (0,3) AND Code1 ≠ "0000"`). So the field is now `*string`, with `nullStrPtr` preserving what `nullStrVal` collapsed. The two internal consumers (`TransportedScopes` #1751, `relayEntry.scope`) only care about non-empty named scopes and are unchanged in behaviour. ## 2. New: a Scope column on the packets table The scope was only reachable one packet at a time by opening the detail pane. It now has its own sortable column between Type and Observer, visible by default. The default view is **Group by Hash**, served by mappers that did not carry `scope_name` at all — so the column would have been empty in exactly the view most people look at. Both grouped paths now select and emit it: `groupedTxsToPage` in the store, and the dedicated grouped query in the DB fallback (v3 and legacy shapes). Rendering lives in `scopeCellHtml` (`public/app.js`, next to `transportBadge`) and is used on all three row-render sites — group header, expanded children, flat rows — so the column and the detail pane cannot drift apart. **Sorting** pins the empties last in both directions, as the nodes table already does for `default_scope`. Only ~8% of packets carry a scope, so an ascending sort would otherwise bury every scoped row under a wall of dashes. **Filtering**: `packet-filter.js` gains a `scope` field, so the cell is click-to-filter like Type and Observer, and `scope == "#be"` works in the filter bar. **Column prefs**: a `packets-known-cols` companion key. The `packets-visible-cols` array alone cannot distinguish "this column did not exist when you saved" from "you unchecked it", so any new column arrives silently hidden for every returning visitor. Keys absent from `known-cols` get the default treatment; keys the visitor actually hid stay hidden — there is a test for that second half specifically. ## Tests Each watched fail first. **Go** (`cmd/server/packet_scope_name_test.go`) - `txToMap` unit tests for all three states, including a JSON round-trip so a typed nil `*string` cannot pass as `null` - end-to-end through `/api/packets/{hash}` - `groupedTxsToPage` unit + end-to-end through `/api/packets?groupByHash=true`, across **both** the store-backed and DB-fallback paths - `transported_scopes_1751_test.go`: the "no scope" guard now covers both non-values (nil and a pointer to `""`) **Frontend** - `test-frontend-helpers.js`: `scopeCellHtml` three states + escaping - `test-packet-filter.js`: `scope` matching, case-insensitivity, and `FIELDS` registration - `test-packets-scope-column.js` (new Playwright e2e): header position, default visibility, one cell per row, em dash on non-transport rows, empties-last sorting, the Columns toggle, and the prefs backfill ## Verification Deployed and checked against a live instance: ``` /api/packets?groupByHash=true&limit=500 → scope_name present on 500/500, 59 with a matched region, 1 unknown-scope test-packets-scope-column.js → 7 passed, 0 failed cd cmd/server && go test ./... → ok ``` Two pre-existing failures, unrelated and equally red on an unmodified checkout: `test-e2e-playwright.js` "Customizer open does not overwrite server home config" and `test-observer-iata-1188-e2e.js` (timeout on `[data-loaded="true"]`). 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
231 lines
8.0 KiB
Go
231 lines
8.0 KiB
Go
package main
|
|
|
|
import (
|
|
"encoding/json"
|
|
"net/http/httptest"
|
|
"testing"
|
|
"time"
|
|
|
|
"github.com/gorilla/mux"
|
|
)
|
|
|
|
// The packet detail panel (public/packets.js) renders a "Scope" row gated on
|
|
// `pkt.scope_name != null`, distinguishing three states that transmissions.scope_name
|
|
// encodes: SQL NULL (not transport-scoped, row hidden), "" (transport-scoped but the
|
|
// region did not match a configured key → "unknown scope") and "#name" (matched
|
|
// region). txToMap is the shape /api/packets and /api/packets/{id} serve from the
|
|
// in-memory store, so it must carry all three states through.
|
|
|
|
func TestTxToMapScopeNameMatchedRegion(t *testing.T) {
|
|
scope := "#belgium"
|
|
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: &scope})
|
|
if m["scope_name"] != "#belgium" {
|
|
t.Errorf("scope_name = %#v, want %q", m["scope_name"], "#belgium")
|
|
}
|
|
}
|
|
|
|
func TestTxToMapScopeNameUnknownScope(t *testing.T) {
|
|
scope := ""
|
|
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: &scope})
|
|
v, ok := m["scope_name"]
|
|
if !ok {
|
|
t.Fatal("scope_name key missing for a transport-scoped packet with an unmatched region")
|
|
}
|
|
if v != "" {
|
|
t.Errorf("scope_name = %#v, want %q (frontend renders this as 'unknown scope')", v, "")
|
|
}
|
|
}
|
|
|
|
func TestTxToMapScopeNameNotTransportScoped(t *testing.T) {
|
|
m := txToMap(&StoreTx{ID: 1, Hash: "aa", ScopeName: nil})
|
|
if m["scope_name"] != nil {
|
|
t.Errorf("scope_name = %#v, want nil for a non-transport-scoped packet", m["scope_name"])
|
|
}
|
|
// A typed nil *string in the map would marshal as null but compare non-nil in
|
|
// Go; assert the JSON the browser actually receives.
|
|
b, err := json.Marshal(m)
|
|
if err != nil {
|
|
t.Fatalf("marshal: %v", err)
|
|
}
|
|
var decoded map[string]interface{}
|
|
if err := json.Unmarshal(b, &decoded); err != nil {
|
|
t.Fatalf("unmarshal: %v", err)
|
|
}
|
|
if decoded["scope_name"] != nil {
|
|
t.Errorf("JSON scope_name = %#v, want null", decoded["scope_name"])
|
|
}
|
|
}
|
|
|
|
// TestPacketDetailExposesScopeName is the end-to-end guard: the packet-detail
|
|
// endpoint is served from the in-memory store, so scope_name must survive the
|
|
// SQL scan (nullStrPtr) and the map conversion (txToMap) with all three states
|
|
// intact. Before this test, txToMap dropped the field entirely and the Scope row
|
|
// only ever rendered for packets old enough to fall through to the DB.
|
|
func TestPacketDetailExposesScopeName(t *testing.T) {
|
|
db := setupTestDB(t)
|
|
if _, err := db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil {
|
|
t.Fatalf("add scope_name column: %v", err)
|
|
}
|
|
db.hasScopeName = true
|
|
|
|
now := time.Now().UTC().Format(time.RFC3339)
|
|
// route_type 1 = FLOOD (never transport-scoped → NULL); 0 = TRANSPORT_FLOOD.
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
|
|
VALUES ('AABB', 'aaaaaaaaaaaaaaa1', ?, 1, 4)`, now); err != nil {
|
|
t.Fatalf("insert unscoped: %v", err)
|
|
}
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
|
|
VALUES ('AABB', 'aaaaaaaaaaaaaaa2', ?, 0, 4, '')`, now); err != nil {
|
|
t.Fatalf("insert unknown-scope: %v", err)
|
|
}
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
|
|
VALUES ('AABB', 'aaaaaaaaaaaaaaa3', ?, 0, 4, '#belgium')`, now); err != nil {
|
|
t.Fatalf("insert matched-scope: %v", err)
|
|
}
|
|
|
|
srv := NewServer(db, &Config{Port: 3000}, NewHub())
|
|
store := NewPacketStore(db, nil)
|
|
if err := store.Load(); err != nil {
|
|
t.Fatalf("store.Load: %v", err)
|
|
}
|
|
if !store.WaitIndexesReady(5 * time.Second) {
|
|
t.Fatal("background indexes never became ready")
|
|
}
|
|
srv.store = store
|
|
router := mux.NewRouter()
|
|
srv.RegisterRoutes(router)
|
|
|
|
cases := []struct {
|
|
name string
|
|
hash string
|
|
want interface{}
|
|
}{
|
|
{"not transport-scoped", "aaaaaaaaaaaaaaa1", nil},
|
|
{"transport-scoped, region unmatched", "aaaaaaaaaaaaaaa2", ""},
|
|
{"transport-scoped, region matched", "aaaaaaaaaaaaaaa3", "#belgium"},
|
|
}
|
|
for _, tc := range cases {
|
|
t.Run(tc.name, func(t *testing.T) {
|
|
if store.GetPacketByHash(tc.hash) == nil {
|
|
t.Fatalf("precondition: %s not in store (would hit the DB fallback)", tc.hash)
|
|
}
|
|
req := httptest.NewRequest("GET", "/api/packets/"+tc.hash, nil)
|
|
w := httptest.NewRecorder()
|
|
router.ServeHTTP(w, req)
|
|
if w.Code != 200 {
|
|
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
|
}
|
|
var body map[string]interface{}
|
|
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
|
|
t.Fatalf("unmarshal: %v", err)
|
|
}
|
|
pkt, ok := body["packet"].(map[string]interface{})
|
|
if !ok {
|
|
t.Fatal("expected packet object")
|
|
}
|
|
if got := pkt["scope_name"]; got != tc.want {
|
|
t.Errorf("scope_name = %#v, want %#v", got, tc.want)
|
|
}
|
|
})
|
|
}
|
|
}
|
|
|
|
// --- grouped view ---
|
|
//
|
|
// The Packets tab defaults to "Group by Hash", which is served by a separate
|
|
// mapper (groupedTxsToPage in the store, a dedicated query in the DB fallback).
|
|
// The Scope column reads scope_name off those rows, so both paths must carry it.
|
|
|
|
func TestGroupedTxsToPageCarriesScopeName(t *testing.T) {
|
|
matched := "#belgium"
|
|
unmatched := ""
|
|
txs := []*StoreTx{
|
|
{ID: 1, Hash: "aa", ScopeName: &matched},
|
|
{ID: 2, Hash: "bb", ScopeName: &unmatched},
|
|
{ID: 3, Hash: "cc", ScopeName: nil},
|
|
}
|
|
res := groupedTxsToPage(txs, len(txs), 0, len(txs))
|
|
want := []interface{}{"#belgium", "", nil}
|
|
for i, w := range want {
|
|
if got := res.Packets[i]["scope_name"]; got != w {
|
|
t.Errorf("packet %d: scope_name = %#v, want %#v", i, got, w)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestGroupedPacketsEndpointExposesScopeName(t *testing.T) {
|
|
db := setupTestDB(t)
|
|
if _, err := db.conn.Exec(`ALTER TABLE transmissions ADD COLUMN scope_name TEXT DEFAULT NULL`); err != nil {
|
|
t.Fatalf("add scope_name column: %v", err)
|
|
}
|
|
db.hasScopeName = true
|
|
if _, err := db.conn.Exec(`DELETE FROM transmissions`); err != nil {
|
|
t.Fatalf("clear transmissions: %v", err)
|
|
}
|
|
|
|
now := time.Now().UTC().Format(time.RFC3339)
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type)
|
|
VALUES ('AABB', 'bbbbbbbbbbbbbbb1', ?, 1, 4)`, now); err != nil {
|
|
t.Fatalf("insert unscoped: %v", err)
|
|
}
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
|
|
VALUES ('AABB', 'bbbbbbbbbbbbbbb2', ?, 0, 4, '')`, now); err != nil {
|
|
t.Fatalf("insert unknown-scope: %v", err)
|
|
}
|
|
if _, err := db.conn.Exec(`INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, scope_name)
|
|
VALUES ('AABB', 'bbbbbbbbbbbbbbb3', ?, 0, 4, '#belgium')`, now); err != nil {
|
|
t.Fatalf("insert matched-scope: %v", err)
|
|
}
|
|
|
|
want := map[string]interface{}{
|
|
"bbbbbbbbbbbbbbb1": nil,
|
|
"bbbbbbbbbbbbbbb2": "",
|
|
"bbbbbbbbbbbbbbb3": "#belgium",
|
|
}
|
|
|
|
// Both the store-backed path and the DB-only fallback must agree.
|
|
for _, withStore := range []bool{true, false} {
|
|
name := "store"
|
|
if !withStore {
|
|
name = "db"
|
|
}
|
|
t.Run(name, func(t *testing.T) {
|
|
srv := NewServer(db, &Config{Port: 3000}, NewHub())
|
|
if withStore {
|
|
store := NewPacketStore(db, nil)
|
|
if err := store.Load(); err != nil {
|
|
t.Fatalf("store.Load: %v", err)
|
|
}
|
|
if !store.WaitIndexesReady(5 * time.Second) {
|
|
t.Fatal("background indexes never became ready")
|
|
}
|
|
srv.store = store
|
|
}
|
|
router := mux.NewRouter()
|
|
srv.RegisterRoutes(router)
|
|
|
|
req := httptest.NewRequest("GET", "/api/packets?groupByHash=true&limit=50", nil)
|
|
w := httptest.NewRecorder()
|
|
router.ServeHTTP(w, req)
|
|
if w.Code != 200 {
|
|
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
|
|
}
|
|
var body struct {
|
|
Packets []map[string]interface{} `json:"packets"`
|
|
}
|
|
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
|
|
t.Fatalf("unmarshal: %v", err)
|
|
}
|
|
if len(body.Packets) != 3 {
|
|
t.Fatalf("expected 3 grouped packets, got %d", len(body.Packets))
|
|
}
|
|
for _, p := range body.Packets {
|
|
h, _ := p["hash"].(string)
|
|
if got := p["scope_name"]; got != want[h] {
|
|
t.Errorf("%s: scope_name = %#v, want %#v", h, got, want[h])
|
|
}
|
|
}
|
|
})
|
|
}
|
|
}
|