feat(#1290): use firmware repeat:on|off hint to exclude listener-only observers from disambiguator (#1624)

Closes #1290.

cross-stack: justified — backend persists firmware-side `repeat` hint to
a new observers column, frontend surfaces the listener/repeater status
as a badge on the observers list and node-detail Heard By table per the
issue's UI acceptance criterion.

## What

Firmware 1.16 publishes a `repeat: on|off` flag in the MQTT `/status`
JSON (confirmed by @cwichura on the issue thread — see
[`MQTTMessageBuilder.cpp:58`](https://github.com/agessaman/MeshCore/blob/b45373a31f111fb0de98bb3b168226d09ceadc47/src/helpers/MQTTMessageBuilder.cpp#L58)
in `agessaman/MeshCore mqtt-bridge-implementation-flex`). Listener-only
observers (`repeat:off`) by firmware contract never relay packets, so
they cannot legitimately be a hop in someone else's resolved path. This
PR plumbs the hint end-to-end so the disambiguator stops considering
them.

## How

* **`internal/dbschema`**: idempotent `can_relay INTEGER DEFAULT 1`
migration on `observers`, plus `AssertReady` probe (server fatal-logs if
absent). Mirrored in `cmd/ingestor/db.go` `CREATE TABLE` for fresh DBs.
Annotated `PREFLIGHT: async=true` — `DEFAULT 1` is constant so SQLite
does this as a metadata-only schema rewrite.
* **`cmd/ingestor`**: `extractObserverMeta` accepts `repeat` as bool,
case-insensitive string (`on|off|true|false|yes|no`), or numeric `0|1`.
Missing field → `nil` → `COALESCE` preserves the existing column value
(back-compat with legacy observers). Plumbed through `UpsertObserverAt`
and the prepared upsert statement.
* **`cmd/server`**: `GetNonRelayObserverPubkeys` + new
`prefixMap.markNonRelay` drop matching candidates inside
`pm.resolveWithContext` at the top of the resolver, so all 4 tiers see
the pruned candidate set. `ObserverResp.CanRelay` is surfaced on
`/api/observers` and `/api/observers/{id}`. `GetNodeHealth` enriches
per-observer rows with `can_relay` so the node-detail badge renders.
Probe-and-fall-back when the `can_relay` column is absent (legacy test
fixtures).
* **`public/`**: listener vs repeater pill on observers list, observer
detail `Relay` stat card, and node-detail `Heard By` table. CSS uses
existing theme vars.

## Test

Added `TestResolveWithContext_ExcludesNonRelayObservers_Issue1290` in
`cmd/server/resolve_non_relay_1290_test.go` covering all three required
cases:
* `repeat:off` pubkey → not a candidate (assertion failed in red commit
`5f7fdb96`, passes after green `f12911dc`)
* `repeat:on` pubkey → still a candidate (regression guard)
* legacy obs (no field) → still a candidate (back-compat)

Red→green proof:
```
$ git log --oneline origin/master..HEAD
f12911dc feat(#1290): exclude listener-only observers from path-hop disambiguator
5f7fdb96 test(#1290): red — assert listener-only observers excluded from path-hop candidates
```

Full server + ingestor + dbschema + migrate test suites pass locally.

## Acceptance checklist (from #1290)

* [x] Ingestor parses `repeat` field (boolean OR string `on|off`)
* [x] Field persisted on `observers` table (new `can_relay BOOLEAN`
column, idempotent migration via `internal/dbschema`)
* [x] Server's disambiguator (`pm.resolveWithContext`) excludes
`can_relay=false` observer-nodes from path-hop candidate set
* [x] UI badge on observers list + node detail page indicating
"listener" vs "repeater"
* [x] Backward compat: legacy observers default to `can_relay=true`
* [x] Test: `repeat:off` → NOT a candidate
* [x] Test: `repeat:on` → IS a candidate
* [x] Test: legacy → IS a candidate

## Out of scope (preserved per issue)

Backfilling already-resolved paths is left as a follow-up. No
firmware/broker changes.

---------

Co-authored-by: Kpa-clawbot <bot@kpa-clawbot.local>
Co-authored-by: openclaw-bot <bot@openclaw>
This commit is contained in:
Kpa-clawbot
2026-06-08 01:27:13 -07:00
committed by GitHub
parent fa02f23a40
commit a4776557ae
15 changed files with 599 additions and 14 deletions
+29 -7
View File
@@ -197,7 +197,9 @@ func applySchema(db *sql.DB) error {
last_packet_at TEXT DEFAULT NULL,
clock_skew_seconds INTEGER DEFAULT NULL,
clock_skew_count_24h INTEGER DEFAULT 0,
clock_last_naive_at TEXT DEFAULT NULL
clock_last_naive_at TEXT DEFAULT NULL,
can_relay INTEGER DEFAULT 1,
can_relay_seen INTEGER DEFAULT 0
);
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
@@ -727,8 +729,8 @@ func (s *Store) prepareStatements() error {
}
s.stmtUpsertObserver, err = s.db.Prepare(`
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor)
VALUES (?, ?, ?, ?, ?, 1, ?, ?, ?, ?, ?, ?, ?)
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor, can_relay, can_relay_seen)
VALUES (?, ?, ?, ?, ?, 1, ?, ?, ?, ?, ?, ?, ?, COALESCE(?, 1), CASE WHEN ? IS NULL THEN 0 ELSE 1 END)
ON CONFLICT(id) DO UPDATE SET
name = COALESCE(?, name),
iata = COALESCE(?, iata),
@@ -740,7 +742,9 @@ func (s *Store) prepareStatements() error {
radio = COALESCE(?, radio),
battery_mv = COALESCE(?, battery_mv),
uptime_secs = COALESCE(?, uptime_secs),
noise_floor = COALESCE(?, noise_floor)
noise_floor = COALESCE(?, noise_floor),
can_relay = COALESCE(?, can_relay),
can_relay_seen = CASE WHEN ? IS NULL THEN can_relay_seen ELSE 1 END
`)
if err != nil {
return err
@@ -973,6 +977,13 @@ type ObserverMeta struct {
RecvErrors *int // cumulative CRC/decode failures since boot
PacketsSent *int // cumulative packets sent since boot
PacketsRecv *int // cumulative packets received since boot
// CanRelay reflects the firmware 1.16 /status `repeat` flag (#1290).
// nil means the firmware did not send the field — caller must
// preserve the existing observers.can_relay value (default 1).
// true → relay-capable (`repeat:on`); false → listener-only
// (`repeat:off`), which causes the server-side disambiguator to
// exclude this observer's pubkey from path-hop candidate sets.
CanRelay *bool
}
// UpsertObserver inserts or updates an observer using the current wall-clock
@@ -995,7 +1006,7 @@ func (s *Store) UpsertObserverAt(id, name, iata string, meta *ObserverMeta, last
normalizedIATA := strings.TrimSpace(strings.ToUpper(iata))
var model, firmware, clientVersion, radio interface{}
var batteryMv, uptimeSecs, noiseFloor interface{}
var batteryMv, uptimeSecs, noiseFloor, canRelay interface{}
if meta != nil {
if meta.Model != nil {
model = *meta.Model
@@ -1018,11 +1029,22 @@ func (s *Store) UpsertObserverAt(id, name, iata string, meta *ObserverMeta, last
if meta.NoiseFloor != nil {
noiseFloor = *meta.NoiseFloor
}
// Issue #1290: nil → leave DB column unchanged (COALESCE in
// the prepared stmt); 0/1 written when firmware provided
// the `repeat` field. INSERT branch defaults to 1 via the
// COALESCE in the VALUES clause.
if meta.CanRelay != nil {
if *meta.CanRelay {
canRelay = 1
} else {
canRelay = 0
}
}
}
_, err := s.stmtUpsertObserver.Exec(
id, name, normalizedIATA, lastSeen, lastSeen, model, firmware, clientVersion, radio, batteryMv, uptimeSecs, noiseFloor,
name, normalizedIATA, ingestNow, lastSeen, model, firmware, clientVersion, radio, batteryMv, uptimeSecs, noiseFloor,
id, name, normalizedIATA, lastSeen, lastSeen, model, firmware, clientVersion, radio, batteryMv, uptimeSecs, noiseFloor, canRelay, canRelay,
name, normalizedIATA, ingestNow, lastSeen, model, firmware, clientVersion, radio, batteryMv, uptimeSecs, noiseFloor, canRelay, canRelay,
)
if err != nil {
s.Stats.WriteErrors.Add(1)
+31
View File
@@ -1124,6 +1124,37 @@ func extractObserverMeta(msg map[string]interface{}) *ObserverMeta {
}
}
// Issue #1290: firmware 1.16 publishes a `repeat` flag at the top
// level of the /status JSON (MQTTMessageBuilder.cpp:58 — see
// agessaman/MeshCore mqtt-bridge-implementation-flex). Accept
// either a boolean or a case-insensitive `on|off|true|false|1|0`
// string. Missing field → leave CanRelay nil; the writer preserves
// the prior column value (default 1, back-compat).
if v, ok := msg["repeat"]; ok && v != nil {
switch t := v.(type) {
case bool:
b := t
meta.CanRelay = &b
hasData = true
case string:
s := strings.ToLower(strings.TrimSpace(t))
switch s {
case "on", "true", "1", "yes":
b := true
meta.CanRelay = &b
hasData = true
case "off", "false", "0", "no":
b := false
meta.CanRelay = &b
hasData = true
}
case float64:
b := t != 0
meta.CanRelay = &b
hasData = true
}
}
if !hasData {
return nil
}
+101 -4
View File
@@ -12,6 +12,7 @@ import (
"sync"
"time"
"github.com/meshcore-analyzer/dbschema"
"github.com/meshcore-analyzer/geofilter"
_ "modernc.org/sqlite"
)
@@ -251,6 +252,13 @@ type Observer struct {
ClockSkewSeconds *int64 `json:"clock_skew_seconds"`
ClockSkewCount24h int `json:"clock_skew_count_24h"`
ClockLastNaiveAt *string `json:"clock_last_naive_at"`
// Issue #1290: firmware 1.16 `repeat: on|off` flag persisted by the
// ingestor. true = relay-capable, false = listener-only, nil =
// unknown (legacy observer that never sent the field — drives the
// tri-state UI badge so legacy rows don't masquerade as confirmed
// repeaters). The ingestor sets can_relay_seen=1 only when it has
// an explicit value; the read layer returns nil when seen=0.
CanRelay *bool `json:"can_relay,omitempty"`
}
// Transmission represents a row from the transmissions table.
@@ -1148,9 +1156,24 @@ func (db *DB) getObservationsForTransmissions(txIDs []int) map[int][]map[string]
// GetObservers returns active observers (not soft-deleted) sorted by last_seen DESC.
func (db *DB) GetObservers() ([]Observer, error) {
// Issue #1290: can_relay is read via COALESCE(can_relay, 1). The
// column is added by internal/dbschema; older test fixtures and
// pre-migration DBs may lack it, so we probe and fall back.
// PR #1624 MAJOR-2: can_relay_seen is the tri-state sentinel — 1
// means the ingestor explicitly wrote a value, 0 means "unknown"
// and the server returns CanRelay=nil so the UI shows no badge.
canRelayClause := "COALESCE(can_relay, 1)"
canRelaySeenClause := "0"
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay"); !hasCol {
canRelayClause = "1"
}
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay_seen"); hasCol {
canRelaySeenClause = "COALESCE(can_relay_seen, 0)"
}
rows, err := db.conn.Query(`SELECT id, name, iata, last_seen, first_seen, packet_count,
model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor, last_packet_at,
clock_skew_seconds, clock_skew_count_24h, clock_last_naive_at
clock_skew_seconds, clock_skew_count_24h, clock_last_naive_at,
` + canRelayClause + `, ` + canRelaySeenClause + `
FROM observers WHERE inactive IS NULL OR inactive = 0 ORDER BY last_seen DESC`)
if err != nil {
return nil, err
@@ -1163,11 +1186,16 @@ func (db *DB) GetObservers() ([]Observer, error) {
var batteryMv, uptimeSecs, clockSkewSec sql.NullInt64
var clockSkewCount sql.NullInt64
var noiseFloor sql.NullFloat64
var canRelay, canRelaySeen int
if err := rows.Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount,
&o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor, &o.LastPacketAt,
&clockSkewSec, &clockSkewCount, &o.ClockLastNaiveAt); err != nil {
&clockSkewSec, &clockSkewCount, &o.ClockLastNaiveAt, &canRelay, &canRelaySeen); err != nil {
continue
}
if canRelaySeen != 0 {
b := canRelay != 0
o.CanRelay = &b
}
if batteryMv.Valid {
v := int(batteryMv.Int64)
o.BatteryMv = &v
@@ -1190,22 +1218,91 @@ func (db *DB) GetObservers() ([]Observer, error) {
return observers, nil
}
// GetNonRelayObserverPubkeys returns the lowercase observer.id pubkeys
// for observers that have advertised `repeat:off` (#1290). The server's
// path-hop disambiguator consumes this to exclude listener-only nodes
// from the candidate set. Inactive observers are excluded for
// consistency with GetObservers; reactivation flips can_relay only on
// the next status message.
func (db *DB) GetNonRelayObserverPubkeys() ([]string, error) {
// Graceful no-op when can_relay column is absent (legacy DB / older
// test fixture). Avoids noisy schema-degradation log spam.
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay"); !hasCol {
return nil, nil
}
rows, err := db.conn.Query(`SELECT LOWER(id) FROM observers
WHERE COALESCE(can_relay, 1) = 0
AND (inactive IS NULL OR inactive = 0)`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []string
for rows.Next() {
var pk string
if err := rows.Scan(&pk); err == nil && pk != "" {
out = append(out, pk)
}
}
return out, rows.Err()
}
// GetCanRelaySeenObserverPubkeys returns the lowercase observer.id
// pubkeys for which the ingestor has explicitly written a repeat-field
// value (can_relay_seen=1). PR #1624 MAJOR-2: the badge surface uses
// this to render tri-state — observers NOT in this set are "unknown"
// and the UI shows no badge.
func (db *DB) GetCanRelaySeenObserverPubkeys() ([]string, error) {
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay_seen"); !hasCol {
return nil, nil
}
rows, err := db.conn.Query(`SELECT LOWER(id) FROM observers
WHERE COALESCE(can_relay_seen, 0) = 1
AND (inactive IS NULL OR inactive = 0)`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []string
for rows.Next() {
var pk string
if err := rows.Scan(&pk); err == nil && pk != "" {
out = append(out, pk)
}
}
return out, rows.Err()
}
// GetObserverByID returns a single observer.
func (db *DB) GetObserverByID(id string) (*Observer, error) {
var o Observer
var batteryMv, uptimeSecs, clockSkewSec sql.NullInt64
var clockSkewCount sql.NullInt64
var noiseFloor sql.NullFloat64
var canRelay, canRelaySeen int
canRelayClause := "COALESCE(can_relay, 1)"
canRelaySeenClause := "0"
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay"); !hasCol {
canRelayClause = "1"
}
if hasCol, _ := dbschema.TableHasColumn(db.conn, "observers", "can_relay_seen"); hasCol {
canRelaySeenClause = "COALESCE(can_relay_seen, 0)"
}
err := db.conn.QueryRow(`SELECT id, name, iata, last_seen, first_seen, packet_count,
model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor, last_packet_at,
clock_skew_seconds, clock_skew_count_24h, clock_last_naive_at
clock_skew_seconds, clock_skew_count_24h, clock_last_naive_at,
`+canRelayClause+`, `+canRelaySeenClause+`
FROM observers WHERE id = ?`, id).
Scan(&o.ID, &o.Name, &o.IATA, &o.LastSeen, &o.FirstSeen, &o.PacketCount,
&o.Model, &o.Firmware, &o.ClientVersion, &o.Radio, &batteryMv, &uptimeSecs, &noiseFloor, &o.LastPacketAt,
&clockSkewSec, &clockSkewCount, &o.ClockLastNaiveAt)
&clockSkewSec, &clockSkewCount, &o.ClockLastNaiveAt, &canRelay, &canRelaySeen)
if err != nil {
return nil, err
}
if canRelaySeen != 0 {
b := canRelay != 0
o.CanRelay = &b
}
if batteryMv.Valid {
v := int(batteryMv.Int64)
o.BatteryMv = &v
@@ -0,0 +1,93 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
)
// Issue #1290 (MAJOR-1, adversarial review of PR #1624) — regression guard.
// GetNonRelayObserverPubkeys() returns LOWER(id); the disambiguator
// (pm.nonRelay) also uses lowercase. GetNodeHealth previously used
// UPPERCASE for both insert and lookup which happens to work by symmetry,
// but any refactor that changes how pkt.ObserverID is normalized would
// silently break the badge. This test pins lowercase as the convention by
// seeding an observer.id with mixed-case packet ObserverID and asserting
// the listener badge is rendered for the matching observer in HeardBy.
func TestNodeHealth_CanRelayCaseInsensitive_Issue1290(t *testing.T) {
srv, router := setupTestServer(t)
// DB row: observer id is the canonical LOWERCASE pubkey with can_relay=0.
const obsIDLower = "deadbeefcafe1290"
const obsIDMixed = "DeadBeefCafe1290" // packet observer-id w/ mixed case
const nodePubkey = "aabbccdd11223344" // seeded by seedTestData
now := time.Now().UTC().Format(time.RFC3339)
// The test fixture's observers table predates the can_relay migration;
// add both columns (matches dbschema migrations).
for _, ddl := range []string{
`ALTER TABLE observers ADD COLUMN can_relay INTEGER DEFAULT 1`,
`ALTER TABLE observers ADD COLUMN can_relay_seen INTEGER DEFAULT 0`,
} {
if _, err := srv.store.db.conn.Exec(ddl); err != nil {
t.Fatalf("alter: %v", err)
}
}
if _, err := srv.store.db.conn.Exec(
`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, can_relay, can_relay_seen)
VALUES (?, 'ListenerOnly', 'SJC', ?, '2026-01-01T00:00:00Z', 1, 0, 1)`,
obsIDLower, now); err != nil {
t.Fatalf("seed observer: %v", err)
}
// In-memory packet with the MIXED-case observer id so the badge resolver
// must lower-case both sides to match against the lower-cased pubkey set.
snr := 7.0
srv.store.mu.Lock()
if srv.store.byNode == nil {
srv.store.byNode = make(map[string][]*StoreTx)
}
srv.store.byNode[nodePubkey] = append(srv.store.byNode[nodePubkey], &StoreTx{
Hash: "1290casebadge00",
FirstSeen: now,
SNR: &snr,
ObservationCount: 1,
ObserverID: obsIDMixed,
ObserverName: "ListenerOnly",
})
srv.store.mu.Unlock()
req := httptest.NewRequest(http.MethodGet, "/api/nodes/"+nodePubkey+"/health", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
}
var body map[string]interface{}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("json: %v", err)
}
obs, ok := body["observers"].([]interface{})
if !ok {
t.Fatalf("expected observers array, got %T", body["observers"])
}
var found bool
for _, raw := range obs {
row, ok := raw.(map[string]interface{})
if !ok {
continue
}
if row["observer_id"] != obsIDMixed {
continue
}
found = true
if row["can_relay"] != false {
t.Errorf("listener observer with can_relay=0 + mixed-case ObserverID: expected can_relay=false, got %v", row["can_relay"])
}
}
if !found {
t.Fatalf("did not find observer %q in HeardBy rows; got %v", obsIDMixed, obs)
}
}
@@ -0,0 +1,114 @@
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
// Issue #1290 (MAJOR-2, adversarial review of PR #1624) — tri-state badge.
//
// The badge surface needs to distinguish three states:
// 1. legacy observer (never sent `repeat` field) → unknown → no badge
// 2. firmware confirmed `repeat:on` → "Repeater"
// 3. firmware confirmed `repeat:off` → "Listener"
//
// Previously `CanRelay bool` defaulted to false in Go even when the row
// was the legacy DEFAULT 1, conflating "confirmed repeater" with
// "unknown". This pins the API surface to *bool + JSON omitempty so the
// frontend tri-state render works.
func TestObservers_CanRelayTriState_Issue1290(t *testing.T) {
srv, router := setupTestServer(t)
// Add the can_relay column (matches dbschema migration) PLUS the
// can_relay_seen tracking column so the read layer can distinguish
// "ingestor explicitly wrote a value" from "default sentinel".
for _, ddl := range []string{
`ALTER TABLE observers ADD COLUMN can_relay INTEGER DEFAULT 1`,
`ALTER TABLE observers ADD COLUMN can_relay_seen INTEGER DEFAULT 0`,
} {
if _, err := srv.store.db.conn.Exec(ddl); err != nil {
t.Fatalf("alter: %v", err)
}
}
now := time.Now().UTC().Format(time.RFC3339)
// Legacy: never received repeat field. can_relay=DEFAULT 1, seen=0.
if _, err := srv.store.db.conn.Exec(
`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES ('legacy-obs', 'Legacy', 'SJC', ?, '2026-01-01T00:00:00Z', 1)`, now); err != nil {
t.Fatalf("seed legacy: %v", err)
}
// Repeater: ingestor wrote can_relay=1, seen=1.
if _, err := srv.store.db.conn.Exec(
`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, can_relay, can_relay_seen)
VALUES ('rep-obs', 'Repeater', 'SFO', ?, '2026-01-01T00:00:00Z', 1, 1, 1)`, now); err != nil {
t.Fatalf("seed repeater: %v", err)
}
// Listener: ingestor wrote can_relay=0, seen=1.
if _, err := srv.store.db.conn.Exec(
`INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, can_relay, can_relay_seen)
VALUES ('lst-obs', 'Listener', 'OAK', ?, '2026-01-01T00:00:00Z', 1, 0, 1)`, now); err != nil {
t.Fatalf("seed listener: %v", err)
}
req := httptest.NewRequest(http.MethodGet, "/api/observers?nocache=1", nil)
w := httptest.NewRecorder()
router.ServeHTTP(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200, got %d (body: %s)", w.Code, w.Body.String())
}
var body struct {
Observers []map[string]interface{} `json:"observers"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatalf("json: %v", err)
}
rows := map[string]map[string]interface{}{}
for _, o := range body.Observers {
if id, _ := o["id"].(string); id != "" {
rows[id] = o
}
}
// Legacy: can_relay key must be absent (JSON omitempty for nil *bool).
legacy, ok := rows["legacy-obs"]
if !ok {
ids := make([]string, 0, len(rows))
for k := range rows {
ids = append(ids, k)
}
t.Fatalf("legacy-obs missing from response; got ids: %v", ids)
}
if _, has := legacy["can_relay"]; has {
t.Errorf("legacy observer (never sent repeat) should have can_relay omitted (unknown); got can_relay=%v", legacy["can_relay"])
}
// Repeater: can_relay must be true.
if v := rows["rep-obs"]["can_relay"]; v != true {
t.Errorf("repeater observer: expected can_relay=true, got %v", v)
}
// Listener: can_relay must be false.
if v, has := rows["lst-obs"]["can_relay"]; !has || v != false {
t.Errorf("listener observer: expected can_relay=false, got %v (present=%v)", v, has)
}
// And the raw JSON must not contain the legacy observer's can_relay key
// (defense against a future ObserverResp change that hardcodes false).
raw := w.Body.String()
if idx := strings.Index(raw, `"id":"legacy-obs"`); idx >= 0 {
// scan its row only — observers are JSON-array-ordered objects.
end := strings.Index(raw[idx:], "}")
if end > 0 {
rowStr := raw[idx : idx+end]
if strings.Contains(rowStr, `"can_relay"`) {
t.Errorf("legacy observer raw JSON unexpectedly contains can_relay key: %s", rowStr)
}
}
}
}
+43
View File
@@ -0,0 +1,43 @@
package main
import (
"testing"
)
// Issue #1290 — exclude observers that advertised `repeat:off` (listener-only)
// from the path-hop disambiguator candidate set. Three cases:
// 1. repeat:off pubkey → NOT a candidate
// 2. repeat:on pubkey → IS a candidate (regression guard)
// 3. legacy / no field → IS a candidate (back-compat preserve current behavior)
func TestResolveWithContext_ExcludesNonRelayObservers_Issue1290(t *testing.T) {
nodes := []nodeInfo{
{Role: "repeater", PublicKey: "a1aaaaaa", Name: "RealRepeater"},
{Role: "repeater", PublicKey: "a1bbbbbb", Name: "ListenerOnly"},
}
// Case 1: marked non-relay → excluded from candidate set.
pm := buildPrefixMap(nodes)
pm.markNonRelay([]string{"a1bbbbbb"})
ni, conf, _ := pm.resolveWithContext("a1bbbbbb", nil, nil)
if ni != nil {
t.Fatalf("case repeat:off — expected nil (listener-only excluded), got name=%q confidence=%q", ni.Name, conf)
}
if conf != "no_match" {
t.Fatalf("case repeat:off — expected no_match confidence after exclusion, got %q", conf)
}
// Case 2: repeat:on (i.e. not in nonRelay set) → still resolves.
pm2 := buildPrefixMap(nodes)
pm2.markNonRelay([]string{"a1bbbbbb"})
ni2, _, _ := pm2.resolveWithContext("a1aaaaaa", nil, nil)
if ni2 == nil || ni2.Name != "RealRepeater" {
t.Fatalf("case repeat:on — expected RealRepeater, got %+v", ni2)
}
// Case 3: legacy back-compat — no markNonRelay call → behavior unchanged.
pm3 := buildPrefixMap(nodes)
ni3, _, _ := pm3.resolveWithContext("a1bbbbbb", nil, nil)
if ni3 == nil || ni3.Name != "ListenerOnly" {
t.Fatalf("case legacy — expected ListenerOnly (back-compat), got %+v", ni3)
}
}
+2
View File
@@ -2534,6 +2534,7 @@ func (s *Server) buildObserversDefaultResponse() (ObserverListResponse, error) {
LastPacketAt: o.LastPacketAt,
PacketsLastHour: plh,
Lat: lat, Lon: lon, NodeRole: nodeRole,
CanRelay: o.CanRelay,
}
applyObserverNaiveClock(&resp, o, nowTime)
result = append(result, resp)
@@ -2578,6 +2579,7 @@ func (s *Server) handleObserverDetail(w http.ResponseWriter, r *http.Request) {
NoiseFloor: obs.NoiseFloor,
LastPacketAt: obs.LastPacketAt,
PacketsLastHour: plh,
CanRelay: obs.CanRelay,
}
applyObserverNaiveClock(&resp, obs, time.Now().UTC())
return resp
+90
View File
@@ -6121,6 +6121,28 @@ func (s *PacketStore) getAllNodes() []nodeInfo {
type prefixMap struct {
m map[string][]nodeInfo
// nonRelay holds lowercase pubkeys of observer-known nodes that have
// advertised `repeat:off` in their MQTT /status message (issue #1290).
// Such nodes are pure listeners and must never be selected as a
// path-hop candidate by resolveWithContext, since by firmware
// contract they do not forward packets. nil/empty preserves the
// pre-#1290 behavior (every prefix-matching node is a candidate).
nonRelay map[string]struct{}
}
// markNonRelay registers a set of lowercase pubkeys as listener-only.
// Called by the server when wiring the prefix map after reading the
// observers table's can_relay column. Issue #1290.
func (pm *prefixMap) markNonRelay(pubkeys []string) {
if pm == nil {
return
}
if pm.nonRelay == nil {
pm.nonRelay = make(map[string]struct{}, len(pubkeys))
}
for _, pk := range pubkeys {
pm.nonRelay[strings.ToLower(pk)] = struct{}{}
}
}
// maxPrefixLen caps prefix map entries. MeshCore path hops use 26 char
@@ -6172,6 +6194,17 @@ func (s *PacketStore) getCachedNodesAndPM() ([]nodeInfo, *prefixMap) {
nodes := s.getAllNodes()
pm := buildPrefixMap(nodes)
// Issue #1290: exclude observers that advertised `repeat:off` from
// the path-hop candidate set. Failure is non-fatal — we log via the
// schema-degradation channel and proceed with an empty filter (i.e.
// pre-#1290 behavior).
if s.db != nil && s.db.conn != nil {
if pks, err := s.db.GetNonRelayObserverPubkeys(); err == nil {
pm.markNonRelay(pks)
} else {
s.logSchemaDegradationOnce("observers.can_relay read failed; path-hop disambiguator will not filter listener-only observers: " + err.Error())
}
}
s.cacheMu.Lock()
s.nodeCache = nodes
@@ -6235,6 +6268,25 @@ func (pm *prefixMap) resolve(hop string) *nodeInfo {
func (pm *prefixMap) resolveWithContext(hop string, contextPubkeys []string, graph *NeighborGraph) (*nodeInfo, string, float64) {
h := strings.ToLower(hop)
candidates := pm.m[h]
// Issue #1290: drop observer-known listener-only nodes from the
// candidate set. By firmware contract a node that advertises
// `repeat:off` in its MQTT /status will never relay a packet, so it
// cannot legitimately be a hop in someone else's path. Filtering
// here shrinks ambiguous candidate sets without affecting any
// upstream caller (the returned shape and confidence labels are
// preserved; only no_match becomes more likely when the only
// matching prefix belonged to a listener). Empty pm.nonRelay
// preserves the pre-#1290 behavior exactly (back-compat).
if len(pm.nonRelay) > 0 && len(candidates) > 0 {
filtered := candidates[:0:0]
for i := range candidates {
if _, isListener := pm.nonRelay[strings.ToLower(candidates[i].PublicKey)]; isListener {
continue
}
filtered = append(filtered, candidates[i])
}
candidates = filtered
}
if len(candidates) == 0 {
return nil, "no_match", 0
}
@@ -8848,6 +8900,34 @@ func (s *PacketStore) GetNodeHealth(pubkey string) (map[string]interface{}, erro
}
observerRows := make([]map[string]interface{}, 0)
// Issue #1290: surface listener/repeater hint on node detail by
// looking up can_relay for each observer that heard this node.
// One-shot fetch of the non-relay set keeps this O(observers) on
// rare events; nil on error degrades to "neither badge" client-side.
// Issue #1290: keep this set lowercase to match the convention used
// by the resolver (cmd/server/store.go pm.nonRelay) and by
// GetNonRelayObserverPubkeys (which already returns LOWER(id)).
// Two case conventions on the same upstream string would be a
// latent regression waiting for any refactor that touches the
// observer-id normalization layer.
nonRelaySet := map[string]struct{}{}
// PR #1624 MAJOR-2: tri-state badge needs to distinguish "confirmed
// repeater" (seen=1, can_relay=1) from "unknown" (seen=0). Build
// the set of observers we have NO repeat-field record for so the
// badge is nil/omitted for them — matches nodes.js:679 tri-state.
seenSet := map[string]struct{}{}
if s.db != nil && s.db.conn != nil {
if pks, err := s.db.GetNonRelayObserverPubkeys(); err == nil {
for _, pk := range pks {
nonRelaySet[strings.ToLower(pk)] = struct{}{}
}
}
if pks, err := s.db.GetCanRelaySeenObserverPubkeys(); err == nil {
for _, pk := range pks {
seenSet[strings.ToLower(pk)] = struct{}{}
}
}
}
for id, o := range observerStats {
var avgSnr, avgRssi interface{}
if o.snrCount > 0 {
@@ -8856,9 +8936,19 @@ func (s *PacketStore) GetNodeHealth(pubkey string) (map[string]interface{}, erro
if o.rssiCount > 0 {
avgRssi = o.rssiSum / float64(o.rssiCount)
}
idLower := strings.ToLower(id)
var canRelay interface{} // nil = unknown (no repeat field ever)
if _, seen := seenSet[idLower]; seen {
if _, isListener := nonRelaySet[idLower]; isListener {
canRelay = false
} else {
canRelay = true
}
}
observerRows = append(observerRows, map[string]interface{}{
"observer_id": id, "observer_name": o.name,
"avgSnr": avgSnr, "avgRssi": avgRssi, "packetCount": o.count,
"can_relay": canRelay,
})
}
sort.Slice(observerRows, func(i, j int) bool {
+5
View File
@@ -909,6 +909,11 @@ type ObserverResp struct {
ClockSkewSeconds interface{} `json:"clock_skew_seconds"`
ClockSkewCount24h int `json:"clock_skew_count_24h"`
ClockLastNaiveAt interface{} `json:"clock_last_naive_at"`
// Issue #1290: firmware 1.16 `repeat` flag — true=repeater,
// false=listener-only, nil=unknown (legacy observer never sent the
// field). UI tri-state badge renders nothing when nil so legacy
// rows don't masquerade as confirmed repeaters (PR #1624 MAJOR-2).
CanRelay *bool `json:"can_relay,omitempty"`
}
type ObserverListResponse struct {
+64
View File
@@ -82,6 +82,12 @@ func Apply(rw *sql.DB, logf Logger) error {
if err := ensureObserverNaiveClockColumns(rw, logf); err != nil {
return fmt.Errorf("ensure observers naive-clock columns: %w", err)
}
if err := ensureObserverCanRelayColumn(rw, logf); err != nil {
return fmt.Errorf("ensure observers.can_relay: %w", err)
}
if err := ensureObserverCanRelaySeenColumn(rw, logf); err != nil {
return fmt.Errorf("ensure observers.can_relay_seen: %w", err)
}
return nil
}
@@ -138,6 +144,19 @@ func AssertReady(ro *sql.DB) error {
mustCol("observers", "clock_skew_seconds")
mustCol("observers", "clock_skew_count_24h")
mustCol("observers", "clock_last_naive_at")
// Issue #1290: firmware 1.16 publishes a `repeat: on|off` flag in
// the MQTT /status JSON. Ingestor persists it as can_relay; server
// reads it to filter listener-only observers out of the path-hop
// disambiguator candidate set. Default 1 preserves prior behavior
// for legacy observers that never sent the field.
mustCol("observers", "can_relay")
// Issue #1290 follow-up (PR #1624 MAJOR-2): tri-state badge. The
// can_relay column defaults to 1 at INSERT, so we cannot distinguish
// "firmware confirmed repeater" from "legacy observer that never
// sent a repeat field". can_relay_seen=1 means the ingestor wrote
// an explicit value; can_relay_seen=0 means leave the UI badge
// unset (unknown state).
mustCol("observers", "can_relay_seen")
if len(missing) > 0 {
return fmt.Errorf("schema not migrated by ingestor; restart ingestor first. missing: %s",
@@ -522,3 +541,48 @@ func ensureObserverNaiveClockColumns(rw *sql.DB, logf Logger) error {
}
return nil
}
// ensureObserverCanRelayColumn adds the can_relay column to observers.
// Firmware 1.16 publishes a `repeat: on|off` flag in the MQTT /status
// JSON (#1290); the ingestor parses it and writes 0/1 here. The server's
// path-hop disambiguator (cmd/server/store.go pm.resolveWithContext)
// excludes observers with can_relay=0 from the candidate set. Default 1
// preserves prior behavior for legacy observers (no repeat field).
func ensureObserverCanRelayColumn(rw *sql.DB, logf Logger) error {
has, err := TableHasColumn(rw, "observers", "can_relay")
if err != nil {
return err
}
if has {
return nil
}
// PREFLIGHT: async=true reason="single-column ALTER on observers (low-cardinality, ~1k rows in prod); DEFAULT 1 is a constant so SQLite does the rewrite as a metadata-only schema update, no row scan"
if _, err := rw.Exec("ALTER TABLE observers ADD COLUMN can_relay INTEGER DEFAULT 1"); err != nil {
return err
}
logf("[dbschema] added can_relay column to observers")
return nil
}
// ensureObserverCanRelaySeenColumn adds the can_relay_seen tracking column.
// Issue #1290 follow-up (PR #1624 MAJOR-2): can_relay defaults to 1 at
// INSERT, which conflates "confirmed repeater" with "legacy observer
// that never sent the repeat field". can_relay_seen=1 is written by
// the ingestor whenever the firmware actually provided the field; the
// server's read layer returns CanRelay=nil whenever seen=0 so the UI
// can render the tri-state badge (no badge for unknown).
func ensureObserverCanRelaySeenColumn(rw *sql.DB, logf Logger) error {
has, err := TableHasColumn(rw, "observers", "can_relay_seen")
if err != nil {
return err
}
if has {
return nil
}
// PREFLIGHT: async=true reason="single-column ALTER on observers (low-cardinality, ~1k rows in prod); DEFAULT 0 is a constant so SQLite does the rewrite as a metadata-only schema update, no row scan"
if _, err := rw.Exec("ALTER TABLE observers ADD COLUMN can_relay_seen INTEGER DEFAULT 0"); err != nil {
return err
}
logf("[dbschema] added can_relay_seen column to observers")
return nil
}
+1 -1
View File
@@ -676,7 +676,7 @@
</tr></thead>
<tbody>
${observers.map(o => `<tr>
<td data-value="${escapeHtml((o.observer_name || o.observer_id || '').toLowerCase())}" style="font-weight:600">${escapeHtml(o.observer_name || o.observer_id)}</td>
<td data-value="${escapeHtml((o.observer_name || o.observer_id || '').toLowerCase())}" style="font-weight:600">${escapeHtml(o.observer_name || o.observer_id)}${o.can_relay === false ? ' <span class="badge-listener" title="Firmware reported repeat:off — excluded from path-hop disambiguator (#1290)">listener</span>' : (o.can_relay === true ? ' <span class="badge-repeater" title="Firmware reported repeat:on — eligible as a path hop">repeater</span>' : '')}</td>
<td data-value="${escapeHtml((o.iata || '').toLowerCase())}">${o.iata ? escapeHtml(o.iata) : '—'}</td>
<td data-value="${o.packetCount || 0}">${o.packetCount}</td>
<td data-value="${o.avgSnr != null ? Number(o.avgSnr) : ''}">${o.avgSnr != null ? Number(o.avgSnr).toFixed(1) + ' dB' : '—'}</td>
+4
View File
@@ -180,6 +180,10 @@ window.ObserverDetailNaiveBanner = {
<div class="stat-label">Status</div>
<div class="stat-value"><span class="health-dot ${statusCls}"></span> ${statusLabel}</div>
</div>
<div class="stat-card">
<div class="stat-label">Relay</div>
<div class="stat-value">${obs.can_relay === false ? '<span class="badge-listener" title="Firmware reported repeat:off — excluded from path-hop disambiguator (#1290)">listener</span>' : (obs.can_relay === true ? '<span class="badge-repeater" title="Firmware reported repeat:on — eligible as a path hop">repeater</span>' : '<span class="text-muted" title="No repeat field received yet — unknown until firmware publishes a /status">—</span>')}</div>
</div>
<div class="stat-card">
<div class="stat-label">Region</div>
<div class="stat-value">${obs.iata ? '<span class="badge-region">' + escapeHtml(obs.iata) + '</span>' : '—'}</div>
+1 -1
View File
@@ -297,7 +297,7 @@ window.ObserversSummary = (function () {
const shape = h.cls === 'health-green' ? '●' : h.cls === 'health-yellow' ? '▲' : '✕';
return `<tr style="cursor:pointer" tabindex="0" role="row" data-action="navigate" data-value="#/observers/${encodeURIComponent(o.id)}" onclick="location.hash='#/observers/${encodeURIComponent(o.id)}'">
<td><span class="health-dot ${h.cls}" title="${h.label}">${shape}</span> ${h.label}</td>
<td class="mono">${escapeHtml(o.name || o.id)}${window.ObserversNaiveChip.render(o)}</td>
<td class="mono">${escapeHtml(o.name || o.id)}${window.ObserversNaiveChip.render(o)}${o.can_relay === false ? ' <span class="badge-listener" title="Firmware reported repeat:off — listener-only; excluded from path-hop disambiguator (issue #1290)">listener</span>' : (o.can_relay === true ? ' <span class="badge-repeater" title="Firmware reported repeat:on — eligible as a path hop">repeater</span>' : '')}</td>
<td>${o.iata ? `<span class="badge-region">${o.iata}</span>` : '—'}</td>
<td>${timeAgo(o.last_seen)}</td>
<td>${o.last_packet_at ? timeAgo(o.last_packet_at) : '<span class="text-muted">—</span>'}</td>
+18
View File
@@ -1146,6 +1146,24 @@ body.scroll-locked { overflow: hidden; }
font-size: 10px; font-weight: 700; font-family: var(--mono);
background: var(--nav-bg); color: var(--nav-text); letter-spacing: .5px;
}
/* Issue #1290: listener vs repeater badge on observers list + detail.
* Driven by ObserverResp.can_relay (false = firmware reported repeat:off).
* Server-side disambiguator excludes listener observers from path-hop
* candidates; this badge surfaces the same hint to operators. Colors use
* existing theme vars so the badge tracks the active palette. */
.badge-listener,
.badge-repeater {
display: inline-block; padding: 1px 5px; border-radius: 4px;
font-size: 10px; font-weight: 700; font-family: var(--mono);
letter-spacing: .5px; margin-left: 4px; vertical-align: middle;
}
.badge-listener {
background: var(--transport-badge-bg, #f59e0b20);
color: var(--transport-badge-fg, #d97706);
}
.badge-repeater {
background: var(--nav-bg); color: var(--nav-text);
}
/* Observer IATA pill rendered inline next to observer name on packets (#1188).
* Visually similar to .badge-region but distinct so the row badge and the
* inline-with-observer badge can be styled independently in future themes. */
+3 -1
View File
@@ -179,7 +179,9 @@ test('observers.js renderRow: observer name cell escapes o.name', () => {
// Capture just the <td class="mono">${ ... o.name || o.id ... }${chip}</td>.
const html = evalTemplate(
'public/observers.js',
/(<td class="mono">\$\{[^}]*o\.name[^}]*\}\$\{window\.ObserversNaiveChip\.render\(o\)\}<\/td>)/,
// Allow optional trailing content (e.g. listener/repeater badge added by #1290)
// between the naive chip and the closing </td>.
/(<td class="mono">\$\{[^}]*o\.name[^}]*\}\$\{window\.ObserversNaiveChip\.render\(o\)\}[^`]*?<\/td>)/,
{ o: { name: TAG_PAYLOAD + ATTR_PAYLOAD, id: 'obs-1' },
window: { ObserversNaiveChip: { render: () => '' } } }
);