Files
meshcore-analyzer/cmd/ingestor/source_status_test.go
T
efd66ea3f5 feat(mqtt): per-source status endpoint + Observers panel (#1682)
## Summary

Adds MQTT source status visibility per #1043 acceptance criteria:

- **Ingestor:** per-source counter registry
(`cmd/ingestor/source_status.go`) tracking `connected`,
`lastConnectUnix`, `lastDisconnectUnix`, `lastPacketUnix`,
`connectCount`, `disconnectCount`, `packetsTotal`, `packetsLast5m`
(sliding 5-min window via per-second buckets keyed by unix second — no
stale-leak), `lastError`. Wired at the existing OnConnect /
ConnectionLost / DefaultPublish callsites alongside the liveness
watchdog. Idempotent registration so counters survive reconnects.
Snapshot emitted in the existing stats file under `source_statuses`
(additive, `omitempty`).
- **Backend:** new `GET /api/mqtt/status` handler reads the ingestor
stats file and returns the per-source list. **Broker passwords are
masked** via a regex over the `scheme://user:pass@host` form (covers
mqtt/mqtts/tcp/ssl/ws/wss). Mask is also applied to `lastError` as
defense-in-depth (broker libs occasionally quote the failing URL).
OpenAPI completeness gate satisfied with a `routeDescriptions` entry.
- **Frontend:** small self-contained panel
(`public/mqtt-status-panel.js`) mounted above the Observers table.
Auto-refreshes every 10s, color-codes each row (green = connected +
recent packet, yellow = connected idle, red = disconnected), and tears
down its timer on SPA route change.

## TDD

- Red commit `f19a93b5` — stub `/api/mqtt/status` handler + assertion
test that the broker password is `****`-redacted. Test fails on the
assertion (handler passes the URL through verbatim). Compile-clean —
assertion-fail, not build-fail.
- Green commit `77042e41` — `maskBrokerURL` helper + table-driven unit
tests across all schemes + handler rewires to mask both `Broker` and
`LastError`.
- Subsequent commits land the ingestor wiring and the frontend panel.

## Tests

```
$ cd cmd/server && go test -run 'TestMqttStatus|TestMaskBrokerURL' -v ./...
PASS: TestMqttStatus_MasksBrokerPassword
PASS: TestMqttStatus_EmptyWhenNoStatsFile
PASS: TestMaskBrokerURL_Patterns (10 subtests)

$ cd cmd/ingestor && go test -run 'TestSourceStatus|TestSnapshotSourceStatuses' -v ./...
PASS: TestSourceStatus_BasicLifecycle
PASS: TestSourceStatus_Disconnect
PASS: TestSnapshotSourceStatuses_ReturnsAll

$ node test-mqtt-status-panel.js
7 passed, 0 failed
```

Full `go test ./...` clean in both `cmd/server` and `cmd/ingestor`.

## Preflight overrides

- `cross-stack`: justified — issue #1043 is intrinsically full-stack
(ingestor stats → server endpoint → observers panel). Per-stack split
would land an unreachable endpoint or a fetch with no backend.
- `check-xss-sinks` (public/mqtt-status-panel.js:55): justified — the
flagged `innerHTML=` is a fully-static literal (empty-state placeholder,
no payload data interpolated). All payload-bearing `innerHTML=` sites in
this file run through `escapeHTML` (defined in the same file); the test
`renderPanel never echoes a plaintext password (defense-in-depth)`
exercises the rendered HTML against payload strings.

## Acceptance criteria

- [x] `/api/mqtt/status` returns per-source connection state —
`cmd/server/mqtt_status.go`
- [x] UI panel shows all configured sources with live status —
`public/mqtt-status-panel.js`
- [x] Connection state updates on reconnect/disconnect events —
`MarkConnect` / `MarkDisconnect` wired in `cmd/ingestor/main.go`
- [x] Broker URLs don't expose passwords in the API response —
`maskBrokerURL` + 13 test cases
- [x] Works with 1-N sources — registry is keyed per-source, snapshot
iterates the map

**Partial fix for #1043** — per-packet `mqtt_source` attribution (the
issue's "Follow-up" section) is **deferred** per the `mc-bot-triaged:v1`
triage and the autofix comment ("Per-packet attribution deferred to
follow-up issue"). That work requires a new observation-row column and
DB schema migration, both explicitly out of scope for this PR.

Refs #1043

---------

Co-authored-by: openclaw-bot <bot@openclaw.local>
2026-06-12 08:11:02 -07:00

117 lines
3.6 KiB
Go

package main
import (
"errors"
"testing"
"time"
)
// TestSourceStatus_BasicLifecycle exercises the counter wiring used by
// the /api/mqtt/status server-side endpoint (#1043).
func TestSourceStatus_BasicLifecycle(t *testing.T) {
resetSourceStatusRegistry()
defer resetSourceStatusRegistry()
s := RegisterSourceStatus("local", "mqtt://broker.example.com:1883")
if s == nil {
t.Fatal("RegisterSourceStatus returned nil")
}
// Re-registration is idempotent.
if s2 := RegisterSourceStatus("local", "mqtt://other"); s2 != s {
t.Fatal("RegisterSourceStatus not idempotent")
}
now := time.Unix(1_700_000_000, 0)
s.MarkConnect(now)
s.MarkPacket(now)
s.MarkPacket(now.Add(1 * time.Second))
s.MarkPacket(now.Add(2 * time.Second))
snap := s.snapshot(now.Add(3 * time.Second))
if !snap.Connected {
t.Error("snapshot.Connected = false, want true after MarkConnect")
}
if snap.PacketsTotal != 3 {
t.Errorf("PacketsTotal = %d, want 3", snap.PacketsTotal)
}
if snap.PacketsLast5m != 3 {
t.Errorf("PacketsLast5m = %d, want 3", snap.PacketsLast5m)
}
if snap.ConnectCount != 1 {
t.Errorf("ConnectCount = %d, want 1", snap.ConnectCount)
}
if snap.LastConnectUnix != now.Unix() {
t.Errorf("LastConnectUnix = %d, want %d", snap.LastConnectUnix, now.Unix())
}
if snap.Broker != "mqtt://broker.example.com:1883" {
t.Errorf("Broker = %q, want raw URL passthrough (server masks)", snap.Broker)
}
// After 5 minutes idle, sliding window must be empty.
snap2 := s.snapshot(now.Add(6 * time.Minute))
if snap2.PacketsLast5m != 0 {
t.Errorf("PacketsLast5m after 6m idle = %d, want 0", snap2.PacketsLast5m)
}
if snap2.PacketsTotal != 3 {
t.Errorf("PacketsTotal must be lifetime-cumulative, got %d", snap2.PacketsTotal)
}
}
func TestSourceStatus_Disconnect(t *testing.T) {
resetSourceStatusRegistry()
defer resetSourceStatusRegistry()
s := RegisterSourceStatus("disco", "mqtt://x:1883")
now := time.Unix(1_700_000_100, 0)
s.MarkConnect(now)
s.MarkDisconnect(now.Add(time.Minute), nil)
snap := s.snapshot(now.Add(2 * time.Minute))
if snap.Connected {
t.Error("snapshot.Connected = true after MarkDisconnect, want false")
}
if snap.DisconnectCount != 1 {
t.Errorf("DisconnectCount = %d, want 1", snap.DisconnectCount)
}
}
func TestSnapshotSourceStatuses_ReturnsAll(t *testing.T) {
resetSourceStatusRegistry()
defer resetSourceStatusRegistry()
RegisterSourceStatus("a", "mqtt://a")
RegisterSourceStatus("b", "mqtt://b")
snaps := SnapshotSourceStatuses(time.Now())
if len(snaps) != 2 {
t.Errorf("len(snaps) = %d, want 2", len(snaps))
}
}
// TestSourceStatus_MarkConnectClearsLastError asserts MarkConnect wipes
// any prior sticky error (#1682 munger r1 review). Otherwise the UI sees
// connected=true alongside a stale "connection refused" string.
func TestSourceStatus_MarkConnectClearsLastError(t *testing.T) {
resetSourceStatusRegistry()
defer resetSourceStatusRegistry()
s := RegisterSourceStatus("sticky", "mqtt://x:1883")
now := time.Unix(1_700_000_200, 0)
s.MarkConnect(now)
s.MarkDisconnect(now.Add(time.Second), errors.New("connection refused"))
snap := s.snapshot(now.Add(2 * time.Second))
if snap.LastError == "" {
t.Fatalf("precondition: expected lastError after MarkDisconnect, got empty")
}
// Reconnect — lastError must clear.
s.MarkConnect(now.Add(3 * time.Second))
snap = s.snapshot(now.Add(4 * time.Second))
if snap.LastError != "" {
t.Errorf("snapshot.LastError = %q after MarkConnect, want empty (sticky-error regression)", snap.LastError)
}
if !snap.Connected {
t.Errorf("snapshot.Connected = false after MarkConnect, want true")
}
}