Files
meshcore-analyzer/cmd/ingestor/relay_touch_test.go
T
6528c7ba3e fix(#1854): move relay last_seen touch to the ingestor — server writes have been no-ops since mode=ro (#1855)
Fixes #1854. Refs #1598, #1611, #1845.

## The bug

`cmd/server/db.go:54` opens SQLite `mode=ro` (#1283/#1289).
`touchRelayLastSeen` → `TouchNodeLastSeen` issues `UPDATE nodes SET
last_seen` on that handle. It has failed on every call since, with the
error discarded at the call site:

```go
if err := s.db.TouchNodeLastSeen(pk, ts); err == nil {
        s.lastSeenTouched[pk] = now
}
```

`nodes.last_seen` has therefore tracked ADVERT arrivals only. Verified
on live.saarmesh.de (1388 nodes): 1362 have `last_seen` within one
minute of their own most recent ADVERT. Reproduced directly with the
server's DSN in #1854.

Secondary effect: `lastSeenTouched` is populated only in the success
branch, so the debounce never engaged — the server retried the failing
UPDATE for every resolved pubkey in every decode window.

## The fix

The writer moves to `cmd/ingestor`, which owns `nodes` per #1283/#1287
and since #1547 already resolves hop prefixes to full pubkeys for
`observations.resolved_path`. The touch hooks into that existing
resolution point, so there is no new IPC surface and no second resolver.
Only unambiguously resolved hops qualify — a 1-byte prefix collision
cannot keep a silent node alive.

I considered the `internal/mbcapqueue` snapshot handoff used for
#903/#1324 and did not need it: that pattern exists because the
capability computation lives in the server's analytics cycle. Path
resolution already happens in the ingestor, so a file handoff would add
a hop for nothing.

`Store.TouchRelayNodes`:

- monotonic guard in SQL (`last_seen IS NULL OR last_seen < ?`) —
out-of-order ingest never rewinds
- 5-minute debounce keyed on `rxTime`, matching the interval the server
intended
- UPDATE only — unknown pubkeys never create rows
- unparsable `rxTime` is a no-op rather than writing garbage into the
node directory
- `Stats.RelayTouches` for `/api/perf` visibility
- debounce records the *attempt*, not the row match, so an unknown
pubkey is not retried per observation

## Server-side removal

`touchRelayLastSeen`, `DB.TouchNodeLastSeen`, the `lastSeenTouched` map
and the now-unused `allResolvedPKs` decode-window map are deleted.
`readonly_invariant_test.go` gains `UPDATE\s+nodes\s+SET\s+last_seen`.

`cmd/server/touch_last_seen_test.go` and two tests in
`resolved_index_test.go` go with it. Worth stating why they were green
for months: they build their `PacketStore` on `setupTestDB`, which opens
read-write. The production constraint is the one thing they did not
reproduce, which is why the added invariant regex — not a replacement
unit test — is the right guard here.

## Tests

Five tests in `cmd/ingestor/relay_touch_test.go`, committed red first
(573bbde3) with a stubbed `TouchRelayNodes` so the suite compiles and
reds on assertions:

```
--- FAIL: TestTouchRelayNodes_AdvancesLastSeen
    last_seen = "2026-07-01T00:00:00Z", want "2026-07-10T12:00:00Z"
    RelayTouches = 0, want 1
--- FAIL: TestTouchRelayNodes_Debounces
    RelayTouches = 0, want 1 (second touch should be debounced)
```

Coverage: `AdvancesLastSeen` (core regression), `NeverGoesBackwards`
(monotonic), `Debounces` (write amplification on the hot path),
`IgnoresEmptyAndUnknown` (unresolved hops must not create rows),
`MalformedTimestamp`.

`cmd/ingestor`: full suite green, 100.7s.

`cmd/server`: green for the invariant and the affected packages, but the
suite is order-dependent on master today. Unmodified `upstream/master`
produced 8 failures on this machine (`TestHandleNodePaths_*`,
`TestHandleAnalytics*`, `TestComputeAnalyticsDistanceLockHoldDuration`);
this branch produced 5, and the set shifts between runs. All pass in
isolation. Untouched by this change — flagging rather than papering
over, and happy to open a separate issue if that is not already known.

## Impact on the open threads

This is the backend half of #1598. The frontend work there keys on relay
recency; that signal was never being written, so the two changes are
complementary rather than alternatives. It also removes the eviction
problem I raised in #1845 without touching `MoveStaleNodes`: once
`last_seen` reflects relay activity, the existing `last_seen < cutoff`
predicate stops evicting nodes that are carrying traffic.

Not addressed here: the duplicate-row behaviour between `nodes` and
`inactive_nodes` (609 keys in both on my deployment), which is an
independent defect and wants its own change.

## Verification offer

I run a 1100-repeater MeshCore deployment and can run this against
production traffic and report `RelayTouches` plus the resulting
`last_seen` distribution before/after, if that is useful for review.

---------

Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-09-02 10:27:40 +02:00

147 lines
5.7 KiB
Go

package main
import (
"testing"
"time"
)
// Issue #1598 / #1611 — the relay-aware last_seen touch.
//
// History: the server had touchRelayLastSeen (cmd/server/store.go), which
// called TouchNodeLastSeen → UPDATE nodes SET last_seen. Since #1283/#1289
// the server opens SQLite with mode=ro, so that UPDATE has failed with
// "attempt to write a readonly database" on every call, and the error was
// discarded at the call site. Net effect: nodes.last_seen has tracked
// ADVERT arrivals only, and relay participation has never refreshed it.
//
// The writer lives in the ingestor, which since #1547 already resolves hop
// prefixes to full pubkeys for observations.resolved_path. These tests pin
// the touch to that existing resolution point.
// helper: seed a node so the prefix index can resolve a hop to it.
func seedRelayNode(t *testing.T, s *Store, pubkey, name, lastSeen string) {
t.Helper()
if err := s.UpsertNode(pubkey, name, "repeater", nil, nil, lastSeen); err != nil {
t.Fatalf("seed node %s: %v", name, err)
}
}
func nodeLastSeen(t *testing.T, s *Store, pubkey string) string {
t.Helper()
var ls string
if err := s.db.QueryRow(`SELECT COALESCE(last_seen,'') FROM nodes WHERE public_key=?`, pubkey).Scan(&ls); err != nil {
t.Fatalf("read last_seen for %s: %v", pubkey, err)
}
return ls
}
// TestTouchRelayNodes_AdvancesLastSeen is the core regression: a node that
// appears as a resolved relay hop must have its last_seen advanced, even
// though it sent no ADVERT of its own.
func TestTouchRelayNodes_AdvancesLastSeen(t *testing.T) {
store := newTestStore(t)
const relay = "aa11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
seedRelayNode(t, store, relay, "RelayOnly", "2026-07-01T00:00:00Z")
rxTime := "2026-07-10T12:00:00Z"
store.touchRelayNodesLocked([]string{relay}, rxTime)
got := nodeLastSeen(t, store, relay)
if got != rxTime {
t.Errorf("last_seen = %q, want %q — relay participation did not refresh the node", got, rxTime)
}
if n := store.Stats.RelayTouches.Load(); n != 1 {
t.Errorf("RelayTouches = %d, want 1", n)
}
}
// TestTouchRelayNodes_NeverGoesBackwards guards the monotonic invariant.
// Out-of-order ingest (a late observation with an older rxTime) must not
// rewind a node's last_seen.
func TestTouchRelayNodes_NeverGoesBackwards(t *testing.T) {
store := newTestStore(t)
const relay = "bb11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
seedRelayNode(t, store, relay, "Backbone", "2026-07-10T12:00:00Z")
store.touchRelayNodesLocked([]string{relay}, "2026-07-09T00:00:00Z")
if got := nodeLastSeen(t, store, relay); got != "2026-07-10T12:00:00Z" {
t.Errorf("last_seen went backwards: got %q, want 2026-07-10T12:00:00Z", got)
}
}
// TestTouchRelayNodes_Debounces pins the write-amplification guard. The
// ingest path is hot; a backbone repeater appears in thousands of paths per
// hour and must not produce one UPDATE per observation.
func TestTouchRelayNodes_Debounces(t *testing.T) {
store := newTestStore(t)
const relay = "cc11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
seedRelayNode(t, store, relay, "Chatty", "2026-07-01T00:00:00Z")
base := time.Date(2026, 7, 10, 12, 0, 0, 0, time.UTC)
store.touchRelayNodesLocked([]string{relay}, base.Format(time.RFC3339))
// Second hit two minutes later — inside the debounce window, no write.
store.touchRelayNodesLocked([]string{relay}, base.Add(2*time.Minute).Format(time.RFC3339))
if n := store.Stats.RelayTouches.Load(); n != 1 {
t.Errorf("RelayTouches = %d, want 1 (second touch should be debounced)", n)
}
if got := nodeLastSeen(t, store, relay); got != base.Format(time.RFC3339) {
t.Errorf("last_seen = %q, want %q", got, base.Format(time.RFC3339))
}
// Past the debounce window the write goes through again.
later := base.Add(6 * time.Minute)
store.touchRelayNodesLocked([]string{relay}, later.Format(time.RFC3339))
if n := store.Stats.RelayTouches.Load(); n != 2 {
t.Errorf("RelayTouches = %d, want 2 after debounce window elapsed", n)
}
if got := nodeLastSeen(t, store, relay); got != later.Format(time.RFC3339) {
t.Errorf("last_seen = %q, want %q", got, later.Format(time.RFC3339))
}
}
// TestTouchRelayNodes_IgnoresEmptyAndUnknown covers the unresolved-hop case:
// resolvePathWithContext yields nil for ambiguous or unknown prefixes, and
// unknown pubkeys must not create rows.
func TestTouchRelayNodes_IgnoresEmptyAndUnknown(t *testing.T) {
store := newTestStore(t)
store.touchRelayNodesLocked(nil, "2026-07-10T12:00:00Z")
store.touchRelayNodesLocked([]string{}, "2026-07-10T12:00:00Z")
store.touchRelayNodesLocked([]string{""}, "2026-07-10T12:00:00Z")
store.touchRelayNodesLocked([]string{"ff99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbb"}, "2026-07-10T12:00:00Z")
var count int
if err := store.db.QueryRow(`SELECT COUNT(*) FROM nodes`).Scan(&count); err != nil {
t.Fatal(err)
}
if count != 0 {
t.Errorf("nodes count = %d, want 0 — touch must never insert rows", count)
}
if n := store.Stats.RelayTouches.Load(); n != 0 {
t.Errorf("RelayTouches = %d, want 0", n)
}
}
// TestTouchRelayNodes_MalformedTimestamp: rxTime that does not parse must be
// a no-op rather als writing a garbage timestamp into the node directory.
func TestTouchRelayNodes_MalformedTimestamp(t *testing.T) {
store := newTestStore(t)
const relay = "dd11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
seedRelayNode(t, store, relay, "Fine", "2026-07-01T00:00:00Z")
store.touchRelayNodesLocked([]string{relay}, "not-a-timestamp")
if got := nodeLastSeen(t, store, relay); got != "2026-07-01T00:00:00Z" {
t.Errorf("last_seen = %q, want it unchanged on malformed rxTime", got)
}
if n := store.Stats.RelayTouches.Load(); n != 0 {
t.Errorf("RelayTouches = %d, want 0", n)
}
}