mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-09-14 02:05:38 +00:00
fix(#1854): move relay last_seen touch to the ingestor — server writes have been no-ops since mode=ro (#1855)
Fixes #1854. Refs #1598, #1611, #1845.
## The bug
`cmd/server/db.go:54` opens SQLite `mode=ro` (#1283/#1289).
`touchRelayLastSeen` → `TouchNodeLastSeen` issues `UPDATE nodes SET
last_seen` on that handle. It has failed on every call since, with the
error discarded at the call site:
```go
if err := s.db.TouchNodeLastSeen(pk, ts); err == nil {
s.lastSeenTouched[pk] = now
}
```
`nodes.last_seen` has therefore tracked ADVERT arrivals only. Verified
on live.saarmesh.de (1388 nodes): 1362 have `last_seen` within one
minute of their own most recent ADVERT. Reproduced directly with the
server's DSN in #1854.
Secondary effect: `lastSeenTouched` is populated only in the success
branch, so the debounce never engaged — the server retried the failing
UPDATE for every resolved pubkey in every decode window.
## The fix
The writer moves to `cmd/ingestor`, which owns `nodes` per #1283/#1287
and since #1547 already resolves hop prefixes to full pubkeys for
`observations.resolved_path`. The touch hooks into that existing
resolution point, so there is no new IPC surface and no second resolver.
Only unambiguously resolved hops qualify — a 1-byte prefix collision
cannot keep a silent node alive.
I considered the `internal/mbcapqueue` snapshot handoff used for
#903/#1324 and did not need it: that pattern exists because the
capability computation lives in the server's analytics cycle. Path
resolution already happens in the ingestor, so a file handoff would add
a hop for nothing.
`Store.TouchRelayNodes`:
- monotonic guard in SQL (`last_seen IS NULL OR last_seen < ?`) —
out-of-order ingest never rewinds
- 5-minute debounce keyed on `rxTime`, matching the interval the server
intended
- UPDATE only — unknown pubkeys never create rows
- unparsable `rxTime` is a no-op rather than writing garbage into the
node directory
- `Stats.RelayTouches` for `/api/perf` visibility
- debounce records the *attempt*, not the row match, so an unknown
pubkey is not retried per observation
## Server-side removal
`touchRelayLastSeen`, `DB.TouchNodeLastSeen`, the `lastSeenTouched` map
and the now-unused `allResolvedPKs` decode-window map are deleted.
`readonly_invariant_test.go` gains `UPDATE\s+nodes\s+SET\s+last_seen`.
`cmd/server/touch_last_seen_test.go` and two tests in
`resolved_index_test.go` go with it. Worth stating why they were green
for months: they build their `PacketStore` on `setupTestDB`, which opens
read-write. The production constraint is the one thing they did not
reproduce, which is why the added invariant regex — not a replacement
unit test — is the right guard here.
## Tests
Five tests in `cmd/ingestor/relay_touch_test.go`, committed red first
(573bbde3) with a stubbed `TouchRelayNodes` so the suite compiles and
reds on assertions:
```
--- FAIL: TestTouchRelayNodes_AdvancesLastSeen
last_seen = "2026-07-01T00:00:00Z", want "2026-07-10T12:00:00Z"
RelayTouches = 0, want 1
--- FAIL: TestTouchRelayNodes_Debounces
RelayTouches = 0, want 1 (second touch should be debounced)
```
Coverage: `AdvancesLastSeen` (core regression), `NeverGoesBackwards`
(monotonic), `Debounces` (write amplification on the hot path),
`IgnoresEmptyAndUnknown` (unresolved hops must not create rows),
`MalformedTimestamp`.
`cmd/ingestor`: full suite green, 100.7s.
`cmd/server`: green for the invariant and the affected packages, but the
suite is order-dependent on master today. Unmodified `upstream/master`
produced 8 failures on this machine (`TestHandleNodePaths_*`,
`TestHandleAnalytics*`, `TestComputeAnalyticsDistanceLockHoldDuration`);
this branch produced 5, and the set shifts between runs. All pass in
isolation. Untouched by this change — flagging rather than papering
over, and happy to open a separate issue if that is not already known.
## Impact on the open threads
This is the backend half of #1598. The frontend work there keys on relay
recency; that signal was never being written, so the two changes are
complementary rather than alternatives. It also removes the eviction
problem I raised in #1845 without touching `MoveStaleNodes`: once
`last_seen` reflects relay activity, the existing `last_seen < cutoff`
predicate stops evicting nodes that are carrying traffic.
Not addressed here: the duplicate-row behaviour between `nodes` and
`inactive_nodes` (609 keys in both on my deployment), which is an
independent defect and wants its own change.
## Verification offer
I run a 1100-repeater MeshCore deployment and can run this against
production traffic and report `RelayTouches` plus the resulting
`last_seen` distribution before/after, if that is useful for review.
---------
Co-authored-by: SaarMesh-Bot <300107934+SaarMesh-Bot@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
co-authored by
SaarMesh-Bot
Claude
parent
176bb53335
commit
6528c7ba3e
@@ -28,6 +28,9 @@ type DBStats struct {
|
||||
ObserverUpserts atomic.Int64
|
||||
WriteErrors atomic.Int64
|
||||
SignatureDrops atomic.Int64
|
||||
// RelayTouches counts nodes.last_seen refreshes driven by relay
|
||||
// participation rather than an ADVERT (#1598).
|
||||
RelayTouches atomic.Int64
|
||||
// WALCommits tracks every successful tx.Commit() that may have flushed
|
||||
// WAL pages.
|
||||
WALCommits atomic.Int64
|
||||
@@ -79,6 +82,7 @@ type Store struct {
|
||||
stmtGetObserverRowid *sql.Stmt
|
||||
stmtUpdateObserverLastSeen *sql.Stmt
|
||||
stmtUpdateNodeTelemetry *sql.Stmt
|
||||
stmtTouchNodeLastSeen *sql.Stmt
|
||||
stmtUpsertMetrics *sql.Stmt
|
||||
|
||||
sampleIntervalSec int
|
||||
@@ -93,8 +97,26 @@ type Store struct {
|
||||
// by the context-aware resolver (#1560). Rebuilt on startup and
|
||||
// once per neighbor-edges builder tick (60s).
|
||||
neighborGraph neighborGraphHolder
|
||||
|
||||
// relayTouched is the debounce map for touchRelayNodesLocked:
|
||||
// pubkey -> rxTime of the last last_seen write (#1598). Guarded by
|
||||
// writerMu, which InsertTransmission holds for its whole body.
|
||||
relayTouched map[string]time.Time
|
||||
}
|
||||
|
||||
// relayTouchDebounce is the minimum interval between two last_seen writes
|
||||
// for the same relay node. A backbone repeater appears in thousands of
|
||||
// paths per hour; without this the ingest path would issue one UPDATE per
|
||||
// observation for no added freshness.
|
||||
const relayTouchDebounce = 5 * time.Minute
|
||||
|
||||
// relayTouchedMaxEntries caps the debounce map. One entry per node ever
|
||||
// seen relaying — ~10^3-10^4 on real deployments — but a long-lived
|
||||
// process on a large mesh should not grow it without bound. On overflow
|
||||
// we drop entries older than two debounce windows, which can only cause
|
||||
// an extra UPDATE, never a missed one.
|
||||
const relayTouchedMaxEntries = 50000
|
||||
|
||||
// OpenStore opens or creates a SQLite DB at the given path, applying the
|
||||
// v3 schema that is compatible with the Node.js server.
|
||||
func OpenStore(dbPath string) (*Store, error) {
|
||||
@@ -779,6 +801,12 @@ func (s *Store) prepareStatements() error {
|
||||
// backfill completes. Recorded in _migrations under
|
||||
// "tx_last_seen_backfill_v1".
|
||||
// PREFLIGHT: async=true reason="prepared-statement row-level UPDATE BY PRIMARY KEY (transmissions.id) — single-row touch per observation, indexed by PK, constant-time at any scale. Not a migration."
|
||||
s.stmtTouchNodeLastSeen, err = s.db.Prepare(
|
||||
"UPDATE nodes SET last_seen = ? WHERE public_key = ? AND (last_seen IS NULL OR last_seen < ?)")
|
||||
if err != nil {
|
||||
return fmt.Errorf("preparing touch node last_seen: %w", err)
|
||||
}
|
||||
|
||||
s.stmtBumpTxLastSeen, err = s.db.Prepare("UPDATE transmissions SET last_seen = ? WHERE id = ? AND last_seen < ?")
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -990,6 +1018,10 @@ func (s *Store) InsertTransmission(data *PacketData) (bool, error) {
|
||||
log.Printf("[db] observation insert (non-fatal): %v", err)
|
||||
} else {
|
||||
s.Stats.ObservationsInserted.Add(1)
|
||||
// #1598: a resolved hop proves the node was forwarding traffic at
|
||||
// rxTime. Refresh its last_seen so staleness/eviction logic sees
|
||||
// relay activity, not just ADVERTs.
|
||||
s.touchRelayNodesLocked(resolvedPubkeys(resolved), rxTime)
|
||||
// #1690: bump transmissions.last_seen so cold-load can filter on
|
||||
// effective recency. Conditional `last_seen < ?` so we never go
|
||||
// backwards on out-of-order ingest.
|
||||
@@ -1508,6 +1540,87 @@ func (s *Store) LogStats() {
|
||||
)
|
||||
}
|
||||
|
||||
// touchRelayNodesLocked refreshes nodes.last_seen for nodes observed as
|
||||
// relay hops, so a node that forwards traffic stays fresh even when it
|
||||
// adverts rarely or not at all.
|
||||
//
|
||||
// MUST be called with writerMu held. InsertTransmission holds it for its
|
||||
// entire body, so the debounce map needs no lock of its own; the name
|
||||
// carries the requirement for future callers.
|
||||
//
|
||||
// Ownership (#1283/#1287/#1289): nodes is written by the ingestor only.
|
||||
// The server opens SQLite mode=ro; its former touchRelayLastSeen has been
|
||||
// failing on every call since that refactor and is removed in this change.
|
||||
// cmd/server/readonly_invariant_test.go now guards against reintroduction.
|
||||
//
|
||||
// Callers pass the resolved pubkeys already computed for
|
||||
// observations.resolved_path (#1547/#1560) — only unambiguously resolved
|
||||
// hops reach this function, so a 1-byte prefix collision cannot keep a
|
||||
// silent node alive. resolvedPubkeys already dedups, so no second pass here.
|
||||
//
|
||||
// Never inserts: the UPDATE matches an existing row or does nothing.
|
||||
// Never rewinds: the last_seen guard makes out-of-order ingest a no-op.
|
||||
// UpsertNode's ON CONFLICT clause is monotonic in the same direction
|
||||
// (MAX(MIN(last_seen, ingestNow), rxTime) reduces to MAX(last_seen, rxTime)
|
||||
// for any non-future stored value), so an ADVERT cannot undo a touch.
|
||||
func (s *Store) touchRelayNodesLocked(pubkeys []string, rxTime string) {
|
||||
if len(pubkeys) == 0 || s.stmtTouchNodeLastSeen == nil {
|
||||
return
|
||||
}
|
||||
// Reject unparsable timestamps rather than writing them into the node
|
||||
// directory. Callers hand us the observation rxTime, which comes off
|
||||
// the wire and is not guaranteed well-formed.
|
||||
ts, err := time.Parse(time.RFC3339, rxTime)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
// Same layout UpsertNode uses for last_seen, so the SQL comparisons
|
||||
// above stay lexicographic-equals-chronological.
|
||||
stamp := ts.UTC().Format(time.RFC3339)
|
||||
|
||||
if s.relayTouched == nil {
|
||||
s.relayTouched = make(map[string]time.Time)
|
||||
}
|
||||
if len(s.relayTouched) >= relayTouchedMaxEntries {
|
||||
s.compactRelayTouched(ts)
|
||||
}
|
||||
for _, pk := range pubkeys {
|
||||
if pk == "" {
|
||||
continue
|
||||
}
|
||||
if last, ok := s.relayTouched[pk]; ok && ts.Sub(last) < relayTouchDebounce {
|
||||
continue
|
||||
}
|
||||
res, err := s.stmtTouchNodeLastSeen.Exec(stamp, pk, stamp)
|
||||
if err != nil {
|
||||
s.Stats.WriteErrors.Add(1)
|
||||
continue
|
||||
}
|
||||
// Debounce on attempt, not on row match: an unknown pubkey would
|
||||
// otherwise be retried on every observation it appears in.
|
||||
//
|
||||
// Side effect: a pubkey touched while absent from nodes, then
|
||||
// inserted by an ADVERT moments later, is skipped for the rest of
|
||||
// the window. Harmless — the ADVERT wrote last_seen itself, and it
|
||||
// is newer than anything this window would have written.
|
||||
s.relayTouched[pk] = ts
|
||||
if n, _ := res.RowsAffected(); n > 0 {
|
||||
s.Stats.RelayTouches.Add(n)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// compactRelayTouched drops debounce entries older than two windows.
|
||||
// Caller must hold writerMu.
|
||||
func (s *Store) compactRelayTouched(now time.Time) {
|
||||
cutoff := now.Add(-2 * relayTouchDebounce)
|
||||
for pk, t := range s.relayTouched {
|
||||
if t.Before(cutoff) {
|
||||
delete(s.relayTouched, pk)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MoveStaleNodes moves nodes not seen in nodeDays to the inactive_nodes table.
|
||||
// Returns the number of nodes moved.
|
||||
func (s *Store) MoveStaleNodes(nodeDays int) (int64, error) {
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Issue #1598 / #1611 — the relay-aware last_seen touch.
|
||||
//
|
||||
// History: the server had touchRelayLastSeen (cmd/server/store.go), which
|
||||
// called TouchNodeLastSeen → UPDATE nodes SET last_seen. Since #1283/#1289
|
||||
// the server opens SQLite with mode=ro, so that UPDATE has failed with
|
||||
// "attempt to write a readonly database" on every call, and the error was
|
||||
// discarded at the call site. Net effect: nodes.last_seen has tracked
|
||||
// ADVERT arrivals only, and relay participation has never refreshed it.
|
||||
//
|
||||
// The writer lives in the ingestor, which since #1547 already resolves hop
|
||||
// prefixes to full pubkeys for observations.resolved_path. These tests pin
|
||||
// the touch to that existing resolution point.
|
||||
|
||||
// helper: seed a node so the prefix index can resolve a hop to it.
|
||||
func seedRelayNode(t *testing.T, s *Store, pubkey, name, lastSeen string) {
|
||||
t.Helper()
|
||||
if err := s.UpsertNode(pubkey, name, "repeater", nil, nil, lastSeen); err != nil {
|
||||
t.Fatalf("seed node %s: %v", name, err)
|
||||
}
|
||||
}
|
||||
|
||||
func nodeLastSeen(t *testing.T, s *Store, pubkey string) string {
|
||||
t.Helper()
|
||||
var ls string
|
||||
if err := s.db.QueryRow(`SELECT COALESCE(last_seen,'') FROM nodes WHERE public_key=?`, pubkey).Scan(&ls); err != nil {
|
||||
t.Fatalf("read last_seen for %s: %v", pubkey, err)
|
||||
}
|
||||
return ls
|
||||
}
|
||||
|
||||
// TestTouchRelayNodes_AdvancesLastSeen is the core regression: a node that
|
||||
// appears as a resolved relay hop must have its last_seen advanced, even
|
||||
// though it sent no ADVERT of its own.
|
||||
func TestTouchRelayNodes_AdvancesLastSeen(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
|
||||
const relay = "aa11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
|
||||
seedRelayNode(t, store, relay, "RelayOnly", "2026-07-01T00:00:00Z")
|
||||
|
||||
rxTime := "2026-07-10T12:00:00Z"
|
||||
store.touchRelayNodesLocked([]string{relay}, rxTime)
|
||||
|
||||
got := nodeLastSeen(t, store, relay)
|
||||
if got != rxTime {
|
||||
t.Errorf("last_seen = %q, want %q — relay participation did not refresh the node", got, rxTime)
|
||||
}
|
||||
if n := store.Stats.RelayTouches.Load(); n != 1 {
|
||||
t.Errorf("RelayTouches = %d, want 1", n)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTouchRelayNodes_NeverGoesBackwards guards the monotonic invariant.
|
||||
// Out-of-order ingest (a late observation with an older rxTime) must not
|
||||
// rewind a node's last_seen.
|
||||
func TestTouchRelayNodes_NeverGoesBackwards(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
|
||||
const relay = "bb11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
|
||||
seedRelayNode(t, store, relay, "Backbone", "2026-07-10T12:00:00Z")
|
||||
|
||||
store.touchRelayNodesLocked([]string{relay}, "2026-07-09T00:00:00Z")
|
||||
|
||||
if got := nodeLastSeen(t, store, relay); got != "2026-07-10T12:00:00Z" {
|
||||
t.Errorf("last_seen went backwards: got %q, want 2026-07-10T12:00:00Z", got)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTouchRelayNodes_Debounces pins the write-amplification guard. The
|
||||
// ingest path is hot; a backbone repeater appears in thousands of paths per
|
||||
// hour and must not produce one UPDATE per observation.
|
||||
func TestTouchRelayNodes_Debounces(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
|
||||
const relay = "cc11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
|
||||
seedRelayNode(t, store, relay, "Chatty", "2026-07-01T00:00:00Z")
|
||||
|
||||
base := time.Date(2026, 7, 10, 12, 0, 0, 0, time.UTC)
|
||||
store.touchRelayNodesLocked([]string{relay}, base.Format(time.RFC3339))
|
||||
// Second hit two minutes later — inside the debounce window, no write.
|
||||
store.touchRelayNodesLocked([]string{relay}, base.Add(2*time.Minute).Format(time.RFC3339))
|
||||
|
||||
if n := store.Stats.RelayTouches.Load(); n != 1 {
|
||||
t.Errorf("RelayTouches = %d, want 1 (second touch should be debounced)", n)
|
||||
}
|
||||
if got := nodeLastSeen(t, store, relay); got != base.Format(time.RFC3339) {
|
||||
t.Errorf("last_seen = %q, want %q", got, base.Format(time.RFC3339))
|
||||
}
|
||||
|
||||
// Past the debounce window the write goes through again.
|
||||
later := base.Add(6 * time.Minute)
|
||||
store.touchRelayNodesLocked([]string{relay}, later.Format(time.RFC3339))
|
||||
if n := store.Stats.RelayTouches.Load(); n != 2 {
|
||||
t.Errorf("RelayTouches = %d, want 2 after debounce window elapsed", n)
|
||||
}
|
||||
if got := nodeLastSeen(t, store, relay); got != later.Format(time.RFC3339) {
|
||||
t.Errorf("last_seen = %q, want %q", got, later.Format(time.RFC3339))
|
||||
}
|
||||
}
|
||||
|
||||
// TestTouchRelayNodes_IgnoresEmptyAndUnknown covers the unresolved-hop case:
|
||||
// resolvePathWithContext yields nil for ambiguous or unknown prefixes, and
|
||||
// unknown pubkeys must not create rows.
|
||||
func TestTouchRelayNodes_IgnoresEmptyAndUnknown(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
|
||||
store.touchRelayNodesLocked(nil, "2026-07-10T12:00:00Z")
|
||||
store.touchRelayNodesLocked([]string{}, "2026-07-10T12:00:00Z")
|
||||
store.touchRelayNodesLocked([]string{""}, "2026-07-10T12:00:00Z")
|
||||
store.touchRelayNodesLocked([]string{"ff99887766554433221100ffeeddccbbaa99887766554433221100ffeeddccbb"}, "2026-07-10T12:00:00Z")
|
||||
|
||||
var count int
|
||||
if err := store.db.QueryRow(`SELECT COUNT(*) FROM nodes`).Scan(&count); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if count != 0 {
|
||||
t.Errorf("nodes count = %d, want 0 — touch must never insert rows", count)
|
||||
}
|
||||
if n := store.Stats.RelayTouches.Load(); n != 0 {
|
||||
t.Errorf("RelayTouches = %d, want 0", n)
|
||||
}
|
||||
}
|
||||
|
||||
// TestTouchRelayNodes_MalformedTimestamp: rxTime that does not parse must be
|
||||
// a no-op rather als writing a garbage timestamp into the node directory.
|
||||
func TestTouchRelayNodes_MalformedTimestamp(t *testing.T) {
|
||||
store := newTestStore(t)
|
||||
|
||||
const relay = "dd11223344556677889900aabbccddeeff00112233445566778899aabbccddee"
|
||||
seedRelayNode(t, store, relay, "Fine", "2026-07-01T00:00:00Z")
|
||||
|
||||
store.touchRelayNodesLocked([]string{relay}, "not-a-timestamp")
|
||||
|
||||
if got := nodeLastSeen(t, store, relay); got != "2026-07-01T00:00:00Z" {
|
||||
t.Errorf("last_seen = %q, want it unchanged on malformed rxTime", got)
|
||||
}
|
||||
if n := store.Stats.RelayTouches.Load(); n != 0 {
|
||||
t.Errorf("RelayTouches = %d, want 0", n)
|
||||
}
|
||||
}
|
||||
@@ -111,3 +111,23 @@ func (s *Store) RefreshPrefixIndex() error {
|
||||
s.prefixIdx.store(idx)
|
||||
return nil
|
||||
}
|
||||
|
||||
// resolvedPubkeys flattens a resolved path to the non-nil pubkeys it
|
||||
// contains, deduplicating repeats within the same path. Used by the
|
||||
// relay-aware last_seen touch (#1598); nil entries are unresolved or
|
||||
// ambiguous hops and are deliberately dropped.
|
||||
func resolvedPubkeys(rp []*string) []string {
|
||||
if len(rp) == 0 {
|
||||
return nil
|
||||
}
|
||||
out := make([]string, 0, len(rp))
|
||||
seen := make(map[string]bool, len(rp))
|
||||
for _, p := range rp {
|
||||
if p == nil || *p == "" || seen[*p] {
|
||||
continue
|
||||
}
|
||||
seen[*p] = true
|
||||
out = append(out, *p)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -30,7 +30,6 @@ func newChannelTestStore(packets []*StoreTx) *PacketStore {
|
||||
spIndex: make(map[string]int),
|
||||
spTxIndex: make(map[string][]*StoreTx),
|
||||
advertPubkeys: make(map[string]int),
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
clockSkew: NewClockSkewEngine(),
|
||||
}
|
||||
ps.byPayloadType[5] = packets
|
||||
|
||||
@@ -2638,16 +2638,6 @@ func (db *DB) GetMetricsSummary(since string) ([]MetricsSummaryRow, error) {
|
||||
// (PruneOldMetrics / RemoveStaleObservers removed in #1283 — see note
|
||||
// above the MetricsSample type. Ingestor owns these writes now.)
|
||||
|
||||
// TouchNodeLastSeen updates last_seen for a node identified by full public key.
|
||||
// Only updates if the new timestamp is newer than the existing value (or NULL).
|
||||
// Returns nil even if no rows are affected (node doesn't exist).
|
||||
func (db *DB) TouchNodeLastSeen(pubkey string, timestamp string) error {
|
||||
_, err := db.conn.Exec(
|
||||
"UPDATE nodes SET last_seen = ? WHERE public_key = ? AND (last_seen IS NULL OR last_seen < ?)",
|
||||
timestamp, pubkey, timestamp,
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
// GetDroppedPackets returns recently dropped packets, newest first.
|
||||
func (db *DB) GetDroppedPackets(limit int, observerID, nodePubkey string) ([]map[string]interface{}, error) {
|
||||
|
||||
@@ -29,12 +29,28 @@ func TestServerSourceHasNoCachedRWCalls(t *testing.T) {
|
||||
regexp.MustCompile(`\bcachedRW\s*\(`),
|
||||
regexp.MustCompile(`mode=rw`),
|
||||
regexp.MustCompile(`sql\.Open\([^)]*\?[^)]*_journal_mode=WAL[^)]*\)`),
|
||||
// #1324 follow-up: PR #903's persistMultibyteCapability moved
|
||||
// to cmd/ingestor — the server may NEVER UPDATE these columns
|
||||
// (it opens mode=ro since #1289). Server publishes a snapshot
|
||||
// file via internal/mbcapqueue; the ingestor applies it.
|
||||
regexp.MustCompile(`UPDATE\s+nodes\s+SET\s+multibyte_`),
|
||||
regexp.MustCompile(`UPDATE\s+inactive_nodes\s+SET\s+multibyte_`),
|
||||
// The node directory is ingestor-owned (#1283/#1287); the
|
||||
// server opens mode=ro since #1289 and may never write it.
|
||||
//
|
||||
// This deliberately matches ANY column rather than naming
|
||||
// them. The column-specific form is what let #1598 through:
|
||||
// #1324 added `UPDATE nodes SET multibyte_` after relocating
|
||||
// PR #903's writer, then touchRelayLastSeen introduced a
|
||||
// second write shape (`SET last_seen`) that the grep did not
|
||||
// cover. It failed on every call for months with the error
|
||||
// discarded at the call site, so nodes.last_seen silently
|
||||
// degraded into an advert-age proxy. Enumerating shapes does
|
||||
// not scale — forbid the table instead.
|
||||
//
|
||||
// Writers live in cmd/ingestor: Store.TouchRelayNodes (#1598)
|
||||
// and RunMultibyteCapPersist (#1324, fed by a snapshot the
|
||||
// server publishes via internal/mbcapqueue).
|
||||
// Shapes are normalised before matching (see nodeTableWritePattern):
|
||||
// optional OR-conflict clause, optional quoting, optional alias.
|
||||
nodeTableWritePattern(`UPDATE(\s+OR\s+\w+)?`, `SET`),
|
||||
nodeTableWritePattern(`INSERT\s+(OR\s+\w+\s+)?INTO`, ``),
|
||||
nodeTableWritePattern(`REPLACE\s+INTO`, ``),
|
||||
nodeTableWritePattern(`DELETE\s+FROM`, ``),
|
||||
regexp.MustCompile(`\bpersistMultibyteCapability\s*\(`),
|
||||
regexp.MustCompile(`\bmaybePersistMultibyteCapability\s*\(`),
|
||||
}
|
||||
@@ -164,3 +180,33 @@ func TestPacketStoreHasNoMultibytePersistMethods(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// nodeTableWritePattern builds a matcher for DML against the
|
||||
// ingestor-owned node directory. verb is the leading keyword(s); trailer
|
||||
// is what must follow the table name (e.g. SET for UPDATE), or empty.
|
||||
//
|
||||
// Covers the shapes a plain `UPDATE nodes SET` regex misses:
|
||||
//
|
||||
// UPDATE OR REPLACE nodes SET ...
|
||||
// UPDATE "nodes" SET ... / `nodes` / [nodes]
|
||||
// UPDATE nodes AS n SET ...
|
||||
// REPLACE INTO nodes ...
|
||||
//
|
||||
// Residual gap, stated rather than papered over: SQL assembled at
|
||||
// runtime (fmt.Sprintf("UPDATE %s SET ...", tbl)) cannot be caught by
|
||||
// source grepping. That is what TestServerDBConnIsReadOnly and
|
||||
// TestServerDBHasNoWriteMethods are for — the handle physically cannot
|
||||
// write and the write helpers do not exist on the type. This test is the
|
||||
// cheap first line that names the offending file and line; those two are
|
||||
// the structural backstop.
|
||||
func nodeTableWritePattern(verb, trailer string) *regexp.Regexp {
|
||||
const table = "[\"`\\[]?(nodes|inactive_nodes)[\"`\\]]?"
|
||||
const alias = `(\s+(AS\s+)?[a-z]\w*)?`
|
||||
expr := `(?i)` + verb + `\s+` + table + alias
|
||||
if trailer != "" {
|
||||
expr += `\s+` + trailer
|
||||
} else {
|
||||
expr += `\b`
|
||||
}
|
||||
return regexp.MustCompile(expr)
|
||||
}
|
||||
|
||||
@@ -221,28 +221,6 @@ func TestAddToByNode_WithoutResolvedPathField(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestTouchRelayLastSeen_WithoutResolvedPathField verifies relay last_seen is
|
||||
// still updated via explicit pubkey list.
|
||||
func TestTouchRelayLastSeen_WithoutResolvedPathField(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "relay_pk", "R1", "REPEATER")
|
||||
|
||||
s := &PacketStore{
|
||||
db: db,
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
}
|
||||
|
||||
s.touchRelayLastSeen([]string{"relay_pk"}, time.Now())
|
||||
|
||||
var lastSeen sql.NullString
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay_pk").Scan(&lastSeen)
|
||||
if !lastSeen.Valid {
|
||||
t.Fatal("expected last_seen to be set")
|
||||
}
|
||||
}
|
||||
|
||||
// TestWebSocketBroadcast_IncludesResolvedPath verifies broadcast maps carry resolved_path
|
||||
// from the decode-window, not from struct fields.
|
||||
func TestWebSocketBroadcast_IncludesResolvedPath(t *testing.T) {
|
||||
@@ -662,30 +640,6 @@ func TestDecodeWindow_LockHoldTimeBounded(t *testing.T) {
|
||||
|
||||
// --- Integration / regression tests ---
|
||||
|
||||
// TestRepeaterLiveness_StillAccurate verifies touchRelayLastSeen still works
|
||||
// with the new pubkey-list interface.
|
||||
func TestRepeaterLiveness_StillAccurate(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "r1", "Relay1", "REPEATER")
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "r2", "Relay2", "REPEATER")
|
||||
|
||||
s := &PacketStore{
|
||||
db: db,
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
}
|
||||
|
||||
s.touchRelayLastSeen([]string{"r1", "r2"}, time.Now())
|
||||
|
||||
var ls1, ls2 sql.NullString
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "r1").Scan(&ls1)
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "r2").Scan(&ls2)
|
||||
if !ls1.Valid || !ls2.Valid {
|
||||
t.Error("expected both relays to have last_seen updated")
|
||||
}
|
||||
}
|
||||
|
||||
// --- Benchmarks ---
|
||||
|
||||
// BenchmarkResolvedPubkeyIndex_Memory measures index memory at different cardinalities.
|
||||
|
||||
+2
-46
@@ -364,9 +364,6 @@ type PacketStore struct {
|
||||
// Updated incrementally during Load/Ingest/Evict — avoids JSON parsing in GetPerfStoreStats.
|
||||
advertPubkeys map[string]int // pubkey → number of advert packets referencing it
|
||||
|
||||
// Debounce map for touchRelayLastSeen: pubkey → last time we wrote last_seen to DB.
|
||||
// Limits DB writes to at most 1 per node per 5 minutes.
|
||||
lastSeenTouched map[string]time.Time
|
||||
|
||||
// Resolved path membership index: xxhash → []txID (forward) and txID → []hashes (reverse).
|
||||
// Replaces per-StoreTx/StoreObs ResolvedPath []*string field (#800).
|
||||
@@ -669,7 +666,6 @@ func NewPacketStore(db *DB, cfg *PacketStoreConfig, cacheTTLs ...map[string]inte
|
||||
spIndex: make(map[string]int, 4096),
|
||||
spTxIndex: make(map[string][]*StoreTx, 4096),
|
||||
advertPubkeys: make(map[string]int),
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
clockSkew: NewClockSkewEngine(),
|
||||
useResolvedPathIndex: true,
|
||||
areaNodeCache: make(map[string]map[string]bool),
|
||||
@@ -1745,31 +1741,6 @@ func (s *PacketStore) addToByNode(tx *StoreTx, pubkey string) bool {
|
||||
return isNew
|
||||
}
|
||||
|
||||
// touchRelayLastSeen updates last_seen in the DB for relay nodes that appear
|
||||
// in resolved paths. Debounced to at most 1 write per node per 5 minutes.
|
||||
// resolvedPubkeys is the pre-extracted list from the decode window.
|
||||
// Must be called under s.mu write lock (reads/writes lastSeenTouched).
|
||||
func (s *PacketStore) touchRelayLastSeen(resolvedPubkeys []string, now time.Time) {
|
||||
if s.db == nil || len(resolvedPubkeys) == 0 {
|
||||
return
|
||||
}
|
||||
const debounceInterval = 5 * time.Minute
|
||||
|
||||
ts := now.UTC().Format(time.RFC3339)
|
||||
seen := make(map[string]bool, len(resolvedPubkeys))
|
||||
for _, pk := range resolvedPubkeys {
|
||||
if pk == "" || seen[pk] {
|
||||
continue
|
||||
}
|
||||
seen[pk] = true
|
||||
if last, ok := s.lastSeenTouched[pk]; ok && now.Sub(last) < debounceInterval {
|
||||
continue
|
||||
}
|
||||
if err := s.db.TouchNodeLastSeen(pk, ts); err == nil {
|
||||
s.lastSeenTouched[pk] = now
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// trackAdvertPubkey increments the advertPubkeys refcount for ADVERT packets.
|
||||
// Must be called under s.mu write lock.
|
||||
@@ -2731,10 +2702,8 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
|
||||
// carmack #1) — one Load per ingest call, not one per row.
|
||||
cachedGraph := s.graph.Load()
|
||||
|
||||
// Decode-window tracking: resolved pubkeys per-tx for touchRelayLastSeen,
|
||||
// and resolved paths per-obs for broadcast/persist.
|
||||
var broadcastRP map[int][]*string // obsID → resolved path (for broadcast/persist)
|
||||
allResolvedPKs := make(map[int][]string) // txID → all resolved pubkeys (for touchRelayLastSeen)
|
||||
// Decode-window tracking: resolved paths per-obs for broadcast/persist.
|
||||
var broadcastRP map[int][]*string // obsID → resolved path (for broadcast/persist)
|
||||
|
||||
hopsSeen := make(map[string]bool) // reused across observations; cleared per use
|
||||
|
||||
@@ -2825,11 +2794,6 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
|
||||
}
|
||||
broadcastRP[*r.obsID] = rpForBroadcast
|
||||
}
|
||||
// Collect resolved pubkeys per-tx for touchRelayLastSeen
|
||||
if len(resolvedPubkeys) > 0 {
|
||||
allResolvedPKs[r.txID] = append(allResolvedPKs[r.txID], resolvedPubkeys...)
|
||||
}
|
||||
|
||||
tx.Observations = append(tx.Observations, obs)
|
||||
tx.obsKeys[dk] = true
|
||||
if obs.ObserverID != "" && !tx.observerSet[obs.ObserverID] {
|
||||
@@ -2857,14 +2821,6 @@ func (s *PacketStore) IngestNewFromDB(sinceID, limit int) ([]map[string]interfac
|
||||
pickBestObservation(tx)
|
||||
}
|
||||
|
||||
// Phase 2 of #660: update last_seen in DB for relay nodes seen in resolved_path.
|
||||
now := time.Now()
|
||||
for txID := range broadcastTxs {
|
||||
if pks, ok := allResolvedPKs[txID]; ok {
|
||||
s.touchRelayLastSeen(pks, now)
|
||||
}
|
||||
}
|
||||
|
||||
// Incrementally update precomputed subpath index with new transmissions
|
||||
for _, tx := range broadcastTxs {
|
||||
if addTxToSubpathIndexFull(s.spIndex, s.spTxIndex, tx) {
|
||||
|
||||
@@ -1,126 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"database/sql"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
_ "modernc.org/sqlite"
|
||||
)
|
||||
|
||||
func TestTouchNodeLastSeen_UpdatesDB(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
// Insert a node with no last_seen
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "abc123", "relay1", "REPEATER")
|
||||
|
||||
err := db.TouchNodeLastSeen("abc123", "2026-04-12T04:00:00Z")
|
||||
if err != nil {
|
||||
t.Fatalf("TouchNodeLastSeen returned error: %v", err)
|
||||
}
|
||||
|
||||
var lastSeen sql.NullString
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "abc123").Scan(&lastSeen)
|
||||
if !lastSeen.Valid || lastSeen.String != "2026-04-12T04:00:00Z" {
|
||||
t.Fatalf("expected last_seen=2026-04-12T04:00:00Z, got %v", lastSeen)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTouchNodeLastSeen_DoesNotGoBackwards(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role, last_seen) VALUES (?, ?, ?, ?)",
|
||||
"abc123", "relay1", "REPEATER", "2026-04-12T05:00:00Z")
|
||||
|
||||
// Try to set an older timestamp
|
||||
err := db.TouchNodeLastSeen("abc123", "2026-04-12T04:00:00Z")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
|
||||
var lastSeen string
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "abc123").Scan(&lastSeen)
|
||||
if lastSeen != "2026-04-12T05:00:00Z" {
|
||||
t.Fatalf("last_seen went backwards: got %s", lastSeen)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTouchNodeLastSeen_NonExistentNode(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
// Should not error for non-existent node
|
||||
err := db.TouchNodeLastSeen("nonexistent", "2026-04-12T04:00:00Z")
|
||||
if err != nil {
|
||||
t.Fatalf("unexpected error for non-existent node: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTouchRelayLastSeen_Debouncing(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
db.conn.Exec("INSERT INTO nodes (public_key, name, role) VALUES (?, ?, ?)", "relay1", "R1", "REPEATER")
|
||||
|
||||
s := &PacketStore{
|
||||
db: db,
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
}
|
||||
|
||||
// After #800, touchRelayLastSeen takes a []string of pubkeys (from decode-window)
|
||||
pks := []string{"relay1"}
|
||||
|
||||
now := time.Now()
|
||||
s.touchRelayLastSeen(pks, now)
|
||||
|
||||
// Verify it was written
|
||||
var lastSeen sql.NullString
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
|
||||
if !lastSeen.Valid {
|
||||
t.Fatal("expected last_seen to be set after first touch")
|
||||
}
|
||||
|
||||
// Reset last_seen to check debounce prevents second write
|
||||
db.conn.Exec("UPDATE nodes SET last_seen = NULL WHERE public_key = ?", "relay1")
|
||||
|
||||
// Call again within 5 minutes — should be debounced (no write)
|
||||
s.touchRelayLastSeen(pks, now.Add(2*time.Minute))
|
||||
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
|
||||
if lastSeen.Valid {
|
||||
t.Fatal("expected debounce to prevent second write within 5 minutes")
|
||||
}
|
||||
|
||||
// Call after 5 minutes — should write again
|
||||
s.touchRelayLastSeen(pks, now.Add(6*time.Minute))
|
||||
db.conn.QueryRow("SELECT last_seen FROM nodes WHERE public_key = ?", "relay1").Scan(&lastSeen)
|
||||
if !lastSeen.Valid {
|
||||
t.Fatal("expected write after debounce interval expired")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTouchRelayLastSeen_SkipsEmptyPubkeys(t *testing.T) {
|
||||
db := setupTestDB(t)
|
||||
defer db.Close()
|
||||
|
||||
s := &PacketStore{
|
||||
db: db,
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
}
|
||||
|
||||
// Empty pubkeys — should not panic or error
|
||||
s.touchRelayLastSeen([]string{}, time.Now())
|
||||
s.touchRelayLastSeen(nil, time.Now())
|
||||
}
|
||||
|
||||
func TestTouchRelayLastSeen_NilDB(t *testing.T) {
|
||||
s := &PacketStore{
|
||||
db: nil,
|
||||
lastSeenTouched: make(map[string]time.Time),
|
||||
}
|
||||
|
||||
// Should not panic with nil db
|
||||
s.touchRelayLastSeen([]string{"abc"}, time.Now())
|
||||
}
|
||||
Reference in New Issue
Block a user