Files
meshcore-analyzer/cmd/server/advert_stats.go
T
n30nex b9fb3d244c feat(analytics): split adverts using recorded route evidence (#2088)
Red commits: `dc7dcbe`, `76e3217` ([assertion
failures](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37059977385)).

Fixes #2041; supplies #2085's backend contract.

Records at most two route bits per retained advert. Node flood counts
now use the same evidence, including mixed and transport-flood adverts.
Analytics failures log/count errors while core observation, path and
liveness writes continue. The unrelated recent-advert limit clamp is
removed.

Direct adverts (empty path) describes the observed remaining path across
valid hash widths. Firmware can forward an advert down to an empty path;
this cannot establish original send mode or RF distance. The `zero_hop`
API spelling remains compatible.

Successful post-upgrade evidence writes survive observation replacement
and restart. Older overwritten frames and failed writes can leave gaps.
The UI explains automatic background backfill.

## Measured validation

Synthetic Windows workloads; timings are not production guarantees:

- 300K retained adverts, one-hour window: median 64.35→39.84 ms; 13.98
MB→145 KB allocated. Seven-day allocations unchanged; five paired
samples.
- 1M transmissions/11M observations: backfill 5m43s. Simple 10 Hz
writer: all 3,423 observations persisted; p99 28 ms, maximum 323 ms.
- Simulated pre-backfill cursor: retained evidence ready 4m21s; full
catch-up 6m41s, 122 cache invalidations. Normal completed-migration
startup bulk-loads current evidence.

Portable opt-in scale tests and window benchmarks are included. All 183
frontend suites, full server tests, targeted races, lint and real-API
desktop/mobile checks passed. Full [Linux
CI](https://github.com/Kpa-clawbot/CoreScope/actions/runs/37063210229)
passed at `36def9c`; three independent reviews are clear. Local Windows
testing hit the unchanged symlink-privilege test. No configuration
changes.

Preflight overrides: external script/OpenClaw profile unavailable;
repository checks and Chromium used.
2026-10-03 12:16:55 +02:00

85 lines
3.1 KiB
Go

package main
import (
"errors"
"time"
"github.com/meshcore-analyzer/packetpath"
)
// floodAdvertEntry is one advert transmission originated by a node, reduced to
// what the windowed flood-advert count needs: first-seen timestamp, known route evidence
// and packet hash (for dedup across re-ingests / multi-observer rows).
type floodAdvertEntry struct {
ts string
mask uint8
hash string
}
// countFloodAdverts counts distinct adverts with known flood evidence whose
// first-seen lies within the past windowHours. Entries
// with unparseable timestamps are skipped, matching relay-liveness behaviour;
// entries without a hash fall back to their timestamp as the dedup key.
func countFloodAdverts(entries []floodAdvertEntry, now time.Time, windowHours float64) int {
cutoff := now.Add(-time.Duration(windowHours * float64(time.Hour)))
seen := map[string]struct{}{}
for _, e := range entries {
if e.mask&packetpath.AdvertFlood == 0 {
continue
}
t, ok := parseRelayTS(e.ts)
if !ok || !t.After(cutoff) {
continue
}
key := e.hash
if key == "" {
key = e.ts
}
seen[key] = struct{}{}
}
return len(seen)
}
// CountFloodAdvertsForNode counts distinct adverts with recorded flood evidence,
// including mixed and transport-flood frames, independently of the first route.
// It is a lower bound while backfill is pending or evidence writes have failed.
// With no evidence table the count is unavailable, not a proven zero.
//
// SQL filters known flood evidence before the row cap. The date-only floor has
// one day of slack for mixed timestamp formats; the exact window stays in Go.
//
// The row cap is a pure safety valve on per-request allocation: it applies to
// flood adverts inside the floor window only, and 50000 in ~8 days is ~4 per
// minute - any node past it is unambiguously a spammer whether the count
// saturates or not. (An exact COUNT cannot move into SQL because the precise
// window check needs parseRelayTS over the mixed first_seen formats.)
// floodAdvertRowCap is the production row cap; tests pass a smaller cap
// directly, so there is no mutable package state to race on.
const floodAdvertRowCap = 50000
func (db *DB) CountFloodAdvertsForNode(pubkey string, windowHours float64, rowCap int) (int, error) {
if !db.advertEvidencePresent() {
return 0, errors.New("advert route evidence unavailable")
}
floor := time.Now().UTC().Add(-time.Duration(windowHours*float64(time.Hour))).AddDate(0, 0, -1).Format("2006-01-02")
rows, err := db.conn.Query(
`SELECT COALESCE(first_seen, ''), ?, COALESCE(hash, '') FROM transmissions t
WHERE from_pubkey=? AND payload_type=? AND first_seen>=?
AND EXISTS(SELECT 1 FROM advert_route_evidence e WHERE e.tx_id=t.id AND e.bit=?)
ORDER BY id DESC LIMIT ?`,
packetpath.AdvertFlood, pubkey, payloadTypeAdvert, floor, packetpath.AdvertFlood, rowCap)
if err != nil {
return 0, err
}
defer rows.Close()
var entries []floodAdvertEntry
for rows.Next() {
var e floodAdvertEntry
if err := rows.Scan(&e.ts, &e.mask, &e.hash); err != nil {
return 0, err
}
entries = append(entries, e)
}
return countFloodAdverts(entries, time.Now(), windowHours), rows.Err()
}