* feat(db): observer adverts on a partial index once observations carry their payload type
* fix(db): keep today's adverts statement unguarded and rate-limit the gate's read warning
* feat(background): batched, resumable payload-type backfill for untyped observations
* fix(background): cap each payload-type backfill statement at 50,000 rows
* fix(db): reject a non-positive backfill row cap
* perf(db): probe rollups per hour and IATA for region-sized stats lists
* fix(db): keep the stats probes inside each rollup table's hour range
* test(db): push the far-future until to year 9999 so a missing path-stats end bound times out
* perf(db): row-comparison trace cursor and a since-only telemetry statement
* test(db): pin the empty trace cursor tag and run the trace pagination tests in CI
* test(planguard): stats, adverts, traces and telemetry statements leave pending; diff cases
* test(planguard): window-bounded advert roles, short quiet page, per-observer telemetry cursors
* feat(db): packet type domains, row-cursor indexes and the inverted-heard index
* feat(api): exact page cursors for packets, nodes and observers
* feat(db): normalise packet list filters before choosing a statement
* feat(db): one packets statement per filter shape with an exact row cursor
* fix(db): let the window statements decide since after until
* feat(db): exact keyset cursors for nodes and observers; observers page by last_seen
* feat(api): pageCursor on packets, nodes and observers; it wins over cursor
* test(planguard): packets statements and node/observer keysets leave pending; diff and storediff cases
* chore: bump version to 2.1.0
* test(db): turn off JIT in the keyset cursor tests
* perf(db): drop small-table indexes that block HOT updates
* perf(db): touch known channels by id instead of upserting
* feat: preload channel ids at boot for ingest and maintenance
* feat(db): background_state table and the (hop_count, iata) route index
* perf(db): one /routes statement per filter shape, hop pair on custom plans
* feat(db): keyset route retention and grace slices at exact cutoffs
* feat(background): hourly route retention with a bounded grace walk and watermark
* fix(background): bound the grace walk at microsecond cutoffs
* feat(db): stamp-free reconfirm sweep over the route primary key
* feat(background): pace route reconfirm at routes.reconfirm_rows_per_hour, no stamps
* test(routes): drive route retention end to end and tidy comments
* fix(profiling): sample during the hourly route retention run
* perf(routes): keep hop_count indexes out of route search order
* perf(stats): count node types without node_iatas when unfiltered
* perf(db): drop the unused idx_known_routes_iata_last_seen
* fix(db): bound boot migration lock waits and name the blockers
* test(db): pin the lock-retry rebuild and tidy comments
* perf(routes): batch cross-IATA and hash route search (#216)
One search fired a /routes/cross request per IATA pair (138k on 2026-10-08). Route search now takes an optional iatas list and runs a fixed number of index-backed queries. /info advertises built-in minimum client versions.
* chore: bump version to 2.0.4
Release the changes on `dev` as v2.0.2: authenticated MeshMapper imports and the traffic-sorted observer directory, including rolling seven-day counts from available history. Bump the API version and regenerate Swagger.
MeshMapper integration requires an API key. Regional admins’ existing per-region or grouped-region coverage keys also work for these APIs. Deployments should set `MESHMAPPER_API_KEY` and restart Beacon.
Validation: `go build ./...`, `gofmt -l .` (empty), `go vet ./...`, `go test ./...`, `govulncheck ./...`, and race tests for MeshMapper and scopestore passed in a clean checkout. PostgreSQL integration tests require CI because Docker is unavailable locally.
Hourly activity read raw rows only 3h back, so a longer rollup backlog left
hours drawn as zero. The raw tail now starts at the newest complete rollup
hour, capped at the 24h raw holdback, and the response reports rolledUntil
and rawFrom so clients can show anything still uncovered.
Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).
Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
Region-filtered /scopes and /stats/scopes now list manual scopes configured
for the region and imported scopes whose MeshMapper catalogue includes the
IATA, instead of inferring membership from observed traffic. Filtered
/scopes returns string[] like the unfiltered call.
Manual scopes now require a region in config.
Bucketed airtime and receive errors used MAX - MIN per bucket, so one zeroed,
replayed or garbage row counted the whole running total (e.g. 47,737 s RX in a
6 h bucket). Buckets are now summed from per-row increments with dips, stale
replays and physically impossible jumps rejected. Ingest skips uptime-only
stats and a migration deletes the rows already stored.
The four-way GROUP BY over node_short_ids ran once per 1000-route batch, up to 750 times a run, and could eat the whole 5s batch budget. The task now fetches the set once and passes it to each batch.
Storing a full candidate snapshot on every observation was the wrong call. The same node lists got copied thousands of times, about 22 GB on prod. Resolve endpoints live like path hops and drop the column.
The payload-breakdown and top-observers matviews stored a single 7-day
total per group, so the 24h/7d/30d selector did nothing: payload ignored
the window entirely (the store discarded `since`) and top-observers only
varied which rows cleared the last_heard cutoff, never the counts.
Bucket both views by hour over a 30-day horizon and sum the buckets in
range, mirroring mv_hourly_iata_stats. The stats endpoints now honour the
requested window while still reading a small precomputed table.
MrAlders0n