* feat(db): observer adverts on a partial index once observations carry their payload type
* fix(db): keep today's adverts statement unguarded and rate-limit the gate's read warning
* feat(background): batched, resumable payload-type backfill for untyped observations
* fix(background): cap each payload-type backfill statement at 50,000 rows
* fix(db): reject a non-positive backfill row cap
* perf(db): probe rollups per hour and IATA for region-sized stats lists
* fix(db): keep the stats probes inside each rollup table's hour range
* test(db): push the far-future until to year 9999 so a missing path-stats end bound times out
* perf(db): row-comparison trace cursor and a since-only telemetry statement
* test(db): pin the empty trace cursor tag and run the trace pagination tests in CI
* test(planguard): stats, adverts, traces and telemetry statements leave pending; diff cases
* test(planguard): window-bounded advert roles, short quiet page, per-observer telemetry cursors
* feat(db): packet type domains, row-cursor indexes and the inverted-heard index
* feat(api): exact page cursors for packets, nodes and observers
* feat(db): normalise packet list filters before choosing a statement
* feat(db): one packets statement per filter shape with an exact row cursor
* fix(db): let the window statements decide since after until
* feat(db): exact keyset cursors for nodes and observers; observers page by last_seen
* feat(api): pageCursor on packets, nodes and observers; it wins over cursor
* test(planguard): packets statements and node/observer keysets leave pending; diff and storediff cases
* chore: bump version to 2.1.0
* test(db): turn off JIT in the keyset cursor tests
* perf(db): drop small-table indexes that block HOT updates
* perf(db): touch known channels by id instead of upserting
* feat: preload channel ids at boot for ingest and maintenance
* feat(db): five pgx pools with fixed session settings and Prometheus tracing
* feat(guards): per-family observe/enforce guards with client slots, deadlines and a shadow budget
* fix(guards): attribute only guard deadlines and classify like the handlers
* feat(db): route API statements by sqlc name; class C always unnamed
* feat(admin): statement statistics by sqlc name; create pg_stat_statements where allowed
* fix(db): skip pg_stat_statements when no schema is creatable
* feat(background): index health against post-rebuild baselines, Postgres activity, job and ingest metrics
* feat: separate API, ingest and maintenance stores; guards on the public API; metrics listener
* fix(db): keep ingest writes waiting, size the ingest pool to the host, tidy
* test(db): give the statements probe a query id of its own
* feat(db): background_state table and the (hop_count, iata) route index
* perf(db): one /routes statement per filter shape, hop pair on custom plans
* feat(db): keyset route retention and grace slices at exact cutoffs
* feat(background): hourly route retention with a bounded grace walk and watermark
* fix(background): bound the grace walk at microsecond cutoffs
* feat(db): stamp-free reconfirm sweep over the route primary key
* feat(background): pace route reconfirm at routes.reconfirm_rows_per_hour, no stamps
* test(routes): drive route retention end to end and tidy comments
* fix(profiling): sample during the hourly route retention run
* perf(routes): keep hop_count indexes out of route search order
* perf(stats): count node types without node_iatas when unfiltered
* perf(db): drop the unused idx_known_routes_iata_last_seen
* fix(db): bound boot migration lock waits and name the blockers
* test(db): pin the lock-retry rebuild and tidy comments
Top advertisers ranked by flood + direct, so chatty companions with few flood adverts topped the list. sort=flood|direct is now required and applied before the limit; client floors raised to app 1.0.0 / web 2.0.4.
* perf(routes): batch cross-IATA and hash route search (#216)
One search fired a /routes/cross request per IATA pair (138k on 2026-10-08). Route search now takes an optional iatas list and runs a fixed number of index-backed queries. /info advertises built-in minimum client versions.
* chore: bump version to 2.0.4
Release the changes on `dev` as v2.0.2: authenticated MeshMapper imports and the traffic-sorted observer directory, including rolling seven-day counts from available history. Bump the API version and regenerate Swagger.
MeshMapper integration requires an API key. Regional admins’ existing per-region or grouped-region coverage keys also work for these APIs. Deployments should set `MESHMAPPER_API_KEY` and restart Beacon.
Validation: `go build ./...`, `gofmt -l .` (empty), `go vet ./...`, `go test ./...`, `govulncheck ./...`, and race tests for MeshMapper and scopestore passed in a clean checkout. PostgreSQL integration tests require CI because Docker is unavailable locally.
Hourly activity read raw rows only 3h back, so a longer rollup backlog left
hours drawn as zero. The raw tail now starts at the newest complete rollup
hour, capped at the 24h raw holdback, and the response reports rolledUntil
and rawFrom so clients can show anything still uncovered.
GET /traces takes an optional cursorTag so tags sharing a millisecond survive
paging. Migrations run under an advisory lock and record each file in the
same implicit transaction, so a crash can't strand a fresh database.
Overview windowHours counts the hours it covers, hourly observer activity
fills unrolled hours from raw rows, and integration fixtures are UTC.
Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).
Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
Region-filtered /scopes and /stats/scopes now list manual scopes configured
for the region and imported scopes whose MeshMapper catalogue includes the
IATA, instead of inferring membership from observed traffic. Filtered
/scopes returns string[] like the unfiltered call.
Manual scopes now require a region in config.
Bucketed airtime and receive errors used MAX - MIN per bucket, so one zeroed,
replayed or garbage row counted the whole running total (e.g. 47,737 s RX in a
6 h bucket). Buckets are now summed from per-row increments with dips, stale
replays and physically impossible jumps rejected. Ingest skips uptime-only
stats and a migration deletes the rows already stored.
The four-way GROUP BY over node_short_ids ran once per 1000-route batch, up to 750 times a run, and could eat the whole 5s batch budget. The task now fetches the set once and passes it to each batch.
Storing a full candidate snapshot on every observation was the wrong call. The same node lists got copied thousands of times, about 22 GB on prod. Resolve endpoints live like path hops and drop the column.
MrAlders0n