Read-only servers now restore the saved MeshMapper scope catalogues (a read-only
step), so region scope lists are no longer empty. A statement that hits the
pool's statement_timeout answers 503 with Retry-After on route evidence and the
read-model pages instead of 500. Read-model trust logs when it comes back after
a drop, and when repair runs drop it.
* feat(db): drop six lookup-table indexes that no plan reads
* feat(db): drop known_routes.last_reconfirmed_at, which nothing reads
* feat(metrics): report how much of each database sequence is used
* refactor(db): retire the legacy API pool; slow region and message statements get their own two connections
* feat(config): database.slow_pool_size sizes the slow pool
* refactor(db): route the two former read-model fallbacks to the slow pool
* fix(metrics): sequence headroom reads any schema name and marks cycling sequences
* feat(metrics): count the 503s pages answer, by reason and model; warn about a busy slow pool once a minute
* refactor(db): expose the adverts page mapping and the original channel message pages for oracles
* refactor(db): name the read-model fallbacks the slow pool runs; count only what this tree refuses
* feat(config): the slow pool defaults to 8 connections while it also serves untrusted read-model pages
* feat(db): packet_sites table and indexes
* feat(ingest): keep each packet's site row in the observation insert
* feat(db): packet_sites backfill, reconcile and audit
* feat(db): region packet pages from packet_sites when trusted, with an exact page cursor
* feat(db): packet_sites front sweeper with plan and liveness metrics
* feat: boot repair, reconcile, audit, sweep and first-time verification for packet_sites
* fix(db): read no site range when no packet was first heard in the window
* fix(db): read the window guard's maxima through indexes
* test(planguard): packet_sites statements active, data-derived cases, diff and storediff
* docs(api): pageCursor applies to region packet pages
* fix(db): drop packet_sites trust again after the unsited fallback write
* fix(db): the packet_sites audit refills a pair at its newest hearing
* fix(db): bounded region reads fill the page within the request, else today's statement answers
* fix(planguard): tie_cut needs both pages at the limit and members of other identities
* fix: packet_sites sweep without JIT, one verification VACUUM, read-only checks to the newest id
* docs(api): cursor and pageCursor on region pages served without nextPageCursor
* test: drop local baselines of entries the copy skips; plain race schedule names
* test(planguard): storediff walks of the bounded region shapes; local baselines after the overlay rebuild
* fix(db): drop message-site trust again after the unseeded insert
* fix(db): a region page cursor's fallback to today's statement continues from its exact position
* feat(db): packet type domains, row-cursor indexes and the inverted-heard index
* feat(api): exact page cursors for packets, nodes and observers
* feat(db): normalise packet list filters before choosing a statement
* feat(db): one packets statement per filter shape with an exact row cursor
* fix(db): let the window statements decide since after until
* feat(db): exact keyset cursors for nodes and observers; observers page by last_seen
* feat(api): pageCursor on packets, nodes and observers; it wins over cursor
* test(planguard): packets statements and node/observer keysets leave pending; diff and storediff cases
* chore: bump version to 2.1.0
* test(db): turn off JIT in the keyset cursor tests
* feat(planguard): read sqlc statement text and load a strict plan corpus
* fix(planguard): fail on unextracted statements, keep fragments mode-neutral
* feat(planguard): plan assertions, scoped sort exemptions and narrow retired-index verdicts
* fix(planguard): an errored run always fails
* fix(planguard): keep subplan conds off their parent scan; fail on unanalyzed budgets
* feat(planguard): read-only plan runner, reports and the run command
* fix(planguard): validate identifiers in emitted scripts; prove read-only runs
* feat(planguard): end-state corpus, planted regressions and the generated cached-plan policy
* fix(planguard): derive observer cases from data; mark legacy generic plans
* fix(planguard): bound the dense-region route resolvers; test policy refusals
* fix(planguard): re-measure the dense-region route cases; cover every policy refusal
* feat(planguard): synthetic fixture built through the migrations, checked in CI
* ci(planguard): build the fixture once; guard fixture data for identifiers
* fix(planguard): hash every fixture generator input; harden the fixture guard
* feat(planguard): snapshot diff over every column, named result classes and cursor walks
* fix(planguard): bound the relaxed diff classes; derive the cursor tie from data
* fix(planguard): refuse multi-statement text in the corpus runner
* feat: read-only server mode and an HTTP page diff between two builds
* fix(planguard): refuse escape strings and psql meta-commands in corpus text
* fix: storediff follows each build's own cursor; prove read-only gates via the log
* fix(planguard): match the PostgreSQL lexer on high-bit bytes; check every emitted line
* feat(admin): daily read-only plan check on the maintenance pool, its metric and report
* feat(planguard): daily prod check behind planguard.daily_check
* fix(planguard): cancel fresh runs at shutdown; expose a never-ran check
* ci(planguard): pin the fixture server and collation; refresh baselines
* fix(planguard): calibrate route budgets on one dataset; skip entries whose index is absent
* feat(db): five pgx pools with fixed session settings and Prometheus tracing
* feat(guards): per-family observe/enforce guards with client slots, deadlines and a shadow budget
* fix(guards): attribute only guard deadlines and classify like the handlers
* feat(db): route API statements by sqlc name; class C always unnamed
* feat(admin): statement statistics by sqlc name; create pg_stat_statements where allowed
* fix(db): skip pg_stat_statements when no schema is creatable
* feat(background): index health against post-rebuild baselines, Postgres activity, job and ingest metrics
* feat: separate API, ingest and maintenance stores; guards on the public API; metrics listener
* fix(db): keep ingest writes waiting, size the ingest pool to the host, tidy
* test(db): give the statements probe a query id of its own
Top advertisers ranked by flood + direct, so chatty companions with few flood adverts topped the list. sort=flood|direct is now required and applied before the limit; client floors raised to app 1.0.0 / web 2.0.4.
* perf(routes): batch cross-IATA and hash route search (#216)
One search fired a /routes/cross request per IATA pair (138k on 2026-10-08). Route search now takes an optional iatas list and runs a fixed number of index-backed queries. /info advertises built-in minimum client versions.
* chore: bump version to 2.0.4
* chore: add .github/CODEOWNERS for main branch policy
Signed-off-by: MrAlders0n <MrAlders0n@smal.ca>
* chore: add .github/workflows/main-source-guard.yml for main branch policy
Signed-off-by: MrAlders0n <MrAlders0n@smal.ca>
* feat(meshmapper): authenticate API imports (#211)
* feat(observers): add traffic-sorted directory (#212)
* fix(observers): rank available traffic over rolling seven days
* chore: bump version to 2.1.0
* chore: set release version to 2.0.2
* feat(api): add /info with mobile min app version (#214)
* chore: bump version to 2.0.3
---------
Signed-off-by: MrAlders0n <MrAlders0n@smal.ca>
Release the changes on `dev` as v2.0.2: authenticated MeshMapper imports and the traffic-sorted observer directory, including rolling seven-day counts from available history. Bump the API version and regenerate Swagger.
MeshMapper integration requires an API key. Regional admins’ existing per-region or grouped-region coverage keys also work for these APIs. Deployments should set `MESHMAPPER_API_KEY` and restart Beacon.
Validation: `go build ./...`, `gofmt -l .` (empty), `go vet ./...`, `go test ./...`, `govulncheck ./...`, and race tests for MeshMapper and scopestore passed in a clean checkout. PostgreSQL integration tests require CI because Docker is unavailable locally.
The cursor was the last row's last_seen in ms, so routes sharing that
millisecond (common within one upsert batch) never came back. GET /routes
takes an optional cursorId, backed by keyset indexes; cursor alone keeps
working as before.
Lookups turned every error into a 404, and region filters into a 400, so a
transient database failure looked final to clients that don't retry 4xx.
Missing rows stay 404/400; anything else is a 500. Empty /regions and
/iatas lists are [] rather than null.
GET /traces takes an optional cursorTag so tags sharing a millisecond survive
paging. Migrations run under an advisory lock and record each file in the
same implicit transaction, so a crash can't strand a fresh database.
Overview windowHours counts the hours it covers, hourly observer activity
fills unrolled hours from raw rows, and integration fixtures are UTC.
Warn at startup when limits are on without trusted_proxies, and once when a
request carries forwarding headers Beacon ignores. Reject short or whitespace
admin keys and negative durations. Key default-window stats by hour, return
500 for scope lookup failures, and validate IATA path params.
Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).
Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
Region-filtered /scopes and /stats/scopes now list manual scopes configured
for the region and imported scopes whose MeshMapper catalogue includes the
IATA, instead of inferring membership from observed traffic. Filtered
/scopes returns string[] like the unfiltered call.
Manual scopes now require a region in config.
srv.Shutdown never cancelled the request context, so a SIGTERM mid-export left the plaintext dump and pg_service.conf in TMPDIR. The handler now derives its context from the app context and the router removes staging dirs older than 20 minutes at startup.
* feat(admin): update runtime CORS origins consistently
* test(admin): preserve rate-limit headers after CORS updates
* test(admin): preserve router connection budget after rebase
Refresh last_heard on duplicate observations too, capped at hourly, so
steady traffic can't age a channel out of the filter while its packets
stay retained (dedup key has no heard_at). Guard the seed join against
the docker /dev/shm cap like the trace one, and drop the unused
last_heard index column so the upserts stay HOT.
The IATA filter ran a correlated EXISTS over packets/observations with
ILIKE, which skipped the iata index and took ~7s live. Keep a small
channel_iatas table at ingest (like node_iatas) and filter against it.
Also honor the iatas= param so multi-site regions stop getting the
global list. Filter now ages out with packet retention rather than
matching any retained packet.
MrAlders0n