Commit Graph
221 Commits
Author SHA1 Message Date
n30nex 5374ee55a3 Merge upstream packet hash search into the dev1 topology backend 2026-10-15 04:37:35 -04:00
MrAlders0n 632b83c4b9 feat(api): search packets by hash prefix (#227) 2026-10-14 17:04:22 -04:00
n30nex 1d3f7ce621 Add bounded topology snapshots and validate TRACE route evidence 2026-10-13 04:21:34 -04:00
MrAlders0n 7297215555 feat(meshmapper): import repeater site details for node detail (#221) 2026-10-11 14:18:26 -04:00
MrAlders0n d788e82ab4 feat(stats): rank top advertisers by flood or direct (#218)
Top advertisers ranked by flood + direct, so chatty companions with few flood adverts topped the list. sort=flood|direct is now required and applied before the limit; client floors raised to app 1.0.0 / web 2.0.4.
2026-10-10 09:38:36 -04:00
MrAlders0n 9bbf8efee5 release: v2.0.4 (#217)
* perf(routes): batch cross-IATA and hash route search (#216)

One search fired a /routes/cross request per IATA pair (138k on 2026-10-08). Route search now takes an optional iatas list and runs a fixed number of index-backed queries. /info advertises built-in minimum client versions.

* chore: bump version to 2.0.4
2026-10-09 22:17:23 -04:00
MrAlders0n 041d9c1f45 release: v2.0.2
Release the changes on `dev` as v2.0.2: authenticated MeshMapper imports and the traffic-sorted observer directory, including rolling seven-day counts from available history. Bump the API version and regenerate Swagger.

MeshMapper integration requires an API key. Regional admins’ existing per-region or grouped-region coverage keys also work for these APIs. Deployments should set `MESHMAPPER_API_KEY` and restart Beacon.

Validation: `go build ./...`, `gofmt -l .` (empty), `go vet ./...`, `go test ./...`, `govulncheck ./...`, and race tests for MeshMapper and scopestore passed in a clean checkout. PostgreSQL integration tests require CI because Docker is unavailable locally.
2026-10-05 18:35:41 -04:00
MrAlders0n ff8758b932 fix(db): read observer activity raw rows back to the rollup watermark
Hourly activity read raw rows only 3h back, so a longer rollup backlog left
hours drawn as zero. The raw tail now starts at the newest complete rollup
hour, capped at the 24h raw holdback, and the response reports rolledUntil
and rawFrom so clients can show anything still uncovered.
2026-10-02 19:03:46 -04:00
MrAlders0n 4ad869d711 fix(db): page known routes by (last_seen, id)
The cursor was the last row's last_seen in ms, so routes sharing that
millisecond (common within one upsert batch) never came back. GET /routes
takes an optional cursorId, backed by keyset indexes; cursor alone keeps
working as before.
2026-10-02 19:03:46 -04:00
MrAlders0n ec0ecd321b refactor(db): move the heard-IATA lookup into sqlc 2026-10-02 16:53:32 -04:00
MrAlders0n 2f4f92e31e fix(db): tie-safe trace paging, atomic migrations and honest stats windows
GET /traces takes an optional cursorTag so tags sharing a millisecond survive
paging. Migrations run under an advisory lock and record each file in the
same implicit transaction, so a crash can't strand a fresh database.
Overview windowHours counts the hours it covers, hourly observer activity
fills unrolled hours from raw rows, and integration fixtures are UTC.
2026-10-02 16:53:32 -04:00
MrAlders0n ec0037860c fix(meshmapper): count cut-short requests and qualify groups by heard IATAs
Record each attempt before sending so a request lost to a deadline still
waits out MeshMapper's limit, and stop holding the directory lock across
HTTP. Groups now qualify only through heard or configured IATAs, and one
missing country list only skips pruning.
2026-10-02 16:53:32 -04:00
MrAlders0n 2042c1617f feat(stats): roll hourly scope observers and advertising nodes 2026-10-02 15:24:33 -04:00
MrAlders0n 53d9f86dd4 feat(stats): split /stats/scopes packet counts by hour 2026-10-02 13:57:54 -04:00
MrAlders0n 3e0b6c074c feat(stats): report the longest path per hour in /stats/paths 2026-10-02 13:57:54 -04:00
MrAlders0n 98006a9364 feat(stats): serve historical stats from hourly rollups
Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).

Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
2026-10-02 07:43:56 -04:00
MrAlders0n 6cd03e73da fix(db): refresh saved-route hash prefix on re-hearing 2026-10-01 12:14:36 -04:00
MrAlders0n 119be44c0d feat(channels): filter channel list by keyKnown
Also seed the cursor integration test with MeshMapper membership; it still
used channel_iatas after channel lists moved to region scoping.
2026-10-01 09:06:50 -04:00
MrAlders0n 7392fdd2d6 fix(meshmapper): create group member IATAs and import zone names and centers 2026-09-30 23:21:44 -04:00
MrAlders0n f8ee1ac884 feat(meshmapper): import regional channels and scope channel lists by region 2026-09-30 23:09:40 -04:00
MrAlders0n 38b1a206b1 fix(meshmapper): respect MeshMapper's per-endpoint rate limits 2026-09-30 22:56:10 -04:00
MrAlders0n a970484b00 feat(meshmapper): import zone groups as regions 2026-09-30 22:56:10 -04:00
MrAlders0n bfda0d5281 feat(meshmapper): discover scope sources from the zone list 2026-09-30 22:46:00 -04:00
MrAlders0n 9a4e17b270 feat(meshmapper): import zones for every known IATA 2026-09-30 22:23:37 -04:00
MrAlders0n 2f49ffb3a3 feat(api): filter scope lists by region and catalogue membership
Region-filtered /scopes and /stats/scopes now list manual scopes configured
for the region and imported scopes whose MeshMapper catalogue includes the
IATA, instead of inferring membership from observed traffic. Filtered
/scopes returns string[] like the unfiltered call.

Manual scopes now require a region in config.
2026-09-30 22:05:04 -04:00
MrAlders0n 3ac4035942 refactor(db): flatten migrations into a 2.0.0 baseline
2.0.0 is a fresh deploy, so the 1.x chain becomes one schema file. Startup now refuses a pre-2.0.0 database instead of applying the baseline on top.
2026-09-30 21:37:28 -04:00
MrAlders0n 14354b03dc fix(telemetry): stop partial and bogus stats inflating bucketed counters
Bucketed airtime and receive errors used MAX - MIN per bucket, so one zeroed,
replayed or garbage row counted the whole running total (e.g. 47,737 s RX in a
6 h bucket). Buckets are now summed from per-row increments with dips, stale
replays and physically impossible jumps rejected. Ingest skips uptime-only
stats and a migration deletes the rows already stored.
2026-09-30 16:32:11 -04:00
MrAlders0n 0e242574fb feat(meshmapper): import region boundaries from the Zones API
Imported boundaries override borderFile on the border map and for foreign marking, falling back to the manual border when MeshMapper has none.
2026-09-30 12:38:36 -04:00
MrAlders0n 689bc2326a fix(ingest): clear a node's location on an explicit 0/0 advert
A repeater that advertises 0/0 is hiding its position. It was stored as a literal 0/0 that every consumer had to special-case; now the stored latitude, longitude and location source are cleared, an omitted location still preserves the stored one, and the nodeUpdate event sends explicit null lat/lng so clients can clear too. Migration 043 drops rows stored as 0/0 before this change.
2026-09-30 11:29:01 -04:00
MrAlders0n 50d35d8626 fix(api): emit observer statusMetadata as JSON 2026-09-30 10:31:08 -04:00
MrAlders0n 3b548900a0 fix(api): tolerate client clock skew and drop the dead route evidence cursor 2026-09-30 10:31:08 -04:00
MrAlders0n 1c2841a3c9 perf(db): compute route ambiguity once per reconfirm run
The four-way GROUP BY over node_short_ids ran once per 1000-route batch, up to 750 times a run, and could eat the whole 5s batch budget. The task now fetches the set once and passes it to each batch.
2026-09-30 10:31:08 -04:00
MrAlders0n 44b9a0f6d8 ci: run packet summary and reconfirm tests, read analytics on 039 views 2026-09-30 10:31:08 -04:00
MrAlders0n c9b7010082 chore: regenerate sqlc and document saved-route evidence 2026-09-30 07:48:21 -04:00
n30nex 8f6032877b test(channels): preserve the existing catch-up response shape 2026-09-30 07:19:03 -04:00
n30nex 07335e6625 test(channels): use the documented catch-up cursor parameter 2026-09-30 07:19:03 -04:00
n30nex 7d3a3e289e test(channels): verify scope metadata through all HTTP message routes 2026-09-30 07:19:03 -04:00
n30nex abb6a3b9cf test(channels): give stored packet fixtures explicit receive times 2026-09-30 07:19:03 -04:00
n30nex 2abed6c195 feat(channels): expose recorded packet scope in message delivery 2026-09-30 07:19:03 -04:00
n30nex 83cbaff1cf fix: address release review follow-ups for #174 2026-09-30 07:19:03 -04:00
n30nex 574bba4e13 fix: address maintainer review for #174 2026-09-30 07:19:03 -04:00
n30nex c18e059e0d feat(scopes): import cached regional MeshMapper catalogues 2026-09-30 07:19:03 -04:00
n30nex 85c9123e9a fix: address maintainer review for #172 2026-09-30 07:19:03 -04:00
n30nex c448a04ea0 feat(routes): add bounded retained observation evidence 2026-09-30 07:19:03 -04:00
n30nex b8a8188fbc test: mirror coalesced observer payload types 2026-09-30 07:19:03 -04:00
n30nex dea5395783 fix: address release review follow-ups for #169 2026-09-30 07:19:02 -04:00
n30nex d3c330e8fb fix: address maintainer review for #169 2026-09-30 07:19:02 -04:00
n30nex 5977e42c5c test(observers): exercise non-finite raw and archived samples 2026-09-30 07:19:02 -04:00
n30nex 47342abc99 fix(observers): keep non-finite signal samples out of JSON 2026-09-30 07:19:02 -04:00
n30nex 5fbba24a4a fix(observers): separate packet metrics from presence updates 2026-09-30 07:19:02 -04:00