Commit Graph
212 Commits
Author SHA1 Message Date
MrAlders0n ec0ecd321b refactor(db): move the heard-IATA lookup into sqlc 2026-10-02 16:53:32 -04:00
MrAlders0n 2f4f92e31e fix(db): tie-safe trace paging, atomic migrations and honest stats windows
GET /traces takes an optional cursorTag so tags sharing a millisecond survive
paging. Migrations run under an advisory lock and record each file in the
same implicit transaction, so a crash can't strand a fresh database.
Overview windowHours counts the hours it covers, hourly observer activity
fills unrolled hours from raw rows, and integration fixtures are UTC.
2026-10-02 16:53:32 -04:00
MrAlders0n ec0037860c fix(meshmapper): count cut-short requests and qualify groups by heard IATAs
Record each attempt before sending so a request lost to a deadline still
waits out MeshMapper's limit, and stop holding the directory lock across
HTTP. Groups now qualify only through heard or configured IATAs, and one
missing country list only skips pruning.
2026-10-02 16:53:32 -04:00
MrAlders0n 2042c1617f feat(stats): roll hourly scope observers and advertising nodes 2026-10-02 15:24:33 -04:00
MrAlders0n 53d9f86dd4 feat(stats): split /stats/scopes packet counts by hour 2026-10-02 13:57:54 -04:00
MrAlders0n 3e0b6c074c feat(stats): report the longest path per hour in /stats/paths 2026-10-02 13:57:54 -04:00
MrAlders0n 98006a9364 feat(stats): serve historical stats from hourly rollups
Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).

Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
2026-10-02 07:43:56 -04:00
MrAlders0n 6cd03e73da fix(db): refresh saved-route hash prefix on re-hearing 2026-10-01 12:14:36 -04:00
MrAlders0n 119be44c0d feat(channels): filter channel list by keyKnown
Also seed the cursor integration test with MeshMapper membership; it still
used channel_iatas after channel lists moved to region scoping.
2026-10-01 09:06:50 -04:00
MrAlders0n 7392fdd2d6 fix(meshmapper): create group member IATAs and import zone names and centers 2026-09-30 23:21:44 -04:00
MrAlders0n f8ee1ac884 feat(meshmapper): import regional channels and scope channel lists by region 2026-09-30 23:09:40 -04:00
MrAlders0n 38b1a206b1 fix(meshmapper): respect MeshMapper's per-endpoint rate limits 2026-09-30 22:56:10 -04:00
MrAlders0n a970484b00 feat(meshmapper): import zone groups as regions 2026-09-30 22:56:10 -04:00
MrAlders0n bfda0d5281 feat(meshmapper): discover scope sources from the zone list 2026-09-30 22:46:00 -04:00
MrAlders0n 9a4e17b270 feat(meshmapper): import zones for every known IATA 2026-09-30 22:23:37 -04:00
MrAlders0n 2f49ffb3a3 feat(api): filter scope lists by region and catalogue membership
Region-filtered /scopes and /stats/scopes now list manual scopes configured
for the region and imported scopes whose MeshMapper catalogue includes the
IATA, instead of inferring membership from observed traffic. Filtered
/scopes returns string[] like the unfiltered call.

Manual scopes now require a region in config.
2026-09-30 22:05:04 -04:00
MrAlders0n 3ac4035942 refactor(db): flatten migrations into a 2.0.0 baseline
2.0.0 is a fresh deploy, so the 1.x chain becomes one schema file. Startup now refuses a pre-2.0.0 database instead of applying the baseline on top.
2026-09-30 21:37:28 -04:00
MrAlders0n 14354b03dc fix(telemetry): stop partial and bogus stats inflating bucketed counters
Bucketed airtime and receive errors used MAX - MIN per bucket, so one zeroed,
replayed or garbage row counted the whole running total (e.g. 47,737 s RX in a
6 h bucket). Buckets are now summed from per-row increments with dips, stale
replays and physically impossible jumps rejected. Ingest skips uptime-only
stats and a migration deletes the rows already stored.
2026-09-30 16:32:11 -04:00
MrAlders0n 0e242574fb feat(meshmapper): import region boundaries from the Zones API
Imported boundaries override borderFile on the border map and for foreign marking, falling back to the manual border when MeshMapper has none.
2026-09-30 12:38:36 -04:00
MrAlders0n 689bc2326a fix(ingest): clear a node's location on an explicit 0/0 advert
A repeater that advertises 0/0 is hiding its position. It was stored as a literal 0/0 that every consumer had to special-case; now the stored latitude, longitude and location source are cleared, an omitted location still preserves the stored one, and the nodeUpdate event sends explicit null lat/lng so clients can clear too. Migration 043 drops rows stored as 0/0 before this change.
2026-09-30 11:29:01 -04:00
MrAlders0n 50d35d8626 fix(api): emit observer statusMetadata as JSON 2026-09-30 10:31:08 -04:00
MrAlders0n 3b548900a0 fix(api): tolerate client clock skew and drop the dead route evidence cursor 2026-09-30 10:31:08 -04:00
MrAlders0n 1c2841a3c9 perf(db): compute route ambiguity once per reconfirm run
The four-way GROUP BY over node_short_ids ran once per 1000-route batch, up to 750 times a run, and could eat the whole 5s batch budget. The task now fetches the set once and passes it to each batch.
2026-09-30 10:31:08 -04:00
MrAlders0n 44b9a0f6d8 ci: run packet summary and reconfirm tests, read analytics on 039 views 2026-09-30 10:31:08 -04:00
MrAlders0n c9b7010082 chore: regenerate sqlc and document saved-route evidence 2026-09-30 07:48:21 -04:00
n30nex 8f6032877b test(channels): preserve the existing catch-up response shape 2026-09-30 07:19:03 -04:00
n30nex 07335e6625 test(channels): use the documented catch-up cursor parameter 2026-09-30 07:19:03 -04:00
n30nex 7d3a3e289e test(channels): verify scope metadata through all HTTP message routes 2026-09-30 07:19:03 -04:00
n30nex abb6a3b9cf test(channels): give stored packet fixtures explicit receive times 2026-09-30 07:19:03 -04:00
n30nex 2abed6c195 feat(channels): expose recorded packet scope in message delivery 2026-09-30 07:19:03 -04:00
n30nex 83cbaff1cf fix: address release review follow-ups for #174 2026-09-30 07:19:03 -04:00
n30nex 574bba4e13 fix: address maintainer review for #174 2026-09-30 07:19:03 -04:00
n30nex c18e059e0d feat(scopes): import cached regional MeshMapper catalogues 2026-09-30 07:19:03 -04:00
n30nex 85c9123e9a fix: address maintainer review for #172 2026-09-30 07:19:03 -04:00
n30nex c448a04ea0 feat(routes): add bounded retained observation evidence 2026-09-30 07:19:03 -04:00
n30nex b8a8188fbc test: mirror coalesced observer payload types 2026-09-30 07:19:03 -04:00
n30nex dea5395783 fix: address release review follow-ups for #169 2026-09-30 07:19:02 -04:00
n30nex d3c330e8fb fix: address maintainer review for #169 2026-09-30 07:19:02 -04:00
n30nex 5977e42c5c test(observers): exercise non-finite raw and archived samples 2026-09-30 07:19:02 -04:00
n30nex 47342abc99 fix(observers): keep non-finite signal samples out of JSON 2026-09-30 07:19:02 -04:00
n30nex 5fbba24a4a fix(observers): separate packet metrics from presence updates 2026-09-30 07:19:02 -04:00
n30nex fd7c6b3eb7 feat(db): retain 30-day analytics independently of packet expiry (#167) 2026-09-30 07:16:27 -04:00
MrAlders0n db30c9b573 fix(db): batch route reconfirmation to avoid ingest stalls (#180) 2026-09-28 08:41:48 -04:00
MrAlders0n 91b4b457b5 perf(db): resolve packet endpoints at read time (#163)
Storing a full candidate snapshot on every observation was the wrong call. The same node lists got copied thousands of times, about 22 GB on prod. Resolve endpoints live like path hops and drop the column.
2026-09-25 13:59:07 -04:00
MrAlders0n 3dee2098ba perf(db): batch retention deletes and tune autovacuum (#162)
Default packets.retention drops to 7 days.
2026-09-25 11:38:04 -04:00
n30nex c02317a4ac feat(packets): summarize ACK and trace references (#161) 2026-09-23 20:18:48 -04:00
n30nex 8a3fa7e6bf feat(stats): add bounded path and hash-width analytics (#159) 2026-09-23 20:18:41 -04:00
n30nex 6107578c25 feat(stats): add bounded reception signal analytics (#157) 2026-09-23 20:16:22 -04:00
n30nex cf5ff7a974 feat(stats): compare flood packets reported by two observers (#152)
* feat(stats): compare distinct flood packets across observers

* test(stats): use the migrated observation deduplication key
2026-09-19 10:36:39 -04:00
n30nex dab228e9bb feat(logging): configure levels and structured runtime output (#144)
* feat(logging): configure levels and structured runtime output

* fix(logging): bound request-derived log fields

* fix(logging): retain malformed IATA source attributes safely

* test(logging): capture structured REST rate-limit records
2026-09-19 10:36:26 -04:00