Stats endpoints read hourly rollup tables filled once per closed hour by one
advisory-locked task, instead of materialized views that re-scanned every
observation on each refresh. IATA-set tables give exact distinct counts for any
IATA set or region. Cleanup holds raw rows for unrolled hours (up to 24h).
Adds /stats/series; ingest keeps packet observation counts, observer last IATA,
channel message counts and trace tag summaries. 001_baseline.sql is rewritten,
so pre-release 2.0 databases must be recreated. See docs/historical-stats.md.
Storing a full candidate snapshot on every observation was the wrong call. The same node lists got copied thousands of times, about 22 GB on prod. Resolve endpoints live like path hops and drop the column.
A packet repeats once per observer in the site scan, so grouping can
collapse a scan_depth window to fewer packets than the page asked for.
That short page set hasMore=false and stranded all older history: YOW
dead-ended after 45 packets, ~6 minutes back.
Report scan saturation and the floor the scans covered, and treat a
saturated short page as more data. Clamp the cursor to the floor so
paging on cannot skip the band the scan never read.
Quiet sites probed hundreds of thousands of packets to fill a page.
Walk idx_observations_iata_heard per requested site and dedup instead.
Filtered lists now order by site-local recency; the cursor follows.
text[] from string_to_array can't match the CHAR(3) indexes and hides
the values from the planner. Handlers already have a slice; stop
joining to CSV in the store layer and pass a typed array everywhere.
MrAlders0n