Refresh last_heard on duplicate observations too, capped at hourly, so
steady traffic can't age a trace out of the filter (dedup key has no
heard_at). Drop the unused last_heard index column so upserts stay HOT,
and share one retention cutoff across the cleanup deletes.
The trace list joined every trace packet to ~50M observations to apply
the IATA filter; the parallel hash join overran docker's 64MB /dev/shm
and the filtered list errored. Keep a small trace_iatas table at ingest
(like channel_iatas) and group packets by tag instead. Filtered stats
are now per tag rather than per matching packet.
The IATA filter ran a correlated EXISTS over packets/observations with
ILIKE, which skipped the iata index and took ~7s live. Keep a small
channel_iatas table at ingest (like node_iatas) and filter against it.
Also honor the iatas= param so multi-site regions stop getting the
global list. Filter now ages out with packet retention rather than
matching any retained packet.
move refresh views and delete tasks to scheduler
add new config for refresh and cleanup intervals
new deployments can set these in minutes to see data faster
Enot (ded) Skelly