Refresh last_heard on duplicate observations too, capped at hourly, so
steady traffic can't age a trace out of the filter (dedup key has no
heard_at). Drop the unused last_heard index column so upserts stay HOT,
and share one retention cutoff across the cleanup deletes.
The trace list joined every trace packet to ~50M observations to apply
the IATA filter; the parallel hash join overran docker's 64MB /dev/shm
and the filtered list errored. Keep a small trace_iatas table at ingest
(like channel_iatas) and group packets by tag instead. Filtered stats
are now per tag rather than per matching packet.
Refresh last_heard on duplicate observations too, capped at hourly, so
steady traffic can't age a channel out of the filter while its packets
stay retained (dedup key has no heard_at). Guard the seed join against
the docker /dev/shm cap like the trace one, and drop the unused
last_heard index column so the upserts stay HOT.
The IATA filter ran a correlated EXISTS over packets/observations with
ILIKE, which skipped the iata index and took ~7s live. Keep a small
channel_iatas table at ingest (like node_iatas) and filter against it.
Also honor the iatas= param so multi-site regions stop getting the
global list. Filter now ages out with packet retention rather than
matching any retained packet.
any route observer where all hops are high confidence
are now stored and available at /routes
you can also search for a src dest route and it will
return any recored route subset that can link the src
and dest hash
generated code, reviewed and tested
this adds an optional lookup table for allowed IATA based on either country,
conintent or both. if the IATA is in either it is allowed.
gives operators the option to consume from global mqtt brokers but filter
to their area of interest
new config for transport scope keys seeded to db
new transport keystore to lookup matching keys and tag
- packets with scope key
- nodes with default scope
- observers with all supported scopes it forwards
allow filtering by region scope key on: packets, nodes, observers
Enot (ded) Skelly