Fixes#2024.
`s.packets` is declared "sorted by first_seen ASC (oldest first; newest
at tail)" (`cmd/server/store.go:177`), and retention eviction depends on
it: `evictStaleInternal` walks from the head and stops at the first
transmission inside the window. A slice out of order is therefore
**under-evicted silently** rather than failing loudly.
## What breaks it
The background chunk loader. Chunks are windowed on `last_seen` (#1690),
so a transmission first heard weeks ago and heard again recently arrives
in a *recent* chunk carrying its old `first_seen`. The chunk was then
put in front of the slice:
```go
s.packets = append(localPackets, s.packets...)
```
and never re-sorted, so the next chunk, which covers an older window,
was prepended in front of it and left that ancient row sitting behind
newer ones. `LoadChunked` re-sorts after its own load; the background
merge did not. That asymmetry is the whole bug.
It is not a corner case. On a production database, of the **236080**
transmissions in a 14 day window, **2071** have a `first_seen` more than
a day older than their `last_seen`, and **1848** more than a week.
This matters more since #2035: with the accounting fixed, `maxMemoryMB`
actually triggers, and a walk that stops early works against it.
## The fix
`mergeChunkIntoPackets` merges the two sorted runs linearly. Re-sorting
the whole slice was not an option: this runs under `s.mu` once per
chunk, so it would sort hundreds of thousands of packets while ingest
waits for the lock. The chunk already arrives sorted, since the chunk
query ends in `ORDER BY t.first_seen ASC`, so the `sort.SliceIsSorted`
guard is a contract check costing one linear pass that never sorts in
production.
## Covered
- `TestMergeChunkIntoPackets_KeepsFirstSeenOrder` pins the merge against
an interleaving, deliberately unsorted chunk.
- `BenchmarkMergeChunkIntoPackets` guards the linear cost, against a
future simplification back into a sort.
The server suite runs under `-race` in CI and is green.
## Not covered, and I would rather say it than let the PR imply
otherwise
There is **no integration test driving `loadChunk` end to end**. I wrote
one and dropped it: a faithful seed database for that path needs more of
the schema and more of the loader's preconditions than the fix itself is
worth. Two CI rounds in, the seed was still loading zero packets (the
first attempt failed at `OpenDB` on a missing `nodes` table, the second
on the window). Both attempts are in this branch's history rather than
rewritten away.
So the end-to-end claim rests on the code path quoted above and on the
production measurement, not on a test that exercises it. The unit test
covers the function where the logic now lives, which is the part that
can regress.
Also not verified locally: `cmd/server` needs cgo for the #1992 driver
and this machine has no C toolchain, so CI is the check.
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
