Joel Claw and Joel Claw
ac6fbaf9f3
perf: reuse ctx buffer in resolvePathForObs, cache ReadMemStats per store ( #1873 )
...
## Problem
Three hot-path inefficiencies causing excess CPU and memory allocations:
### 1. `filterTxSlice` starts with nil slice
`filterTxSlice` is called on the full `s.packets` slice (50k+ packets)
for every query that doesn't hit a fast-path index. Starting with `var
result []*StoreTx` means Go's append does ~15 growth+copy cycles
(1→2→4→8→...→32768→65536) before reaching steady state.
### 2. `resolvePathForObs` allocates per hop
Each hop in the path resolution loop allocates a new `ctx` slice
(`make([]string, len(contextPKs), len(contextPKs)+2)`). For a 5-hop
path, that's 5 allocations per observation. With 500+ observations per
ingest batch, that's 2500+ small allocations.
### 3. `estimatedMemoryMB` calls `runtime.ReadMemStats` without caching
`runtime.ReadMemStats()` triggers a STW (stop-the-world) pause. It's
called from stats/debug endpoints (`GetStoreStats`, `GetPerfStoreStats`)
that may be polled frequently. The routes.go layer already caches this
with a 5s TTL, but the store layer doesn't.
## Fix
1. **Pre-allocate `filterTxSlice`**: `make([]*StoreTx, 0, n/2)` — the 2x
over-allocation is cheaper than repeated growth+copy.
2. **Reuse ctx buffer**: Allocate one `ctx` buffer before the hop loop,
reset to base length each iteration with `ctx = ctx[:ctxLen]`.
3. **Cache `ReadMemStats`**: 5-second TTL cache matching the routes.go
pattern. Uses a package-level mutex (not on `PacketStore`) to avoid
adding a field.
## Testing
- `go build` passes
- No behavior change — same results, fewer allocations
---------
Co-authored-by: Joel Claw <358739783+Joel-Claw@users.noreply.github.com >
2026-09-02 23:20:39 +02:00
..
2026-04-21 09:09:39 -07:00
2026-04-03 13:51:13 -07:00
2026-07-08 22:14:41 -07:00
2026-07-08 22:14:41 -07:00
2026-05-21 14:00:15 -07:00
2026-06-12 21:02:59 +00:00
2026-06-12 21:02:59 +00:00
2026-06-12 21:02:59 +00:00
2026-04-05 14:50:40 -07:00
2026-05-21 14:00:15 -07:00
2026-05-03 17:56:42 -07:00
2026-05-03 17:56:42 -07:00
2026-07-03 02:21:08 -07:00
2026-05-18 22:51:23 -07:00
2026-05-18 22:51:23 -07:00
2026-05-18 22:51:23 -07:00
2026-06-27 22:03:05 -07:00
2026-06-04 03:21:26 -07:00
2026-04-12 18:09:23 -07:00
2026-06-04 13:14:09 +00:00
2026-06-04 13:14:09 +00:00
2026-09-02 10:27:40 +02:00
2026-04-20 21:46:34 -07:00
2026-05-25 17:45:32 -07:00
2026-05-25 22:16:14 -07:00
2026-06-07 03:43:29 -07:00
2026-06-07 03:43:29 -07:00
2026-06-07 03:43:29 -07:00
2026-06-07 03:43:29 -07:00
2026-09-02 18:34:59 +02:00
2026-06-03 13:58:04 -07:00
2026-06-06 22:45:05 -07:00
2026-09-02 09:50:18 +02:00
2026-09-02 09:50:18 +02:00
2026-09-02 09:50:18 +02:00
2026-05-18 12:27:44 -07:00
2026-09-02 09:50:18 +02:00
2026-09-02 09:50:18 +02:00
2026-05-21 14:00:15 -07:00
2026-05-21 11:39:49 -07:00
2026-05-21 11:39:49 -07:00
2026-05-21 11:39:49 -07:00
2026-05-19 23:53:41 -07:00
2026-06-06 22:45:05 -07:00
2026-09-02 09:50:09 +02:00
2026-05-30 13:22:41 -07:00
2026-05-30 13:22:41 -07:00
2026-05-30 13:22:41 -07:00
2026-06-19 11:37:16 -07:00
2026-06-27 22:03:05 -07:00
2026-06-27 22:03:05 -07:00
2026-09-02 14:21:42 +02:00
2026-06-04 14:41:00 -07:00
2026-09-02 13:10:43 +00:00
2026-09-02 11:07:04 +02:00
2026-09-02 18:34:59 +02:00
2026-06-12 19:10:44 -07:00
2026-05-15 22:34:21 -07:00
2026-05-15 22:34:21 -07:00
2026-09-02 10:27:56 +02:00
2026-05-05 01:16:57 -07:00
2026-05-05 01:16:57 -07:00
2026-06-29 15:53:59 -07:00
2026-05-21 14:00:15 -07:00
2026-04-16 00:09:36 -07:00
2026-05-16 20:56:52 +00:00
2026-06-04 16:27:48 -07:00
2026-05-05 01:58:52 -07:00
2026-05-19 23:53:41 -07:00
2026-03-31 01:10:56 -07:00
2026-05-31 14:54:21 -07:00
2026-06-22 10:07:49 -07:00
2026-05-29 02:42:21 -07:00
2026-06-27 22:03:05 -07:00
2026-04-18 11:52:22 -07:00
2026-04-18 11:52:22 -07:00
2026-05-19 23:53:41 -07:00
2026-06-07 09:28:51 -07:00
2026-04-03 13:11:59 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-06-11 10:10:12 -07:00
2026-06-11 10:10:12 -07:00
2026-05-15 16:21:14 +00:00
2026-05-16 19:55:00 +00:00
2026-06-04 23:48:47 -07:00
2026-05-15 20:24:55 -07:00
2026-05-15 22:46:25 -07:00
2026-06-12 12:47:53 -07:00
2026-06-06 22:45:05 -07:00
2026-06-06 20:46:42 -07:00
2026-04-20 22:15:02 -07:00
2026-05-21 14:00:15 -07:00
2026-04-21 04:51:24 +00:00
2026-05-02 20:35:15 -07:00
2026-05-17 16:13:11 +00:00
2026-05-19 08:08:28 -07:00
2026-05-18 23:19:27 -07:00
2026-06-30 17:19:56 -07:00
2026-06-25 05:05:46 -07:00
2026-06-30 17:19:56 -07:00
2026-06-11 07:38:36 -07:00
2026-06-11 07:38:36 -07:00
2026-06-11 07:38:36 -07:00
2026-06-06 22:44:59 -07:00
2026-06-11 11:36:49 -07:00
2026-06-11 11:36:49 -07:00
2026-06-11 11:36:49 -07:00
2026-06-04 14:41:22 -07:00
2026-09-02 09:50:14 +02:00
2026-05-28 15:06:30 -07:00
2026-05-28 15:06:30 -07:00
2026-04-20 23:10:33 -07:00
2026-09-02 10:27:48 +02:00
2026-06-30 15:16:31 -07:00
2026-06-12 08:11:02 -07:00
2026-09-02 10:27:48 +02:00
2026-05-25 22:35:35 -07:00
2026-05-03 08:56:09 -07:00
2026-05-21 14:00:15 -07:00
2026-06-09 01:24:46 -07:00
2026-09-02 09:50:09 +02:00
2026-04-02 23:45:03 -07:00
2026-05-15 09:16:39 -07:00
2026-05-15 09:16:39 -07:00
2026-05-29 02:42:21 -07:00
2026-05-29 02:42:21 -07:00
2026-05-29 02:42:21 -07:00
2026-05-29 02:42:21 -07:00
2026-05-16 10:14:44 -07:00
2026-09-02 11:07:18 +02:00
2026-09-02 11:07:18 +02:00
2026-06-12 11:38:43 -07:00
2026-09-02 23:20:39 +02:00
2026-05-19 23:53:41 -07:00
2026-05-19 23:53:41 -07:00
2026-05-05 01:41:00 -07:00
2026-05-05 01:41:00 -07:00
2026-04-17 23:43:05 +00:00
2026-06-08 01:27:13 -07:00
2026-06-09 00:27:56 -07:00
2026-06-09 03:23:48 -07:00
2026-06-09 03:23:48 -07:00
2026-06-09 00:27:56 -07:00
2026-06-11 10:10:12 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-05-18 09:22:27 -07:00
2026-05-18 07:36:33 -07:00
2026-04-03 13:33:26 -07:00
2026-09-02 11:07:08 +02:00
2026-09-02 11:07:08 +02:00
2026-05-01 23:11:27 -07:00
2026-09-02 09:50:26 +02:00
2026-05-29 01:08:12 -07:00
2026-05-29 01:08:12 -07:00
2026-05-29 02:42:21 -07:00
2026-05-29 02:42:21 -07:00
2026-06-08 01:27:13 -07:00
2026-06-12 01:52:12 -07:00
2026-06-19 11:37:16 -07:00
2026-07-07 00:12:44 -07:00
2026-04-05 15:05:20 -07:00
2026-07-08 22:14:41 -07:00
2026-09-02 18:34:59 +02:00
2026-05-17 16:13:11 +00:00
2026-05-25 22:32:00 -07:00
2026-06-04 23:48:47 -07:00
2026-05-15 22:46:28 -07:00
2026-05-15 22:46:28 -07:00
2026-05-15 22:46:28 -07:00
2026-05-15 22:46:28 -07:00
2026-05-15 22:46:28 -07:00
2026-09-02 09:50:09 +02:00
2026-09-02 09:50:09 +02:00
2026-09-02 14:34:20 +02:00
2026-06-29 15:53:59 -07:00
2026-05-28 15:02:59 -07:00
2026-05-28 14:55:59 -07:00
2026-05-25 06:03:10 +00:00
2026-05-02 11:15:25 -07:00
2026-05-08 16:29:23 -07:00
2026-05-08 16:29:23 -07:00
2026-05-08 16:29:23 -07:00
2026-05-08 16:29:23 -07:00
2026-06-07 09:28:51 -07:00
2026-05-05 17:56:56 -07:00
2026-06-07 09:28:51 -07:00
2026-04-01 19:26:11 -07:00
2026-06-06 22:45:05 -07:00
2026-05-15 09:16:39 -07:00
2026-09-02 14:22:30 +02:00
2026-06-27 22:03:05 -07:00
2026-06-27 22:03:05 -07:00
2026-05-03 19:50:01 -07:00
2026-06-22 10:07:49 -07:00
2026-06-22 10:07:49 -07:00
2026-05-21 11:39:43 -07:00
2026-05-20 20:57:02 -07:00
2026-06-13 00:10:59 -07:00
2026-05-28 15:01:58 -07:00
2026-09-02 18:34:59 +02:00
2026-06-06 20:46:42 -07:00
2026-06-06 20:46:42 -07:00
2026-07-08 15:13:39 -07:00
2026-09-02 18:34:59 +02:00
2026-05-20 20:57:02 -07:00
2026-05-20 20:57:02 -07:00
2026-05-15 16:21:14 +00:00
2026-05-15 09:16:39 -07:00
2026-06-08 01:27:13 -07:00
2026-05-17 16:42:01 -07:00
2026-09-02 10:27:40 +02:00
2026-06-04 15:37:37 -07:00
2026-05-18 07:36:28 -07:00
2026-05-03 17:56:12 -07:00
2026-05-21 14:00:15 -07:00
2026-09-02 11:07:08 +02:00
2026-09-02 11:07:08 +02:00
2026-06-30 17:19:56 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-06-19 11:37:16 -07:00
2026-05-15 16:21:14 +00:00
2026-05-03 17:40:54 -07:00
2026-06-03 13:58:04 -07:00
2026-06-30 18:21:21 -07:00
2026-04-20 23:10:33 -07:00
2026-06-28 13:48:47 -07:00
2026-04-21 09:09:39 -07:00
2026-09-02 23:20:39 +02:00
2026-05-29 02:42:21 -07:00
2026-06-06 21:59:23 -07:00
2026-05-03 17:41:22 -07:00
2026-05-03 17:41:22 -07:00
2026-05-21 14:00:15 -07:00
2026-04-20 19:55:00 -07:00
2026-06-27 22:03:05 -07:00
2026-09-02 18:34:59 +02:00
2026-09-02 09:50:09 +02:00
2026-06-27 22:03:05 -07:00
2026-06-27 22:03:05 -07:00
2026-06-27 22:03:05 -07:00
2026-06-27 22:03:05 -07:00
2026-06-29 05:48:58 -07:00
2026-03-30 03:42:11 +00:00
2026-06-29 05:48:58 -07:00
2026-09-02 14:21:42 +02:00