mirror of
https://github.com/Kpa-clawbot/meshcore-analyzer.git
synced 2026-04-25 19:22:13 +00:00
81ef51cc5c
## Problem On busy meshes (325K+ transmissions, 50 observers), the distance index rebuild runs on **every ingest poll** (~1s interval), computing haversine distances for 1M+ hop records. Each rebuild takes 2-3 seconds but new observations arrive faster than it can finish, creating a CPU hot loop that starves the HTTP server. Discovered on the Cascadia Mesh instance where `corescope-server` was consuming 15 minutes of CPU time in 10 minutes of uptime, the API was completely unresponsive, and health checks were timing out. ### Server logs showing the hot loop: ``` [store] Built distance index: 1797778 hop records, 207072 path records [store] Built distance index: 1797806 hop records, 207075 path records [store] Built distance index: 1797811 hop records, 207075 path records [store] Built distance index: 1797820 hop records, 207075 path records ``` Every 2 seconds, nonstop. ## Root Cause `IngestNewObservations` calls `buildDistanceIndex()` synchronously whenever `pickBestObservation` selects a longer path. With 50 observers sending observations every second, paths change on nearly every poll cycle, triggering a full rebuild each time. ## Fix - Mark distance index dirty on path changes instead of rebuilding inline - Rebuild at most every **30 seconds** (configurable via `distLast` timer) - Set `distLast` after initial `Load()` to prevent immediate re-rebuild on first ingest - Distance data is at most 30s stale — acceptable for an analytics view ## Testing - `go build`, `go vet`, `go test` all pass - No behavioral change for the initial load or the analytics API response shape - Distance data freshness goes from real-time to 30s max staleness --------- Co-authored-by: Kpa-clawbot <259247574+Kpa-clawbot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: you <you@example.com>