The poller only queried WHERE t.id > sinceID, which missed new
observations added to transmissions already in the store. The trace
page was correct because it always queries the DB directly.
Add IngestNewObservations() that polls observations by o.id watermark,
adds them to existing StoreTx entries, re-picks best observation, and
invalidates analytics caches. The Poller now tracks both lastTxID and
lastObsID watermarks.
Includes tests for v3, v2, dedup, best-path re-pick, and
GetMaxObservationID.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Go staging packets page wasn't live-updating because the deployed
binary was stale (built before the #162 fix). Rebuilding from current
source fixed the issue — broadcasts now fire correctly.
Added two permanent diagnostic log lines:
- [poller] IngestNewFromDB: logs when new transmissions are found
- [broadcast] sending N packets to M clients: logs each broadcast batch
These log lines make it easy to verify the broadcast pipeline is working
and would have caught this stale-deployment issue immediately.
Verified on VM: WS clients receive packets with nested 'packet' field,
all Go tests pass.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The frontend packets.js filters WS messages with m.data?.packet and
extracts m.data.packet for live rendering. Node's server.js includes
a packet sub-object (packet: fullPacket) in the broadcast data, but
Go's IngestNewFromDB built the data flat without a nested packet field.
This caused the Go staging packets page to never live-update via WS
even though messages were being sent — they were silently filtered out
by packets.js.
Fix: build the packet fields map separately, then create the broadcast
map with both top-level fields (for live.js) and nested packet (for
packets.js). Also fixes the fallback DB-direct poller path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Streams transmissions + observations from SQLite at startup into
5 indexed in-memory structures. QueryPackets and QueryGroupedPackets
now serve from RAM (<10ms) instead of hitting SQLite (2.3s).
- store.go: PacketStore with byHash, byTxID, byObsID, byObserver, byNode indexes
- main.go: create + load store at startup
- routes.go: dispatch to store for packet/stats endpoints
- websocket.go: poller ingests new transmissions into store
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Node.js upgrades WS at /, Go was only at /ws. Now the static file
handler checks for Upgrade header first and routes to WebSocket.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
