Commit Graph
10 Commits
Author SHA1 Message Date
Kpa-clawbotandCopilot afe16db960 feat(go-server): in-memory packet store — port of packet-store.js
Streams transmissions + observations from SQLite at startup into
5 indexed in-memory structures. QueryPackets and QueryGroupedPackets
now serve from RAM (<10ms) instead of hitting SQLite (2.3s).

- store.go: PacketStore with byHash, byTxID, byObsID, byObserver, byNode indexes
- main.go: create + load store at startup
- routes.go: dispatch to store for packet/stats endpoints
- websocket.go: poller ingests new transmissions into store

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 08:52:07 -07:00
Kpa-clawbotandCopilot 616af26981 fix: Go ingestor normalize mqtt:// to tcp:// and mqtts:// to ssl:// for paho
Paho MQTT client uses tcp:// and ssl:// schemes, not mqtt:// and mqtts://.
Also properly configure TLS for mqtts connections with InsecureSkipVerify
when rejectUnauthorized is false.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 07:56:57 -07:00
Kpa-clawbotandCopilot dbac8e9d52 fix: Go server since/until filter uses observation timestamp, not first_seen
The frontend sends ISO timestamps to filter by observation time.
Go was filtering by transmission first_seen which missed packets
with recent observations but old first_seen. Now converts ISO to
unix epoch and queries the observations table directly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 07:53:59 -07:00
Kpa-clawbotandCopilot 5c68605f2c feat(go-server): full API parity with Node.js server
Performance:
- QueryGroupedPackets: 8s → <100ms (transmissions table, not packets_v VIEW)

Field parity:
- /api/stats: totalNodes uses 7-day window, added totalNodesAllTime
- /api/stats: role counts filtered by 7-day (matching Node.js)
- /api/nodes: role counts use all-time (matching Node.js)
- /api/packets/🆔 path field returns parsed path_json hops
- /api/packets: added multi-node filter (?nodes=pk1,pk2)
- /api/observers: packetsLastHour, lat, lon, nodeRole computed
- /api/observers/🆔 packetsLastHour computed
- /api/nodes/bulk-health: per-node stats from SQL

Tests updated with dynamic timestamps for 7-day filter compat.
All tests pass, go vet clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 02:11:33 -07:00
Kpa-clawbotandCopilot e18a73e1f2 feat: Go server API parity with Node.js — response shapes, perf, computed fields
- Packets query rewired from packets_v VIEW (9s) to direct table joins (~50ms)
- Packet response: added first_seen, observation_count; removed created_at, score
- Node response: added last_heard, hash_size, hash_size_inconsistent
- Schema-aware v2/v3 detection for observer_idx vs observer_id
- All Go tests passing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 01:50:46 -07:00
Kpa-clawbotandCopilot 842b49e8c4 perf: fast-path count for unfiltered /api/packets (skip packets_v scan)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 01:29:22 -07:00
Kpa-clawbotandCopilot b2e6c8105b fix: handle WebSocket upgrade at root path (client connects to ws://host/)
Node.js upgrades WS at /, Go was only at /ws. Now the static file
handler checks for Upgrade header first and routes to WebSocket.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 01:25:35 -07:00
Kpa-clawbotandCopilot 742ed86596 feat: add Go web server (cmd/server/) — full API + WebSocket + static files
35+ REST endpoints matching Node.js server, WebSocket broadcast,
static file serving with SPA fallback, config.json support.
Uses modernc.org/sqlite (pure Go, no CGO required).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 01:16:59 -07:00
Kpa-clawbotandCopilot e89c2bfe1f test: add comprehensive Go test coverage for ingestor (80%) and server (90%)
- ingestor: add config_test.go (LoadConfig, env overrides, legacy MQTT)
- ingestor: add main_test.go (toFloat64, firstNonEmpty, handleMessage, advertRole)
- ingestor: extend decoder_test.go (short buffer errors, edge cases, all payload types)
- ingestor: extend db_test.go (empty hash, timestamp updates, BuildPacketData, schema)
- server: add config_test.go (LoadConfig, LoadTheme, health thresholds, ResolveDBPath)
- server: add helpers_test.go (writeJSON/Error, queryInt, mergeMap, round, percentile, spaHandler)
- server: extend db_test.go (all query functions, filters, channel messages, node health)
- server: extend routes_test.go (all endpoints, error paths, analytics, observer analytics)
- server: extend websocket_test.go (multi-client, buffer full, poller cycle)

Coverage: ingestor 48% -> 80%, server 52% -> 90%

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 00:07:44 -07:00
Kpa-clawbotandCopilot 488ead617d feat: add standalone Go MQTT ingestor (cmd/ingestor/)
First step of Go rewrite — separates MQTT ingestion from the Node.js
web server. Single static binary (no CGO) that connects to MQTT
brokers, decodes MeshCore packets, and writes to the shared SQLite DB.

Ported from JS:
- decoder.js → decoder.go (header, path, all payload types, adverts)
- computeContentHash → Go (SHA-256, path-independent)
- db.js v3 schema → db.go (transmissions, observations, nodes, observers)
- server.js MQTT logic → main.go (multi-broker, reconnect, IATA filter)

25 Go tests passing (golden fixtures from production + schema compat).
No existing JS files modified.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-26 23:22:26 -07:00