Commit Graph

7 Commits

Author SHA1 Message Date
Kpa-clawbot
2435f2eaaf fix: observation timestamps, leaked fields, perf path normalization
- #178: Use strftime ISO 8601 format instead of datetime() for observation
  timestamps in all SQL queries (v3 + v2 views). Add normalizeTimestamp()
  helper for non-v3 paths that may store space-separated timestamps.

- #179: Strip internal fields (decoded_json, direction, payload_type,
  raw_hex, route_type, score, created_at) from ObservationResp. Only
  expose id, transmission_id, observer_id, observer_name, snr, rssi,
  path_json, timestamp — matching Node.js parity.

- #180: Remove _parsedDecoded and _parsedPath from node detail
  recentAdverts response. These internal/computed fields were leaking
  to the API. Updated golden shapes.json accordingly.

- #181: Use mux route template (GetPathTemplate) for perf stats path
  normalization, converting {param} to :param for Node.js parity.
  Fallback to hex regex for unmatched routes. Compile regexes once at
  package level instead of per-request.

fixes #178, fixes #179, fixes #180, fixes #181

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 18:09:36 -07:00
Kpa-clawbot
64bf3744e2 fix: channels stale latest message from observation-timestamp ordering, fixes #171
db.GetChannels() queried packets_v (observation-level rows) ordered by
observation timestamp and always overwrote lastMessage. When an older
message had a later re-observation, it would overwrite the correct
latest message with stale data.

Fix: query transmissions table directly (one row per unique message)
ordered by first_seen. This ensures lastMessage always reflects the
most recently sent message, not the most recently observed one.

Also fix db.GetChannelMessages() to use first_seen ordering with
schema-aware queries (v2/v3), and add missing distCache/subpathCache
invalidation on packet ingestion.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 16:01:54 -07:00
Kpa-clawbot
2f5404edc3 fix: close last parity gaps in /api/perf and /api/nodes/:pubkey
- db.go: Add freelistMB (PRAGMA freelist_count * page_size) and walPages
  (PRAGMA wal_checkpoint(PASSIVE)) to GetDBSizeStats
- store.go: Add advertByObserver count to GetPerfStoreStats indexes
  (count distinct pubkeys with ADVERT observations)
- db.go: Add getObservationsForTransmissions helper; enrich
  GetRecentTransmissionsForNode results with observations array,
  _parsedPath, and _parsedDecoded
- db_test.go: Add second ADVERT with different hash_size to seed data
  so hash_sizes_seen is populated; enrich decoded_json with full
  ADVERT fields; update count assertions for new seed row

fixes #151, fixes #152

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 11:57:35 -07:00
Kpa-clawbot
93dbe0e909 fix(go): add runtime stats to /api/perf and /api/health, fixes #143
- /api/perf: add goRuntime (heap, GC, goroutines, CPU), packetStore
  stats (totalLoaded, observations, index sizes, estimatedMB),
  sqlite stats (dbSizeMB, walSizeMB, row counts), real RF cache
  hit/miss tracking, and endpoint sorting by total time spent
- /api/health: add memory.heapMB, goRuntime (goroutines, gcPauses,
  numCPU), real packetStore packet count and estimatedMB, real
  cache stats from RF cache; remove hardcoded-zero eventLoop
- store.go: add cacheHits/cacheMisses tracking in GetAnalyticsRF,
  GetPerfStoreStats() and GetCacheStats() methods
- db.go: add path field to DB struct, GetDBSizeStats() for file
  sizes and row counts
- Tests: verify new fields in health/perf endpoints, add
  TestGetDBSizeStats, wire up PacketStore in test server setup

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 10:45:00 -07:00
Kpa-clawbot
5c68605f2c feat(go-server): full API parity with Node.js server
Performance:
- QueryGroupedPackets: 8s → <100ms (transmissions table, not packets_v VIEW)

Field parity:
- /api/stats: totalNodes uses 7-day window, added totalNodesAllTime
- /api/stats: role counts filtered by 7-day (matching Node.js)
- /api/nodes: role counts use all-time (matching Node.js)
- /api/packets/🆔 path field returns parsed path_json hops
- /api/packets: added multi-node filter (?nodes=pk1,pk2)
- /api/observers: packetsLastHour, lat, lon, nodeRole computed
- /api/observers/🆔 packetsLastHour computed
- /api/nodes/bulk-health: per-node stats from SQL

Tests updated with dynamic timestamps for 7-day filter compat.
All tests pass, go vet clean.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 02:11:33 -07:00
Kpa-clawbot
e18a73e1f2 feat: Go server API parity with Node.js — response shapes, perf, computed fields
- Packets query rewired from packets_v VIEW (9s) to direct table joins (~50ms)
- Packet response: added first_seen, observation_count; removed created_at, score
- Node response: added last_heard, hash_size, hash_size_inconsistent
- Schema-aware v2/v3 detection for observer_idx vs observer_id
- All Go tests passing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 01:50:46 -07:00
Kpa-clawbot
e89c2bfe1f test: add comprehensive Go test coverage for ingestor (80%) and server (90%)
- ingestor: add config_test.go (LoadConfig, env overrides, legacy MQTT)
- ingestor: add main_test.go (toFloat64, firstNonEmpty, handleMessage, advertRole)
- ingestor: extend decoder_test.go (short buffer errors, edge cases, all payload types)
- ingestor: extend db_test.go (empty hash, timestamp updates, BuildPacketData, schema)
- server: add config_test.go (LoadConfig, LoadTheme, health thresholds, ResolveDBPath)
- server: add helpers_test.go (writeJSON/Error, queryInt, mergeMap, round, percentile, spaHandler)
- server: extend db_test.go (all query functions, filters, channel messages, node health)
- server: extend routes_test.go (all endpoints, error paths, analytics, observer analytics)
- server: extend websocket_test.go (multi-client, buffer full, poller cycle)

Coverage: ingestor 48% -> 80%, server 52% -> 90%

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-03-27 00:07:44 -07:00