#156: The Go WebSocket broadcast from IngestNewFromDB was missing the
'decoded' field (with header.payloadTypeName) that live.js needs to
display packet types. Added decoded object with payloadTypeName
resolution, plus observer_id, observer_name, snr, rssi, path_json,
and observation_count fields to match the Node.js broadcast shape.
#157-160: Analytics endpoints (hash-sizes, topology, channels) were
recomputing everything on every request. Added:
- TTL caching (15s) for topology, hash-sizes, and channels endpoints
(matching the existing RF cache pattern)
- Cached node list + prefix map shared across analytics (30s TTL)
- Lazy-cached parsed path JSON on StoreTx (parse once, read many)
- Cache invalidation on new data ingestion
- Global payloadTypeNames map (avoids per-call allocation)
Fixes#156, fixes#157, fixes#158, fixes#159, fixes#160
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
fixes#154: Go analytics channels showed single 'ch?' because
channelHash is a JSON number (from decoder.js) but the Go struct
declared it as string. json.Unmarshal failed on every packet.
Changed to interface{} with proper type conversion. Also fixed
chKey to use hash (not name) for grouping, matching Node.js.
fixes#155: uniqueNodes in topology analytics used hop resolution
count (phantom hops inflated it). Both Node.js and Go now use
db.getStats().totalNodes (7-day active window), matching /api/stats.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- db.go: Add freelistMB (PRAGMA freelist_count * page_size) and walPages
(PRAGMA wal_checkpoint(PASSIVE)) to GetDBSizeStats
- store.go: Add advertByObserver count to GetPerfStoreStats indexes
(count distinct pubkeys with ADVERT observations)
- db.go: Add getObservationsForTransmissions helper; enrich
GetRecentTransmissionsForNode results with observations array,
_parsedPath, and _parsedDecoded
- db_test.go: Add second ADVERT with different hash_size to seed data
so hash_sizes_seen is populated; enrich decoded_json with full
ADVERT fields; update count assertions for new seed row
fixes#151, fixes#152
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
When engine=go, the perf page now renders Go-specific runtime stats
(goroutines, GC collections, GC pause times, heap breakdown, CPUs)
instead of the misleading Node.js Event Loop metrics. Falls back to
the existing Node UI when engine is not 'go' or goRuntime data is
missing. Includes color-coded GC pause thresholds.
fixes#153
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Frontend reads ps.inMemory.toLocaleString() which crashed because
the Go response was missing inMemory, sqliteOnly, maxPackets, maxMB,
evicted, inserts, queries fields. Added all + atomic counters.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The frontend perf dashboard crashes with toLocaleString error because
the Go /api/perf endpoint was missing fields that Node returns:
- status: "ok"
- uptimeHuman: formatted "Xh Ym" string
- websocket: { clients: N } via hub.ClientCount()
The Go /api/health endpoint already had all three fields.
fixes#150
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Fix#148: channels endpoint returned null for msgLengths when no
decrypted messages exist. Initialize msgLengths as make([]int, 0)
in store path and guard channels slice in DB fallback path.
- Fix#149: nodes endpoint always returned hash_size=null and
hash_size_inconsistent=false. Add GetNodeHashSizeInfo() to
PacketStore that scans advert packets to compute per-node hash
size, flip-flop detection, and sizes_seen. Enrich nodes in both
handleNodes and handleNodeDetail with computed hash data.
fixes#148, fixes#149
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Go's /api/health was missing the eventLoop object that Node.js provides.
The perf.js frontend reads health.eventLoop.p95Ms which crashed with
'Cannot read properties of undefined' when served by the Go server.
Adds eventLoop field using GC pause data from runtime.MemStats.PauseNs
(last 256 pauses) to compute p50Ms, p95Ms, p99Ms, currentLagMs, maxLagMs
— matching the Node.js response shape exactly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Go analytics topology endpoint was counting every unique hop string
from packet paths (including unresolved 1-byte hex prefixes) as a unique
node, inflating the count from ~540 to 6502. Now resolves each hop via
the prefix map and deduplicates by public key, matching the Node.js
behavior.
fixes#146
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- /api/perf: add goRuntime (heap, GC, goroutines, CPU), packetStore
stats (totalLoaded, observations, index sizes, estimatedMB),
sqlite stats (dbSizeMB, walSizeMB, row counts), real RF cache
hit/miss tracking, and endpoint sorting by total time spent
- /api/health: add memory.heapMB, goRuntime (goroutines, gcPauses,
numCPU), real packetStore packet count and estimatedMB, real
cache stats from RF cache; remove hardcoded-zero eventLoop
- store.go: add cacheHits/cacheMisses tracking in GetAnalyticsRF,
GetPerfStoreStats() and GetCacheStats() methods
- db.go: add path field to DB struct, GetDBSizeStats() for file
sizes and row counts
- Tests: verify new fields in health/perf endpoints, add
TestGetDBSizeStats, wire up PacketStore in test server setup
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Implement all analytics endpoints from in-memory PacketStore instead of
returning stubs/empty data. Each handler now matches the Node.js response
shape field-by-field.
Endpoints fixed:
- /api/analytics/topology (#135): full hop distribution, top repeaters,
top pairs, hops-vs-SNR, per-observer reachability, cross-observer
comparison, best path analysis
- /api/analytics/distance (#137): haversine distance computation,
category stats (R↔R, C↔R, C↔C), distance histogram, top hops/paths,
distance over time
- /api/analytics/hash-sizes (#136): hash size distribution from raw_hex
path byte parsing, hourly breakdown, top hops, multi-byte node tracking
- /api/analytics/hash-issues (#138): hash-sizes data now populated so
frontend collision tab can compute inconsistent sizes and collision risk
- /api/analytics/route-patterns (#134): subpaths and subpath-detail now
compute from in-memory store with hop resolution
- /api/nodes/bulk-health (#140): switched from N per-node SQL queries to
in-memory PacketStore lookups with observer stats
- /api/channels (#142): response shape already correct via GetChannels;
analytics/channels now returns topSenders, channelTimeline, msgLengths
- /api/analytics/channels: full channel analytics with sender tracking,
timeline, and message length distribution
All handlers fall back to DB/stubs when store is nil (test compat).
All 42+ existing Go tests pass. go vet clean.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Style .version-badge anchor elements to use --nav-text-muted color
instead of browser-default blue. Adds hover state using --nav-text.
Works with both light and dark themes via existing CSS variables.
fixes#139
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Go server and ingestor tests now run with -coverprofile
- Coverage percentages parsed and printed in CI output
- Badge JSON files generated (.badges/go-server-coverage.json,
.badges/go-ingestor-coverage.json) matching existing format
- Badges uploaded as artifacts from go-build job, downloaded
in test job, and published alongside existing Node.js badges
- Coverage summary table added to GitHub Step Summary
fixes#141
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Commit hash is now an <a> linking to GitHub commit (full hash in URL, 7-char display)
- Version tag only shown on prod (port 80/443 or no port), linked to GitHub release
- Staging (non-standard port) shows commit + engine only, no version noise
- Detect prod vs staging via location.port
- Updated tests: 16 cases covering prod/staging/links/edge cases
- Bumped cache busters
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add formatVersionBadge() that renders version, short commit hash, and
engine as a single badge in the nav stats area. Format: v2.6.0 · abc1234 [go].
Skips commit when 'unknown' or missing. Truncates commit to 7 chars.
Replaces the standalone engine badge call in updateNavStats().
8 unit tests cover all edge cases (missing fields, v-prefix dedup,
unknown commit, truncation).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Previous approach invalidated cache on every ingest (every 1s with live
mesh data). Now uses TTL-only expiry (15s). Separate cache mutex avoids
data race with main store RWMutex.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Node.js: reads version from package.json, commit from .git-commit file
or git rev-parse --short HEAD at runtime, with unknown fallback.
Go: uses -ldflags build-time variables (Version, Commit) with fallback
to .git-commit file and git command at runtime.
Dockerfile: copies .git-commit if present (CI bakes it before build).
Dockerfile.go: passes APP_VERSION and GIT_COMMIT as build args to ldflags.
deploy.yml: writes GITHUB_SHA to .git-commit before docker build steps.
docker-compose.yml: passes build args to Go staging build.
Tests updated to verify version and commit fields in both endpoints.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cache the computed RF analytics result for 15 seconds.
1.2M observation scan takes ~140ms; cached response <1ms.
Cache invalidated when new packets are ingested.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build and deploy the Go staging container (port 82) after Node staging
is healthy. Uses continue-on-error so Go staging failures don't block
the Node.js deploy. Health-checks the Go container for up to 60s and
verifies /api/stats returns the engine field.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Move per-transmission work (hash indexing, type resolution, packet sizes)
outside the per-observation loop. Cache SNR dereference, pre-resolve type
name once per transmission. Reduces redundant map lookups from 1.2M to 52K.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both backends now return an 'engine' field ('node' or 'go') in
/api/stats and /api/health responses so the frontend can display
which backend is running.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Show [go] or [node] badge in the nav stats bar when /api/stats
returns an engine field. Gracefully hidden when field is absent.
- Add formatEngineBadge() to app.js (top-level, testable)
- Add .engine-badge CSS class using CSS variables
- Add 5 unit tests in test-frontend-helpers.js
- Bump cache busters
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add byPayloadType index to PacketStore for O(1) type-5 lookups
- Channels scan reduced from 52K to ~17K packets (3x fewer iterations)
- Use struct-based JSON decoding (avoids map[string]interface{} allocations)
- Pre-allocate snrVals/rssiVals/scatterAll with capacity hints for 1.2M obs
- Remove second-pass time.Parse loop (1.2M calls) in RF analytics
Track min/max timestamps as strings during first pass instead
- Index also populated during IngestNewFromDB for new packets
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace SQLite-backed handlers for /api/channels, /api/channels/:hash/messages,
/api/analytics/rf, and /api/analytics/channels with in-memory PacketStore queries.
Before (SQLite via packets_v VIEW on 1.2M rows):
/api/channels 7.2s
/api/channels/:hash/msgs 8.2s
/api/analytics/rf 4.2s
After (in-memory scan of ~50K transmissions):
Target: all under 100ms
Three new PacketStore methods:
- GetChannels(region) — filters payload_type 5 + decoded type CHAN
- GetChannelMessages(hash, limit, offset) — deduplicates by sender+hash
- GetAnalyticsRF(region) — full RF stats with histograms, scatter, per-type SNR
All handlers fall back to DB queries when store is nil (test compat).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add detail-collapsed class to split-layout initial HTML so the empty
right panel is hidden before any packet is selected. The class is
already removed when a packet row is clicked and re-added when the
close button is pressed.
Add 3 tests verifying the detail pane starts collapsed and that
open/close toggling is wired correctly.
Bump cache busters.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Streams transmissions + observations from SQLite at startup into
5 indexed in-memory structures. QueryPackets and QueryGroupedPackets
now serve from RAM (<10ms) instead of hitting SQLite (2.3s).
- store.go: PacketStore with byHash, byTxID, byObsID, byObserver, byNode indexes
- main.go: create + load store at startup
- routes.go: dispatch to store for packet/stats endpoints
- websocket.go: poller ingests new transmissions into store
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add go-build job to deploy.yml that builds and tests cmd/server and cmd/ingestor
- Go job gates the Node.js test job and deploy job
- Re-enable frontend coverage detection (was hardcoded to false)
- Remove stale temp files from repo root (recover-delta.sh, merge.sh, replacements.txt, reps.txt)
- Add temp scripts and Go build artifacts to .gitignore
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Change staging-go HTTP port from 81 to 82 (via STAGING_GO_HTTP_PORT)
to avoid conflict with CI's Node.js staging on port 81
- Change staging-go MQTT port from 1884 to 1885 (via STAGING_GO_MQTT_PORT)
to avoid conflict with Node.js staging MQTT on port 1884
- Add MQTT_BROKER=mqtt://localhost:1883 env var so Go ingestor connects
to its own internal mosquitto instead of unreachable prod external IP
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add #/compare page that lets users select two observers and compare
which packets each sees. Fetches last 24h of packets per observer,
computes set diff client-side using O(n) Set lookups. Shows summary
cards (both/only-A/only-B), stacked bar, type breakdown, and tabbed
detail tables. URL is shareable via ?a=ID1&b=ID2 query params.
- New file: public/compare.js (comparePacketSets + page module)
- Added compare button to observers page header
- 11 new tests for comparePacketSets (87 total frontend tests)
- Cache busters bumped
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
