dandri/meshcore-analyzer
1
0
Fork 0
mirror of https://github.com/Kpa-clawbot/meshcore-analyzer.git synced 2026-06-04 10:36:43 +00:00
Code Issues Packages Projects Releases Wiki Activity
253 Commits 84 Branches 30 Tags
311db0285d6a6db5e32c6f3076281c19e8a2c060
Commit Graph

253 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
you 311db0285d fix: bump all cache busters - browser was serving stale JS 2026-03-20 07:58:57 +00:00
you 01df7f7871 fix: observer column min-width 70px to prevent zero-width squeeze 2026-03-20 07:57:43 +00:00
you 90fa755e7d fix: observer pages scroll - calc(100vh - 56px) for nav bar 2026-03-20 07:53:24 +00:00
you 039d1fc28f fix: resolve observer names from observers table in packets view 
observer_name on packets is often NULL; now cross-references
the loaded observers list to display friendly names
 2026-03-20 07:51:26 +00:00
you d8c0e3a156 fix: only backfill observer name if observer already exists 
Prevents ADVERT nodes from being created as observers
 2026-03-20 07:47:37 +00:00
you bee124e6d2 fix: shrink donut chart, show observer name in dropdown 2026-03-20 07:44:20 +00:00
you fd919a2a80 fix: spark bar invisible in observers table - max-width:0 override 2026-03-20 07:39:46 +00:00
you f58728118d feat: observer detail page with analytics 
- GET /api/observers/:id — observer metadata + packet count
- GET /api/observers/:id/analytics — timeline, type breakdown, nodes heard, SNR distribution
- observer-detail.js — info cards, 4 Chart.js charts, recent packets table
- Observers list rows now clickable to navigate to detail
- Time range selector (24h, 3d, 7d, 30d)
 2026-03-20 07:37:36 +00:00
you 4aa78305d3 feat: backfill observer names from ADVERT pubkey cross-reference 2026-03-20 07:32:46 +00:00
you 8659cda7b7 fix: remove PII from seed data - no real names/coords 2026-03-20 07:30:48 +00:00
you 2713d501b4 feat: MQTT topic arrays, IATA filtering, observer status parsing 
- mqttSources[].topics is now an array of topic patterns
- mqttSources[].iataFilter optionally restricts to specific regions
- meshcore/<region>/<id>/status topic parsed for observer metadata:
  name, model, firmware, client_version, radio, battery, uptime, noise_floor
- New observer columns with auto-migration for existing DBs
- Status updates don't inflate packet_count (separate updateObserverStatus)
 2026-03-20 07:29:01 +00:00
you 4ff72935ca feat: multi-broker MQTT support 
config.mqttSources array allows multiple MQTT brokers with independent
topics, credentials, and TLS settings. Legacy config.mqtt still works
for backward compatibility.
 2026-03-20 07:18:49 +00:00
you a756517647 fix: set XDG_DATA_HOME so Caddy persists certs to mounted volume 2026-03-20 07:13:52 +00:00
you 74983d3f74 ci: switch to self-hosted runner — no SSH, no secrets, no exposed ports 2026-03-20 07:07:01 +00:00
you ab35ced2bf ci: auto-deploy to VM on push to master via GitHub Actions 2026-03-20 07:03:36 +00:00
you ff86a78480 style: proper themed copy-link button matching existing detail action buttons 2026-03-20 06:59:10 +00:00
you 2a076dfb1d feat: shareable URLs for channels — update URL on selection, accept route param 
- selectChannel updates URL to #/channels/<hash>
- init accepts routeParam and auto-selects channel
- Search results use new URL format instead of ?ch= query param
 2026-03-20 06:51:54 +00:00
you fceff15e2f feat: update URL bar when selecting a packet for easy sharing 2026-03-20 06:49:20 +00:00
you 9c87f0040e docs: update README — fix duplicate heading, add Docker/perf files to project structure 2026-03-20 06:47:07 +00:00
you 395abc2585 feat: standalone packet detail page at #/packet/ID 
- New route #/packet/123 shows full packet detail on its own page
- Back link to packets list
- Copy Link button now generates #/packet/ID URLs
- Reuses existing renderDetail() for consistent display
 2026-03-20 06:44:18 +00:00
you e82e4fe05f fix: copy link URL format — use #/packets/id/N not query param 2026-03-20 06:42:12 +00:00
you 6cf9793706 feat: copy link button in packet detail pane 2026-03-20 06:41:36 +00:00
you 1772b34e8f fix: copy all JS files in Dockerfile — was missing decoder.js 2026-03-20 06:18:04 +00:00
you fea8a7e0b5 feat: add Caddy to Docker container — automatic HTTPS 
- Caddy reverse proxies :80/:443 → Node :3000
- Mount custom Caddyfile for your domain → auto Let's Encrypt TLS
- Caddy certs persisted in /data/caddy volume
- Ports: 80 (HTTP), 443 (HTTPS), 1883 (MQTT)
 2026-03-20 06:07:38 +00:00
you 2e486e2a66 feat: Docker packaging — single container with Mosquitto + Node 
- Dockerfile: Alpine + Node 22 + Mosquitto + supervisord
- Auto-copies config.example.json if no config.json mounted
- Named volume for data persistence (SQLite + Mosquitto)
- Ports: 3000 (web), 1883 (MQTT)
- .dockerignore excludes data, config, git, benchmarks
- README updated with Docker quickstart
 2026-03-20 06:06:15 +00:00
you f0c29b38f1 chore: bump perf.js cache buster 2026-03-20 05:47:29 +00:00
you 46d9b690ee fix: close if(health) block in perf dashboard — was swallowing all content 2026-03-20 05:47:11 +00:00
you 2e51e5f743 feat: system health + SWR stats in perf dashboard 
Perf page now shows: heap usage, RSS, event loop p95/max/current,
WS client count, stale-while-revalidate hits, recompute count.
Color-coded: green/yellow/red based on thresholds.
 2026-03-20 05:45:51 +00:00
you 11b398cfe1 feat: stale-while-revalidate cache + /api/health telemetry 
Cache: entries stay valid for 2× TTL as stale. First request after
TTL serves stale data while recompute runs (guarded: one at a time).
No more cache stampedes.

/api/health returns:
- Process memory (RSS, heap)
- Event loop lag (p50/p95/p99/max, sampled every 1s)
- Cache stats (hit rate, stale hits, recomputes)
- WebSocket client count
- Packet store size
- Recent slow queries
 2026-03-20 05:43:32 +00:00
you f4ac789ee9 release: v2.1.0 — Performance 
Two-layer caching: in-memory packet store + TTL response cache.
All packet reads from RAM, SQLite write-only.

Highlights:
- Bulk Health: 7,059ms → 1ms (7,059×)
- Node Analytics: 381ms → 1ms (381×)
- Topology: 685ms → 2ms (342×)
- RF Analytics: 253ms → 1ms (253×)
- Channels: 206ms → 1ms (206×)
- Node Health/Detail: 133-195ms → 1ms

Architecture:
- In-memory packet store with Map indexes (byNode, byHash, byObserver)
- Ring buffer with configurable max (1GB default, ~2.3M packets)
- Smart cache invalidation (packet bursts don't nuke analytics)
- Pre-warm all heavy endpoints on startup
- Eliminated every LIKE '%pubkey%' full-table scan
- All TTLs configurable via config.json
- A/B benchmark script included
- Favicon added
 2026-03-20 05:38:23 +00:00
you 2a2a80b4ea chore: add A/B benchmark script, remove worker thread experiments 2026-03-20 05:37:08 +00:00
you 6dd077be13 feat: add favicon — mesh network icon (SVG + ICO) 2026-03-20 05:36:32 +00:00
you 2b3597dff1 fix: null guard getElementById in animatePacket 
Elements don't exist yet when replayRecent fires during init.
 2026-03-20 05:34:04 +00:00
you 77b7b218b1 perf: channels endpoint — single pass, no sort, no double filter 
Was doing two pktStore.filter() calls + sort on each. Now single
loop over all packets with inline type check.
 2026-03-20 05:31:03 +00:00
you 0a499745ec perf: pre-warm all heavy analytics endpoints on startup 
Sequential self-requests after subpath pre-warm completes.
RF, topology, channels, hash-sizes, bulk-health all cached
before any user hits the page.
 2026-03-20 05:29:10 +00:00
you c83eb099c9 perf: stop calling db.getNode() in health/analytics endpoints 
db.getNode() does a 4-way LIKE scan for recent packets we don't even
use. Direct SELECT on primary key instead. Saves ~110ms per call.
 2026-03-20 05:17:06 +00:00
you cd01da5a64 perf: hash-sizes analytics reads from memory store 
Last remaining full-table scan on packets from SQLite.
All packet reads now go through pktStore (in-memory).
 2026-03-20 05:13:13 +00:00
you 0b4590e48d perf: node detail uses in-memory packet index 
Was doing 4-way LIKE scan for recent packets (~130ms).
Now reads from pktStore.byNode, slices last 20.
 2026-03-20 05:12:00 +00:00
you f5d377e396 perf: node health uses in-memory packet store 
Was doing 6 LIKE scans on SQLite (~169ms). Now reads from
pktStore.byNode index, single pass over packets.
 2026-03-20 05:11:00 +00:00
you dc703ebf28 perf: node analytics uses in-memory packet store 
Was doing 7 separate LIKE scans on SQLite (~552ms). Now reads from
pktStore.byNode index and computes all aggregations in JS.
Also added cache with TTL.nodeAnalytics.
 2026-03-20 05:09:49 +00:00
you 89c1e84924 perf: bulk-health uses in-memory packet store index 
Was doing 50-pattern LIKE OR scan on all packets in SQLite (~2s).
Now reads from pktStore.byNode index — O(1) lookup per node.
 2026-03-20 05:08:23 +00:00
you 50b6124325 revert: remove background refresh jobs — blocks event loop 
Node.js is single-threaded. A 5s subpath computation in a background
timer blocks ALL concurrent requests. Stats endpoint went from 3ms
to 1.2s because it was waiting for a background refresh to finish.

Pre-warm on startup + long TTLs (30min-1hr) is sufficient. At most
one user per hour eats a cold compute cost.
 2026-03-20 04:56:57 +00:00
you f08756a6ac perf: background refresh jobs — recompute expensive caches before TTL expires 
No user ever hits a cold compute path. Background timers fire at 80%
of each TTL, hitting endpoints with ?nocache=1 to force recomputation
and re-cache the result.

Jobs: RF (24min), topology (24min), channels (24min), hash-sizes (48min),
subpaths ×4 (48min), bulk-health (8min).
 2026-03-20 04:52:06 +00:00
you c2bc07bb4a feat: live A/B benchmark — launches SQLite-only vs in-memory servers 
NO_MEMORY_STORE=1 env var makes packet-store fall through to SQLite
for all reads. Benchmark spins up both servers on temp ports and
compares: SQLite cold, Memory cold, Memory cached.

Results on 27K packets (ARM64):
  Subpaths 5-8: SQLite 4.7s → cached 1.1ms (4,273×)
  Bulk health:  SQLite 1.8s → cached 1.7ms (1,059×)
  Topology:     SQLite 1.1s → cached 3.0ms (367×)
  Channels:     SQLite 617ms → cached 1.9ms (325×)
  RF Analytics: SQLite 448ms → cached 1.6ms (280×)
 2026-03-20 04:47:31 +00:00
you e589fd959a feat: benchmark compares against pre-optimization baseline 
Stores pre-optimization /api/perf measurements (pure SQLite, 27K packets)
in benchmark-baseline.json. Benchmark suite auto-loads and shows side-by-side:

Highlights:
  Subpaths 5-8 hop: 6,190ms → 1.1ms (5,627× faster)
  Hash sizes:          430ms → 1.3ms (331× faster)
  Topology:            697ms → 2.8ms (249× faster)
  RF analytics:        272ms → 1.6ms (170× faster), 1MB → 22KB
  Packets:              78ms → 3ms (26× faster)
  Channels:             60ms → 1.5ms (40× faster)
  Bulk health:       1,610ms → 67ms (24× faster)
 2026-03-20 04:30:18 +00:00
you 706227b106 feat: add Perf dashboard to nav bar, show packet store stats 
Perf page now accessible from main nav (⚡ Perf).
Shows in-memory packet store metrics: packets in RAM, memory used/limit,
queries served, live inserts, evictions, index sizes.
 2026-03-20 04:25:05 +00:00
you 44f9a95ec5 feat: benchmark suite + nocache bypass for cold compute testing 
node benchmark.js [--runs N] [--json]
Adds ?nocache=1 query param to bypass server cache for benchmarking.
Tests all 21 endpoints cached vs cold, shows speedup comparison.
 2026-03-20 04:23:34 +00:00
you b481df424f docs: add PERFORMANCE.md with before/after benchmarks 2026-03-20 04:21:53 +00:00
you 2edcca77f1 perf: RF endpoint from 1MB to ~15KB — server-side histograms, scatter downsampled to 500pts 
Was sending 27K raw SNR/RSSI/size values (420KB) + 27K scatter points (763KB).
Now: histograms computed server-side (20-25 bins), scatter downsampled
to max 500 evenly-spaced points. Client histogram() accepts both formats.
 2026-03-20 04:17:25 +00:00
you cd678d492d perf: grouped mode also updates client-side from WS — zero API fetches 
Existing groups get count/observer_count incremented, latest timestamp
updated, longest path kept. New hashes get a new group prepended.
Expanded children updated inline. No more /api/packets re-fetch on
any incoming packet in either mode.
 2026-03-20 04:15:04 +00:00