Compare commits

..

296 Commits

Author SHA1 Message Date
you 95b59d1792 fix: recent packets deeplink uses route path not query param 2026-03-20 22:34:00 +00:00
you e7aa4246ac fix: live node panel deeplinks — full detail, observer names, recent packets
- Full Detail links to #/nodes/<pubkey> (was #/nodes?selected= which just showed list)
- Heard By observer names link to #/observers/<id>
- Recent Packets link to #/packets?hash=<hash>
2026-03-20 22:17:13 +00:00
you f1aa6caf93 fix: packet expand shows observations, heard-by uses correct field name
- pktToggleGroup fetches ?expand=observations and maps them as children
- Live page heard-by uses o.packetCount (was o.count → undefined)
2026-03-20 22:02:54 +00:00
you a882aae681 M5: Frontend updates for dedup — observation_count badges, totalTransmissions
- packets.js: Show observation_count badge (👁 N) on grouped rows
- nodes.js: Use totalTransmissions (fallback totalPackets), show observation badges on recent packets
- home.js: Use totalTransmissions for network stats
- node-analytics.js: Use totalTransmissions for throughput display
- analytics.js: Use totalTransmissions for overview stats and node rankings
- live.js: Use totalTransmissions in node detail, show observation badges in feed and recent packets
- style.css: Add .badge-obs style for observation count badges
- index.html: Bump cache busters on all changed JS/CSS files

All changes have backward compat fallbacks to totalPackets.
2026-03-20 21:31:10 +00:00
you aa35164252 M4: API response changes for dedup-normalize
- GET /api/packets: returns transmissions with observation_count, strip
  observations[] by default (use ?expand=observations to include)
- GET /api/packets/🆔 includes observation_count and observations[]
- GET /api/nodes/:pubkey/health: stats.totalTransmissions + totalObservations
  (totalPackets kept for backward compat)
- GET /api/nodes/bulk-health: same transmission/observation split
- WebSocket broadcast: includes observation_count
- db.js getStats(): adds totalTransmissions count
- All backward-compatible: old field names preserved alongside new ones
2026-03-20 20:49:34 +00:00
you 84f33aef7b M3: Restructure in-memory store around transmissions
- load() reads from transmissions JOIN observations (with legacy fallback)
- byHash now maps hash → single transmission object (1:1)
- byNode maps pubkey → [transmissions] (deduped, no inflated observations)
- byTransmission is the primary data structure
- byId maps observation IDs for backward-compat packet detail links
- byObserver still maps observer_id → [observations]
- getSiblings() returns observations from transmission
- findPacketsForNode() returns unique transmissions
- query()/queryGrouped() work with transmission-centric model
- All returned objects maintain backward-compatible fields
- SQLite-only fallback path (NO_MEMORY_STORE=1) unchanged
- Tested: 11.6K transmissions from 37.5K observations (3.2x dedup)
2026-03-20 20:44:32 +00:00
you baa60cac0f M2: Dual-write ingest to transmissions/observations tables
- Add transmissions and observations schema to db.js init
- Add insertTransmission() function: upsert transmission by hash,
  always insert observation row
- All 6 pktStore.insert() call sites in server.js now also call
  db.insertTransmission() with try/catch (non-fatal on error)
- packet-store.js: add byTransmission Map index (hash → transmission
  with observations array) for future M3 query migration
- Existing insertPacket() and all read paths unchanged
2026-03-20 20:29:03 +00:00
you d7e415daa7 fix: raw_hex NOT NULL in transmissions schema — deleted 4 junk test rows 2026-03-20 20:24:13 +00:00
you 2c6148fd2d Add dedup migration script (Milestone 1)
Creates transmissions and observations tables from existing packets table.
- Groups packets by hash → 1 transmission per unique hash
- Creates 1 observation per original packet row with FK to transmission
- Idempotent: drops and recreates new tables on each run
- Does NOT modify the original packets table
- Prints stats and verifies counts match

Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio)
2026-03-20 20:22:30 +00:00
you 2feb2c5b94 fix: escapeHtml crashes on numbers — String(s) before .replace() 2026-03-20 19:25:48 +00:00
you 10b11106f6 ci: add pre-deploy JS validation — syntax check + undefined reference detection
Validation runs BEFORE docker build. If it fails, deployment is blocked.
No more broken code reaching production.
2026-03-20 19:24:11 +00:00
you 326d411c4a fix: esc is not defined — use escapeHtml in live.js node detail 2026-03-20 19:23:20 +00:00
you 15a93d5ea4 feat: clickable nodes on live map with slide-in detail panel
- Click any node marker to see name, role, status, location, stats
- Heard By observers and recent packets shown
- Links to full node detail and analytics pages
- Slide-in panel from right with blur background, matches live page style
- Uses shared ROLE_COLORS and HEALTH_THRESHOLDS
2026-03-20 19:21:30 +00:00
you 055467ca43 fix: live map legend uses shared ROLE_COLORS instead of hardcoded wrong colors 2026-03-20 19:17:57 +00:00
you 4f7b02a91c fix: centralize hardcoded values — roles, thresholds, colors, tiles, limits — closes #104
- New public/roles.js shared module: ROLE_COLORS, ROLE_LABELS, ROLE_STYLE,
  ROLE_EMOJI, ROLE_SORT, HEALTH_THRESHOLDS, TILE_DARK/LIGHT, SNR_THRESHOLDS,
  DIST_THRESHOLDS, MAX_HOP_DIST, LIMITS — all configurable via /api/config/roles
- Removed duplicate ROLE_COLORS from map.js, nodes.js, live.js, analytics.js
- Removed duplicate health thresholds from nodes.js, home.js, observer-detail.js
- Deduplicated CartoDB tile URLs (3 copies → 1 in roles.js)
- Removed hardcoded region names from map.js and packets.js
- channels.js uses ROLE_EMOJI/ROLE_LABELS instead of hardcoded emoji chains
- server.js reads healthThresholds from config.json with defaults
- Unknown roles get gray circle fallback instead of crashing
2026-03-20 17:36:41 +00:00
you f0db317051 fix: observer locations case-insensitive match, regions from API not hardcoded
- Observer ID is uppercase, node pubkey is lowercase — added COLLATE NOCASE
- New /api/config/regions endpoint merges config regions + observed IATAs
- map.js and packets.js fetch regions from API instead of hardcoded maps
2026-03-20 16:10:16 +00:00
you 9bf78bd28d feat: add MRY (Monterey) to lincomatic MQTT topics 2026-03-20 15:29:37 +00:00
you 5fe275b3f8 fix: use region-specific MQTT topics instead of wildcards — saves bandwidth 2026-03-20 15:28:54 +00:00
you 74a08d99b0 fix: observer location from nodes table direct join — observer ID = node pubkey 2026-03-20 15:11:57 +00:00
you 76d63ffe75 feat: observers as map markers (purple stars) with computed locations
- Removed dead 'MQTT Connected Only' checkbox (never worked)
- Added 'observer' role type with purple star marker
- Observer locations computed from average of nodes they've seen
- Observer popup with name, IATA, packets, link to detail page
- Role filter checkbox includes observers with count
2026-03-20 14:56:08 +00:00
you 157dc9a979 fix: spark bars use inline spans instead of div — immune to max-width:0 table crush
The div-based spark bar was always getting crushed to 0px by
table-layout:auto + max-width:0 on td. Inline spans with fixed
width survive because they participate in text flow, not block layout.
2026-03-20 14:48:15 +00:00
you 2f07ae2e5c fix: spark bar z-index so it renders above adjacent packet count cell 2026-03-20 14:39:48 +00:00
you 1f9cd3ead1 fix: add config.json mount to CI deploy workflow 2026-03-20 14:29:32 +00:00
you 4b5b801def docs: v2.1.1 — multi-broker MQTT, observer detail, changelog
- Multi-broker MQTT with per-source IATA filtering
- Observer detail pages with charts and status
- Channel key auto-derivation (SHA256)
- Map dark/light tile swap
- Server-side My Nodes filter
- 12 bug fixes including spark bars, packet ordering, PII cleanup
- Docker deploy.sh fixed with config mount
2026-03-20 09:37:22 +00:00
you 8ef65def7d fix: make renderTableRows async (await broke packets page entirely) 2026-03-20 09:33:08 +00:00
you 9ef5c1a809 fix: My Nodes filter uses server-side findPacketsForNode for all packet types
Client was matching field names that only exist on ADVERTs. Now sends
pubkeys to server, which uses findPacketsForNode() (byNode index +
text search) to find ALL packet types referencing those nodes.
2026-03-20 09:30:41 +00:00
you e31e4aa356 chore: remove accidentally committed test DB files, update gitignore 2026-03-20 09:23:25 +00:00
you db884f12eb fix: 4 bugs - spark bars inline style, My Nodes filter field names, duplicate pin button, map dark mode
1. Spark bars: inline style override on td (max-width:none, min-width:80px)
2. My Nodes filter: pubkey→pubKey, to/from→srcPubKey/destPubKey/srcHash/destHash
3. Pin button: guard against duplicates in init, remove in destroy
4. Map page: CartoDB dark/light tiles with MutationObserver theme swap
2026-03-20 09:21:17 +00:00
you 116f0c8dfb fix: add col-spark class to spark bar cells, min-width 80px 2026-03-20 09:14:19 +00:00
you b3e8dcaa93 fix: spark bars crushed by max-width:0 on data-table td
Override max-width for td cells containing spark bars so they render.
2026-03-20 09:12:27 +00:00
you 920eab04c1 fix: sort observer analytics packets newest-first, fix recentPackets slice 2026-03-20 09:10:16 +00:00
you d67b531bf2 fix: pass observer name (msg.origin) to packet insert and observer upsert
Lincomatic MQTT packets include origin field with friendly name but
we were only using it in status handler. Now both packet and observer
records get the name.
2026-03-20 09:07:39 +00:00
you 5a6847bbf4 fix: remove duplicate crypto require that crashed server 2026-03-20 08:54:35 +00:00
you 034c68c154 feat: auto-derive hashtag channel keys from SHA256(name)
Scans DB for known #channel names, derives 16-byte AES keys
algorithmically. No need to manually add hashtag channels to config.
Private channels still need manual config.
2026-03-20 08:53:11 +00:00
you 477dcde82f fix: analytics shows total packets + signal data count separately
RF analytics filtered on snr!=null, showing only 3 packets when most
lincomatic data has no SNR. Now shows total packets prominently and
signal-data count as a separate stat.
2026-03-20 08:42:38 +00:00
you c997318cd2 fix: recentPackets in health endpoint — show 20 newest DESC, not 10 oldest 2026-03-20 08:39:01 +00:00
you fa40ede9e7 fix: findPacketsForNode always resolves name, even when called with pubkey 2026-03-20 08:31:07 +00:00
you 8ce2262813 refactor: single findPacketsForNode() replaces 4 duplicate node lookups
One method resolves name→pubkey, combines byNode index + text search.
Used in: query fast-path, combined-filter path, health endpoint,
analytics endpoint. Bulk-health still uses index-only (perf).
2026-03-20 08:29:45 +00:00
you 90c4c03ac3 fix: node health endpoint searches by name + pubkey, not just byNode index
byNode only has packets where full pubkey appears in decoded_json fields.
Channel messages reference nodes by name, not pubkey. Combined search
finds all 304 packets instead of just 12.
2026-03-20 08:25:06 +00:00
you 87bbd93d12 fix: node-only search path also resolves name→pubkey + text search
The fast-path for single node filter was bypassing the name
resolution, using raw name string on pubkey-indexed byNode map
2026-03-20 08:20:43 +00:00
you e837dba000 fix: node search combines index + text search for name AND pubkey
Previous fix only used index OR text, missing packets that reference
nodes by name in decoded_json
2026-03-20 08:13:27 +00:00
you fa72e6242d fix: node name search returns all packet types, not just adverts
Resolves node name → pubkey, then searches byNode index and paths
table instead of only matching decoded_json text
2026-03-20 08:10:24 +00:00
you 2fcbcd97d1 fix: observer column max-width:none to override td max-width:0 2026-03-20 08:01:15 +00:00
you 311db0285d fix: bump all cache busters - browser was serving stale JS 2026-03-20 07:58:57 +00:00
you 01df7f7871 fix: observer column min-width 70px to prevent zero-width squeeze 2026-03-20 07:57:43 +00:00
you 90fa755e7d fix: observer pages scroll - calc(100vh - 56px) for nav bar 2026-03-20 07:53:24 +00:00
you 039d1fc28f fix: resolve observer names from observers table in packets view
observer_name on packets is often NULL; now cross-references
the loaded observers list to display friendly names
2026-03-20 07:51:26 +00:00
you d8c0e3a156 fix: only backfill observer name if observer already exists
Prevents ADVERT nodes from being created as observers
2026-03-20 07:47:37 +00:00
you bee124e6d2 fix: shrink donut chart, show observer name in dropdown 2026-03-20 07:44:20 +00:00
you fd919a2a80 fix: spark bar invisible in observers table - max-width:0 override 2026-03-20 07:39:46 +00:00
you f58728118d feat: observer detail page with analytics
- GET /api/observers/:id — observer metadata + packet count
- GET /api/observers/:id/analytics — timeline, type breakdown, nodes heard, SNR distribution
- observer-detail.js — info cards, 4 Chart.js charts, recent packets table
- Observers list rows now clickable to navigate to detail
- Time range selector (24h, 3d, 7d, 30d)
2026-03-20 07:37:36 +00:00
you 4aa78305d3 feat: backfill observer names from ADVERT pubkey cross-reference 2026-03-20 07:32:46 +00:00
you 8659cda7b7 fix: remove PII from seed data - no real names/coords 2026-03-20 07:30:48 +00:00
you 2713d501b4 feat: MQTT topic arrays, IATA filtering, observer status parsing
- mqttSources[].topics is now an array of topic patterns
- mqttSources[].iataFilter optionally restricts to specific regions
- meshcore/<region>/<id>/status topic parsed for observer metadata:
  name, model, firmware, client_version, radio, battery, uptime, noise_floor
- New observer columns with auto-migration for existing DBs
- Status updates don't inflate packet_count (separate updateObserverStatus)
2026-03-20 07:29:01 +00:00
you 4ff72935ca feat: multi-broker MQTT support
config.mqttSources array allows multiple MQTT brokers with independent
topics, credentials, and TLS settings. Legacy config.mqtt still works
for backward compatibility.
2026-03-20 07:18:49 +00:00
you a756517647 fix: set XDG_DATA_HOME so Caddy persists certs to mounted volume 2026-03-20 07:13:52 +00:00
you 74983d3f74 ci: switch to self-hosted runner — no SSH, no secrets, no exposed ports 2026-03-20 07:07:01 +00:00
you ab35ced2bf ci: auto-deploy to VM on push to master via GitHub Actions 2026-03-20 07:03:36 +00:00
you ff86a78480 style: proper themed copy-link button matching existing detail action buttons 2026-03-20 06:59:10 +00:00
you 2a076dfb1d feat: shareable URLs for channels — update URL on selection, accept route param
- selectChannel updates URL to #/channels/<hash>
- init accepts routeParam and auto-selects channel
- Search results use new URL format instead of ?ch= query param
2026-03-20 06:51:54 +00:00
you fceff15e2f feat: update URL bar when selecting a packet for easy sharing 2026-03-20 06:49:20 +00:00
you 9c87f0040e docs: update README — fix duplicate heading, add Docker/perf files to project structure 2026-03-20 06:47:07 +00:00
you 395abc2585 feat: standalone packet detail page at #/packet/ID
- New route #/packet/123 shows full packet detail on its own page
- Back link to packets list
- Copy Link button now generates #/packet/ID URLs
- Reuses existing renderDetail() for consistent display
2026-03-20 06:44:18 +00:00
you e82e4fe05f fix: copy link URL format — use #/packets/id/N not query param 2026-03-20 06:42:12 +00:00
you 6cf9793706 feat: copy link button in packet detail pane 2026-03-20 06:41:36 +00:00
you 1772b34e8f fix: copy all JS files in Dockerfile — was missing decoder.js 2026-03-20 06:18:04 +00:00
you fea8a7e0b5 feat: add Caddy to Docker container — automatic HTTPS
- Caddy reverse proxies :80/:443 → Node :3000
- Mount custom Caddyfile for your domain → auto Let's Encrypt TLS
- Caddy certs persisted in /data/caddy volume
- Ports: 80 (HTTP), 443 (HTTPS), 1883 (MQTT)
2026-03-20 06:07:38 +00:00
you 2e486e2a66 feat: Docker packaging — single container with Mosquitto + Node
- Dockerfile: Alpine + Node 22 + Mosquitto + supervisord
- Auto-copies config.example.json if no config.json mounted
- Named volume for data persistence (SQLite + Mosquitto)
- Ports: 3000 (web), 1883 (MQTT)
- .dockerignore excludes data, config, git, benchmarks
- README updated with Docker quickstart
2026-03-20 06:06:15 +00:00
you f0c29b38f1 chore: bump perf.js cache buster 2026-03-20 05:47:29 +00:00
you 46d9b690ee fix: close if(health) block in perf dashboard — was swallowing all content 2026-03-20 05:47:11 +00:00
you 2e51e5f743 feat: system health + SWR stats in perf dashboard
Perf page now shows: heap usage, RSS, event loop p95/max/current,
WS client count, stale-while-revalidate hits, recompute count.
Color-coded: green/yellow/red based on thresholds.
2026-03-20 05:45:51 +00:00
you 11b398cfe1 feat: stale-while-revalidate cache + /api/health telemetry
Cache: entries stay valid for 2× TTL as stale. First request after
TTL serves stale data while recompute runs (guarded: one at a time).
No more cache stampedes.

/api/health returns:
- Process memory (RSS, heap)
- Event loop lag (p50/p95/p99/max, sampled every 1s)
- Cache stats (hit rate, stale hits, recomputes)
- WebSocket client count
- Packet store size
- Recent slow queries
2026-03-20 05:43:32 +00:00
you f4ac789ee9 release: v2.1.0 — Performance
Two-layer caching: in-memory packet store + TTL response cache.
All packet reads from RAM, SQLite write-only.

Highlights:
- Bulk Health: 7,059ms → 1ms (7,059×)
- Node Analytics: 381ms → 1ms (381×)
- Topology: 685ms → 2ms (342×)
- RF Analytics: 253ms → 1ms (253×)
- Channels: 206ms → 1ms (206×)
- Node Health/Detail: 133-195ms → 1ms

Architecture:
- In-memory packet store with Map indexes (byNode, byHash, byObserver)
- Ring buffer with configurable max (1GB default, ~2.3M packets)
- Smart cache invalidation (packet bursts don't nuke analytics)
- Pre-warm all heavy endpoints on startup
- Eliminated every LIKE '%pubkey%' full-table scan
- All TTLs configurable via config.json
- A/B benchmark script included
- Favicon added
2026-03-20 05:38:23 +00:00
you 2a2a80b4ea chore: add A/B benchmark script, remove worker thread experiments 2026-03-20 05:37:08 +00:00
you 6dd077be13 feat: add favicon — mesh network icon (SVG + ICO) 2026-03-20 05:36:32 +00:00
you 2b3597dff1 fix: null guard getElementById in animatePacket
Elements don't exist yet when replayRecent fires during init.
2026-03-20 05:34:04 +00:00
you 77b7b218b1 perf: channels endpoint — single pass, no sort, no double filter
Was doing two pktStore.filter() calls + sort on each. Now single
loop over all packets with inline type check.
2026-03-20 05:31:03 +00:00
you 0a499745ec perf: pre-warm all heavy analytics endpoints on startup
Sequential self-requests after subpath pre-warm completes.
RF, topology, channels, hash-sizes, bulk-health all cached
before any user hits the page.
2026-03-20 05:29:10 +00:00
you c83eb099c9 perf: stop calling db.getNode() in health/analytics endpoints
db.getNode() does a 4-way LIKE scan for recent packets we don't even
use. Direct SELECT on primary key instead. Saves ~110ms per call.
2026-03-20 05:17:06 +00:00
you cd01da5a64 perf: hash-sizes analytics reads from memory store
Last remaining full-table scan on packets from SQLite.
All packet reads now go through pktStore (in-memory).
2026-03-20 05:13:13 +00:00
you 0b4590e48d perf: node detail uses in-memory packet index
Was doing 4-way LIKE scan for recent packets (~130ms).
Now reads from pktStore.byNode, slices last 20.
2026-03-20 05:12:00 +00:00
you f5d377e396 perf: node health uses in-memory packet store
Was doing 6 LIKE scans on SQLite (~169ms). Now reads from
pktStore.byNode index, single pass over packets.
2026-03-20 05:11:00 +00:00
you dc703ebf28 perf: node analytics uses in-memory packet store
Was doing 7 separate LIKE scans on SQLite (~552ms). Now reads from
pktStore.byNode index and computes all aggregations in JS.
Also added cache with TTL.nodeAnalytics.
2026-03-20 05:09:49 +00:00
you 89c1e84924 perf: bulk-health uses in-memory packet store index
Was doing 50-pattern LIKE OR scan on all packets in SQLite (~2s).
Now reads from pktStore.byNode index — O(1) lookup per node.
2026-03-20 05:08:23 +00:00
you 50b6124325 revert: remove background refresh jobs — blocks event loop
Node.js is single-threaded. A 5s subpath computation in a background
timer blocks ALL concurrent requests. Stats endpoint went from 3ms
to 1.2s because it was waiting for a background refresh to finish.

Pre-warm on startup + long TTLs (30min-1hr) is sufficient. At most
one user per hour eats a cold compute cost.
2026-03-20 04:56:57 +00:00
you f08756a6ac perf: background refresh jobs — recompute expensive caches before TTL expires
No user ever hits a cold compute path. Background timers fire at 80%
of each TTL, hitting endpoints with ?nocache=1 to force recomputation
and re-cache the result.

Jobs: RF (24min), topology (24min), channels (24min), hash-sizes (48min),
subpaths ×4 (48min), bulk-health (8min).
2026-03-20 04:52:06 +00:00
you c2bc07bb4a feat: live A/B benchmark — launches SQLite-only vs in-memory servers
NO_MEMORY_STORE=1 env var makes packet-store fall through to SQLite
for all reads. Benchmark spins up both servers on temp ports and
compares: SQLite cold, Memory cold, Memory cached.

Results on 27K packets (ARM64):
  Subpaths 5-8: SQLite 4.7s → cached 1.1ms (4,273×)
  Bulk health:  SQLite 1.8s → cached 1.7ms (1,059×)
  Topology:     SQLite 1.1s → cached 3.0ms (367×)
  Channels:     SQLite 617ms → cached 1.9ms (325×)
  RF Analytics: SQLite 448ms → cached 1.6ms (280×)
2026-03-20 04:47:31 +00:00
you e589fd959a feat: benchmark compares against pre-optimization baseline
Stores pre-optimization /api/perf measurements (pure SQLite, 27K packets)
in benchmark-baseline.json. Benchmark suite auto-loads and shows side-by-side:

Highlights:
  Subpaths 5-8 hop: 6,190ms → 1.1ms (5,627× faster)
  Hash sizes:          430ms → 1.3ms (331× faster)
  Topology:            697ms → 2.8ms (249× faster)
  RF analytics:        272ms → 1.6ms (170× faster), 1MB → 22KB
  Packets:              78ms → 3ms (26× faster)
  Channels:             60ms → 1.5ms (40× faster)
  Bulk health:       1,610ms → 67ms (24× faster)
2026-03-20 04:30:18 +00:00
you 706227b106 feat: add Perf dashboard to nav bar, show packet store stats
Perf page now accessible from main nav ( Perf).
Shows in-memory packet store metrics: packets in RAM, memory used/limit,
queries served, live inserts, evictions, index sizes.
2026-03-20 04:25:05 +00:00
you 44f9a95ec5 feat: benchmark suite + nocache bypass for cold compute testing
node benchmark.js [--runs N] [--json]
Adds ?nocache=1 query param to bypass server cache for benchmarking.
Tests all 21 endpoints cached vs cold, shows speedup comparison.
2026-03-20 04:23:34 +00:00
you b481df424f docs: add PERFORMANCE.md with before/after benchmarks 2026-03-20 04:21:53 +00:00
you 2edcca77f1 perf: RF endpoint from 1MB to ~15KB — server-side histograms, scatter downsampled to 500pts
Was sending 27K raw SNR/RSSI/size values (420KB) + 27K scatter points (763KB).
Now: histograms computed server-side (20-25 bins), scatter downsampled
to max 500 evenly-spaced points. Client histogram() accepts both formats.
2026-03-20 04:17:25 +00:00
you cd678d492d perf: grouped mode also updates client-side from WS — zero API fetches
Existing groups get count/observer_count incremented, latest timestamp
updated, longest path kept. New hashes get a new group prepended.
Expanded children updated inline. No more /api/packets re-fetch on
any incoming packet in either mode.
2026-03-20 04:15:04 +00:00
you 4c6172bc6e perf: WS packets prepend client-side instead of re-fetching entire list
Non-grouped mode: new packets from WebSocket are filtered client-side
and prepended to the table, no API call. Grouped mode still re-fetches
(group counts change). Server broadcast now includes full packet row.
Eliminates repeated /api/packets fetches on every incoming packet.
2026-03-20 04:12:07 +00:00
you d01fa7e17f perf: pre-warm all 4 subpath query variants on startup + dedup concurrent computation 2026-03-20 03:53:10 +00:00
you 35e86c34e0 perf: single-pass subpath computation + startup pre-warm
4 parallel subpath queries were each scanning 27K packets independently
(937ms + 1.99s + 3.09s + 6.19s). Now one shared computation builds all
subpath data, cached for 1hr. Subsequent queries just slice the result.
Pre-warmed on startup so first user never sees a cold call.
2026-03-20 03:51:58 +00:00
you f8638974c7 perf: smart cache invalidation — only channels/observers on packet burst, node/health/analytics expire by TTL, node invalidated on ADVERT only 2026-03-20 03:48:55 +00:00
you 1be6b4f4ad perf: ALL packet reads from RAM — analytics, channels, topology, subpaths, RF, observers
Zero SQLite reads from packets table. Every endpoint that previously
scanned packets now reads from the in-memory PacketStore.
Expected: subpaths from 1.6s to <100ms, topology from 700ms to <50ms,
RF from 270ms to <30ms on cold calls.
2026-03-20 03:43:23 +00:00
you d8d0572abb perf: in-memory packet store — all reads from RAM, SQLite write-only
- PacketStore loads all packets into memory on startup (~11MB for 27K packets)
- Indexed by id, hash, observer, and node pubkey for fast lookups
- /api/packets, /api/packets/timestamps, /api/packets/:id all served from RAM
- MQTT ingest writes to both RAM + SQLite
- Configurable maxMemoryMB (default 1024MB) in config.json packetStore section
- groupByHash queries computed in-memory
- Packet store stats exposed in /api/perf
- Expected: /api/packets goes from 77ms to <1ms
2026-03-20 03:38:37 +00:00
you de658bfb0d perf: configurable cache TTLs via config.json — server + client fetch from /api/config/cache
All cache TTLs now read from config.json cacheTTL section (seconds).
Client fetches config on load via GET /api/config/cache.
config.example.json updated with defaults.
Edit config.json, restart server — no code changes needed to tweak TTLs.
2026-03-20 03:23:58 +00:00
you 720d019a28 perf: align cache TTLs with real data rates — analytics 30min-1hr, nodes 5min, chat 10-15s, stats 10s, server debounce 30s 2026-03-20 03:20:33 +00:00
you ce030c91f7 perf: bump analytics cache to 5min, subpaths to 10min, cache subpath-detail 2026-03-20 02:24:46 +00:00
you 99ef07ca05 fix: debounce client cache invalidation (5s window) — same issue as server 2026-03-20 02:23:14 +00:00
you 141c28231e fix: debounce server cache invalidation (5s window), fix client cache stat reporting 2026-03-20 02:15:18 +00:00
you 2b7ed064d1 perf page: show server + client cache stats 2026-03-20 02:10:27 +00:00
you 415440d36d merge: server + frontend perf optimizations 2026-03-20 02:07:54 +00:00
you 5832c73a0d perf: add TTL cache layer + rewrite bulk-health to single-query
- Add TTLCache class with hit/miss tracking
- Cache all expensive endpoints:
  - analytics/* endpoints: 60s TTL
  - channels: 30s TTL
  - channels/:hash/messages: 15s TTL
  - nodes/:pubkey: 30s TTL
  - nodes/:pubkey/health: 30s TTL
  - observers: 30s TTL
  - bulk-health: 60s TTL
- Invalidate all caches on new packet ingestion (POST + MQTT)
- Rewrite bulk-health from N×5 queries to 1 query + JS matching
- Add cache stats (size, hits, misses, hitRate) to /api/perf
2026-03-20 02:06:23 +00:00
you e98e04553a feat: add frontend API response caching with TTL, in-flight dedup, and WebSocket invalidation
- Replace api() with caching version supporting TTL and request deduplication
- Add appropriate TTLs to all api() call sites across all frontend JS files:
  - /stats: 5s TTL (was called 962 times in 3 min)
  - /nodes/:pubkey: 15s, /health: 30s, /observers: 30s
  - /channels: 15s, messages: 10s
  - /analytics/*: 60s, /bulk-health: 60s, /network-status: 60s
  - /nodes?*: 10s
- Skip caching for real-time endpoints (/packets, /resolve-hops, /perf)
- Invalidate /stats, /nodes, /channels caches on WebSocket messages
- Deduplicate in-flight requests (same path returns same promise)
- Add cache hit rate to window.apiPerf() console debugging
- Update all cache busters in index.html
2026-03-20 02:03:25 +00:00
you 8587286896 merge: performance instrumentation 2026-03-20 01:49:19 +00:00
you 4fff11976e feat: performance instrumentation — server timing middleware, client API tracking, /api/perf endpoint, #/perf dashboard 2026-03-20 01:34:25 +00:00
you 68b79d2d50 release: v2.0.1 — mobile packets UX 2026-03-20 01:19:19 +00:00
you 681cf82cd6 merge: mobile packets improvements (v2.0.1) 2026-03-20 01:19:19 +00:00
you 36598a3623 mobile: hide hash column by default 2026-03-20 01:15:49 +00:00
you 0dc2dd3f25 mobile packets: bottom sheet detail, collapsed filters, smaller table, fewer columns
- Mobile (≤640px) default columns: time, hash, type, details (hide region, observer, path, rpt, size)
- Detail panel hidden on mobile; tapping row opens slide-up bottom sheet (70vh max, close button, drag handle)
- Filter bar collapses to single 'Filters' toggle button on mobile
- Table font 11px, tighter padding, no min-width forcing horizontal scroll
- Panel-right completely hidden on mobile (no split layout)
2026-03-20 01:14:32 +00:00
you 0106c8ebf9 readme: GIF under Live Trace Map, iOS screenshot under Mobile Ready section 2026-03-20 01:09:48 +00:00
you cb4773b426 readme: lead with GIF, resize iOS screenshot to 400px 2026-03-20 01:05:17 +00:00
you 9c6608acc2 add screenshots for README 2026-03-20 01:02:38 +00:00
you dc4e91a348 add CHANGELOG.md 2026-03-20 01:01:40 +00:00
you 78034cbbc0 release: v2.0.0 — analytics, live VCR, mobile, accessibility, 100+ fixes 2026-03-20 01:00:16 +00:00
you 435a19057a fix: mobile VCR bar bottom padding with safe-area + 20px fallback 2026-03-20 00:34:16 +00:00
you 90abb42904 fix: bump feed/legend safe-area offsets to account for two-row VCR on mobile 2026-03-20 00:33:24 +00:00
you 93c7f4c9eb fix: VCR bar + feed/legend offset by safe-area-inset-bottom with 34px fallback for iOS 2026-03-20 00:32:57 +00:00
you 175d9269ec fix: VCR bar respects iOS safe area inset (home indicator) 2026-03-20 00:22:00 +00:00
you 4fc9c25a5d Revert "release: v2.0.0 — analytics, mobile redesign, accessibility, 100+ fixes"
This reverts commit d7f0e0c9fe.
2026-03-20 00:05:51 +00:00
you d7f0e0c9fe release: v2.0.0 — analytics, mobile redesign, accessibility, 100+ fixes 2026-03-20 00:04:30 +00:00
you f054841a99 fix: rotation — JS-driven height (window.innerHeight) on live-page+app, runs immediately + on every resize/orientation/visualViewport change 2026-03-19 23:54:18 +00:00
you 0be9e8b4fd mobile VCR: proper two-row layout — controls+scope+LCD row, full-width timeline row, no horizontal scroll 2026-03-19 23:49:06 +00:00
you 08063c1316 mobile: hide feed+legend, show LCD, fix rotation with visualViewport + forced height recalc 2026-03-19 23:47:43 +00:00
you ffe26f7d03 fix: comprehensive mobile responsive fixes for all pages
Live page (priority):
- Fix feed panel max-height from 180px to 40vh (was clipping content)
- Fix VCR bar: bump breakpoint from 600px to 640px, hide LCD, ensure
  no wrapping, increase touch targets to 44px min
- Remove rogue ≤600px rules that hid feed/legend/toggle entirely
- Fix legend toggle: was using inverted logic (legend-mobile-hidden
  toggled off instead of legend-mobile-visible toggled on)
- Header: constrain width to viewport, reduce padding/font on mobile
- Feed detail card: add max-height 50vh + overflow-y auto to prevent
  clipping off screen
- Disable feed resize handle on mobile (not practical for touch)
- Ensure Leaflet zoom controls clear mobile header

Packets page:
- panel-left gets min-height 50vh + overflow-x with -webkit touch scrolling
- data-table gets min-width 500px so it scrolls horizontally instead of
  collapsing columns to nothing
- panel-right removes max-height 50vh cap (was hiding detail panel)
- Filter bar buttons get 44px min touch target
- Node filter wrap goes full width

Nodes page:
- Node count pills wrap properly
- Smaller font on count pills for narrow screens

Analytics pages:
- analytics-grid goes single column on mobile
- analytics-page reduces padding

Style fixes (global):
- Filter bar gap reduced to 4px on mobile
- All cache busters updated
2026-03-19 23:44:44 +00:00
you d99aa3ac11 mobile live: hide feed+legend, keep LCD visible 2026-03-19 23:41:42 +00:00
you b407fa4f28 fix: remove stray CSS fragment corrupting live.css 2026-03-19 23:40:46 +00:00
you 6cc2f3a8c7 fix: live map tiles swap instantly on theme toggle via MutationObserver 2026-03-19 23:40:09 +00:00
you c62902cd9c fix: legend bottom 12px→58px to clear VCR bar 2026-03-19 23:38:33 +00:00
you 46abc7b11b fix: mobile — dvh for proper viewport height, VCR bar no-wrap, LCD hidden on small screens, orientation triple-invalidate, feed-detail-card bottom aligned 2026-03-19 23:35:09 +00:00
you 80215f9d31 fix: VCR bar properly sized — bigger buttons, taller timeline (28px), comfortable padding 2026-03-19 23:32:52 +00:00
you 25ae36c4b6 fix: VCR bar much thinner — 3px padding, 12px timeline, feed bottom 40px; removed dead CSS 2026-03-19 23:26:52 +00:00
you 6ca5336563 fix: VCR bar — flatten to single row, thinner timeline (16px), remove stacked layout that caused weird gray band 2026-03-19 23:19:19 +00:00
you fda8d73588 UX: move analytics/copy URL buttons to top of node detail; live feed close/resize more visible; VCR bar labeled 2026-03-19 23:12:30 +00:00
you ca21dc5608 feat: packets view — ★ My Nodes toggle filters to only claimed/favorited node packets 2026-03-19 23:09:44 +00:00
you f2b0145da0 fix: claimed nodes always fetched even if not in current page; auto-sync claimed→favorites 2026-03-19 23:07:49 +00:00
you 3048166648 fix: favorites dropdown transparent bg (var(--surface) didn't exist); live map uses light/dark tiles based on theme 2026-03-19 23:05:28 +00:00
you f947af5b01 feat: claimed nodes get visual distinction — blue tint, left accent border, ★ badge 2026-03-19 23:04:41 +00:00
you ad5e12fc45 fix: node analytics — compact layout, add descriptions to every stat card and chart 2026-03-19 23:01:16 +00:00
you 55ee3c6327 fix: network status computed server-side across ALL nodes, not just top 50 2026-03-19 22:57:19 +00:00
you d1a4333b87 fix: move bulk-health route before ALL :pubkey wildcards (not just /health) 2026-03-19 22:54:26 +00:00
you b91ba7e38a fix: live page respects dark/light theme — replace hardcoded colors with CSS variables 2026-03-19 22:52:18 +00:00
you fb1cfae089 fix: move bulk-health route before :pubkey wildcard — Express route ordering 2026-03-19 22:49:24 +00:00
you 3be3a039f1 perf: bulk health endpoint — single API call replaces 50 individual health requests for Nodes tab 2026-03-19 22:46:24 +00:00
you 8549ac4ac9 fix: hash matrix — free prefixes show actual hex code (e.g. 'A7') instead of dots 2026-03-19 22:45:14 +00:00
you a84a8b8bb0 fix: nodes tab — unwrap {nodes} from API response 2026-03-19 22:44:00 +00:00
you 77bc6c9391 fix: hash matrix color scheme — empty=subtle, 1=light green, 2+=orange→red progression 2026-03-19 22:43:21 +00:00
you bab0b6c441 fix: hash matrix — bigger font, remove ⚠ clutter, use · for empty cells; collision risk sorted closest-first 2026-03-19 22:41:01 +00:00
you 2e48e5db2f feat: add Nodes tab to global analytics — status overview, role breakdown, claimed nodes, leaderboards (activity, signal, observers, recent) 2026-03-19 22:40:03 +00:00
you 4060ddd326 fix: claimed (My Mesh) nodes sort to top, then favorites, then rest 2026-03-19 22:36:33 +00:00
you b62c8c7b43 fix: analytics page scroll — add overflow-y:auto to wrapper 2026-03-19 22:35:50 +00:00
you 963778632e fix: favorited (claimed) nodes always sort to top of nodes list 2026-03-19 22:34:50 +00:00
you 21b1cbc332 Add per-node analytics page with charts, stats, and heatmap
- New route: #/nodes/:pubkey/analytics with Chart.js v4 visualizations
- Activity timeline (bar), SNR trend (line), packet type breakdown (doughnut)
- Observer coverage (horizontal bar), hop distribution (bar)
- Uptime heatmap (7x24 CSS grid, GitHub-style)
- Peer interactions table with links to node details
- Stat cards: availability, signal grade, packets/day, relay %, silence
- Time range selector: 24h / 7d / 30d / All
- Server: GET /api/nodes/:pubkey/analytics with full aggregation in SQLite
- Analytics button added to both sidebar and full-screen node views
2026-03-19 22:31:09 +00:00
you 1ecc95db5a fix: merge Recent Adverts + Recent Activity into single Recent Packets section 2026-03-19 22:21:54 +00:00
you d41477d1d8 feat: richer node detail — status badge, avg SNR/hops, observer breakdown table, totalPackets 2026-03-19 22:17:00 +00:00
you 58531e5da7 fix: add QR code to full-screen node detail view, bump cache buster 2026-03-19 22:11:04 +00:00
you 84b817745f fix: restore proper advert-entry markup, bump cache buster 2026-03-19 22:08:43 +00:00
you 989de353b5 fix: add cache busters to all JS and CSS files 2026-03-19 22:07:33 +00:00
you d4c131ec1e debug: Recent Adverts sidebar — plain HTML, no CSS classes, hardcoded colors, to find rendering issue 2026-03-19 22:07:03 +00:00
you 3e38d88bed fix: Recent Adverts shows packet type + observer + explicit text color, handles missing timestamp 2026-03-19 21:57:25 +00:00
you 4431769b00 fix: role-aware status thresholds — repeaters/rooms 24h/72h, companions/sensors 1h/24h 2026-03-19 21:54:02 +00:00
you f8b05f15b9 fix: always show QR code in node detail, add Recent Adverts section to sidebar detail 2026-03-19 21:53:22 +00:00
you 2cf11bea54 fix: map markers use distinct shapes (diamond/circle/square/triangle) + high-contrast colors for accessibility 2026-03-19 21:49:40 +00:00
you fe34fc81a7 fix: remove dead Regions column from nodes table (closes #26) 2026-03-19 21:48:32 +00:00
you cfbdc8a9e0 fix: column resize steals from ALL right columns proportionally, wider grab handle, 50px min 2026-03-19 21:46:02 +00:00
you 8e3a860cb7 fix: restore geographic prefix disambiguation for route overlay 2026-03-19 21:41:58 +00:00
you 08d3fd3539 fix: don't fitBounds on initial load — respect Bay Area default center 2026-03-19 21:39:38 +00:00
you ca96d5dfbc fix: VCR replay paginates — fetches next 10k when buffer exhausted instead of jumping to live 2026-03-19 21:33:55 +00:00
you 6dd258be65 Replace hardcoded section-row background with CSS variable for dark mode
closes #32
2026-03-19 21:32:58 +00:00
you 27ba362ace Add empty/error states to data tables with aria-live for accessibility
closes #31
2026-03-19 21:32:54 +00:00
you e44f288dab Add SRI integrity hashes to Leaflet CDN scripts
closes #30
2026-03-19 21:32:48 +00:00
you dac05aff1a Remove dead code: svgLine(), .vcr-clock, .vcr-lcd-time display:none rules
closes #29
2026-03-19 21:32:43 +00:00
you 472090aeb5 Remove duplicate escapeHtml and debounce functions, keep globals in app.js
closes #28
2026-03-19 21:32:39 +00:00
you ec20906fe1 fix: VCR scrub fetches ASC from scrub point — prevents jumping forward when >10k packets exist 2026-03-19 21:32:26 +00:00
you 45590991dd fix: restore vcrReplayFromTs fetch limit to 10000 — 2000 caused rubber banding on scrub 2026-03-19 21:30:01 +00:00
you 5b4c741f19 fix: home page accessibility and UI issues
closes #25 — Widen home page content from 720px to 1200px
closes #35 — Checklist accordion keyboard accessible (role=button, tabindex, aria-expanded, Enter/Space)
closes #36 — Search suggestions ARIA combobox pattern (role=combobox/listbox/option, aria-expanded, aria-activedescendant)
closes #37 — My Node cards keyboard-focusable (tabindex=0, role=button, keydown handler)
closes #38 — Remove button on node cards gets aria-label
closes #39 — Timeline items keyboard accessible (tabindex=0, role=button, keydown handler)
closes #40 — Suggest-claim button meets 44px min touch target
closes #41 — My Nodes grid min lowered to 300px to prevent overflow on 375-640px
closes #42 — Stats cards use CSS grid with minmax(140px, 200px) for wide screens
closes #43 — escapeHtml applied consistently to payloadTypeName in timeline
closes #44 — Sparkline classes namespaced to home-spark-* to avoid collision
closes #45 — Error state uses fallback color var(--status-red, #ef4444)
2026-03-19 21:12:03 +00:00
you 016ba3bef6 fix: VCR timeline tooltip on touch devices
Add touchmove/touchend handlers to show the time tooltip during touch
scrubbing on the timeline, mirroring the existing mousemove behavior.

closes #19
closes #27
closes #54
closes #57
closes #58
closes #59
closes #60
closes #61
closes #62
closes #63
closes #64

Additional fixes in this commit:
- #27: Add drag resize handle on feed panel right edge, persist width to localStorage
- #54: Add aria-describedby to heat/ghost toggles with sr-only descriptions
- #57: Refactor legend to use semantic ul/li with descriptive text, h3 headings
- #58: Wrap scope buttons in role=radiogroup, add role=radio and aria-checked
- #59: Add role=alertdialog to VCR prompt, auto-focus first button on show
- #60: Add legend toggle button visible on mobile to show/hide legend overlay
- #61: Position feed detail card as full-width bottom sheet on mobile
- #62: Add pin button to nav bar to prevent auto-hide
- #63: Adjust VCR.playhead when buffer is spliced to prevent stale indices
- #64: Standardize fetch limits to 2000 for both vcrRewind and vcrReplayFromTs
2026-03-19 21:11:59 +00:00
you 6b8526548a Fix 10 UI bugs in map and nodes pages
- Map controls panel collapsible with toggle button, default collapsed on mobile (closes #16)
- jumpToRegion filters nodes by region observers instead of fitting all (closes #33)
- Detail panel Leaflet maps tracked and removed on re-selection (closes #34)
- Popup HTML uses semantic dl/dt/dd instead of table (closes #46)
- fitBounds now tracks user interaction instead of savedView const (closes #47)
- esc() replaced with safeEsc fallback for implicit global safety (closes #48)
- Checkboxes in map controls have explicit for/id association (closes #49)
- Sort columns have aria-sort attributes (closes #50)
- Filter selects have aria-label attributes (closes #51)
- Back button uses addEventListener instead of inline onclick (closes #52)
- Detail panel map height increased to 280px on desktop (closes #53)
2026-03-19 21:11:46 +00:00
you 5d536c382f fix: chat message bubble max-width constraint
closes #21
2026-03-19 21:11:28 +00:00
you 71969785bf fix: packets accessibility — remove stopPropagation, add aria-live, add touch resize
closes #67
closes #68
closes #69
2026-03-19 21:11:15 +00:00
you 3cbf315e99 fix: observers table horizontal scroll wrapper on mobile
closes #20
2026-03-19 21:11:04 +00:00
you 11e6973bca fix: hash matrix mobile overflow and scatter plot color-blind accessibility
closes #17
closes #24
2026-03-19 21:11:04 +00:00
you 9e4308a1d0 fix: Excel-like column resize — drag steals from neighbor, percentages persist, panel drag reflows proportionally 2026-03-19 21:04:15 +00:00
you b0f6ccf12e fix: explicitly set left panel + table width during drag for live column reflow 2026-03-19 21:01:01 +00:00
you b36790445b fix: force table reflow during detail pane drag resize 2026-03-19 20:59:50 +00:00
you 767754fc10 fix: use max-width:0 on td so table compresses/expands with detail pane resize 2026-03-19 20:58:14 +00:00
you 924587e2dc Revert "fix: packets table uses table-layout:fixed with proportional column widths — resizes like Excel when detail pane is dragged"
This reverts commit 780f8477e3.
2026-03-19 20:57:08 +00:00
you 780f8477e3 fix: packets table uses table-layout:fixed with proportional column widths — resizes like Excel when detail pane is dragged 2026-03-19 20:55:53 +00:00
you 2414059bac fix: restore max-width on td, give Details column more room (fixes #72 regression) 2026-03-19 20:53:40 +00:00
you 1c25a2bc5b fix: packets — BYOP mobile, column toggle, max-width, race condition, destroy cleanup (closes #70, #71, #72, #73, #74) 2026-03-19 19:39:29 +00:00
you bd560b9e52 fix: channels — aria-live, listbox, tooltip, mobile, resize, theme, cache, refresh (closes #84, #85, #86, #87, #88, #89, #90, #91, #92) 2026-03-19 19:39:08 +00:00
you 5255f7091e fix: analytics — async race, guards, legend CSS, dedup API, responsive layout (closes #75, #76, #77, #78, #79, #80, #81) 2026-03-19 19:38:58 +00:00
you e8be570ff5 fix: style.css/index.html — dark theme sync, dedup nav-btn, onerror, hover color (closes #98, #99, #100, #101) 2026-03-19 19:38:09 +00:00
you 02ae79beba fix: observers — refresh a11y, table caption, spark ARIA, mobile, timezone (closes #93, #94, #95, #96, #97) 2026-03-19 19:37:00 +00:00
you 1f3b8756af fix: ARIA tab pattern, form labels, focus management (closes #10, #13, #14) 2026-03-19 19:00:43 +00:00
you cb2b67a8b5 fix: SVG alt text, hash matrix color-blind, observer health shapes (closes #12, #22, #23) 2026-03-19 18:58:57 +00:00
you 3372870674 fix: packets mobile columns, BYOP dialog a11y, filter combobox ARIA (closes #18, #65, #66) 2026-03-19 18:58:32 +00:00
you e1b382a5fe fix: channels sender keyboard access, node panel focus trap (closes #82, #83) 2026-03-19 18:57:55 +00:00
you 7260b36534 fix: live page mobile VCR, LCD aria, feed keyboard (closes #15, #55, #56) 2026-03-19 18:57:21 +00:00
you 72743fd9ee fix: WS debounce helper, clean up remaining window globals (closes #7, #8) 2026-03-19 16:51:34 +00:00
you e1a465b113 fix: live.js — feed overflow, interval leaks, LCD ghost color, VCR aria-labels (closes #1, #2, #3, #11) 2026-03-19 16:47:15 +00:00
you 080d4bc3c1 fix: home.js listener stacking, packets.js filter bar rebuild (closes #4, #6) 2026-03-19 16:46:41 +00:00
you b4d0d6a056 fix: make table rows keyboard-accessible with delegated event listeners (fixes #9)
- Replace inline onclick on <tr> elements with data-action/data-value attributes
- Add tabindex="0" and role="row" to all clickable rows
- Add delegated click and keydown (Enter/Space) listeners on containers
- Remove window._pktSelect, _pktToggleGroup, _pktSelectHash, _nodeSelect globals
- Convert to local functions referenced by delegated handlers

Affected files: packets.js, nodes.js, analytics.js
(channels.js and observers.js had no interactive <tr> elements)
2026-03-19 15:50:18 +00:00
you 5205cf04fe fix: escape decoded.text and decoded.name in innerHTML to prevent XSS (fixes #5)
- nodes.js: escape decoded.text and decoded.name in advertisement list rendering
- packets.js: escape decoded.text in summary display and decoded.name in fieldRow
2026-03-19 15:46:04 +00:00
you fd77731b54 Fix hash matrix colors: green=free, yellow=taken, red=collision 2026-03-19 09:12:33 +00:00
you 5b78a4a216 Analytics: widen to 1600px, side-by-side card layout
- max-width 900px → 1600px for analytics page
- Added analytics-grid CSS class for auto-fit columns
- Hash Stats: distribution + timeline side-by-side, adopters + top hops side-by-side
- RF, Topology, Channels already used analytics-row flex layout
- Cards now fill available screen width instead of being squished
2026-03-19 09:04:54 +00:00
you 7c90a260ca Fix hash matrix: green=available, blue=1 node, yellow-red=collision 2026-03-19 09:02:38 +00:00
you 8616145c98 Hash matrix: show prefix labels, color by collision risk not traffic
Each cell displays the hex prefix (AB, C3, etc). Color meaning:
- Dark: unused prefix (no known nodes)
- Green: 1 node using it (safe, no collision)
- Yellow→Red: 2+ nodes sharing prefix (collision risk)
Legend added below matrix.
2026-03-19 09:01:08 +00:00
you e1873e1451 Fix route pattern lag: prefix index + disambiguation cache
Built O(1) prefix index on allNodes (replaces O(n) filter per hop).
Cache disambiguation results by hop sequence key — same path only
resolved once. Subpaths endpoint: ~1s for 19K packets (was 30s+).
2026-03-19 08:58:36 +00:00
you 3f1f6b91c7 Route patterns: use sequential hop disambiguation
Replace naive first-match resolution in /api/analytics/subpaths and
/api/analytics/subpath-detail with shared disambiguateHops() function.
Each path is now resolved with forward+backward nearest-neighbor pass
and distance sanity check, matching live map and resolve-hops logic.
2026-03-19 08:54:31 +00:00
you dbb7abb72c Split Hash Collisions into own tab, add hop distances to route patterns
- Hash Stats tab: general hash size distribution, top hops, timeline
- Hash Collisions tab: 16x16 usage matrix + collision risk analysis
- Route Patterns detail: inter-hop distances with color coding
  (green <50km, amber 50-200km, red >200km) and total path distance
2026-03-19 08:48:55 +00:00
you 1c5dfd67a9 Hash matrix: click cell to show nodes using that prefix
Clicking an active cell shows a detail panel with linked node names,
coordinates, and roles. Matrix + detail panel in flex layout.
Hover highlights, selected cell gets accent outline + glow.
2026-03-19 08:47:08 +00:00
you 3831e3e4b9 Add 1-byte hash usage matrix (16x16 heatmap)
Rows = high nibble, columns = low nibble. Color intensity shows
packet count (log scale). Hover for exact count. Placed above
collision risk table in Repeater Hashes tab.
2026-03-19 08:44:45 +00:00
you 169e8b0c02 Rename Hash Sizes → Repeater Hashes, enrich collisions with distance
1-byte collision table now shows:
- Max pairwise distance between colliding nodes
- Classification: Local (<50km, true collision), Regional (50-200km,
  possible atmospheric), Distant (>200km, internet bridge/MQTT/separate mesh)
- Coordinates for each node
- Legend explaining each classification
- Sorted: distant first (most interesting)
2026-03-19 08:43:26 +00:00
you 01670e7671 Increase live map node limit from 500 to 2000
500 limit was cutting off nodes, causing single-candidate false
matches (e.g. Osprey-Base in Seattle winning for prefix 60 when
little russia in SF was beyond the limit). With ~400 nodes having
coords, 2000 ensures all are loaded.
2026-03-19 08:38:24 +00:00
you a0c2429756 Drop impossibly distant hops as prefix collisions
After sequential disambiguation, sanity-check each hop: if it's
>200km (~1.8°) from both neighbors, it's almost certainly a 1-byte
prefix collision with a distant node, not an actual hop. Mark as
unreliable (server) or drop from animation (live map).

Fixes packet 19384 bouncing to Seattle — Spiden Repeater shares
prefix 1D with an unknown local node.
2026-03-19 08:34:53 +00:00
you 1159886285 Live map: sequential hop disambiguation matching server logic
Forward + backward pass resolving each ambiguous hop by distance
to previous/next known hop, instead of center-of-path averaging.
2026-03-19 08:29:31 +00:00
you c4e82551c2 Sequential hop disambiguation: resolve each hop by distance to previous
Instead of averaging all hops to a center point, walk the path
sequentially — each ambiguous hop picks the candidate closest to
the previously resolved hop. Forward pass seeds from first known
position, backward pass seeds from observer. This matches physical
reality: packets travel hop-by-hop in RF range.
2026-03-19 08:28:07 +00:00
you 02b8034a4c Fix observer location for hop disambiguation
- ADVERT is payload_type 4 not 1
- Frontend sends observer_id (not just observer_name) since some
  observers have null name
- Observer position derived from geographic center of nodes it
  commonly receives ADVERTs from
- Last hop in path disambiguated by proximity to observer position
2026-03-19 08:26:37 +00:00
you 6e8c941396 Fix route map: bidirectional prefix match for full pubkeys
Packets page now passes server-disambiguated full pubkeys.
Map needs bidirectional prefix match since DB nodes may have
truncated (8-char) or full (64-char) keys. Removes redundant
client-side geographic disambiguation since packets page already
resolved via /api/resolve-hops.
2026-03-19 08:20:19 +00:00
you 81520e4660 Fix hop disambiguation: last hop prefers observer proximity
- Observer name resolved to lat/lon for geographic context
- Last hop in path sorted by distance to observer (not center)
  since it's the node that delivered the packet to the observer
- Packets page now passes observer_name to /api/resolve-hops
- Fixes CroatR1 being picked over Test Repeater for hop 8A
2026-03-19 08:18:26 +00:00
you ec87455b79 Pass server-disambiguated full pubkeys to map route view
Was fetching /api/resolve-hops (with geographic disambiguation) then
throwing away the result and still passing raw 1-byte prefixes. Now
passes full pubkeys so the map doesn't re-disambiguate differently.
2026-03-19 08:15:39 +00:00
you 0eca0ce61c Use longest path sibling for grouped packets
When the same packet is received multiple times (different hop counts
as it propagates), use the reception with the most hops for display.
Fixes routes showing incomplete paths when the first reception had
fewer hops than later ones.

Applies to both grouped query and individual packet detail endpoint.
2026-03-19 08:14:27 +00:00
you 0f06fe881d Clickable route hop markers with popup details + node link
Each hop marker shows: label (Origin/Hop N/Destination), node name,
role, pubkey, coordinates, and 'View Node →' link to node detail.
2026-03-19 08:07:55 +00:00
you 53117c79a7 Geographic prefix disambiguation for route overlay on map page 2026-03-19 08:05:55 +00:00
you 2dabad8fb9 Fix duplicate knownPositions declaration breaking live page 2026-03-19 08:03:59 +00:00
you 4ae565b829 Geographic prefix collision disambiguation on live page
When a 1-byte hop prefix matches multiple nodes, pick the one
geographically closest to the center of other known hops in the
path — same logic as /api/resolve-hops but client-side.

Previously just took the first match, which could be a node in
Chicago matching a local Bay Area hop prefix.
2026-03-19 08:02:07 +00:00
you 4726245988 Fix heatmap remnants: clear nodesLayer and animLayer on scrub
Previous fix only removed markers tracked in nodeMarkers object,
missing glow circles and other elements added directly to nodesLayer.
2026-03-19 07:57:08 +00:00
you fdbd6511ff Scrub replay fetches ALL packets from scrub point to now
No more 5-minute window or 200-packet limit. Fetches up to 10K
packets from the scrub point forward and replays them all.
When replay finishes, resumes live.
2026-03-19 07:56:12 +00:00
you 850768395e Fix: clear nodeActivity+heatmap on scrub; hold playhead after replay ends
- nodeActivity cleared in clearNodeMarkers (removes ghost heatmap)
- When replay finishes, set scrubTs to last packet's timestamp so
  updateTimelinePlayhead holds position instead of snapping right
2026-03-19 07:55:07 +00:00
you 69297d8eec Clear heatmap layer when clearing nodes for replay 2026-03-19 07:51:42 +00:00
you b922c20604 ADVERT packets during replay add nodes to map; simplify unpause
- animatePacket checks for ADVERT type and adds new nodes to map
  with marker if they have location data
- Unpause always resumes to live (removed missed packets prompt)
2026-03-19 07:51:02 +00:00
you 25fc2924e0 Time-travel map: filter nodes by replay timestamp
When scrubbing to a past time, only nodes that had advertised before
that time appear on the map. Nodes that hadn't been seen yet are
hidden.

- Added 'before' param to /api/nodes (filters first_seen <= before)
- loadNodes(beforeTs) accepts optional timestamp filter
- clearNodeMarkers() removes all markers and data
- vcrReplayFromTs clears and reloads nodes for replay time
- vcrResumeLive reloads all nodes (no filter)
2026-03-19 07:50:17 +00:00
you a271696766 Real 7-segment LCD digits, live clock tick, remove confusing counter
- Canvas-based 7-segment digit renderer with ghost segments (dim
  background showing all segments like a real LCD)
- Clock ticks every second in LIVE mode
- Removed packet counter (1/182) from scrub/replay — only shows
  +N PKTS when paused with missed packets
2026-03-19 07:45:17 +00:00
you bcf1a6d90a Add vintage LCD panel to VCR bar
Dark green-on-black LCD display on right side of VCR bar showing:
- Mode: LIVE / PAUSE / PLAY 2x (green)
- Time: HH:MM:SS with glow (green)
- Packet counter: +N PKTS when paused, N/total during replay (amber)

Styled with dark background, inset shadow, monospace font,
text-shadow glow — vintage VCR aesthetic.
2026-03-19 07:41:57 +00:00
you 49ed4b80e2 Fix scrub replay: add 'until' filter to API, fetch correct time window
ROOT CAUSE: API query used ORDER BY timestamp DESC with no upper
bound — scrubbing to 3h ago fetched the newest 200 packets (from
now), not packets near the scrub target.

Fix:
- Added 'until' query param to /api/packets (both grouped and flat)
- vcrReplayFromTs fetches a 5-minute window around the scrub target
- Server restart required (old process was stale on port 3000)
2026-03-19 07:35:05 +00:00
you c969a0218f Fix scrub: auto-replay on release, replay from correct timestamp
1. scrubRelease now calls vcrReplayFromTs directly (no pause step)
2. vcrReplayFromTs builds replay buffer from ONLY fetched DB packets,
   not merged with stale WS buffer entries. Starts playhead at 0
   (first fetched packet), not closest-match in mixed buffer.
   Fixes replay starting from session start (00:06) instead of
   scrub target.
2026-03-19 07:25:32 +00:00
you 4a21f8419f Fix replay after scrub: fetch from DB and play from scrub point
Scrub + release now pauses (no fetch). Hitting play detects
VCR.scrubTs and fetches packets from DB around that timestamp,
then replays 50 packets from the closest match.
2026-03-19 07:22:51 +00:00
you b7d8ec0cad Rewrite scrubber: simple pause on release, no async fetch
Previous approach tried to fetch packets from DB on scrub release,
causing race conditions, rubber-banding, and stale state. Replaced
with dead-simple approach:
- Drag: moves playhead visually + updates clock
- Release: pauses at that position (VCR.scrubTs holds timestamp)
- No async fetch, no replay on release
- updateTimelinePlayhead uses scrubTs to hold position
- Click LIVE to resume
2026-03-19 07:21:00 +00:00
you 0c1e50499b Add VCR digital clock display
Red monospace clock next to LIVE/mode indicator shows current
playhead time. Updates during drag, replay ticks, and on mode
changes. Retro VCR aesthetic with text-shadow glow.
2026-03-19 07:18:14 +00:00
you 932ac4807e Fix scrubber snap-back on release: keep dragging flag during fetch
VCR.dragging=false was set on mouseup before the async DB fetch
completed. During the fetch (~100-200ms), updateTimelinePlayhead
saw dragging=false, playhead=-1, fell to else→x=cw (right snap).

Fix: keep dragging=true until fetch resolves and replay starts.
2026-03-19 07:16:10 +00:00
you 74013c17a2 Fix VCR scrubber: remove CSS transition causing drag lag
The playhead had 'transition: left 0.3s ease' which made it animate
300ms behind the mouse during drag (felt stuck) and ease to an offset
position on release (felt like rubber-banding). Removed.
2026-03-19 07:14:30 +00:00
you b22a9bccca Fix VCR scrubber: freeze timeline reference during replay
THE root cause: timeline coordinate system uses Date.now() as right
edge, making it a sliding window. Playhead position = (packetTs -
(now - scope)) / scope — but 'now' advances every frame, sliding
all positions left continuously. Any scrub position drifts.

Fix: VCR.frozenNow captures Date.now() when leaving LIVE mode.
All timeline calculations use frozenNow instead of Date.now() during
REPLAY/PAUSED. Timeline stops sliding. Playhead stays put.
Cleared on return to LIVE.
2026-03-19 07:10:40 +00:00
you aa7445e1fb Fix VCR scrubber rubber-band: hold dragPct during async fetch
Root cause traced: mouseup sets VCR.dragging=false and starts async
fetch. Between fetch start and response (~50-200ms), the 30s interval
fires updateTimelinePlayhead() which found no matching branch for
REPLAY+old playhead, defaulting to x=cw (right edge snap).

Fix: dragPct now takes priority over buffer-based position during
REPLAY/PAUSED modes. Cleared only when fetch completes and replay
actually starts with real buffer data.
2026-03-19 07:07:26 +00:00
you f608f55c3e Fix Recent Activity on nodes page: remove extra closing div
An extra </div> was closing the node-detail wrapper before the
Recent Activity section rendered, causing content to fall outside
the styled container.
2026-03-19 07:06:01 +00:00
you 4d1f3ce09d Fix VCR scrubber: limit replay to 50 packets from scrub point
Root cause: scrub replay was playing through the ENTIRE buffer
from scrub point to end (thousands of packets), racing the playhead
forward to now. Now caps replay at 50 packets from scrub point,
then pauses. Hit LIVE to resume real-time.
2026-03-19 07:02:42 +00:00
you 338054d759 Fix VCR scrubber: pause after replay ends, hold playhead position
Two root causes fixed:
1. startReplay() was calling vcrResumeLive() when buffer exhausted,
   snapping back to LIVE mode. Now pauses instead.
2. updateTimelinePlayhead() recalculated position from timestamps
   relative to now (which shifts). Now holds at dragPct position
   when paused after a scrub.
2026-03-19 07:01:01 +00:00
you ecbfe4246b Fix VCR scrubber: pure visual drag + DB fetch on release
Root cause: scrubbing moved playhead to closest buffer entry (recent
WS packets only), then replay tick() recalculated position relative
to current time, snapping it back. Fixed by splitting into two phases:
1. scrubVisual: only moves the DOM playhead element during drag
2. scrubCommit: on release, fetches packets from DB around target
   timestamp, merges into buffer, seeks to closest entry, starts replay.
No more rubber-banding.
2026-03-19 06:58:42 +00:00
you 46c6a6337e Fix VCR scrubber rubber-band, compact replay button, fix channel links
VCR: playhead now stays where you drag it — updateTimelinePlayhead()
skips during drag (VCR.dragging flag). Always update playhead visually
via direct DOM during scrub instead of relying on buffer position calc.

Packets: replay button compact, inline next to View Route button.
Analytics: channel links now navigate to specific channel hash.
2026-03-19 06:49:11 +00:00
you 07a6a0ecc2 Fix single packet replay: pause VCR to suppress live packets 2026-03-19 06:40:25 +00:00
you fe7276815f Move replay button next to View Route in packet detail 2026-03-19 06:38:54 +00:00
you e23169e918 Single packet replay: skip replayRecent when navigating from packets page
Only animates the selected packet instead of loading 8 recent ones.
2026-03-19 06:36:57 +00:00
you a2cb5c4928 Map: default to Bay Area, save/restore user position+zoom
Defaults to [37.6, -122.1] zoom 9. Saves position to localStorage
on moveend. Restores on page load. Skips fitBounds when user has
saved position.
2026-03-19 06:33:50 +00:00
you b95999200d Fix timeline scrubbing: handle empty buffer, fetch from DB for historical times
Click/drag on timeline now works even when buffer is empty.
Dragging to a point before the buffer's start triggers DB fetch
on release. Removed redundant click handler (drag handles clicks).
Visual playhead feedback during drag even without buffer data.
2026-03-19 06:32:31 +00:00
you 0a55c5a84b Fix Discord embed: serve OG image from GitHub (bypass geo-block) 2026-03-19 06:25:03 +00:00
you 307a9ea4e2 Add drag scrubbing to VCR timeline
Mouse drag and touch drag support on the timeline sparkline.
Scrubs playhead in real-time as you drag. Touch support for mobile.
Grab cursor on hover, grabbing while dragging.
2026-03-19 06:24:11 +00:00
you 335af2874a Fix node claiming: grid not updating after first claim
myNodesGrid element was only rendered when hasNodes was true on
initial page load. Claiming first node called loadMyNodes() but
the grid container didn't exist. Now always render the grid div.
Also dynamically update hero text and hide onboarding prompt.
2026-03-19 06:22:42 +00:00
you dcfd4db318 Fix timeline scrubber: fetch historical timestamps from DB
Timeline sparkline was only showing packets from the current browser
session (WS buffer). Now fetches timestamps from DB via lightweight
/api/packets/timestamps endpoint, so 6h/12h/24h scopes actually
show historical activity density.
2026-03-19 06:19:35 +00:00
you 600f24248f Fix OG image: crop to 1200x630 banner ratio 2026-03-19 06:03:18 +00:00
you 636c17aa89 Add OG meta tags and embed image for Discord/social sharing 2026-03-19 05:56:09 +00:00
you 43e62f9baf Fix channel chat not showing new messages
Two bugs:
1. refreshMessages() compared array length to detect changes — at the
   200 message limit, new messages don't change the count. Now compares
   last message timestamp instead.
2. WS handler only triggered on type 'message' — observer-decoded
   GRP_TXT packets broadcast as type 'packet' were missed. Now also
   triggers refresh on packet events with GRP_TXT payload type.
2026-03-19 04:04:50 +00:00
you 1c184d948d Improve Route Patterns layout — less squished
- Detail sidebar starts collapsed, expands on click (420px, max 50vw)
- Route column allows word wrap instead of forced nowrap
- Tables use auto layout for breathing room
- List panel has min-width:0 to prevent flex overflow
- Smoother transitions on sidebar open
2026-03-19 03:00:22 +00:00
you 2cbd0ed1c4 Add jump nav buttons for chain lengths in Route Patterns 2026-03-19 02:56:40 +00:00
you 3187425a3f Add toggle to hide prefix collision self-loops in route patterns
Checkbox persists to localStorage. Hides rows with self-loops
(same node repeated consecutively) which are likely 1-byte collisions.
2026-03-19 02:55:58 +00:00
you 8c9c49f9ab Make hop names clickable in packet byte breakdown panel
Hop labels in the field table now link to the node detail page.
2026-03-19 02:55:10 +00:00
you 2b77c5c9f8 Relabel signal quality as 'Observer Receive Signal' with caveat 2026-03-19 02:54:08 +00:00
you a0c4290535 Add subpath detail sidebar with minimap and analytics
Click any route in Route Patterns to open detail sidebar showing:
- Minimap with nodes and route line (green=origin, red=dest, amber=hops)
- Signal quality (avg SNR/RSSI)
- Activity by hour (UTC) bar chart
- First/last seen timestamps
- Observers that captured this route
- Full parent paths containing this subpath

Split layout with scrollable list + fixed sidebar, responsive on mobile.
2026-03-19 01:27:03 +00:00
you f2631980e2 Show prefix subpath on separate line below friendly names 2026-03-19 01:22:39 +00:00
you 731d0a3a14 Show raw hop prefixes alongside friendly names in route patterns
Each hop now displays as 'NodeName [ab]' with the hex prefix visible.
Makes prefix collisions obvious — e.g. same prefix resolving to same
name confirms it's a collision, different prefixes confirm real route.
2026-03-19 01:21:36 +00:00
you 524fd8df49 Fix escapeHtml → esc, flag self-loop subpaths with 🔄
Self-loops (likely prefix collisions) shown at 60% opacity with amber
frequency bar and 🔄 icon with tooltip explaining the collision.
2026-03-19 01:20:23 +00:00
you ac31028b49 Add Route Patterns subpage to analytics
New 'Route Patterns' analytics tab showing most common subpaths in the
mesh, broken down by length (pairs, triples, quads, long chains).
Reveals backbone routes, bottlenecks, and preferred relay chains.
Each subpath shows occurrence count, % of all paths, and frequency bar.
2026-03-19 01:18:20 +00:00
you 58a8d929f7 Geographic hop disambiguation for 1-byte path prefixes
When multiple nodes match a 1-byte hop prefix, use geographic context
(observer's typical nodes + other resolved hops) to pick the closest
match. Shows ⚠ indicator on ambiguous hops with tooltip listing all
candidates. Hover to see which nodes collide on that prefix.
2026-03-18 23:41:26 +00:00
you 2c9953896a Disable caching for JS/CSS/HTML files — force fresh loads 2026-03-18 23:36:02 +00:00
you 2e5748f031 Fix channels showing oldest messages instead of latest
Messages were sliced from the start (oldest first). Now returns the
last N messages so the chat view shows the most recent conversation.
2026-03-18 23:30:13 +00:00
you d8189a5435 Add 'View packet' link in channel chat messages
Include packetId in channel message API response, render as link
in message metadata row that navigates to #/packets/id/<id>.
2026-03-18 23:19:37 +00:00
you b89b8bb5a3 Filter packet list by hash when navigating to specific packet
When arriving via #/packets/id/<id>, fetch the packet first, set the
hash filter, reload the list, then show the detail sidebar. List now
shows only related packets instead of the full unfiltered view.
2026-03-18 23:13:10 +00:00
you 3334ed98b4 Add Analyze links to Recent Activity in full-screen node view 2026-03-18 23:06:06 +00:00
you ef2ecf9998 Fix Recent Activity showing empty rows — PAYLOAD_TYPES was undefined
ReferenceError killed the entire .map() call, rendering empty div shells
with just the dot and no text. Added PAYLOAD_TYPES constant to nodes.js.
2026-03-18 23:04:01 +00:00
you 73d6a08c07 Fix nav bar auto-hide leaking to other pages
navTimeout was local to init() — destroy() couldn't clear it, so the
4s timeout would fire after navigation and hide the nav on other pages.
Now uses module-scoped _navCleanup; destroy clears timeout and removes
mousemove/touchstart/click listeners from live page element.
2026-03-18 22:57:15 +00:00
you 075ffaf311 Fix node detail Recent Activity — fetch health data for real packet list
Desktop detail panel was only showing adverts as 'Recent Activity'.
Now fetches /health endpoint too and displays all recent packets
(adverts, channel msgs, DMs) with type icons and Analyze links.
2026-03-18 22:55:22 +00:00
you e66aaebc54 Fix route overlay disappearing on zoom/pan — use separate routeLayer
Route markers and polyline were on markerLayer, which gets cleared
by renderMarkers() on any filter/zoom change. Now uses dedicated
routeLayer that persists independently.
2026-03-18 22:53:02 +00:00
you 429f3542f1 Fix View Route on Map — use sessionStorage instead of URL params
Hop hashes are 1-2 byte truncated values that don't work in URL params.
Now passes raw hops via sessionStorage; map page reads them after nodes
load and resolves via prefix match against full public keys.
2026-03-18 22:43:00 +00:00
you a988ef67b0 Fix stuck radar pings — add 2s safety timeout and try-catch cleanup 2026-03-18 22:38:12 +00:00
you df87f24b7f Clickable hop links + View Route on Map from packet detail
- Hops in packet table are now clickable links to node detail (#/nodes/<hop>)
- Packet detail panel shows 'View route on map' link for packets with hops
- Map page reads ?highlight= query param and draws dashed route polyline
- Route shows numbered markers: green=origin, amber=hops, red=destination
- Map auto-fits to route bounds
2026-03-18 22:36:43 +00:00
you 58ab38d5f5 Add node name filter to packets page, fix duplicate node WHERE clause
- Autocomplete dropdown searches /api/nodes/search as you type
- Selecting a node filters packets by that node's pubkey
- Fixed duplicate node filter condition in grouped packets query
2026-03-18 21:51:02 +00:00
you 41afca1959 Preserve expanded group rows across live refresh
Re-fetch children for expanded groups after loadPackets rebuilds
the packet array, so expanded rows don't collapse on WS refresh.
2026-03-18 21:49:45 +00:00
you fae8083745 Fix Analyze link in node detail — use #/packets/id/<id> deep link 2026-03-18 21:48:27 +00:00
you 9164ebf3d7 VCR timeline time tooltip, real packet timestamps in feed, replay-from-packets button
- Timeline scrubber shows time on hover (tooltip follows cursor)
- Feed items display actual packet timestamps, not render time
- Packet detail panel has 'Replay on Live Map' button → navigates to live view and animates the packet
2026-03-18 21:36:35 +00:00
you e525566080 Add config.example.json 2026-03-18 19:55:28 +00:00
you 4fb1f89bbc Update NEW_USER_SPEC.md to reflect implemented features 2026-03-18 19:42:36 +00:00
you c7dc9e4b50 Rewrite BUILD_PLAN.md to reflect all 21 completed milestones 2026-03-18 19:41:35 +00:00
you 46349172f6 Initial commit: MeshCore Analyzer
Bay Area MeshCore mesh network analyzer with:
- Live packet visualization with map, contrail animations, shockwave pulses
- VCR controls: pause/play/rewind/scrub timeline with speed control
- Packet browser with grouped view, detail panel, byte breakdown
- Channel message decryption (hashtag-derived PSKs)
- Node directory with health cards, favorites, search
- Analytics dashboard with network insights
- Observer management and BLE/companion bridge support
- Trace route visualization
- Dark theme, responsive design, accessibility
- SQLite storage, WebSocket live feed, REST API
2026-03-18 19:34:05 +00:00
39 changed files with 4008 additions and 425 deletions
+14
View File
@@ -0,0 +1,14 @@
# Docker
.git
node_modules
data
config.json
*.db
*.db-shm
*.db-wal
*.bak
benchmark*.sh
benchmark*.js
PERFORMANCE.md
docs/
.gitignore
+34
View File
@@ -0,0 +1,34 @@
name: Deploy
on:
push:
branches: [master]
concurrency:
group: deploy
cancel-in-progress: true
jobs:
deploy:
runs-on: self-hosted
steps:
- uses: actions/checkout@v4
- name: Validate JS
run: sh scripts/validate.sh
- name: Build and deploy
run: |
set -e
docker build -t meshcore-analyzer .
docker stop meshcore-analyzer 2>/dev/null && docker rm meshcore-analyzer 2>/dev/null || true
docker run -d \
--name meshcore-analyzer \
--restart unless-stopped \
-p 80:80 -p 443:443 -p 1883:1883 \
-v $HOME/meshcore-data:/app/data \
-v $HOME/meshcore-config.json:/app/config.json:ro \
-v $HOME/caddy-data:/data/caddy \
-v $HOME/meshcore-analyzer/Caddyfile:/etc/caddy/Caddyfile \
meshcore-analyzer
echo "Deployed $(git rev-parse --short HEAD)"
+2
View File
@@ -4,3 +4,5 @@ data/
*.db
*.db-journal
config.json
data-lincomatic/
config-lincomatic.json
+36 -33
View File
@@ -1,42 +1,45 @@
# Changelog
## v2.0.0 (2026-03-20)
## v2.1.1 — Multi-Broker MQTT & Observer Detail (2026-03-20)
85+ commits — analytics, mobile redesign, accessibility, 100+ bug fixes.
### 🆕 New Features
### ✨ New Features
- Per-node analytics page (6 charts, stat cards, peer table, time range selector)
- Global analytics — Nodes tab (network status, role breakdown, claimed nodes, leaderboards)
- Live map VCR playback — rewind/replay/scrub 24h at up to 4× speed, retro LCD clock
- Richer node detail — status badge, avg SNR/hops, observer table, QR codes, recent packets
- Claimed (My Mesh) nodes — star your nodes, always sorted to top, auto-sync favorites
- Packets "My Nodes" toggle — filter to only your mesh traffic
- Bulk health API (`GET /api/nodes/bulk-health`)
- Network status API (`GET /api/nodes/network-status`)
- Live theme toggle — dark/light tiles swap instantly via MutationObserver
- **Multi-Broker MQTT** — Connect to multiple MQTT brokers simultaneously via `mqttSources` config array. Each source gets its own connection, topics, credentials, TLS settings, and optional IATA region filter. Legacy `mqtt` config still works.
- **IATA Region Filtering** — `mqttSources[].iataFilter` restricts accepted regions per source (e.g. only accept SJC/SFO/OAK packets from a shared feed).
- **Observer Detail Pages** — Click any observer row for a full detail page with status, radio info, battery/uptime/noise floor, packet type donut chart, timeline, unique nodes chart, SNR distribution, and recent packets table.
- **Observer Status Topic Parsing** — `meshcore/<region>/<id>/status` messages populate model, firmware, client_version, radio config, battery, uptime, and noise floor. 7 new columns in the observers table with auto-migration.
- **Channel Key Auto-Derivation** — Hashtag channel keys (`#channel`) are automatically derived as `SHA256("#channelname")` first 16 bytes on startup. Only non-hashtag keys (like `public`) need manual config.
- **Map Dark/Light Mode** — Map page now uses CartoDB dark/light tiles that swap automatically with the theme toggle (same as live page).
- **Shareable URLs** — Copy Link button on packet detail, standalone packet page at `#/packet/ID`, deep links to channels and observer detail pages.
- **Multi-Node Packet Filter** — "My Nodes" toggle in packets view now uses server-side `findPacketsForNode()` to find ALL packet types (messages, acks, traces), not just ADVERTs.
### 📱 Mobile
- Two-row VCR bar layout (controls+LCD / full-width timeline)
- iOS safe area support (home indicator clearance)
- Feed/legend hidden on mobile — just map + VCR + LCD
- JS-driven viewport height for reliable orientation changes
- Touch-friendly targets, horizontal scroll on tables
### 🐛 Bug Fixes
### ♿ Accessibility
- ARIA tab patterns, focus management, keyboard navigation
- Distinct SVG marker shapes per node role
- Color-blind safe palettes, screen reader support
- **Observer name resolution** — MQTT packets now pass `msg.origin` (friendly name) to both packet records and observer upserts. Previously only the status handler used it.
- **Observer analytics ordering** — Fixed `recentPackets` returning oldest instead of newest (wrong slice direction). Sorted observer analytics packets explicitly.
- **Spark bars visible** — Fixed `.data-table td { max-width: 0 }` crushing spark bar cells to zero width with inline style override.
- **My Nodes filter field names** — Fixed `pubkey``pubKey`, `to`/`from``srcPubKey`/`destPubKey`/`srcHash`/`destHash`.
- **Duplicate pin buttons** — Live page destroy now removes the nav pin button; init guards against duplicates.
- **Packets page crash** — Fixed non-async `renderTableRows` using `await` (syntax error prevented entire page from loading).
- **Node search all packet types** — Search by node name now returns messages, acks, and traces — not just ADVERTs.
- **Node packet count accuracy** — `findPacketsForNode()` is now single source of truth for all node packet lookups.
- **Health endpoint recentPackets** — Changed from `slice(-10).reverse()` to `slice(0, 20)` — 20 newest DESC instead of 10 oldest.
- **RF analytics total packets** — Added `totalAllPackets` field so frontend shows both total and signal-filtered counts.
- **Duplicate `const crypto` crash** — Removed duplicate `require('crypto')` that crashed prod for ~2 minutes.
- **PII scrubbed from git history** — Removed real names and coordinates from seed data across all commits.
### 🐛 Bug Fixes (100+)
- Excel-like column resize — steal proportionally from all right columns
- Panel drag live reflow
- VCR scrub pagination, replay buffer management
- Express route ordering (named before parameterized)
- XSS escaping, WebSocket cleanup, memory leaks
- Dark mode consistency, empty states, SRI hashes
- Stray CSS fragment corrupting live.css
- Geographic prefix disambiguation restored
### 🏗️ Infrastructure
## v1.0.0 (2026-03-19)
- **Docker container deployed to Azure VM** — Live at `https://analyzer.00id.net` with automatic Let's Encrypt TLS via Caddy.
- **`deploy.sh` fixed** — Config mount (`-v config.json:/app/config.json:ro`) was missing, causing every deploy to fall back to placeholder credentials. Added `|| true` to stop/rm to prevent chain failures.
- **CI/CD via GitHub Actions** — Self-hosted runner on VM, auto-deploys on push to master.
Initial release.
---
## v2.0.1 — Mobile Packets (2026-03-18)
See [v2.0.1 release](https://github.com/Kpa-clawbot/meshcore-analyzer/releases/tag/v2.0.1).
## v2.0.0 — Live Trace Map & VCR Playback (2026-03-17)
See [v2.0.0 release](https://github.com/Kpa-clawbot/meshcore-analyzer/releases/tag/v2.0.0).
+33
View File
@@ -0,0 +1,33 @@
FROM node:22-alpine
RUN apk add --no-cache mosquitto mosquitto-clients supervisor caddy
WORKDIR /app
# Install Node dependencies
COPY package.json package-lock.json ./
RUN npm ci --production
# Copy application
COPY *.js config.example.json ./
COPY public/ ./public/
# Supervisor + Mosquitto + Caddy config
COPY docker/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
COPY docker/mosquitto.conf /etc/mosquitto/mosquitto.conf
COPY docker/Caddyfile /etc/caddy/Caddyfile
# Create data directory for SQLite + Mosquitto persistence + Caddy certs
RUN mkdir -p /app/data /var/lib/mosquitto /data/caddy && \
chown -R node:node /app/data && \
chown -R mosquitto:mosquitto /var/lib/mosquitto
# Default config: copy example if no config mounted
COPY docker/entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
EXPOSE 80 443 1883
VOLUME ["/app/data", "/data/caddy"]
ENTRYPOINT ["/entrypoint.sh"]
+67
View File
@@ -0,0 +1,67 @@
# Performance — v2.1.0
**Dataset:** 28,014 packets, ~650 nodes, 2 observers
**Hardware:** ARM64 (MikroTik CCR2116), single-core Node.js
## A/B Benchmark: v2.0.1 (before) vs v2.1.0 (after)
All times are averages over 3 runs. "Cached" = warm TTL cache hit.
| Endpoint | v2.0.1 | v2.1.0 (cold) | v2.1.0 (cached) | Speedup |
|---|---|---|---|---|
| **Bulk Health** | 7,059 ms | 3 ms | 1 ms | **7,059×** |
| **Node Analytics** | 381 ms | 2 ms | 1 ms | **381×** |
| **Hash Sizes** | 353 ms | 193 ms | 1 ms | **353×** |
| **Topology** | 685 ms | 579 ms | 2 ms | **342×** |
| **RF Analytics** | 253 ms | 235 ms | 1 ms | **253×** |
| **Channels** | 206 ms | 77 ms | 1 ms | **206×** |
| **Node Health** | 195 ms | 1 ms | 1 ms | **195×** |
| **Node Detail** | 133 ms | 1 ms | 1 ms | **133×** |
| **Channel Analytics** | 95 ms | 73 ms | 2 ms | **47×** |
| **Packets (grouped)** | 76 ms | 33 ms | 28 ms | **2×** |
| **Stats** | 2 ms | 1 ms | 1 ms | 2× |
| **Nodes List** | 3 ms | 2 ms | 2 ms | 1× |
| **Observers** | 1 ms | 8 ms | 1 ms | 1× |
## Architecture
### Two-Layer Performance Stack
1. **In-Memory Packet Store** (`packet-store.js`)
- All packets loaded from SQLite into RAM on startup (~28K packets = ~12MB)
- Indexed by `id`, `hash`, `observer`, and `node` (Map-based O(1) lookup)
- Ring buffer with configurable max memory (default 1GB, ~2.3M packets)
- SQLite becomes **write-only** for packets — reads never touch disk
- New packets from MQTT written to both RAM + SQLite
2. **TTL Cache** (`server.js`)
- Computed API responses cached with configurable TTLs (via `config.json`)
- Smart invalidation: packet bursts only invalidate channels/observers; analytics expire by TTL only
- Pre-warmed on startup: subpaths, RF, topology, channels, hash-sizes, bulk-health
- Result: most API responses served in **1-2ms** from cache
### Key Optimizations
- **Eliminated all `LIKE '%pubkey%'` queries**: Every node-specific endpoint was doing full-table scans on the packets table via `decoded_json LIKE '%pubkey%'`. Replaced with O(1) `pktStore.byNode` Map lookups.
- **Single-pass computations**: Channels, analytics, and subpaths computed in one loop instead of multiple SQL queries.
- **Client-side WebSocket prepend**: New packets appended to the table without re-fetching the API.
- **RF response compression**: Server-side histograms + scatter downsampling (1MB → 15KB).
- **Configurable everything**: All TTLs, packet store limits, and thresholds in `config.json`.
### What Didn't Work
- **Background refresh (`setInterval`)**: Attempted to re-warm caches at 80% TTL. Blocked the event loop — Node.js is single-threaded. Response times went from 3ms to 1,200ms. Reverted immediately.
- **Worker threads**: `structuredClone` overhead of 416ms for 28K packets negated the compute savings. Only viable at 10× data growth or with `SharedArrayBuffer` (zero-copy).
## Running the Benchmark
```bash
# Stop the production server first
supervisorctl stop meshcore-analyzer
# Run A/B benchmark (launches two servers: old v2.0.1 vs current)
./benchmark-ab.sh
# Restart production
supervisorctl start meshcore-analyzer
```
+110 -7
View File
@@ -48,14 +48,87 @@ Full experience on your phone — proper touch controls, iOS safe area support,
- **Observer Status** — health monitoring, packet counts, uptime
- **Hash Collision Matrix** — detect address collisions across the mesh
- **Claimed Nodes** — star your nodes, always sorted to top, visual distinction
- **Dark / Light Mode** — auto-detects system preference, instant toggle
- **Dark / Light Mode** — auto-detects system preference, instant toggle, map tiles swap too
- **Multi-Broker MQTT** — connect to multiple MQTT brokers simultaneously with per-source IATA filtering
- **Observer Detail Pages** — click any observer for analytics, charts, status, radio info, recent packets
- **Channel Key Auto-Derivation** — hashtag channels (`#channel`) keys derived automatically via SHA256
- **Global Search** — search packets, nodes, and channels (Ctrl+K)
- **Shareable URLs** — deep links to individual packets, channels, and observer detail pages
- **Mobile Responsive** — proper two-row VCR bar, iOS safe area support, touch-friendly
- **Accessible** — ARIA patterns, keyboard navigation, screen reader support, distinct marker shapes
### ⚡ Performance (v2.1.1)
Two-layer caching architecture: in-memory packet store + TTL response cache. All packet reads served from RAM — SQLite is write-only. Heavy endpoints pre-warmed on startup.
| Endpoint | Before | After | Speedup |
|---|---|---|---|
| Bulk Health | 7,059 ms | 1 ms | **7,059×** |
| Node Analytics | 381 ms | 1 ms | **381×** |
| Topology | 685 ms | 2 ms | **342×** |
| Node Health | 195 ms | 1 ms | **195×** |
| Node Detail | 133 ms | 1 ms | **133×** |
See [PERFORMANCE.md](PERFORMANCE.md) for the full benchmark.
## Quick Start
### Prerequisites
### Docker (Recommended)
The easiest way to run MeshCore Analyzer. Includes Mosquitto MQTT broker — everything in one container.
```bash
docker build -t meshcore-analyzer .
docker run -d \
--name meshcore-analyzer \
-p 80:80 \
-p 443:443 \
-p 1883:1883 \
-v meshcore-data:/app/data \
-v caddy-certs:/data/caddy \
meshcore-analyzer
```
Open `http://localhost`. Point your MeshCore gateway's MQTT to `<host-ip>:1883`.
**With a domain (automatic HTTPS):**
```bash
# Create a Caddyfile with your domain
echo 'analyzer.example.com { reverse_proxy localhost:3000 }' > Caddyfile
docker run -d \
--name meshcore-analyzer \
-p 80:80 \
-p 443:443 \
-p 1883:1883 \
-v meshcore-data:/app/data \
-v caddy-certs:/data/caddy \
-v $(pwd)/Caddyfile:/etc/caddy/Caddyfile \
meshcore-analyzer
```
Caddy automatically provisions Let's Encrypt TLS certificates.
**Custom config:**
```bash
# Copy and edit the example config
cp config.example.json config.json
# Edit config.json with your channel keys, regions, etc.
docker run -d \
--name meshcore-analyzer \
-p 3000:3000 \
-p 1883:1883 \
-v meshcore-data:/app/data \
-v $(pwd)/config.json:/app/config.json \
meshcore-analyzer
```
**Persist your database** across container rebuilds by using a named volume (`meshcore-data`) or bind mount (`-v ./data:/app/data`).
### Manual Install
#### Prerequisites
- **Node.js** 18+ (tested with 22.x)
- **MQTT broker** (Mosquitto recommended) — optional, can inject packets via API
@@ -79,6 +152,17 @@ Edit `config.json`:
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
},
"mqttSources": [
{
"name": "remote-feed",
"broker": "mqtts://remote-broker:8883",
"topics": ["meshcore/+/+/packets", "meshcore/+/+/status"],
"username": "user",
"password": "pass",
"rejectUnauthorized": false,
"iataFilter": ["SJC", "SFO", "OAK"]
}
],
"channelKeys": {
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
},
@@ -94,9 +178,16 @@ Edit `config.json`:
| Field | Description |
|-------|-------------|
| `port` | HTTP server port (default: 3000) |
| `mqtt.broker` | MQTT broker URL. Set to `""` to disable MQTT and use API-only mode |
| `mqtt.broker` | Local MQTT broker URL. Set to `""` to disable |
| `mqtt.topic` | MQTT topic pattern for packet ingestion |
| `channelKeys` | Named channel decryption keys (hex). `public` is the default MeshCore public channel |
| `mqttSources` | Array of external MQTT broker connections (optional) |
| `mqttSources[].name` | Friendly name for logging |
| `mqttSources[].broker` | Broker URL (`mqtt://` or `mqtts://` for TLS) |
| `mqttSources[].topics` | Array of MQTT topic patterns to subscribe to |
| `mqttSources[].username` / `password` | Broker credentials |
| `mqttSources[].rejectUnauthorized` | Set `false` for self-signed TLS certs |
| `mqttSources[].iataFilter` | Only accept packets from these IATA regions |
| `channelKeys` | Named channel decryption keys (hex). Hashtag channels auto-derived via SHA256 |
| `defaultRegion` | Default IATA region code for the UI |
| `regions` | Map of IATA codes to human-readable region names |
@@ -158,17 +249,26 @@ Observer Node → USB → meshcoretomqtt → MQTT Broker → Analyzer Server →
```
meshcore-analyzer/
├── config.json # MQTT, channel keys, regions
├── Dockerfile # Single-container build (Node + Mosquitto + Caddy)
├── .dockerignore
├── config.example.json # Example config (copy to config.json)
├── config.json # MQTT, channel keys, regions (gitignored)
├── server.js # Express + WebSocket + MQTT + REST API
├── decoder.js # Custom MeshCore packet decoder
├── db.js # SQLite schema + queries
├── packet-store.js # In-memory packet store (ring buffer, indexed)
├── docker/
│ ├── supervisord.conf # Process manager config
│ ├── mosquitto.conf # MQTT broker config
│ ├── Caddyfile # Default Caddy config (localhost)
│ └── entrypoint.sh # Container entrypoint
├── data/
│ └── meshcore.db # Packet database (auto-created)
├── public/
│ ├── index.html # SPA shell
│ ├── style.css # Theme (light/dark)
│ ├── app.js # Router, WebSocket, utilities
│ ├── packets.js # Packet feed + byte breakdown
│ ├── packets.js # Packet feed + byte breakdown + detail page
│ ├── map.js # Leaflet map with route visualization
│ ├── live.js # Live trace page with VCR playback
│ ├── channels.js # Channel chat
@@ -176,7 +276,10 @@ meshcore-analyzer/
│ ├── analytics.js # Global analytics dashboard
│ ├── node-analytics.js # Per-node analytics with charts
│ ├── traces.js # Packet tracing
── observers.js # Observer status
── observers.js # Observer status
│ ├── observer-detail.js # Observer detail with analytics
│ ├── home.js # Dashboard home page
│ └── perf.js # Performance monitoring dashboard
└── tools/
├── generate-packets.js # Synthetic packet generator
├── e2e-test.js # End-to-end API tests
+131
View File
@@ -0,0 +1,131 @@
#!/bin/bash
# A/B benchmark: old (pre-perf) vs new (current)
# Usage: ./benchmark-ab.sh
set -e
PORT_OLD=13003
PORT_NEW=13004
RUNS=3
DB_PATH="$(pwd)/data/meshcore.db"
OLD_COMMIT="23caae4"
NEW_COMMIT="$(git rev-parse HEAD)"
echo "═══════════════════════════════════════════════════════"
echo " A/B Benchmark: Pre-optimization vs Current"
echo "═══════════════════════════════════════════════════════"
echo "OLD: $OLD_COMMIT (v2.0.1 — before any perf work)"
echo "NEW: $NEW_COMMIT (current)"
echo "Runs per endpoint: $RUNS"
echo ""
# Get a real node pubkey for testing
ORIG_DIR="$(pwd)"
PUBKEY=$(sqlite3 "$DB_PATH" "SELECT public_key FROM nodes ORDER BY last_seen DESC LIMIT 1")
echo "Test node: ${PUBKEY:0:16}..."
echo ""
# Setup old version in temp dir
OLD_DIR=$(mktemp -d)
echo "Cloning old version to $OLD_DIR..."
git worktree add "$OLD_DIR" "$OLD_COMMIT" --quiet 2>/dev/null || {
git worktree add "$OLD_DIR" "$OLD_COMMIT" --detach --quiet
}
# Copy config + db symlink
# Copy config + db + share node_modules
cp config.json "$OLD_DIR/"
mkdir -p "$OLD_DIR/data"
cp "$ORIG_DIR/data/meshcore.db" "$OLD_DIR/data/meshcore.db"
ln -sf "$ORIG_DIR/node_modules" "$OLD_DIR/node_modules"
ENDPOINTS=(
"Stats|/api/stats"
"Packets(50)|/api/packets?limit=50"
"PacketsGrouped|/api/packets?limit=50&groupByHash=true"
"NodesList|/api/nodes?limit=50"
"NodeDetail|/api/nodes/$PUBKEY"
"NodeHealth|/api/nodes/$PUBKEY/health"
"NodeAnalytics|/api/nodes/$PUBKEY/analytics?days=7"
"BulkHealth|/api/nodes/bulk-health?limit=50"
"NetworkStatus|/api/nodes/network-status"
"Channels|/api/channels"
"Observers|/api/observers"
"RF|/api/analytics/rf"
"Topology|/api/analytics/topology"
"ChannelAnalytics|/api/analytics/channels"
"HashSizes|/api/analytics/hash-sizes"
)
bench_endpoint() {
local port=$1 path=$2 runs=$3 nocache=$4
local total=0
for i in $(seq 1 $runs); do
local url="http://127.0.0.1:$port$path"
if [ "$nocache" = "1" ]; then
if echo "$path" | grep -q '?'; then
url="${url}&nocache=1"
else
url="${url}?nocache=1"
fi
fi
local ms=$(curl -s -o /dev/null -w "%{time_total}" "$url" 2>/dev/null)
local ms_int=$(echo "$ms * 1000" | bc | cut -d. -f1)
total=$((total + ms_int))
done
echo $((total / runs))
}
# Launch old server
echo "Starting OLD server (port $PORT_OLD)..."
cd "$OLD_DIR"
PORT=$PORT_OLD node server.js &>/dev/null &
OLD_PID=$!
cd - >/dev/null
# Launch new server
echo "Starting NEW server (port $PORT_NEW)..."
PORT=$PORT_NEW node server.js &>/dev/null &
NEW_PID=$!
# Wait for both
sleep 12 # old server has no memory store; new needs prewarm
# Verify
curl -s "http://127.0.0.1:$PORT_OLD/api/stats" >/dev/null 2>&1 || { echo "OLD server failed to start"; kill $OLD_PID $NEW_PID 2>/dev/null; exit 1; }
curl -s "http://127.0.0.1:$PORT_NEW/api/stats" >/dev/null 2>&1 || { echo "NEW server failed to start"; kill $OLD_PID $NEW_PID 2>/dev/null; exit 1; }
echo ""
echo "Warming up caches on new server..."
for ep in "${ENDPOINTS[@]}"; do
path="${ep#*|}"
curl -s -o /dev/null "http://127.0.0.1:$PORT_NEW$path" 2>/dev/null
done
sleep 2
printf "\n%-22s %9s %9s %9s %9s\n" "Endpoint" "Old(ms)" "New-cold" "New-cache" "Speedup"
printf "%-22s %9s %9s %9s %9s\n" "──────────────────────" "─────────" "─────────" "─────────" "─────────"
for ep in "${ENDPOINTS[@]}"; do
name="${ep%%|*}"
path="${ep#*|}"
old_ms=$(bench_endpoint $PORT_OLD "$path" $RUNS 0)
new_cold=$(bench_endpoint $PORT_NEW "$path" $RUNS 1)
new_cached=$(bench_endpoint $PORT_NEW "$path" $RUNS 0)
if [ "$old_ms" -gt 0 ] && [ "$new_cached" -gt 0 ]; then
speedup="${old_ms}/${new_cached}"
speedup_x=$(echo "scale=0; $old_ms / $new_cached" | bc 2>/dev/null || echo "?")
printf "%-22s %7dms %7dms %7dms %7d×\n" "$name" "$old_ms" "$new_cold" "$new_cached" "$speedup_x"
else
printf "%-22s %7dms %7dms %7dms %9s\n" "$name" "$old_ms" "$new_cold" "$new_cached" "∞"
fi
done
echo ""
echo "═══════════════════════════════════════════════════════"
# Cleanup
kill $OLD_PID $NEW_PID 2>/dev/null
git worktree remove "$OLD_DIR" --force 2>/dev/null
echo "Done."
+246
View File
@@ -0,0 +1,246 @@
#!/usr/bin/env node
'use strict';
/**
* Benchmark suite for meshcore-analyzer.
* Launches two server instances — one with in-memory store, one with pure SQLite —
* and compares performance side by side.
*
* Usage: node benchmark.js [--runs 5] [--json]
*/
const http = require('http');
const { spawn } = require('child_process');
const path = require('path');
const args = process.argv.slice(2);
const RUNS = Number(args.find((a, i) => args[i - 1] === '--runs') || 5);
const JSON_OUT = args.includes('--json');
const PORT_MEM = 13001; // In-memory store
const PORT_SQL = 13002; // SQLite-only
const ENDPOINTS = [
{ name: 'Stats', path: '/api/stats' },
{ name: 'Packets (50)', path: '/api/packets?limit=50' },
{ name: 'Packets (100)', path: '/api/packets?limit=100' },
{ name: 'Packets grouped', path: '/api/packets?limit=100&groupByHash=true' },
{ name: 'Packets filtered', path: '/api/packets?limit=50&type=5' },
{ name: 'Packets timestamps', path: '/api/packets/timestamps?since=2020-01-01' },
{ name: 'Nodes list', path: '/api/nodes?limit=50' },
{ name: 'Node detail', path: '/api/nodes/__FIRST_NODE__' },
{ name: 'Node health', path: '/api/nodes/__FIRST_NODE__/health' },
{ name: 'Bulk health', path: '/api/nodes/bulk-health?limit=50' },
{ name: 'Network status', path: '/api/nodes/network-status' },
{ name: 'Observers', path: '/api/observers' },
{ name: 'Channels', path: '/api/channels' },
{ name: 'RF Analytics', path: '/api/analytics/rf' },
{ name: 'Topology', path: '/api/analytics/topology' },
{ name: 'Channel Analytics', path: '/api/analytics/channels' },
{ name: 'Hash Sizes', path: '/api/analytics/hash-sizes' },
{ name: 'Subpaths 2-hop', path: '/api/analytics/subpaths?minLen=2&maxLen=2&limit=50' },
{ name: 'Subpaths 3-hop', path: '/api/analytics/subpaths?minLen=3&maxLen=3&limit=30' },
{ name: 'Subpaths 4-hop', path: '/api/analytics/subpaths?minLen=4&maxLen=4&limit=20' },
{ name: 'Subpaths 5-8 hop', path: '/api/analytics/subpaths?minLen=5&maxLen=8&limit=15' },
];
function fetch(url) {
return new Promise((resolve, reject) => {
const t0 = process.hrtime.bigint();
const req = http.get(url, (res) => {
let body = '';
res.on('data', c => body += c);
res.on('end', () => {
const ms = Number(process.hrtime.bigint() - t0) / 1e6;
resolve({ ms, bytes: Buffer.byteLength(body), status: res.statusCode, body });
});
});
req.on('error', reject);
req.setTimeout(60000, () => { req.destroy(); reject(new Error('timeout')); });
});
}
function median(arr) { const s = [...arr].sort((a,b)=>a-b); return s[Math.floor(s.length/2)]; }
function p95(arr) { const s = [...arr].sort((a,b)=>a-b); return s[Math.floor(s.length*0.95)]; }
function avg(arr) { return arr.reduce((a,b)=>a+b,0)/arr.length; }
function fmt(ms) { return ms >= 1000 ? (ms/1000).toFixed(1)+'s' : ms.toFixed(1)+'ms'; }
function fmtSize(b) { return b >= 1048576 ? (b/1048576).toFixed(1)+'MB' : b >= 1024 ? (b/1024).toFixed(0)+'KB' : b+'B'; }
function launchServer(port, env = {}) {
return new Promise((resolve, reject) => {
const child = spawn('node', ['server.js'], {
cwd: __dirname,
env: { ...process.env, PORT: String(port), ...env },
stdio: ['ignore', 'pipe', 'pipe'],
});
let started = false;
const timeout = setTimeout(() => { if (!started) { child.kill(); reject(new Error('Server start timeout')); } }, 30000);
child.stdout.on('data', (d) => {
if (!started && (d.toString().includes('listening') || d.toString().includes('running'))) {
started = true; clearTimeout(timeout); resolve(child);
}
});
child.stderr.on('data', (d) => {
if (!started && (d.toString().includes('listening') || d.toString().includes('running'))) {
started = true; clearTimeout(timeout); resolve(child);
}
});
child.on('exit', (code) => { if (!started) { clearTimeout(timeout); reject(new Error(`Server exited with ${code}`)); } });
// Fallback: wait longer (SQLite-only mode pre-warms subpaths ~6s)
setTimeout(() => {
if (!started) {
started = true; clearTimeout(timeout);
resolve(child);
}
}, 15000);
});
}
async function waitForServer(port, maxMs = 20000) {
const t0 = Date.now();
while (Date.now() - t0 < maxMs) {
try {
const r = await fetch(`http://127.0.0.1:${port}/api/stats`);
if (r.status === 200) return true;
} catch {}
await new Promise(r => setTimeout(r, 500));
}
throw new Error(`Server on port ${port} didn't start`);
}
async function benchmarkEndpoints(port, endpoints, nocache = false) {
const results = [];
for (const ep of endpoints) {
const suffix = nocache ? (ep.path.includes('?') ? '&nocache=1' : '?nocache=1') : '';
const url = `http://127.0.0.1:${port}${ep.path}${suffix}`;
// Warm-up
try { await fetch(url); } catch {}
const times = [];
let bytes = 0;
let failed = false;
for (let i = 0; i < RUNS; i++) {
try {
const r = await fetch(url);
if (r.status !== 200) { failed = true; break; }
times.push(r.ms);
bytes = r.bytes;
} catch { failed = true; break; }
}
if (failed || !times.length) {
results.push({ name: ep.name, failed: true });
} else {
results.push({
name: ep.name,
avg: Math.round(avg(times) * 10) / 10,
p50: Math.round(median(times) * 10) / 10,
p95: Math.round(p95(times) * 10) / 10,
bytes
});
}
}
return results;
}
async function run() {
console.log(`\nMeshCore Analyzer Benchmark — ${RUNS} runs per endpoint`);
console.log('Launching servers...\n');
// Launch both servers
let memServer, sqlServer;
try {
console.log(' Starting in-memory server (port ' + PORT_MEM + ')...');
memServer = await launchServer(PORT_MEM, {});
await waitForServer(PORT_MEM);
console.log(' ✅ In-memory server ready');
console.log(' Starting SQLite-only server (port ' + PORT_SQL + ')...');
sqlServer = await launchServer(PORT_SQL, { NO_MEMORY_STORE: '1' });
await waitForServer(PORT_SQL);
console.log(' ✅ SQLite-only server ready\n');
} catch (e) {
console.error('Failed to start servers:', e.message);
if (memServer) memServer.kill();
if (sqlServer) sqlServer.kill();
process.exit(1);
}
// Get first node pubkey
let firstNode = '';
try {
const r = await fetch(`http://127.0.0.1:${PORT_MEM}/api/nodes?limit=1`);
const data = JSON.parse(r.body);
firstNode = data.nodes?.[0]?.public_key || '';
} catch {}
const endpoints = ENDPOINTS.map(e => ({
...e,
path: e.path.replace('__FIRST_NODE__', firstNode),
}));
// Get packet count
try {
const r = await fetch(`http://127.0.0.1:${PORT_MEM}/api/stats`);
const stats = JSON.parse(r.body);
console.log(`Dataset: ${(stats.totalPackets || '?').toLocaleString()} packets\n`);
} catch {}
// Run benchmarks
console.log('Benchmarking in-memory store (nocache for true compute cost)...');
const memResults = await benchmarkEndpoints(PORT_MEM, endpoints, true);
console.log('Benchmarking SQLite-only (nocache)...');
const sqlResults = await benchmarkEndpoints(PORT_SQL, endpoints, true);
// Also test cached in-memory for the full picture
console.log('Benchmarking in-memory store (cached)...');
const memCachedResults = await benchmarkEndpoints(PORT_MEM, endpoints, false);
// Kill servers
memServer.kill();
sqlServer.kill();
if (JSON_OUT) {
console.log(JSON.stringify({ memoryNocache: memResults, sqliteNocache: sqlResults, memoryCached: memCachedResults }, null, 2));
return;
}
// Print results
const W = 94;
console.log(`\n${'═'.repeat(W)}`);
console.log(' 🏁 BENCHMARK RESULTS: SQLite vs In-Memory Store');
console.log(`${'═'.repeat(W)}`);
console.log(`${'Endpoint'.padEnd(24)} ${'SQLite'.padStart(9)} ${'Memory'.padStart(9)} ${'Cached'.padStart(9)} ${'Speedup'.padStart(9)} ${'Size (SQL)'.padStart(10)} ${'Size (Mem)'.padStart(10)}`);
console.log(`${'─'.repeat(24)} ${'─'.repeat(9)} ${'─'.repeat(9)} ${'─'.repeat(9)} ${'─'.repeat(9)} ${'─'.repeat(10)} ${'─'.repeat(10)}`);
for (let i = 0; i < endpoints.length; i++) {
const sql = sqlResults[i];
const mem = memResults[i];
const cached = memCachedResults[i];
if (!sql || sql.failed || !mem || mem.failed) {
console.log(`${endpoints[i].name.padEnd(24)} ${'FAILED'.padStart(9)}`);
continue;
}
const speedup = sql.avg > 0 && mem.avg > 0 ? Math.round(sql.avg / mem.avg) + '×' : '—';
const cachedStr = cached && !cached.failed ? fmt(cached.avg) : '—';
console.log(
`${sql.name.padEnd(24)} ${fmt(sql.avg).padStart(9)} ${fmt(mem.avg).padStart(9)} ${cachedStr.padStart(9)} ${speedup.padStart(9)} ${fmtSize(sql.bytes).padStart(10)} ${fmtSize(mem.bytes).padStart(10)}`
);
}
// Summary
const sqlTotal = sqlResults.filter(r => !r.failed).reduce((s, r) => s + r.avg, 0);
const memTotal = memResults.filter(r => !r.failed).reduce((s, r) => s + r.avg, 0);
console.log(`${'─'.repeat(24)} ${'─'.repeat(9)} ${'─'.repeat(9)} ${'─'.repeat(9)} ${'─'.repeat(9)}`);
console.log(`${'TOTAL'.padEnd(24)} ${fmt(sqlTotal).padStart(9)} ${fmt(memTotal).padStart(9)} ${''.padStart(9)} ${(Math.round(sqlTotal/memTotal)+'×').padStart(9)}`);
console.log(`\n${'═'.repeat(W)}\n`);
}
run().catch(e => { console.error(e); process.exit(1); });
+56 -2
View File
@@ -4,6 +4,35 @@
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
},
"mqttSources": [
{
"name": "local",
"broker": "mqtt://localhost:1883",
"topics": [
"meshcore/+/+/packets",
"meshcore/#"
]
},
{
"name": "lincomatic",
"broker": "mqtts://mqtt.lincomatic.com:8883",
"username": "your-username",
"password": "your-password",
"rejectUnauthorized": false,
"topics": [
"meshcore/SJC/#",
"meshcore/SFO/#",
"meshcore/OAK/#",
"meshcore/MRY/#"
],
"iataFilter": [
"SJC",
"SFO",
"OAK",
"MRY"
]
}
],
"channelKeys": {
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72",
"#test": "9cd8fcf22a47333b591d96a2b848b73f",
@@ -20,7 +49,32 @@
"SJC": "San Jose, US",
"SFO": "San Francisco, US",
"OAK": "Oakland, US",
"MRY": "Monterey, US",
"LAR": "Los Angeles, US"
"MRY": "Monterey, US"
},
"cacheTTL": {
"stats": 10,
"nodeDetail": 300,
"nodeHealth": 300,
"nodeList": 90,
"bulkHealth": 600,
"networkStatus": 600,
"observers": 300,
"channels": 15,
"channelMessages": 10,
"analyticsRF": 1800,
"analyticsTopology": 1800,
"analyticsChannels": 1800,
"analyticsHashSizes": 3600,
"analyticsSubpaths": 3600,
"analyticsSubpathDetail": 3600,
"nodeAnalytics": 60,
"nodeSearch": 10,
"invalidationDebounce": 30,
"_comment": "All values in seconds. Server uses these directly. Client fetches via /api/config/cache."
},
"packetStore": {
"maxMemoryMB": 1024,
"estimatedPacketBytes": 450,
"_comment": "In-memory packet store. maxMemoryMB caps RAM usage. All packets loaded on startup, served from RAM."
}
}
+172 -13
View File
@@ -49,7 +49,14 @@ db.exec(`
iata TEXT,
last_seen TEXT,
first_seen TEXT,
packet_count INTEGER DEFAULT 0
packet_count INTEGER DEFAULT 0,
model TEXT,
firmware TEXT,
client_version TEXT,
radio TEXT,
battery_mv INTEGER,
uptime_secs INTEGER,
noise_floor INTEGER
);
CREATE TABLE IF NOT EXISTS paths (
@@ -64,8 +71,53 @@ db.exec(`
CREATE INDEX IF NOT EXISTS idx_packets_payload_type ON packets(payload_type);
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
CREATE INDEX IF NOT EXISTS idx_observers_last_seen ON observers(last_seen);
CREATE TABLE IF NOT EXISTS transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
hash TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_transmissions_hash ON transmissions(hash);
CREATE INDEX IF NOT EXISTS idx_transmissions_first_seen ON transmissions(first_seen);
CREATE INDEX IF NOT EXISTS idx_transmissions_payload_type ON transmissions(payload_type);
CREATE INDEX IF NOT EXISTS idx_observations_hash ON observations(hash);
CREATE INDEX IF NOT EXISTS idx_observations_transmission_id ON observations(transmission_id);
CREATE INDEX IF NOT EXISTS idx_observations_observer_id ON observations(observer_id);
CREATE INDEX IF NOT EXISTS idx_observations_timestamp ON observations(timestamp);
`);
// --- Migrations for existing DBs ---
const observerCols = db.pragma('table_info(observers)').map(c => c.name);
for (const col of ['model', 'firmware', 'client_version', 'radio', 'battery_mv', 'uptime_secs', 'noise_floor']) {
if (!observerCols.includes(col)) {
const type = ['battery_mv', 'uptime_secs', 'noise_floor'].includes(col) ? 'INTEGER' : 'TEXT';
db.exec(`ALTER TABLE observers ADD COLUMN ${col} ${type}`);
console.log(`[migration] Added observers.${col}`);
}
}
// --- Prepared statements ---
const stmts = {
insertPacket: db.prepare(`
@@ -85,13 +137,35 @@ const stmts = {
advert_count = advert_count + 1
`),
upsertObserver: db.prepare(`
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count)
VALUES (@id, @name, @iata, @last_seen, @first_seen, 1)
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor)
VALUES (@id, @name, @iata, @last_seen, @first_seen, 1, @model, @firmware, @client_version, @radio, @battery_mv, @uptime_secs, @noise_floor)
ON CONFLICT(id) DO UPDATE SET
name = COALESCE(@name, name),
iata = COALESCE(@iata, iata),
last_seen = @last_seen,
packet_count = packet_count + 1
packet_count = packet_count + 1,
model = COALESCE(@model, model),
firmware = COALESCE(@firmware, firmware),
client_version = COALESCE(@client_version, client_version),
radio = COALESCE(@radio, radio),
battery_mv = COALESCE(@battery_mv, battery_mv),
uptime_secs = COALESCE(@uptime_secs, uptime_secs),
noise_floor = COALESCE(@noise_floor, noise_floor)
`),
updateObserverStatus: db.prepare(`
INSERT INTO observers (id, name, iata, last_seen, first_seen, packet_count, model, firmware, client_version, radio, battery_mv, uptime_secs, noise_floor)
VALUES (@id, @name, @iata, @last_seen, @first_seen, 0, @model, @firmware, @client_version, @radio, @battery_mv, @uptime_secs, @noise_floor)
ON CONFLICT(id) DO UPDATE SET
name = COALESCE(@name, name),
iata = COALESCE(@iata, iata),
last_seen = @last_seen,
model = COALESCE(@model, model),
firmware = COALESCE(@firmware, firmware),
client_version = COALESCE(@client_version, client_version),
radio = COALESCE(@radio, radio),
battery_mv = COALESCE(@battery_mv, battery_mv),
uptime_secs = COALESCE(@uptime_secs, uptime_secs),
noise_floor = COALESCE(@noise_floor, noise_floor)
`),
getPacket: db.prepare(`SELECT * FROM packets WHERE id = ?`),
getPathsForPacket: db.prepare(`SELECT * FROM paths WHERE packet_id = ? ORDER BY hop_index`),
@@ -105,6 +179,16 @@ const stmts = {
countNodes: db.prepare(`SELECT COUNT(*) as count FROM nodes`),
countObservers: db.prepare(`SELECT COUNT(*) as count FROM observers`),
countRecentPackets: db.prepare(`SELECT COUNT(*) as count FROM packets WHERE timestamp > ?`),
getTransmissionByHash: db.prepare(`SELECT id, first_seen FROM transmissions WHERE hash = ?`),
insertTransmission: db.prepare(`
INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json)
VALUES (@raw_hex, @hash, @first_seen, @route_type, @payload_type, @payload_version, @decoded_json)
`),
updateTransmissionFirstSeen: db.prepare(`UPDATE transmissions SET first_seen = @first_seen WHERE id = @id`),
insertObservation: db.prepare(`
INSERT INTO observations (transmission_id, hash, observer_id, observer_name, direction, snr, rssi, score, path_json, timestamp)
VALUES (@transmission_id, @hash, @observer_id, @observer_name, @direction, @snr, @rssi, @score, @path_json, @timestamp)
`),
};
// --- Helper functions ---
@@ -129,6 +213,49 @@ function insertPacket(data) {
return stmts.insertPacket.run(d).lastInsertRowid;
}
function insertTransmission(data) {
const hash = data.hash;
if (!hash) return null; // Can't deduplicate without a hash
const timestamp = data.timestamp || new Date().toISOString();
let transmissionId;
const existing = stmts.getTransmissionByHash.get(hash);
if (existing) {
transmissionId = existing.id;
// Update first_seen if this observation is earlier
if (timestamp < existing.first_seen) {
stmts.updateTransmissionFirstSeen.run({ id: transmissionId, first_seen: timestamp });
}
} else {
const result = stmts.insertTransmission.run({
raw_hex: data.raw_hex || '',
hash,
first_seen: timestamp,
route_type: data.route_type ?? null,
payload_type: data.payload_type ?? null,
payload_version: data.payload_version ?? null,
decoded_json: data.decoded_json || null,
});
transmissionId = result.lastInsertRowid;
}
const obsResult = stmts.insertObservation.run({
transmission_id: transmissionId,
hash,
observer_id: data.observer_id || null,
observer_name: data.observer_name || null,
direction: data.direction || null,
snr: data.snr ?? null,
rssi: data.rssi ?? null,
score: data.score ?? null,
path_json: data.path_json || null,
timestamp,
});
return { transmissionId, observationId: obsResult.lastInsertRowid };
}
function insertPath(packetId, hops) {
const tx = db.transaction((hops) => {
for (let i = 0; i < hops.length; i++) {
@@ -159,6 +286,31 @@ function upsertObserver(data) {
iata: data.iata || null,
last_seen: data.last_seen || now,
first_seen: data.first_seen || now,
model: data.model || null,
firmware: data.firmware || null,
client_version: data.client_version || null,
radio: data.radio || null,
battery_mv: data.battery_mv || null,
uptime_secs: data.uptime_secs || null,
noise_floor: data.noise_floor || null,
});
}
function updateObserverStatus(data) {
const now = new Date().toISOString();
stmts.updateObserverStatus.run({
id: data.id,
name: data.name || null,
iata: data.iata || null,
last_seen: data.last_seen || now,
first_seen: data.first_seen || now,
model: data.model || null,
firmware: data.firmware || null,
client_version: data.client_version || null,
radio: data.radio || null,
battery_mv: data.battery_mv || null,
uptime_secs: data.uptime_secs || null,
noise_floor: data.noise_floor || null,
});
}
@@ -212,8 +364,15 @@ function getObservers() {
function getStats() {
const oneHourAgo = new Date(Date.now() - 3600000).toISOString();
// Try to get transmission count from normalized schema
let totalTransmissions = null;
try {
totalTransmissions = db.prepare('SELECT COUNT(*) as count FROM transmissions').get().count;
} catch {}
return {
totalPackets: stmts.countPackets.get().count,
totalTransmissions,
totalObservations: stmts.countPackets.get().count, // legacy packets = observations
totalNodes: stmts.countNodes.get().count,
totalObservers: stmts.countObservers.get().count,
packetsLastHour: stmts.countRecentPackets.get(oneHourAgo).count,
@@ -225,13 +384,13 @@ function seed() {
const now = new Date().toISOString();
const rawHex = '11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172';
upsertObserver({ id: 'obs-sjc-001', name: 'Iavor Observer', iata: 'SJC', last_seen: now, first_seen: now });
upsertObserver({ id: 'obs-seed-001', name: 'Seed Observer', iata: 'UNK', last_seen: now, first_seen: now });
const pktId = insertPacket({
raw_hex: rawHex,
timestamp: now,
observer_id: 'obs-sjc-001',
observer_name: 'Iavor Observer',
observer_id: 'obs-seed-001',
observer_name: 'Seed Observer',
direction: 'rx',
snr: 10.5,
rssi: -85,
@@ -241,17 +400,17 @@ function seed() {
payload_type: 4,
payload_version: 1,
path_json: JSON.stringify(['A1B2', 'C3D4']),
decoded_json: JSON.stringify({ type: 'ADVERT', name: 'Kpa Roof Solar', role: 'repeater', lat: 37.31468, lon: -121.8921 }),
decoded_json: JSON.stringify({ type: 'ADVERT', name: 'Test Repeater', role: 'repeater', lat: 0, lon: 0 }),
});
insertPath(pktId, ['A1B2', 'C3D4']);
upsertNode({
public_key: 'kpa-roof-solar-pubkey',
name: 'Kpa Roof Solar',
public_key: 'seed-test-pubkey',
name: 'Test Repeater',
role: 'repeater',
lat: 37.31468,
lon: -121.8921,
lat: 0,
lon: 0,
last_seen: now,
first_seen: now,
});
@@ -493,4 +652,4 @@ function getNodeAnalytics(pubkey, days) {
};
}
module.exports = { db, insertPacket, insertPath, upsertNode, upsertObserver, getPackets, getPacket, getNodes, getNode, getObservers, getStats, seed, searchNodes, getNodeHealth, getNodeAnalytics };
module.exports = { db, insertPacket, insertTransmission, insertPath, upsertNode, upsertObserver, updateObserverStatus, getPackets, getPacket, getNodes, getNode, getObservers, getStats, seed, searchNodes, getNodeHealth, getNodeAnalytics };
+2 -2
View File
@@ -269,7 +269,7 @@ module.exports = { decodePacket, ROUTE_TYPES, PAYLOAD_TYPES };
// --- Tests ---
if (require.main === module) {
console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Kpa Roof Solar" ===');
console.log('=== Test 1: ADVERT, FLOOD, 5 hops (2-byte hashes), "Test Repeater" ===');
const pkt1 = decodePacket(
'11451000D818206D3AAC152C8A91F89957E6D30CA51F36E28790228971C473B755F244F718754CF5EE4A2FD58D944466E42CDED140C66D0CC590183E32BAF40F112BE8F3F2BDF6012B4B2793C52F1D36F69EE054D9A05593286F78453E56C0EC4A3EB95DDA2A7543FCCC00B939CACC009278603902FC12BCF84B706120526F6F6620536F6C6172'
);
@@ -285,7 +285,7 @@ if (require.main === module) {
assert(pkt1.path.hops[0] === '1000', 'first hop should be 1000');
assert(pkt1.path.hops[1] === 'D818', 'second hop should be D818');
assert(pkt1.transportCodes === null, 'FLOOD has no transport codes');
assert(pkt1.payload.name === 'Kpa Roof Solar', 'name should be "Kpa Roof Solar"');
assert(pkt1.payload.name === 'Test Repeater', 'name should be "Test Repeater"');
console.log('✅ Test 1 passed\n');
console.log('=== Test 2: ADVERT, FLOOD, 0 hops (zero-path) ===');
+11
View File
@@ -0,0 +1,11 @@
# Default Caddyfile — reverse proxy to Node app
# Override by mounting your own: -v ./Caddyfile:/etc/caddy/Caddyfile
#
# For automatic HTTPS, replace :80 with your domain:
# analyzer.example.com {
# reverse_proxy localhost:3000
# }
:80 {
reverse_proxy localhost:3000
}
+9
View File
@@ -0,0 +1,9 @@
#!/bin/sh
# Copy example config if no config.json exists
if [ ! -f /app/config.json ]; then
echo "[entrypoint] No config.json found, copying from config.example.json"
cp /app/config.example.json /app/config.json
fi
exec /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf
+10
View File
@@ -0,0 +1,10 @@
# Mosquitto config for MeshCore Analyzer
listener 1883 0.0.0.0
allow_anonymous true
persistence true
persistence_location /var/lib/mosquitto/
# Logging
log_dest stdout
log_type warning
log_type error
+36
View File
@@ -0,0 +1,36 @@
[supervisord]
nodaemon=true
user=root
logfile=/dev/stdout
logfile_maxbytes=0
pidfile=/var/run/supervisord.pid
[program:mosquitto]
command=/usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
[program:meshcore-analyzer]
command=node /app/server.js
directory=/app
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
environment=NODE_ENV="production"
[program:caddy]
command=/usr/sbin/caddy run --config /etc/caddy/Caddyfile
environment=XDG_DATA_HOME="/data"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "meshcore-analyzer",
"version": "2.0.0",
"version": "2.1.0",
"description": "Community-run alternative to the closed-source `analyzer.letsmesh.net`. MQTT packet collection + open-source web analyzer for the Bay Area MeshCore mesh.",
"main": "index.js",
"scripts": {
+600
View File
@@ -0,0 +1,600 @@
'use strict';
/**
* In-memory packet store — loads transmissions + observations from SQLite on startup,
* serves reads from RAM, writes to both RAM + SQLite.
* M3: Restructured around transmissions (deduped by hash) with observations.
* Caps memory at configurable limit (default 1GB).
*/
class PacketStore {
constructor(dbModule, config = {}) {
this.dbModule = dbModule; // The full db module (has .db, .insertPacket, .getPacket)
this.db = dbModule.db; // Raw better-sqlite3 instance for queries
this.maxBytes = (config.maxMemoryMB || 1024) * 1024 * 1024;
this.estPacketBytes = config.estimatedPacketBytes || 450;
this.maxPackets = Math.floor(this.maxBytes / this.estPacketBytes);
// SQLite-only mode: skip RAM loading, all reads go to DB
this.sqliteOnly = process.env.NO_MEMORY_STORE === '1';
// Primary storage: transmissions sorted by first_seen DESC (newest first)
// Each transmission looks like a packet for backward compat
this.packets = [];
// Indexes
this.byId = new Map(); // observation_id → observation object (backward compat for packet detail links)
this.byHash = new Map(); // hash → transmission object (1:1)
this.byObserver = new Map(); // observer_id → [observation objects]
this.byNode = new Map(); // pubkey → [transmission objects] (deduped)
this.byTransmission = new Map(); // hash → transmission object (same refs as byHash)
// Track which hashes are indexed per node pubkey (avoid dupes in byNode)
this._nodeHashIndex = new Map(); // pubkey → Set<hash>
this.loaded = false;
this.stats = { totalLoaded: 0, totalObservations: 0, evicted: 0, inserts: 0, queries: 0 };
}
/** Load all packets from SQLite into memory */
load() {
if (this.sqliteOnly) {
console.log('[PacketStore] SQLite-only mode (NO_MEMORY_STORE=1) — all reads go to database');
this.loaded = true;
return this;
}
const t0 = Date.now();
// Check if normalized schema exists
const hasTransmissions = this.db.prepare(
"SELECT name FROM sqlite_master WHERE type='table' AND name='transmissions'"
).get();
if (hasTransmissions) {
this._loadNormalized();
} else {
this._loadLegacy();
}
this.stats.totalLoaded = this.packets.length;
this.loaded = true;
const elapsed = Date.now() - t0;
console.log(`[PacketStore] Loaded ${this.packets.length} transmissions (${this.stats.totalObservations} observations) in ${elapsed}ms (${Math.round(this.packets.length * this.estPacketBytes / 1024 / 1024)}MB est)`);
return this;
}
/** Load from normalized transmissions + observations tables */
_loadNormalized() {
const rows = this.db.prepare(`
SELECT t.id AS transmission_id, t.raw_hex, t.hash, t.first_seen, t.route_type,
t.payload_type, t.payload_version, t.decoded_json,
o.id AS observation_id, o.observer_id, o.observer_name, o.direction,
o.snr, o.rssi, o.score, o.path_json, o.timestamp AS obs_timestamp
FROM transmissions t
LEFT JOIN observations o ON o.transmission_id = t.id
ORDER BY t.first_seen DESC, o.timestamp DESC
`).all();
for (const row of rows) {
if (this.packets.length >= this.maxPackets && !this.byTransmission.has(row.hash)) break;
let tx = this.byTransmission.get(row.hash);
if (!tx) {
tx = {
id: row.transmission_id,
raw_hex: row.raw_hex,
hash: row.hash,
first_seen: row.first_seen,
timestamp: row.first_seen,
route_type: row.route_type,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
observations: [],
observation_count: 0,
// Filled from first observation for backward compat
observer_id: null,
observer_name: null,
snr: null,
rssi: null,
path_json: null,
direction: null,
};
this.byTransmission.set(row.hash, tx);
this.byHash.set(row.hash, tx);
this.packets.push(tx);
this._indexByNode(tx);
}
if (row.observation_id != null) {
const obs = {
id: row.observation_id,
observer_id: row.observer_id,
observer_name: row.observer_name,
direction: row.direction,
snr: row.snr,
rssi: row.rssi,
score: row.score,
path_json: row.path_json,
timestamp: row.obs_timestamp,
// Carry transmission fields for backward compat
hash: row.hash,
raw_hex: row.raw_hex,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
route_type: row.route_type,
};
tx.observations.push(obs);
tx.observation_count++;
// Fill first observation data into transmission for backward compat
if (tx.observer_id == null && obs.observer_id) {
tx.observer_id = obs.observer_id;
tx.observer_name = obs.observer_name;
tx.snr = obs.snr;
tx.rssi = obs.rssi;
tx.path_json = obs.path_json;
tx.direction = obs.direction;
}
// byId maps observation IDs for packet detail links
this.byId.set(obs.id, obs);
// byObserver
if (obs.observer_id) {
if (!this.byObserver.has(obs.observer_id)) this.byObserver.set(obs.observer_id, []);
this.byObserver.get(obs.observer_id).push(obs);
}
this.stats.totalObservations++;
}
}
}
/** Fallback: load from legacy packets table */
_loadLegacy() {
const rows = this.db.prepare(
'SELECT * FROM packets ORDER BY timestamp DESC'
).all();
for (const row of rows) {
if (this.packets.length >= this.maxPackets) break;
this._indexLegacy(row);
}
}
/** Index a legacy packet row (old flat structure) — builds transmission + observation */
_indexLegacy(pkt) {
let tx = this.byTransmission.get(pkt.hash);
if (!tx) {
tx = {
id: pkt.id,
raw_hex: pkt.raw_hex,
hash: pkt.hash,
first_seen: pkt.timestamp,
timestamp: pkt.timestamp,
route_type: pkt.route_type,
payload_type: pkt.payload_type,
decoded_json: pkt.decoded_json,
observations: [],
observation_count: 0,
observer_id: pkt.observer_id,
observer_name: pkt.observer_name,
snr: pkt.snr,
rssi: pkt.rssi,
path_json: pkt.path_json,
direction: pkt.direction,
};
this.byTransmission.set(pkt.hash, tx);
this.byHash.set(pkt.hash, tx);
this.packets.push(tx);
this._indexByNode(tx);
}
if (pkt.timestamp < tx.first_seen) {
tx.first_seen = pkt.timestamp;
tx.timestamp = pkt.timestamp;
}
const obs = {
id: pkt.id,
observer_id: pkt.observer_id,
observer_name: pkt.observer_name,
direction: pkt.direction,
snr: pkt.snr,
rssi: pkt.rssi,
score: pkt.score,
path_json: pkt.path_json,
timestamp: pkt.timestamp,
hash: pkt.hash,
raw_hex: pkt.raw_hex,
payload_type: pkt.payload_type,
decoded_json: pkt.decoded_json,
route_type: pkt.route_type,
};
tx.observations.push(obs);
tx.observation_count++;
this.byId.set(pkt.id, obs);
if (pkt.observer_id) {
if (!this.byObserver.has(pkt.observer_id)) this.byObserver.set(pkt.observer_id, []);
this.byObserver.get(pkt.observer_id).push(obs);
}
this.stats.totalObservations++;
}
/** Extract node pubkeys from decoded_json and index transmission in byNode */
_indexByNode(tx) {
if (!tx.decoded_json) return;
try {
const decoded = JSON.parse(tx.decoded_json);
const keys = new Set();
if (decoded.pubKey) keys.add(decoded.pubKey);
if (decoded.destPubKey) keys.add(decoded.destPubKey);
if (decoded.srcPubKey) keys.add(decoded.srcPubKey);
for (const k of keys) {
if (!this._nodeHashIndex.has(k)) this._nodeHashIndex.set(k, new Set());
if (this._nodeHashIndex.get(k).has(tx.hash)) continue; // already indexed
this._nodeHashIndex.get(k).add(tx.hash);
if (!this.byNode.has(k)) this.byNode.set(k, []);
this.byNode.get(k).push(tx);
}
} catch {}
}
/** Remove oldest transmissions when over memory limit */
_evict() {
while (this.packets.length > this.maxPackets) {
const old = this.packets.pop();
this.byHash.delete(old.hash);
this.byTransmission.delete(old.hash);
// Remove observations from byId and byObserver
for (const obs of old.observations) {
this.byId.delete(obs.id);
if (obs.observer_id && this.byObserver.has(obs.observer_id)) {
const arr = this.byObserver.get(obs.observer_id).filter(o => o.id !== obs.id);
if (arr.length) this.byObserver.set(obs.observer_id, arr); else this.byObserver.delete(obs.observer_id);
}
}
// Skip node index cleanup (expensive, low value)
this.stats.evicted++;
}
}
/** Insert a new packet (to both memory and SQLite) */
insert(packetData) {
const id = this.dbModule.insertPacket(packetData);
const row = this.dbModule.getPacket(id);
if (row && !this.sqliteOnly) {
// Update or create transmission in memory
let tx = this.byTransmission.get(row.hash);
if (!tx) {
tx = {
id: row.id,
raw_hex: row.raw_hex,
hash: row.hash,
first_seen: row.timestamp,
timestamp: row.timestamp,
route_type: row.route_type,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
observations: [],
observation_count: 0,
observer_id: row.observer_id,
observer_name: row.observer_name,
snr: row.snr,
rssi: row.rssi,
path_json: row.path_json,
direction: row.direction,
};
this.byTransmission.set(row.hash, tx);
this.byHash.set(row.hash, tx);
this.packets.unshift(tx); // newest first
this._indexByNode(tx);
} else {
// Update first_seen if earlier
if (row.timestamp < tx.first_seen) {
tx.first_seen = row.timestamp;
tx.timestamp = row.timestamp;
}
}
// Add observation
const obs = {
id: row.id,
observer_id: row.observer_id,
observer_name: row.observer_name,
direction: row.direction,
snr: row.snr,
rssi: row.rssi,
score: row.score,
path_json: row.path_json,
timestamp: row.timestamp,
hash: row.hash,
raw_hex: row.raw_hex,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
route_type: row.route_type,
};
tx.observations.push(obs);
tx.observation_count++;
// Update transmission's display fields if this is first observation
if (tx.observations.length === 1) {
tx.observer_id = obs.observer_id;
tx.observer_name = obs.observer_name;
tx.snr = obs.snr;
tx.rssi = obs.rssi;
tx.path_json = obs.path_json;
}
this.byId.set(obs.id, obs);
if (obs.observer_id) {
if (!this.byObserver.has(obs.observer_id)) this.byObserver.set(obs.observer_id, []);
this.byObserver.get(obs.observer_id).push(obs);
}
this.stats.totalObservations++;
this._evict();
this.stats.inserts++;
}
return id;
}
/**
* Find ALL packets referencing a node — by pubkey index + name + pubkey text search.
* Returns unique transmissions (deduped).
* @param {string} nodeIdOrName - pubkey or friendly name
* @param {Array} [fromPackets] - packet array to filter (defaults to this.packets)
* @returns {{ packets: Array, pubkey: string, nodeName: string }}
*/
findPacketsForNode(nodeIdOrName, fromPackets) {
let pubkey = nodeIdOrName;
let nodeName = nodeIdOrName;
// Always resolve to get both pubkey and name
try {
const row = this.db.prepare("SELECT public_key, name FROM nodes WHERE public_key = ? OR name = ? LIMIT 1").get(nodeIdOrName, nodeIdOrName);
if (row) { pubkey = row.public_key; nodeName = row.name || nodeIdOrName; }
} catch {}
// Combine: index hits + text search
const indexed = this.byNode.get(pubkey);
const hashSet = indexed ? new Set(indexed.map(t => t.hash)) : new Set();
const source = fromPackets || this.packets;
const packets = source.filter(t =>
hashSet.has(t.hash) ||
(t.decoded_json && (t.decoded_json.includes(nodeName) || t.decoded_json.includes(pubkey)))
);
return { packets, pubkey, nodeName };
}
/** Count transmissions and observations for a node */
countForNode(pubkey) {
const txs = this.byNode.get(pubkey) || [];
let observations = 0;
for (const tx of txs) observations += tx.observation_count;
return { transmissions: txs.length, observations };
}
/** Query packets with filters — all from memory (or SQLite in fallback mode) */
query({ limit = 50, offset = 0, type, route, region, observer, hash, since, until, node, order = 'DESC' } = {}) {
this.stats.queries++;
if (this.sqliteOnly) return this._querySQLite({ limit, offset, type, route, region, observer, hash, since, until, node, order });
let results = this.packets;
// Use indexes for single-key filters when possible
if (hash && !type && !route && !region && !observer && !since && !until && !node) {
const tx = this.byHash.get(hash);
results = tx ? [tx] : [];
} else if (observer && !type && !route && !region && !hash && !since && !until && !node) {
// For observer filter, find unique transmissions where any observation matches
results = this._transmissionsForObserver(observer);
} else if (node && !type && !route && !region && !observer && !hash && !since && !until) {
results = this.findPacketsForNode(node).packets;
} else {
// Apply filters sequentially
if (type !== undefined) {
const t = Number(type);
results = results.filter(p => p.payload_type === t);
}
if (route !== undefined) {
const r = Number(route);
results = results.filter(p => p.route_type === r);
}
if (observer) results = this._transmissionsForObserver(observer, results);
if (hash) {
const tx = this.byHash.get(hash);
results = tx ? results.filter(p => p.hash === hash) : [];
}
if (since) results = results.filter(p => p.timestamp > since);
if (until) results = results.filter(p => p.timestamp < until);
if (region) {
const regionObservers = new Set();
try {
const obs = this.db.prepare('SELECT id FROM observers WHERE iata = ?').all(region);
obs.forEach(o => regionObservers.add(o.id));
} catch {}
results = results.filter(p =>
p.observations.some(o => regionObservers.has(o.observer_id))
);
}
if (node) {
results = this.findPacketsForNode(node, results).packets;
}
}
const total = results.length;
// Sort
if (order === 'ASC') {
results = results.slice().sort((a, b) => {
if (a.timestamp < b.timestamp) return -1;
if (a.timestamp > b.timestamp) return 1;
return 0;
});
}
// Default DESC — packets array is already sorted newest-first
// Paginate
const paginated = results.slice(Number(offset), Number(offset) + Number(limit));
return { packets: paginated, total };
}
/** Find unique transmissions that have at least one observation from given observer */
_transmissionsForObserver(observerId, fromTransmissions) {
if (fromTransmissions) {
return fromTransmissions.filter(tx =>
tx.observations.some(o => o.observer_id === observerId)
);
}
// Use byObserver index: get observations, then unique transmissions
const obs = this.byObserver.get(observerId) || [];
const seen = new Set();
const result = [];
for (const o of obs) {
if (!seen.has(o.hash)) {
seen.add(o.hash);
const tx = this.byTransmission.get(o.hash);
if (tx) result.push(tx);
}
}
return result;
}
/** Query with groupByHash — now trivial since packets ARE transmissions */
queryGrouped({ limit = 50, offset = 0, type, route, region, observer, hash, since, until, node } = {}) {
this.stats.queries++;
if (this.sqliteOnly) return this._queryGroupedSQLite({ limit, offset, type, route, region, observer, hash, since, until, node });
// Get filtered transmissions
const { packets: filtered, total: filteredTotal } = this.query({
limit: 999999, offset: 0, type, route, region, observer, hash, since, until, node
});
// Already grouped by hash — just format for backward compat
const sorted = filtered.map(tx => ({
hash: tx.hash,
count: tx.observation_count,
observer_count: new Set(tx.observations.map(o => o.observer_id).filter(Boolean)).size,
latest: tx.observations.length ? tx.observations.reduce((max, o) => o.timestamp > max ? o.timestamp : max, tx.observations[0].timestamp) : tx.timestamp,
observer_id: tx.observer_id,
observer_name: tx.observer_name,
path_json: tx.path_json,
payload_type: tx.payload_type,
raw_hex: tx.raw_hex,
decoded_json: tx.decoded_json,
observation_count: tx.observation_count,
})).sort((a, b) => b.latest.localeCompare(a.latest));
const total = sorted.length;
const paginated = sorted.slice(Number(offset), Number(offset) + Number(limit));
return { packets: paginated, total };
}
/** Get timestamps for sparkline */
getTimestamps(since) {
if (this.sqliteOnly) {
return this.db.prepare('SELECT timestamp FROM packets WHERE timestamp > ? ORDER BY timestamp ASC').all(since).map(r => r.timestamp);
}
const results = [];
for (const p of this.packets) {
if (p.timestamp <= since) break;
results.push(p.timestamp);
}
return results.reverse();
}
/** Get a single packet by ID — checks observation IDs first (backward compat) */
getById(id) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets WHERE id = ?').get(id) || null;
return this.byId.get(id) || null;
}
/** Get all siblings of a packet (same hash) — returns observations array */
getSiblings(hash) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets WHERE hash = ? ORDER BY timestamp DESC').all(hash);
const tx = this.byTransmission.get(hash);
return tx ? tx.observations : [];
}
/** Get all transmissions (backward compat — returns packets array) */
all() {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets ORDER BY timestamp DESC').all();
return this.packets;
}
/** Get all transmissions matching a filter function */
filter(fn) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets ORDER BY timestamp DESC').all().filter(fn);
return this.packets.filter(fn);
}
/** Memory stats */
getStats() {
return {
...this.stats,
inMemory: this.sqliteOnly ? 0 : this.packets.length,
sqliteOnly: this.sqliteOnly,
maxPackets: this.maxPackets,
estimatedMB: this.sqliteOnly ? 0 : Math.round(this.packets.length * this.estPacketBytes / 1024 / 1024),
maxMB: Math.round(this.maxBytes / 1024 / 1024),
indexes: {
byHash: this.byHash.size,
byObserver: this.byObserver.size,
byNode: this.byNode.size,
byTransmission: this.byTransmission.size,
}
};
}
/** SQLite fallback: query with filters */
_querySQLite({ limit, offset, type, route, region, observer, hash, since, until, node, order }) {
const where = []; const params = [];
if (type !== undefined) { where.push('payload_type = ?'); params.push(Number(type)); }
if (route !== undefined) { where.push('route_type = ?'); params.push(Number(route)); }
if (observer) { where.push('observer_id = ?'); params.push(observer); }
if (hash) { where.push('hash = ?'); params.push(hash); }
if (since) { where.push('timestamp > ?'); params.push(since); }
if (until) { where.push('timestamp < ?'); params.push(until); }
if (region) { where.push('observer_id IN (SELECT id FROM observers WHERE iata = ?)'); params.push(region); }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('(decoded_json LIKE ? OR id IN (SELECT packet_id FROM paths WHERE node_hash = ?))'); params.push('%' + pk + '%', pk.substring(0, 8)); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
const w = where.length ? 'WHERE ' + where.join(' AND ') : '';
const total = this.db.prepare(`SELECT COUNT(*) as c FROM packets ${w}`).get(...params).c;
const packets = this.db.prepare(`SELECT * FROM packets ${w} ORDER BY timestamp ${order === 'ASC' ? 'ASC' : 'DESC'} LIMIT ? OFFSET ?`).all(...params, limit, offset);
return { packets, total };
}
/** SQLite fallback: grouped query */
_queryGroupedSQLite({ limit, offset, type, route, region, observer, hash, since, until, node }) {
const where = []; const params = [];
if (type !== undefined) { where.push('payload_type = ?'); params.push(Number(type)); }
if (route !== undefined) { where.push('route_type = ?'); params.push(Number(route)); }
if (observer) { where.push('observer_id = ?'); params.push(observer); }
if (hash) { where.push('hash = ?'); params.push(hash); }
if (since) { where.push('timestamp > ?'); params.push(since); }
if (until) { where.push('timestamp < ?'); params.push(until); }
if (region) { where.push('observer_id IN (SELECT id FROM observers WHERE iata = ?)'); params.push(region); }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('(decoded_json LIKE ? OR id IN (SELECT packet_id FROM paths WHERE node_hash = ?))'); params.push('%' + pk + '%', pk.substring(0, 8)); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
const w = where.length ? 'WHERE ' + where.join(' AND ') : '';
const sql = `SELECT hash, COUNT(*) as count, COUNT(DISTINCT observer_id) as observer_count,
MAX(timestamp) as latest, MIN(observer_id) as observer_id, MIN(observer_name) as observer_name,
MIN(path_json) as path_json, MIN(payload_type) as payload_type, MIN(raw_hex) as raw_hex,
MIN(decoded_json) as decoded_json
FROM packets ${w} GROUP BY hash ORDER BY latest DESC LIMIT ? OFFSET ?`;
const packets = this.db.prepare(sql).all(...params, limit, offset);
const countSql = `SELECT COUNT(DISTINCT hash) as c FROM packets ${w}`;
const total = this.db.prepare(countSql).get(...params).c;
return { packets, total };
}
}
module.exports = PacketStore;
+33 -21
View File
@@ -40,7 +40,15 @@
return svg;
}
function histogram(values, bins, color, w = 800, h = 180) {
function histogram(data, bins, color, w = 800, h = 180) {
// Support pre-computed histogram from server { bins: [{x, w, count}], min, max }
if (data && data.bins && Array.isArray(data.bins)) {
const buckets = data.bins.map(b => b.count);
const labels = data.bins.map(b => b.x.toFixed(1));
return { svg: barChart(buckets, labels, color, w, h), buckets, labels };
}
// Legacy: raw values array
const values = data;
const min = Math.min(...values), max = Math.max(...values);
const step = (max - min) / bins;
const buckets = Array(bins).fill(0);
@@ -101,10 +109,10 @@
try {
_analyticsData = {};
const [hashData, rfData, topoData, chanData] = await Promise.all([
api('/analytics/hash-sizes'),
api('/analytics/rf'),
api('/analytics/topology'),
api('/analytics/channels'),
api('/analytics/hash-sizes', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/rf', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/topology', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/channels', { ttl: CLIENT_TTL.analyticsRF }),
]);
_analyticsData = { hashData, rfData, topoData, chanData };
renderTab('overview');
@@ -142,10 +150,14 @@
el.innerHTML = `
<div class="stats-grid">
<div class="stat-card">
<div class="stat-value">${rf.totalPackets.toLocaleString()}</div>
<div class="stat-label">Total Packets</div>
<div class="stat-value">${(rf.totalTransmissions || rf.totalAllPackets || rf.totalPackets).toLocaleString()}</div>
<div class="stat-label">Total Transmissions</div>
<div class="stat-spark">${sparkSvg(rf.packetsPerHour.map(h=>h.count), 'var(--accent)')}</div>
</div>
<div class="stat-card">
<div class="stat-value">${rf.totalPackets.toLocaleString()}</div>
<div class="stat-label">Observations with Signal</div>
</div>
<div class="stat-card">
<div class="stat-value">${topo.uniqueNodes}</div>
<div class="stat-label">Unique Nodes</div>
@@ -747,7 +759,7 @@
</div>
`;
let allNodes = [];
try { const nd = await api('/nodes?limit=2000'); allNodes = nd.nodes || []; } catch {}
try { const nd = await api('/nodes?limit=2000', { ttl: CLIENT_TTL.nodeList }); allNodes = nd.nodes || []; } catch {}
renderHashMatrix(data.topHops, allNodes);
renderCollisions(data.topHops, allNodes);
}
@@ -938,10 +950,10 @@
el.innerHTML = '<div class="text-center text-muted" style="padding:40px">Analyzing route patterns…</div>';
try {
const [d2, d3, d4, d5] = await Promise.all([
api('/analytics/subpaths?minLen=2&maxLen=2&limit=50'),
api('/analytics/subpaths?minLen=3&maxLen=3&limit=30'),
api('/analytics/subpaths?minLen=4&maxLen=4&limit=20'),
api('/analytics/subpaths?minLen=5&maxLen=8&limit=15')
api('/analytics/subpaths?minLen=2&maxLen=2&limit=50', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=3&maxLen=3&limit=30', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=4&maxLen=4&limit=20', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=5&maxLen=8&limit=15', { ttl: CLIENT_TTL.analyticsRF })
]);
function renderTable(data, title) {
@@ -1032,7 +1044,7 @@
panel.classList.remove('collapsed');
panel.innerHTML = '<div class="text-center text-muted" style="padding:40px">Loading…</div>';
try {
const data = await api('/analytics/subpath-detail?hops=' + encodeURIComponent(hopsStr));
const data = await api('/analytics/subpath-detail?hops=' + encodeURIComponent(hopsStr), { ttl: CLIENT_TTL.analyticsRF });
renderSubpathDetail(panel, data);
} catch (e) {
panel.innerHTML = `<div class="text-muted">Error: ${e.message}</div>`;
@@ -1117,7 +1129,7 @@
// Render minimap
if (hasMap && typeof L !== 'undefined') {
const map = L.map('subpathMap', { zoomControl: false, attributionControl: false });
L.tileLayer('https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png', { maxZoom: 18 }).addTo(map);
L.tileLayer(getTileUrl(), { maxZoom: 18 }).addTo(map);
const latlngs = [];
nodesWithLoc.forEach((n, i) => {
@@ -1141,9 +1153,9 @@
el.innerHTML = '<div style="padding:40px;text-align:center;color:var(--text-muted)">Loading node analytics…</div>';
try {
const [nodesResp, bulkHealth, netStatus] = await Promise.all([
api('/nodes?limit=200&sortBy=lastSeen'),
api('/nodes/bulk-health?limit=50'),
api('/nodes/network-status')
api('/nodes?limit=200&sortBy=lastSeen', { ttl: CLIENT_TTL.nodeList }),
api('/nodes/bulk-health?limit=50', { ttl: CLIENT_TTL.analyticsRF }),
api('/nodes/network-status', { ttl: CLIENT_TTL.analyticsRF })
]);
const nodes = nodesResp.nodes || nodesResp;
const myNodes = JSON.parse(localStorage.getItem('meshcore-my-nodes') || '[]');
@@ -1155,7 +1167,7 @@
const enriched = nodes.filter(n => healthMap[n.public_key]).map(n => ({ ...n, health: { stats: healthMap[n.public_key].stats, observers: healthMap[n.public_key].observers } }));
// Compute rankings
const byPackets = [...enriched].sort((a, b) => (b.health.stats.totalPackets || 0) - (a.health.stats.totalPackets || 0));
const byPackets = [...enriched].sort((a, b) => (b.health.stats.totalTransmissions || b.health.stats.totalPackets || 0) - (a.health.stats.totalTransmissions || a.health.stats.totalPackets || 0));
const bySnr = [...enriched].filter(n => n.health.stats.avgSnr != null).sort((a, b) => b.health.stats.avgSnr - a.health.stats.avgSnr);
const byObservers = [...enriched].sort((a, b) => (b.health.observers?.length || 0) - (a.health.observers?.length || 0));
const byRecent = [...enriched].filter(n => n.health.stats.lastHeard).sort((a, b) => new Date(b.health.stats.lastHeard) - new Date(a.health.stats.lastHeard));
@@ -1170,7 +1182,7 @@
return myKeys.has(n.public_key) ? ' <span style="color:var(--accent);font-size:10px">★ MINE</span>' : '';
}
const ROLE_COLORS = { repeater: '#dc2626', companion: '#2563eb', room: '#16a34a', sensor: '#d97706' };
// ROLE_COLORS from shared roles.js
el.innerHTML = `
<div class="analytics-section">
@@ -1211,7 +1223,7 @@
return `<tr>
<td>${nodeLink(n)}</td>
<td><span class="badge" style="background:${(ROLE_COLORS[n.role]||'#6b7280')}20;color:${ROLE_COLORS[n.role]||'#6b7280'}">${n.role}</span></td>
<td>${s.totalPackets || 0}</td>
<td>${s.totalTransmissions || s.totalPackets || 0}</td>
<td>${s.avgSnr != null ? s.avgSnr.toFixed(1) + ' dB' : '—'}</td>
<td>${n.health.observers?.length || 0}</td>
<td>${s.lastHeard ? timeAgo(s.lastHeard) : '—'}</td>
@@ -1228,7 +1240,7 @@
<td>${i + 1}</td>
<td>${nodeLink(n)}${claimedBadge(n)}</td>
<td><span class="badge" style="background:${(ROLE_COLORS[n.role]||'#6b7280')}20;color:${ROLE_COLORS[n.role]||'#6b7280'}">${n.role}</span></td>
<td>${n.health.stats.totalPackets || 0}</td>
<td>${n.health.stats.totalTransmissions || n.health.stats.totalPackets || 0}</td>
<td>${n.health.stats.packetsToday || 0}</td>
<td><a href="#/nodes/${encodeURIComponent(n.public_key)}/analytics" class="analytics-link">📊</a></td>
</tr>`).join('')}
+101 -10
View File
@@ -11,12 +11,81 @@ function payloadTypeName(n) { return PAYLOAD_TYPES[n] || 'UNKNOWN'; }
function payloadTypeColor(n) { return PAYLOAD_COLORS[n] || 'unknown'; }
// --- Utilities ---
async function api(path) {
const res = await fetch('/api' + path);
if (!res.ok) throw new Error(`API ${res.status}: ${path}`);
return res.json();
const _apiPerf = { calls: 0, totalMs: 0, log: [], cacheHits: 0 };
const _apiCache = new Map();
const _inflight = new Map();
// Client-side TTLs (ms) — loaded from server config, with defaults
const CLIENT_TTL = {
stats: 10000, nodeDetail: 240000, nodeHealth: 240000, nodeList: 90000,
bulkHealth: 300000, networkStatus: 300000, observers: 120000,
channels: 15000, channelMessages: 10000, analyticsRF: 300000,
analyticsTopology: 300000, analyticsChannels: 300000, analyticsHashSizes: 300000,
analyticsSubpaths: 300000, analyticsSubpathDetail: 300000,
nodeAnalytics: 60000, nodeSearch: 10000
};
// Fetch server cache config and use as client TTLs (server values are in seconds)
fetch('/api/config/cache').then(r => r.json()).then(cfg => {
for (const [k, v] of Object.entries(cfg)) {
if (k in CLIENT_TTL && typeof v === 'number') CLIENT_TTL[k] = v * 1000;
}
}).catch(() => {});
async function api(path, { ttl = 0, bust = false } = {}) {
const t0 = performance.now();
if (!bust && ttl > 0) {
const cached = _apiCache.get(path);
if (cached && Date.now() < cached.expires) {
_apiPerf.calls++;
_apiPerf.cacheHits++;
_apiPerf.log.push({ path, ms: 0, time: Date.now(), cached: true });
if (_apiPerf.log.length > 200) _apiPerf.log.shift();
return cached.data;
}
}
// Deduplicate in-flight requests
if (_inflight.has(path)) return _inflight.get(path);
const promise = (async () => {
const res = await fetch('/api' + path);
if (!res.ok) throw new Error(`API ${res.status}: ${path}`);
const data = await res.json();
const ms = performance.now() - t0;
_apiPerf.calls++;
_apiPerf.totalMs += ms;
_apiPerf.log.push({ path, ms: Math.round(ms), time: Date.now() });
if (_apiPerf.log.length > 200) _apiPerf.log.shift();
if (ms > 500) console.warn(`[SLOW API] ${path} took ${Math.round(ms)}ms`);
if (ttl > 0) _apiCache.set(path, { data, expires: Date.now() + ttl });
return data;
})();
_inflight.set(path, promise);
promise.finally(() => _inflight.delete(path));
return promise;
}
function invalidateApiCache(prefix) {
for (const key of _apiCache.keys()) {
if (key.startsWith(prefix || '')) _apiCache.delete(key);
}
}
// Expose for console debugging: apiPerf()
window.apiPerf = function() {
const byPath = {};
_apiPerf.log.forEach(e => {
if (!byPath[e.path]) byPath[e.path] = { count: 0, totalMs: 0, maxMs: 0 };
byPath[e.path].count++;
byPath[e.path].totalMs += e.ms;
if (e.ms > byPath[e.path].maxMs) byPath[e.path].maxMs = e.ms;
});
const rows = Object.entries(byPath).map(([p, s]) => ({
path: p, count: s.count, avgMs: Math.round(s.totalMs / s.count), maxMs: s.maxMs,
totalMs: Math.round(s.totalMs)
})).sort((a, b) => b.totalMs - a.totalMs);
console.table(rows);
const hitRate = _apiPerf.calls ? Math.round(_apiPerf.cacheHits / _apiPerf.calls * 100) : 0;
const misses = _apiPerf.calls - _apiPerf.cacheHits;
console.log(`Cache: ${_apiPerf.cacheHits} hits / ${misses} misses (${hitRate}% hit rate)`);
return { calls: _apiPerf.calls, avgMs: Math.round(_apiPerf.totalMs / (misses || 1)), cacheHits: _apiPerf.cacheHits, cacheMisses: misses, cacheHitRate: hitRate, endpoints: rows };
};
function timeAgo(iso) {
if (!iso) return '—';
const s = Math.floor((Date.now() - new Date(iso).getTime()) / 1000);
@@ -140,6 +209,15 @@ function connectWS() {
ws.onmessage = (e) => {
try {
const msg = JSON.parse(e.data);
// Debounce cache invalidation — don't nuke on every packet
if (!api._invalidateTimer) {
api._invalidateTimer = setTimeout(() => {
api._invalidateTimer = null;
invalidateApiCache('/stats');
invalidateApiCache('/nodes');
invalidateApiCache('/channels');
}, 5000);
}
wsListeners.forEach(fn => fn(msg));
} catch {}
};
@@ -150,8 +228,8 @@ function offWS(fn) { wsListeners = wsListeners.filter(f => f !== fn); }
/* Global escapeHtml — used by multiple pages */
function escapeHtml(s) {
if (!s) return '';
return s.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
if (s == null) return '';
return String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
}
/* Global debounce */
@@ -205,6 +283,16 @@ function navigate() {
basePage = 'node-analytics';
}
// Special route: packet/123 → standalone packet detail page
if (basePage === 'packet' && routeParam) {
basePage = 'packet-detail';
}
// Special route: observers/ID → observer detail page
if (basePage === 'observers' && routeParam) {
basePage = 'observer-detail';
}
// Update nav active state
document.querySelectorAll('.nav-link[data-route]').forEach(el => {
el.classList.toggle('active', el.dataset.route === basePage);
@@ -217,7 +305,10 @@ function navigate() {
const app = document.getElementById('app');
if (pages[basePage]?.init) {
const t0 = performance.now();
pages[basePage].init(app, routeParam);
const ms = performance.now() - t0;
if (ms > 100) console.warn(`[SLOW PAGE] ${basePage} init took ${Math.round(ms)}ms`);
app.classList.remove('page-enter'); void app.offsetWidth; app.classList.add('page-enter');
} else {
app.innerHTML = `<div style="padding:40px;text-align:center;color:#6b7280"><h2>${route}</h2><p>Page not yet implemented.</p></div>`;
@@ -290,9 +381,9 @@ window.addEventListener('DOMContentLoaded', () => {
favDropdown.innerHTML = '<div class="fav-dd-loading">Loading...</div>';
const items = await Promise.all(favs.map(async (pk) => {
try {
const h = await api('/nodes/' + pk + '/health');
const h = await api('/nodes/' + pk + '/health', { ttl: CLIENT_TTL.nodeHealth });
const age = h.stats.lastHeard ? Date.now() - new Date(h.stats.lastHeard).getTime() : null;
const status = age === null ? '🔴' : age < 3600000 ? '🟢' : age < 86400000 ? '🟡' : '🔴';
const status = age === null ? '🔴' : age < HEALTH_THRESHOLDS.nodeDegradedMs ? '🟢' : age < HEALTH_THRESHOLDS.nodeSilentMs ? '🟡' : '🔴';
return '<a href="#/nodes/' + pk + '" class="fav-dd-item" data-key="' + pk + '">'
+ '<span class="fav-dd-status">' + status + '</span>'
+ '<span class="fav-dd-name">' + (h.node.name || truncate(pk, 12)) + '</span>'
@@ -378,7 +469,7 @@ window.addEventListener('DOMContentLoaded', () => {
const chList = Array.isArray(channels) ? channels : [];
for (const c of chList) {
if (c.name && c.name.toLowerCase().includes(q.toLowerCase())) {
html += `<div class="search-result-item" onclick="location.hash='#/channels?ch=${c.channel_hash}';document.getElementById('searchOverlay').classList.add('hidden')">
html += `<div class="search-result-item" onclick="location.hash='#/channels/${c.channel_hash}';document.getElementById('searchOverlay').classList.add('hidden')">
<span class="search-result-type">Channel</span>${c.name}</div>`;
}
}
@@ -394,7 +485,7 @@ window.addEventListener('DOMContentLoaded', () => {
// --- Nav Stats ---
async function updateNavStats() {
try {
const stats = await api('/stats');
const stats = await api('/stats', { ttl: CLIENT_TTL.stats });
const el = document.getElementById('navStats');
if (el) {
el.innerHTML = `<span class="stat-val">${stats.totalPackets}</span> pkts · <span class="stat-val">${stats.totalNodes}</span> nodes · <span class="stat-val">${stats.totalObservers}</span> obs`;
+14 -9
View File
@@ -18,7 +18,7 @@
if (cached && !cached.fetchedAt) return cached; // legacy null entries
}
try {
const data = await api('/nodes/search?q=' + encodeURIComponent(name));
const data = await api('/nodes/search?q=' + encodeURIComponent(name), { ttl: CLIENT_TTL.channelMessages });
// Try exact match first, then case-insensitive, then contains
const nodes = data.nodes || [];
const match = nodes.find(n => n.name === name)
@@ -40,7 +40,8 @@
tip.id = 'chNodeTooltip';
tip.className = 'ch-node-tooltip';
tip.setAttribute('role', 'tooltip');
const role = node.is_repeater ? '📡 Repeater' : node.is_room ? '🏠 Room' : node.is_sensor ? '🌡 Sensor' : '📻 Companion';
const roleKey = node.role || (node.is_repeater ? 'repeater' : node.is_room ? 'room' : node.is_sensor ? 'sensor' : 'companion');
const role = (ROLE_EMOJI[roleKey] || '●') + ' ' + (ROLE_LABELS[roleKey] || roleKey);
const lastSeen = node.last_seen ? timeAgo(node.last_seen) : 'unknown';
tip.innerHTML = `<div class="ch-tooltip-name">${escapeHtml(node.name)}</div>
<div class="ch-tooltip-role">${role}</div>
@@ -110,10 +111,11 @@
}
try {
const detail = await api('/nodes/' + encodeURIComponent(node.public_key));
const detail = await api('/nodes/' + encodeURIComponent(node.public_key), { ttl: CLIENT_TTL.nodeDetail });
const n = detail.node;
const adverts = detail.recentAdverts || [];
const role = n.is_repeater ? '📡 Repeater' : n.is_room ? '🏠 Room' : n.is_sensor ? '🌡 Sensor' : '📻 Companion';
const roleKey = n.role || (n.is_repeater ? 'repeater' : n.is_room ? 'room' : n.is_sensor ? 'sensor' : 'companion');
const role = (ROLE_EMOJI[roleKey] || '●') + ' ' + (ROLE_LABELS[roleKey] || roleKey);
const lastSeen = n.last_seen ? timeAgo(n.last_seen) : 'unknown';
panel.innerHTML = `<div class="ch-node-panel-header">
@@ -211,7 +213,7 @@
});
}
function init(app) {
function init(app, routeParam) {
app.innerHTML = `<div class="ch-layout">
<div class="ch-sidebar" aria-label="Channel list">
<div class="ch-sidebar-header">
@@ -235,7 +237,9 @@
</div>
</div>`;
loadChannels();
loadChannels().then(() => {
if (routeParam) selectChannel(routeParam);
});
// #89: Sidebar resize handle
(function () {
@@ -389,7 +393,7 @@
async function loadChannels(silent) {
try {
const data = await api('/channels');
const data = await api('/channels', { ttl: CLIENT_TTL.channels });
channels = (data.channels || []).sort((a, b) => (b.lastActivity || '').localeCompare(a.lastActivity || ''));
renderChannelList();
} catch (e) {
@@ -438,6 +442,7 @@
async function selectChannel(hash) {
selectedHash = hash;
history.replaceState(null, '', `#/channels/${hash}`);
renderChannelList();
const ch = channels.find(c => c.hash === hash);
const name = ch?.name || `Channel ${hash}`;
@@ -451,7 +456,7 @@
msgEl.innerHTML = '<div class="ch-loading">Loading messages…</div>';
try {
const data = await api(`/channels/${hash}/messages?limit=200`);
const data = await api(`/channels/${hash}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
messages = data.messages || [];
renderMessages();
scrollToBottom();
@@ -466,7 +471,7 @@
if (!msgEl) return;
const wasAtBottom = msgEl.scrollHeight - msgEl.scrollTop - msgEl.clientHeight < 60;
try {
const data = await api(`/channels/${selectedHash}/messages?limit=200`);
const data = await api(`/channels/${selectedHash}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
const newMsgs = data.messages || [];
// #92: Use message ID/hash for change detection instead of count + timestamp
var _getLastId = function (arr) { var m = arr.length ? arr[arr.length - 1] : null; return m ? (m.id || m.packetId || m.timestamp || '') : ''; };
Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

+13
View File
@@ -0,0 +1,13 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
<rect width="32" height="32" rx="6" fill="#1a1a2e"/>
<circle cx="16" cy="8" r="3" fill="#00d4ff"/>
<circle cx="7" cy="22" r="3" fill="#00d4ff"/>
<circle cx="25" cy="22" r="3" fill="#00d4ff"/>
<circle cx="16" cy="18" r="2" fill="#00ff88"/>
<line x1="16" y1="8" x2="7" y2="22" stroke="#00d4ff" stroke-width="1" opacity="0.6"/>
<line x1="16" y1="8" x2="25" y2="22" stroke="#00d4ff" stroke-width="1" opacity="0.6"/>
<line x1="7" y1="22" x2="25" y2="22" stroke="#00d4ff" stroke-width="1" opacity="0.6"/>
<line x1="16" y1="8" x2="16" y2="18" stroke="#00ff88" stroke-width="1" opacity="0.5"/>
<line x1="7" y1="22" x2="16" y2="18" stroke="#00ff88" stroke-width="1" opacity="0.5"/>
<line x1="25" y1="22" x2="16" y2="18" stroke="#00ff88" stroke-width="1" opacity="0.5"/>
</svg>

After

Width:  |  Height:  |  Size: 851 B

+8 -8
View File
@@ -146,7 +146,7 @@
if (!q) { suggest.classList.remove('open'); input.setAttribute('aria-expanded', 'false'); input.setAttribute('aria-activedescendant', ''); return; }
searchTimeout = setTimeout(async () => {
try {
const data = await api('/nodes/search?q=' + encodeURIComponent(q));
const data = await api('/nodes/search?q=' + encodeURIComponent(q), { ttl: CLIENT_TTL.nodeSearch });
const nodes = data.nodes || [];
if (!nodes.length) {
suggest.innerHTML = '<div class="suggest-empty">No nodes found</div>';
@@ -247,13 +247,13 @@
const cards = await Promise.all(myNodes.map(async (mn) => {
try {
const h = await api('/nodes/' + encodeURIComponent(mn.pubkey) + '/health');
const h = await api('/nodes/' + encodeURIComponent(mn.pubkey) + '/health', { ttl: CLIENT_TTL.nodeHealth });
const node = h.node || {};
const stats = h.stats || {};
const obs = h.observers || [];
const age = stats.lastHeard ? Date.now() - new Date(stats.lastHeard).getTime() : null;
const status = age === null ? 'silent' : age < 3600000 ? 'healthy' : age < 86400000 ? 'degraded' : 'silent';
const status = age === null ? 'silent' : age < HEALTH_THRESHOLDS.nodeDegradedMs ? 'healthy' : age < HEALTH_THRESHOLDS.nodeSilentMs ? 'degraded' : 'silent';
const statusDot = status === 'healthy' ? '🟢' : status === 'degraded' ? '🟡' : '🔴';
const statusText = status === 'healthy' ? 'Active' : status === 'degraded' ? 'Degraded' : 'Silent';
const name = node.name || mn.name || truncate(mn.pubkey, 12);
@@ -369,11 +369,11 @@
// ==================== STATS ====================
async function loadStats() {
try {
const s = await api('/stats');
const s = await api('/stats', { ttl: CLIENT_TTL.nodeSearch });
const el = document.getElementById('homeStats');
if (!el) return;
el.innerHTML = `
<div class="home-stat"><div class="val">${s.totalPackets ?? '—'}</div><div class="lbl">Packets</div></div>
<div class="home-stat"><div class="val">${s.totalTransmissions ?? s.totalPackets ?? '—'}</div><div class="lbl">Transmissions</div></div>
<div class="home-stat"><div class="val">${s.totalNodes ?? '—'}</div><div class="lbl">Nodes</div></div>
<div class="home-stat"><div class="val">${s.totalObservers ?? '—'}</div><div class="lbl">Observers</div></div>
<div class="home-stat"><div class="val">${s.packetsLast24h ?? '—'}</div><div class="lbl">Last 24h</div></div>
@@ -391,7 +391,7 @@
if (journey) journey.classList.remove('visible');
try {
const h = await api('/nodes/' + encodeURIComponent(pubkey) + '/health');
const h = await api('/nodes/' + encodeURIComponent(pubkey) + '/health', { ttl: CLIENT_TTL.nodeHealth });
const node = h.node || {};
const stats = h.stats || {};
const packets = h.recentPackets || [];
@@ -403,8 +403,8 @@
if (stats.lastHeard) {
const ageMs = Date.now() - new Date(stats.lastHeard).getTime();
const ago = timeAgo(stats.lastHeard);
if (ageMs < 3600000) { status = 'healthy'; color = 'green'; statusMsg = `Last heard ${ago}`; }
else if (ageMs < 86400000) { status = 'degraded'; color = 'yellow'; statusMsg = `Last heard ${ago}`; }
if (ageMs < HEALTH_THRESHOLDS.nodeDegradedMs) { status = 'healthy'; color = 'green'; statusMsg = `Last heard ${ago}`; }
else if (ageMs < HEALTH_THRESHOLDS.nodeSilentMs) { status = 'degraded'; color = 'yellow'; statusMsg = `Last heard ${ago}`; }
else { statusMsg = `Last heard ${ago}`; }
}
+19 -13
View File
@@ -2,6 +2,8 @@
<html lang="en">
<head>
<meta charset="UTF-8">
<link rel="icon" href="favicon.ico" type="image/x-icon">
<link rel="icon" href="favicon.svg" type="image/svg+xml">
<meta name="viewport" content="width=device-width, initial-scale=1.0, viewport-fit=cover">
<title>MeshCore Analyzer</title>
@@ -20,9 +22,9 @@
<meta name="twitter:title" content="MeshCore Analyzer">
<meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis.">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1773963867">
<link rel="stylesheet" href="style.css?v=1774042199">
<link rel="stylesheet" href="home.css">
<link rel="stylesheet" href="live.css?v=1773966856">
<link rel="stylesheet" href="live.css?v=1774034490">
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
crossorigin="anonymous">
@@ -51,6 +53,7 @@
<a href="#/traces" class="nav-link" data-route="traces">Traces</a>
<a href="#/observers" class="nav-link" data-route="observers">Observers</a>
<a href="#/analytics" class="nav-link" data-route="analytics">Analytics</a>
<a href="#/perf" class="nav-link" data-route="perf">⚡ Perf</a>
</div>
</div>
<div class="nav-right">
@@ -76,16 +79,19 @@
<main id="app" role="main"></main>
<script src="vendor/qrcode.js"></script>
<script src="app.js?v=1774079160"></script>
<script src="home.js?v=1774079160"></script>
<script src="packets.js?v=1773961784"></script>
<script src="map.js?v=1774079160" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774079160" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1773961950" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774079160" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1773961035" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1773964458" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774079160" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1773961276" onerror="console.error('Failed to load:', this.src)"></script>
<script src="roles.js?v=1774028201"></script>
<script src="app.js?v=1774034748"></script>
<script src="home.js?v=1774042199"></script>
<script src="packets.js?v=1774044174"></script>
<script src="map.js?v=1774028201" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774028201" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774042199" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1773972187" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774042199" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774046040" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774018095" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774028201" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774042199" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1773985649" onerror="console.error('Failed to load:', this.src)"></script>
</body>
</html>
+20
View File
@@ -100,6 +100,26 @@
background: color-mix(in srgb, var(--text) 14%, transparent);
}
/* ---- Node Detail Panel ---- */
.live-node-detail {
top: 60px;
right: 12px;
width: 320px;
max-height: calc(100vh - 140px);
overflow-y: auto;
background: color-mix(in srgb, var(--surface-1) 95%, transparent);
backdrop-filter: blur(12px);
border-radius: 10px;
border: 1px solid var(--border);
box-shadow: 0 4px 24px rgba(0, 0, 0, 0.5);
transition: transform 0.2s ease, opacity 0.2s ease;
}
.live-node-detail.hidden {
transform: translateX(340px);
opacity: 0;
pointer-events: none;
}
/* ---- Feed ---- */
.live-feed {
bottom: 12px;
+112 -22
View File
@@ -30,10 +30,7 @@
timelineFetchedScope: 0, // last fetched scope to avoid redundant fetches
};
const ROLE_COLORS = {
repeater: '#3b82f6', companion: '#06b6d4', room: '#a855f7',
sensor: '#f59e0b', unknown: '#6b7280'
};
// ROLE_COLORS loaded from shared roles.js (includes 'unknown')
const TYPE_COLORS = {
ADVERT: '#22c55e', GRP_TXT: '#3b82f6', TXT_MSG: '#f59e0b', ACK: '#6b7280',
@@ -601,6 +598,10 @@
<button class="feed-hide-btn" id="feedHideBtn" title="Hide feed">✕</button>
</div>
<button class="feed-show-btn hidden" id="feedShowBtn" title="Show feed">📋</button>
<div class="live-overlay live-node-detail hidden" id="liveNodeDetail">
<button class="feed-hide-btn" id="nodeDetailClose" title="Close">✕</button>
<div id="nodeDetailContent"></div>
</div>
<button class="legend-toggle-btn hidden" id="legendToggleBtn" aria-label="Show legend" title="Show legend">🎨</button>
<div class="live-overlay live-legend" id="liveLegend" role="region" aria-label="Map legend">
<h3 class="legend-title">PACKET TYPES</h3>
@@ -612,12 +613,7 @@
<li><span class="live-dot" style="background:#ec4899" aria-hidden="true"></span> Trace — Route trace</li>
</ul>
<h3 class="legend-title" style="margin-top:8px">NODE ROLES</h3>
<ul class="legend-list">
<li><span class="live-dot" style="background:#3b82f6" aria-hidden="true"></span> Repeater</li>
<li><span class="live-dot" style="background:#06b6d4" aria-hidden="true"></span> Companion</li>
<li><span class="live-dot" style="background:#a855f7" aria-hidden="true"></span> Room</li>
<li><span class="live-dot" style="background:#f59e0b" aria-hidden="true"></span> Sensor</li>
</ul>
<ul class="legend-list" id="roleLegendList"></ul>
</div>
<!-- VCR Bar -->
@@ -656,15 +652,13 @@
const isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
const DARK_TILES = 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
const LIGHT_TILES = 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png';
let tileLayer = L.tileLayer(isDark ? DARK_TILES : LIGHT_TILES, { maxZoom: 19 }).addTo(map);
let tileLayer = L.tileLayer(isDark ? TILE_DARK : TILE_LIGHT, { maxZoom: 19 }).addTo(map);
// Swap tiles when theme changes
const _themeObs = new MutationObserver(function () {
const dark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
tileLayer.setUrl(dark ? DARK_TILES : LIGHT_TILES);
tileLayer.setUrl(dark ? TILE_DARK : TILE_LIGHT);
});
_themeObs.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
L.control.zoom({ position: 'topright' }).addTo(map);
@@ -749,6 +743,23 @@
});
}
// Populate role legend from shared roles.js
const roleLegendList = document.getElementById('roleLegendList');
if (roleLegendList) {
for (const role of (window.ROLE_SORT || ['repeater', 'companion', 'room', 'sensor', 'observer'])) {
const li = document.createElement('li');
li.innerHTML = `<span class="live-dot" style="background:${ROLE_COLORS[role] || '#6b7280'}" aria-hidden="true"></span> ${(ROLE_LABELS[role] || role).replace(/s$/, '')}`;
roleLegendList.appendChild(li);
}
}
// Node detail panel
const nodeDetailPanel = document.getElementById('liveNodeDetail');
const nodeDetailContent = document.getElementById('nodeDetailContent');
document.getElementById('nodeDetailClose').addEventListener('click', () => {
nodeDetailPanel.classList.add('hidden');
});
// Feed panel resize handle (#27)
const savedFeedWidth = localStorage.getItem('live-feed-width');
if (savedFeedWidth) feedEl.style.width = savedFeedWidth + 'px';
@@ -908,8 +919,8 @@
const topNav = document.querySelector('.top-nav');
if (topNav) { topNav.style.position = 'fixed'; topNav.style.width = '100%'; topNav.style.zIndex = '1100'; }
_navCleanup = { timeout: null, fn: null, pinned: false };
// Add pin button to nav
if (topNav) {
// Add pin button to nav (guard against duplicate)
if (topNav && !document.getElementById('navPinBtn')) {
const pinBtn = document.createElement('button');
pinBtn.id = 'navPinBtn';
pinBtn.className = 'nav-pin-btn';
@@ -966,6 +977,80 @@
}, 2000);
}
async function showNodeDetail(pubkey) {
const panel = document.getElementById('liveNodeDetail');
const content = document.getElementById('nodeDetailContent');
panel.classList.remove('hidden');
content.innerHTML = '<div style="padding:20px;color:var(--text-muted)">Loading…</div>';
try {
const [data, healthData] = await Promise.all([
api('/nodes/' + encodeURIComponent(pubkey), { ttl: 30 }),
api('/nodes/' + encodeURIComponent(pubkey) + '/health', { ttl: 30 }).catch(() => null)
]);
const n = data.node;
const h = healthData || {};
const stats = h.stats || {};
const observers = h.observers || [];
const recent = h.recentPackets || [];
const roleColor = ROLE_COLORS[n.role] || '#6b7280';
const roleLabel = (ROLE_LABELS[n.role] || n.role || 'unknown').replace(/s$/, '');
const hasLoc = n.lat != null && n.lon != null;
const lastSeen = n.last_seen ? timeAgo(n.last_seen) : '—';
const thresholds = window.getHealthThresholds ? getHealthThresholds(n.role) : { degradedMs: 3600000, silentMs: 86400000 };
const ageMs = n.last_seen ? Date.now() - new Date(n.last_seen).getTime() : Infinity;
const statusDot = ageMs < thresholds.degradedMs ? 'health-green' : ageMs < thresholds.silentMs ? 'health-yellow' : 'health-red';
const statusLabel = ageMs < thresholds.degradedMs ? 'Online' : ageMs < thresholds.silentMs ? 'Degraded' : 'Offline';
let html = `
<div style="padding:16px;">
<div style="display:flex;align-items:center;gap:8px;margin-bottom:12px;">
<span class="${statusDot}" style="font-size:18px">●</span>
<h3 style="margin:0;font-size:16px;font-weight:700;">${escapeHtml(n.name || 'Unknown')}</h3>
</div>
<div style="margin-bottom:12px;">
<span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;background:${roleColor};color:#fff;">${roleLabel.toUpperCase()}</span>
<span style="color:var(--text-muted);font-size:12px;margin-left:8px;">${statusLabel}</span>
</div>
<div style="font-size:12px;color:var(--text-muted);margin-bottom:8px;">
<code style="font-size:10px;word-break:break-all;">${escapeHtml(n.public_key)}</code>
</div>
<table style="font-size:12px;width:100%;border-collapse:collapse;">
<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Last Seen</td><td>${lastSeen}</td></tr>
<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Adverts</td><td>${n.advert_count || 0}</td></tr>
${hasLoc ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Location</td><td>${n.lat.toFixed(5)}, ${n.lon.toFixed(5)}</td></tr>` : ''}
${stats.avgSnr != null ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Avg SNR</td><td>${stats.avgSnr.toFixed(1)} dB</td></tr>` : ''}
${stats.avgHops != null ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Avg Hops</td><td>${stats.avgHops.toFixed(1)}</td></tr>` : ''}
${stats.totalTransmissions || stats.totalPackets ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Total Packets</td><td>${stats.totalTransmissions || stats.totalPackets}</td></tr>` : ''}
</table>`;
if (observers.length) {
html += `<h4 style="font-size:12px;margin:12px 0 6px;color:var(--text-muted);">Heard By</h4>
<div style="font-size:11px;">` +
observers.map(o => `<div style="padding:2px 0;"><a href="#/observers/${encodeURIComponent(o.observer_id)}" style="color:var(--accent);text-decoration:none;">${escapeHtml(o.observer_name || o.observer_id.slice(0, 12))}</a> — ${o.packetCount || o.count || 0} pkts</div>`).join('') +
'</div>';
}
if (recent.length) {
html += `<h4 style="font-size:12px;margin:12px 0 6px;color:var(--text-muted);">Recent Packets</h4>
<div style="font-size:11px;max-height:200px;overflow-y:auto;">` +
recent.slice(0, 10).map(p => `<div style="padding:2px 0;display:flex;justify-content:space-between;">
<a href="#/packets/${encodeURIComponent(p.hash || '')}" style="color:var(--accent);text-decoration:none;">${escapeHtml(p.payload_type || '?')}${p.observation_count > 1 ? ' <span class="badge badge-obs" style="font-size:9px">👁 ' + p.observation_count + '</span>' : ''}</a>
<span style="color:var(--text-muted)">${p.timestamp ? timeAgo(p.timestamp) : '—'}</span>
</div>`).join('') +
'</div>';
}
html += `<div style="margin-top:12px;display:flex;gap:8px;">
<a href="#/nodes/${encodeURIComponent(n.public_key)}" style="font-size:12px;color:var(--accent);">Full Detail →</a>
<a href="#/nodes/${encodeURIComponent(n.public_key)}/analytics" style="font-size:12px;color:var(--accent);">📊 Analytics</a>
</div></div>`;
content.innerHTML = html;
} catch (e) {
content.innerHTML = `<div style="padding:20px;color:var(--text-muted);">Error: ${e.message}</div>`;
}
}
async function loadNodes(beforeTs) {
try {
const url = beforeTs
@@ -980,7 +1065,7 @@
addNodeMarker(n);
}
});
document.getElementById('liveNodeCount').textContent = Object.keys(nodeMarkers).length;
const _el2 = document.getElementById('liveNodeCount'); if (_el2) _el2.textContent = Object.keys(nodeMarkers).length;
} catch (e) { console.error('Failed to load nodes:', e); }
}
@@ -1014,6 +1099,8 @@
permanent: false, direction: 'top', offset: [0, -10], className: 'live-tooltip'
});
marker.on('click', () => showNodeDetail(n.public_key));
marker._glowMarker = glow;
marker._baseColor = color;
marker._baseSize = size;
@@ -1059,14 +1146,14 @@
if (msg.type === 'packet') bufferPacket(msg.data);
} catch {}
};
ws.onclose = () => setTimeout(connectWS, 3000);
ws.onclose = () => setTimeout(connectWS, WS_RECONNECT_MS);
ws.onerror = () => {};
}
function animatePacket(pkt) {
packetCount++;
pktTimestamps.push(Date.now());
document.getElementById('livePktCount').textContent = packetCount;
const _el = document.getElementById('livePktCount'); if (_el) _el.textContent = packetCount;
const decoded = pkt.decoded || {};
const header = decoded.header || {};
@@ -1086,7 +1173,7 @@
const n = { public_key: key, name: payload.name || key.slice(0,8), role: payload.role || 'unknown', lat: payload.lat, lon: payload.lon };
nodeData[key] = n;
addNodeMarker(n);
document.getElementById('liveNodeCount').textContent = Object.keys(nodeMarkers).length;
const _el2 = document.getElementById('liveNodeCount'); if (_el2) _el2.textContent = Object.keys(nodeMarkers).length;
}
}
@@ -1154,7 +1241,7 @@
// Sanity check: drop hops that are impossibly far from both neighbors (>200km ≈ 1.8°)
// These are almost certainly 1-byte prefix collisions with distant nodes
const MAX_HOP_DIST = 1.8;
// MAX_HOP_DIST from shared roles.js
for (let i = 0; i < raw.length; i++) {
if (!raw[i].known || !raw[i].pos) continue;
const prev = i > 0 && raw[i-1].known && raw[i-1].pos ? raw[i-1].pos : null;
@@ -1387,6 +1474,7 @@
const text = payload.text || payload.name || '';
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = pkt.observation_count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${pkt.observation_count}</span>` : '';
const item = document.createElement('div');
item.className = 'live-feed-item live-feed-enter';
@@ -1396,7 +1484,7 @@
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${hopStr}
${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${new Date(pkt._ts || Date.now()).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit',second:'2-digit'})}</span>
`;
@@ -1466,6 +1554,8 @@
if (appEl) appEl.style.height = '';
const topNav = document.querySelector('.top-nav');
if (topNav) { topNav.classList.remove('nav-autohide'); topNav.style.position = ''; topNav.style.width = ''; topNav.style.zIndex = ''; }
const existingPin = document.getElementById('navPinBtn');
if (existingPin) existingPin.remove();
if (_navCleanup) {
clearTimeout(_navCleanup.timeout);
const livePage = document.querySelector('.live-page');
+79 -24
View File
@@ -8,7 +8,7 @@
let clusterGroup = null;
let nodes = [];
let observers = [];
let filters = { repeater: true, companion: true, room: true, sensor: true, lastHeard: '30d', mqttOnly: false, neighbors: false, clusters: false };
let filters = { repeater: true, companion: true, room: true, sensor: true, observer: true, lastHeard: '30d', neighbors: false, clusters: false };
let wsHandler = null;
let heatLayer = null;
let userHasMoved = false;
@@ -17,16 +17,7 @@
// Safe escape — falls back to identity if app.js hasn't loaded yet
const safeEsc = (typeof esc === 'function') ? esc : function (s) { return s; };
// Distinct shapes + high-contrast WCAG AA colors for each role
const ROLE_STYLE = {
repeater: { color: '#dc2626', shape: 'diamond', radius: 10, weight: 2 }, // red diamond
companion: { color: '#2563eb', shape: 'circle', radius: 8, weight: 2 }, // blue circle
room: { color: '#16a34a', shape: 'square', radius: 9, weight: 2 }, // green square
sensor: { color: '#d97706', shape: 'triangle', radius: 8, weight: 2 }, // amber triangle
};
const ROLE_LABELS = { repeater: 'Repeaters', companion: 'Companions', room: 'Room Servers', sensor: 'Sensors' };
const ROLE_COLORS = { repeater: '#dc2626', companion: '#2563eb', room: '#16a34a', sensor: '#d97706' };
// Roles loaded from shared roles.js (ROLE_STYLE, ROLE_LABELS, ROLE_COLORS globals)
function makeMarkerIcon(role) {
const s = ROLE_STYLE[role] || ROLE_STYLE.companion;
@@ -43,6 +34,19 @@
case 'triangle':
path = `<polygon points="${c},2 ${size-2},${size-2} 2,${size-2}" fill="${s.color}" stroke="#fff" stroke-width="2"/>`;
break;
case 'star': {
// 5-pointed star
const cx = c, cy = c, outer = c - 1, inner = outer * 0.4;
let pts = '';
for (let i = 0; i < 5; i++) {
const aOuter = (i * 72 - 90) * Math.PI / 180;
const aInner = ((i * 72) + 36 - 90) * Math.PI / 180;
pts += `${cx + outer * Math.cos(aOuter)},${cy + outer * Math.sin(aOuter)} `;
pts += `${cx + inner * Math.cos(aInner)},${cy + inner * Math.sin(aInner)} `;
}
path = `<polygon points="${pts.trim()}" fill="${s.color}" stroke="#fff" stroke-width="1.5"/>`;
break;
}
default: // circle
path = `<circle cx="${c}" cy="${c}" r="${c-2}" fill="${s.color}" stroke="#fff" stroke-width="2"/>`;
}
@@ -74,7 +78,6 @@
</fieldset>
<fieldset class="mc-section">
<legend class="mc-label">Filters</legend>
<label for="mcMqtt"><input type="checkbox" id="mcMqtt"> MQTT Connected Only</label>
<label for="mcNeighbors"><input type="checkbox" id="mcNeighbors"> Show direct neighbors</label>
</fieldset>
<fieldset class="mc-section">
@@ -105,10 +108,19 @@
try { const v = JSON.parse(savedView); initCenter = [v.lat, v.lng]; initZoom = v.zoom; } catch {}
}
map = L.map('leaflet-map', { zoomControl: true }).setView(initCenter, initZoom);
L.tileLayer('https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png', {
attribution: '© OpenStreetMap',
const isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
const tileLayer = L.tileLayer(isDark ? TILE_DARK : TILE_LIGHT, {
attribution: '© OpenStreetMap © CartoDB',
maxZoom: 19,
}).addTo(map);
const _mapThemeObs = new MutationObserver(function () {
const dark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
tileLayer.setUrl(dark ? TILE_DARK : TILE_LIGHT);
});
_mapThemeObs.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
// Save position on move
map.on('moveend', () => {
@@ -141,7 +153,6 @@
// Bind controls
document.getElementById('mcClusters').addEventListener('change', e => { filters.clusters = e.target.checked; renderMarkers(); });
document.getElementById('mcHeatmap').addEventListener('change', e => { toggleHeatmap(e.target.checked); });
document.getElementById('mcMqtt').addEventListener('change', e => { filters.mqttOnly = e.target.checked; renderMarkers(); });
document.getElementById('mcNeighbors').addEventListener('change', e => { filters.neighbors = e.target.checked; renderMarkers(); });
document.getElementById('mcLastHeard').addEventListener('change', e => { filters.lastHeard = e.target.value; loadNodes(); });
@@ -245,13 +256,17 @@
async function loadNodes() {
try {
const data = await api(`/nodes?limit=10000&lastHeard=${filters.lastHeard}`);
nodes = data.nodes || [];
buildRoleChecks(data.counts || {});
// Load regions from config + observed IATAs
try { REGION_NAMES = await api('/config/regions', { ttl: 3600 }); } catch {}
// Load observers for jump buttons
const obsData = await api('/observers');
const data = await api(`/nodes?limit=10000&lastHeard=${filters.lastHeard}`, { ttl: CLIENT_TTL.nodeList });
nodes = data.nodes || [];
// Load observers for jump buttons + map markers
const obsData = await api('/observers', { ttl: CLIENT_TTL.observers });
observers = obsData.observers || [];
buildRoleChecks(data.counts || {});
buildJumpButtons();
renderMarkers();
@@ -266,12 +281,14 @@
const el = document.getElementById('mcRoleChecks');
if (!el) return;
el.innerHTML = '';
for (const role of ['repeater', 'companion', 'room', 'sensor']) {
const count = counts[role + 's'] || 0;
const obsCount = observers.filter(o => o.lat && o.lon).length;
const roles = ['repeater', 'companion', 'room', 'sensor', 'observer'];
const shapeMap = { repeater: '◆', companion: '●', room: '■', sensor: '▲', observer: '★' };
for (const role of roles) {
const count = role === 'observer' ? obsCount : (counts[role + 's'] || 0);
const cbId = 'mcRole_' + role;
const lbl = document.createElement('label');
lbl.setAttribute('for', cbId);
const shapeMap = { repeater: '◆', companion: '●', room: '■', sensor: '▲' };
const shape = shapeMap[role] || '●';
lbl.innerHTML = `<input type="checkbox" id="${cbId}" data-role="${role}" ${filters[role] ? 'checked' : ''}> <span style="color:${ROLE_COLORS[role]};font-weight:600;" aria-hidden="true">${shape}</span> ${ROLE_LABELS[role]} <span style="color:var(--text-muted)">(${count})</span>`;
lbl.querySelector('input').addEventListener('change', e => {
@@ -282,7 +299,7 @@
}
}
const REGION_NAMES = { SJC: 'San Jose', SFO: 'San Francisco', OAK: 'Oakland', MTV: 'Mountain View', SCZ: 'Santa Cruz', MRY: 'Monterey', PAO: 'Palo Alto' };
let REGION_NAMES = {};
function buildJumpButtons() {
const el = document.getElementById('mcJumps');
@@ -347,6 +364,44 @@
marker.bindPopup(buildPopup(node), { maxWidth: 280 });
markerLayer.addLayer(marker);
}
// Add observer markers
if (filters.observer) {
for (const obs of observers) {
if (!obs.lat || !obs.lon) continue;
const icon = makeMarkerIcon('observer');
const marker = L.marker([obs.lat, obs.lon], {
icon,
alt: `${obs.name || obs.id} (observer)`,
});
marker.bindPopup(buildObserverPopup(obs), { maxWidth: 280 });
markerLayer.addLayer(marker);
}
}
}
function buildObserverPopup(obs) {
const name = safeEsc(obs.name || obs.id || 'Unknown');
const iata = obs.iata ? `<span class="badge-region">${safeEsc(obs.iata)}</span>` : '';
const lastSeen = obs.last_seen ? timeAgo(obs.last_seen) : '—';
const packets = (obs.packet_count || 0).toLocaleString();
const loc = `${obs.lat.toFixed(5)}, ${obs.lon.toFixed(5)}`;
const roleBadge = `<span style="display:inline-block;padding:2px 8px;border-radius:12px;font-size:11px;font-weight:600;background:${ROLE_COLORS.observer};color:#fff;">OBSERVER</span>`;
return `
<div class="map-popup" style="font-family:var(--font);min-width:180px;">
<h3 style="font-weight:700;font-size:14px;margin:0 0 4px;">${name}</h3>
${roleBadge} ${iata}
<dl style="margin-top:8px;font-size:12px;">
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Location</dt>
<dd style="margin-left:88px;padding:2px 0;">${loc}</dd>
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Last Seen</dt>
<dd style="margin-left:88px;padding:2px 0;">${lastSeen}</dd>
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Packets</dt>
<dd style="margin-left:88px;padding:2px 0;">${packets}</dd>
</dl>
<a href="#/observers/${encodeURIComponent(obs.id || obs.observer_id)}" style="display:block;margin-top:8px;font-size:12px;color:var(--accent);">View Detail →</a>
</div>`;
}
function buildPopup(node) {
+2 -2
View File
@@ -40,7 +40,7 @@
let data;
try {
data = await api('/nodes/' + encodeURIComponent(pubkey) + '/analytics?days=' + days);
data = await api('/nodes/' + encodeURIComponent(pubkey) + '/analytics?days=' + days, { ttl: CLIENT_TTL.nodeAnalytics });
} catch (e) {
container.innerHTML = '<div style="padding:40px;text-align:center;color:#ff6b6b">Failed to load analytics: ' + escapeHtml(e.message) + '</div>';
return;
@@ -55,7 +55,7 @@
<div style="margin-bottom:12px">
<a href="#/nodes/${encodeURIComponent(n.public_key)}" style="color:var(--accent);text-decoration:none;font-size:12px">← Back to ${nodeName}</a>
<h2 style="margin:4px 0 2px;font-size:18px">📊 ${nodeName} — Analytics</h2>
<div style="color:var(--text-muted);font-size:11px">${n.role || 'Unknown role'} · ${s.totalPackets} packets in ${days}d window</div>
<div style="color:var(--text-muted);font-size:11px">${n.role || 'Unknown role'} · ${s.totalTransmissions || s.totalPackets} packets in ${days}d window</div>
</div>
<div class="analytics-time-range" id="timeRangeBtns">
+14 -16
View File
@@ -24,7 +24,7 @@
let wsHandler = null;
let detailMap = null;
const ROLE_COLORS = { repeater: '#3b82f6', room: '#6b7280', companion: '#22c55e', sensor: '#f59e0b' };
// ROLE_COLORS loaded from shared roles.js
const TABS = [
{ key: 'all', label: 'All' },
{ key: 'repeater', label: 'Repeaters' },
@@ -85,8 +85,8 @@
const body = document.getElementById('nodeFullBody');
try {
const [nodeData, healthData] = await Promise.all([
api('/nodes/' + encodeURIComponent(pubkey)),
api('/nodes/' + encodeURIComponent(pubkey) + '/health').catch(() => null)
api('/nodes/' + encodeURIComponent(pubkey), { ttl: CLIENT_TTL.nodeDetail }),
api('/nodes/' + encodeURIComponent(pubkey) + '/health', { ttl: CLIENT_TTL.nodeDetail }).catch(() => null)
]);
const n = nodeData.node;
const adverts = nodeData.recentAdverts || [];
@@ -107,9 +107,7 @@
// Repeaters/rooms: flood advert every 12-24h, so degraded after 24h, silent after 72h
// Companions/sensors: user-initiated adverts, shorter thresholds
const role = (n.role || '').toLowerCase();
const isInfra = role === 'repeater' || role === 'room';
const degradedMs = isInfra ? 86400000 : 3600000; // 24h : 1h
const silentMs = isInfra ? 259200000 : 86400000; // 72h : 24h
const { degradedMs, silentMs } = getHealthThresholds(role);
const statusLabel = statusAge < degradedMs ? '🟢 Active' : statusAge < silentMs ? '🟡 Degraded' : '🔴 Silent';
body.innerHTML = `
@@ -130,7 +128,7 @@
<dl class="detail-meta">
<dt>Last Heard</dt><dd>${lastHeard ? timeAgo(lastHeard) : (n.last_seen ? timeAgo(n.last_seen) : '—')}</dd>
<dt>First Seen</dt><dd>${n.first_seen ? new Date(n.first_seen).toLocaleString() : '—'}</dd>
<dt>Total Packets</dt><dd>${stats.totalPackets || n.advert_count || 0}</dd>
<dt>Total Packets</dt><dd>${stats.totalTransmissions || stats.totalPackets || n.advert_count || 0}${stats.totalObservations && stats.totalObservations !== (stats.totalTransmissions || stats.totalPackets) ? ' <span class="text-muted" style="font-size:0.85em">(seen ' + stats.totalObservations + '×)</span>' : ''}</dd>
<dt>Packets Today</dt><dd>${stats.packetsToday || 0}</dd>
${stats.avgSnr != null ? `<dt>Avg SNR</dt><dd>${stats.avgSnr.toFixed(1)} dB</dd>` : ''}
${stats.avgHops ? `<dt>Avg Hops</dt><dd>${stats.avgHops}</dd>` : ''}
@@ -163,9 +161,10 @@
const obs = p.observer_name || p.observer_id;
const snr = p.snr != null ? ` · SNR ${p.snr}dB` : '';
const rssi = p.rssi != null ? ` · RSSI ${p.rssi}dBm` : '';
const obsBadge = p.observation_count > 1 ? ` <span class="badge badge-obs" title="Seen ${p.observation_count} times">👁 ${p.observation_count}</span>` : '';
return `<div class="node-activity-item">
<span class="node-activity-time">${timeAgo(p.timestamp)}</span>
<span>${typeLabel}${detail}${obs ? ' via ' + escapeHtml(obs) : ''}${snr}${rssi}</span>
<span>${typeLabel}${detail}${obsBadge}${obs ? ' via ' + escapeHtml(obs) : ''}${snr}${rssi}</span>
<a href="#/packets/id/${p.id}" class="ch-analyze-link" style="margin-left:8px;font-size:0.8em">Analyze →</a>
</div>`;
}).join('') : '<div class="text-muted">No recent packets</div>'}
@@ -228,7 +227,7 @@
if (activeTab !== 'all') params.set('role', activeTab);
if (search) params.set('search', search);
if (lastHeard) params.set('lastHeard', lastHeard);
const data = await api('/nodes?' + params);
const data = await api('/nodes?' + params, { ttl: CLIENT_TTL.nodeList });
nodes = data.nodes || [];
counts = data.counts || {};
@@ -238,7 +237,7 @@
const missing = myNodes.filter(mn => !existingKeys.has(mn.pubkey));
if (missing.length) {
const fetched = await Promise.allSettled(
missing.map(mn => api('/nodes/' + encodeURIComponent(mn.pubkey)))
missing.map(mn => api('/nodes/' + encodeURIComponent(mn.pubkey), { ttl: CLIENT_TTL.nodeDetail }))
);
fetched.forEach(r => {
if (r.status === 'fulfilled' && r.value && r.value.public_key) nodes.push(r.value);
@@ -401,8 +400,8 @@
try {
const [data, healthData] = await Promise.all([
api('/nodes/' + encodeURIComponent(pubkey)),
api('/nodes/' + encodeURIComponent(pubkey) + '/health').catch(() => null)
api('/nodes/' + encodeURIComponent(pubkey), { ttl: CLIENT_TTL.nodeDetail }),
api('/nodes/' + encodeURIComponent(pubkey) + '/health', { ttl: CLIENT_TTL.nodeDetail }).catch(() => null)
]);
data.healthData = healthData;
renderDetail(panel, data);
@@ -426,11 +425,9 @@
const lastHeard = stats.lastHeard;
const statusAge = lastHeard ? (Date.now() - new Date(lastHeard).getTime()) : Infinity;
const role = (n.role || '').toLowerCase();
const isInfra = role === 'repeater' || role === 'room';
const degradedMs = isInfra ? 86400000 : 3600000;
const silentMs = isInfra ? 259200000 : 86400000;
const { degradedMs, silentMs } = getHealthThresholds(role);
const statusLabel = statusAge < degradedMs ? '🟢 Active' : statusAge < silentMs ? '🟡 Degraded' : '🔴 Silent';
const totalPackets = stats.totalPackets || n.advert_count || 0;
const totalPackets = stats.totalTransmissions || stats.totalPackets || n.advert_count || 0;
panel.innerHTML = `
<div class="node-detail">
@@ -484,6 +481,7 @@
<span class="advert-dot" style="background:${roleColor}"></span>
<div class="advert-info">
<strong>${timeAgo(a.timestamp)}</strong> ${icon} ${pType}${detail}
${a.observation_count > 1 ? ' <span class="badge badge-obs">👁 ' + a.observation_count + '</span>' : ''}
${obs ? ' via ' + escapeHtml(obs) : ''}
${a.snr != null ? ` · SNR ${a.snr}dB` : ''}${a.rssi != null ? ` · RSSI ${a.rssi}dBm` : ''}
<br><a href="#/packets/id/${a.id}" class="ch-analyze-link">Analyze </a>
+320
View File
@@ -0,0 +1,320 @@
/* === MeshCore Analyzer — observer-detail.js === */
'use strict';
(function () {
const PAYLOAD_LABELS = { 0: 'Request', 1: 'Response', 2: 'Direct Msg', 3: 'ACK', 4: 'Advert', 5: 'Channel Msg', 7: 'Anon Req', 8: 'Path', 9: 'Trace', 11: 'Control' };
const CHART_COLORS = ['#4a9eff', '#ff6b6b', '#51cf66', '#fcc419', '#cc5de8', '#20c997', '#ff922b', '#845ef7', '#f06595', '#339af0'];
let charts = [];
let currentDays = 7;
let currentId = null;
function destroyCharts() {
charts.forEach(c => { try { c.destroy(); } catch {} });
charts = [];
}
function chartDefaults() {
const style = getComputedStyle(document.documentElement);
Chart.defaults.color = style.getPropertyValue('--text-muted').trim() || '#6b7280';
Chart.defaults.borderColor = style.getPropertyValue('--border').trim() || '#e2e5ea';
}
function formatDuration(secs) {
if (!secs) return '—';
const d = Math.floor(secs / 86400);
const h = Math.floor((secs % 86400) / 3600);
const m = Math.floor((secs % 3600) / 60);
if (d > 0) return d + 'd ' + h + 'h';
if (h > 0) return h + 'h ' + m + 'm';
return m + 'm';
}
function init(app, routeParam) {
currentId = routeParam;
if (!currentId) {
app.innerHTML = '<div class="text-center text-muted" style="padding:40px">No observer ID specified.</div>';
return;
}
app.innerHTML = `
<div class="observer-detail-page" style="overflow-y:auto;height:calc(100vh - 56px);padding:16px">
<div class="page-header" style="display:flex;align-items:center;gap:12px;margin-bottom:16px">
<a href="#/observers" class="btn-icon" title="Back to Observers" aria-label="Back">←</a>
<h2 style="margin:0" id="obsTitle">Observer Detail</h2>
<div style="margin-left:auto;display:flex;gap:8px">
<select id="obsDaysSelect" class="time-range-select" aria-label="Time range">
<option value="1">24 Hours</option>
<option value="3">3 Days</option>
<option value="7" selected>7 Days</option>
<option value="30">30 Days</option>
</select>
</div>
</div>
<div id="obsDetailContent"><div class="text-center text-muted" style="padding:40px">Loading…</div></div>
</div>`;
document.getElementById('obsDaysSelect').addEventListener('change', function (e) {
currentDays = parseInt(e.target.value);
loadDetail();
});
loadDetail();
}
function destroy() {
destroyCharts();
currentId = null;
}
async function loadDetail() {
try {
destroyCharts();
chartDefaults();
const [obs, analytics] = await Promise.all([
api('/observers/' + encodeURIComponent(currentId)),
api('/observers/' + encodeURIComponent(currentId) + '/analytics?days=' + currentDays),
]);
renderDetail(obs, analytics);
} catch (e) {
document.getElementById('obsDetailContent').innerHTML =
'<div class="text-muted" style="padding:40px">Error: ' + e.message + '</div>';
}
}
function renderDetail(obs, analytics) {
const el = document.getElementById('obsDetailContent');
if (!el) return;
const title = document.getElementById('obsTitle');
if (title) title.textContent = obs.name || obs.id.substring(0, 16) + '…';
// Parse radio string
let radioHtml = '—';
if (obs.radio) {
const rp = obs.radio.split(',');
radioHtml = rp[0] + ' MHz · SF' + (rp[2] || '?') + ' · BW' + (rp[1] || '?') + ' · CR' + (rp[3] || '?');
}
// Health status
const ago = obs.last_seen ? Date.now() - new Date(obs.last_seen).getTime() : Infinity;
const statusCls = ago < 600000 ? 'health-green' : ago < HEALTH_THRESHOLDS.nodeDegradedMs ? 'health-yellow' : 'health-red';
const statusLabel = ago < 600000 ? 'Online' : ago < HEALTH_THRESHOLDS.nodeDegradedMs ? 'Stale' : 'Offline';
el.innerHTML = `
<div class="obs-info-grid" style="display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));gap:12px;margin-bottom:20px">
<div class="stat-card">
<div class="stat-label">Status</div>
<div class="stat-value"><span class="health-dot ${statusCls}">●</span> ${statusLabel}</div>
</div>
<div class="stat-card">
<div class="stat-label">Region</div>
<div class="stat-value">${obs.iata ? '<span class="badge-region">' + obs.iata + '</span>' : '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Model</div>
<div class="stat-value">${obs.model || '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Firmware</div>
<div class="stat-value" style="font-size:0.8em;word-break:break-all">${obs.firmware || '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Client</div>
<div class="stat-value" style="font-size:0.8em;word-break:break-all">${obs.client_version || '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Radio</div>
<div class="stat-value" style="font-size:0.85em">${radioHtml}</div>
</div>
<div class="stat-card">
<div class="stat-label">Battery</div>
<div class="stat-value">${obs.battery_mv ? obs.battery_mv + ' mV' : '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Uptime</div>
<div class="stat-value">${formatDuration(obs.uptime_secs)}</div>
</div>
<div class="stat-card">
<div class="stat-label">Noise Floor</div>
<div class="stat-value">${obs.noise_floor != null ? obs.noise_floor + ' dBm' : '—'}</div>
</div>
<div class="stat-card">
<div class="stat-label">Total Packets</div>
<div class="stat-value">${(obs.packet_count || 0).toLocaleString()}</div>
</div>
<div class="stat-card">
<div class="stat-label">Packets/Hour</div>
<div class="stat-value">${(obs.packetsLastHour || 0).toLocaleString()}</div>
</div>
<div class="stat-card">
<div class="stat-label">First Seen</div>
<div class="stat-value" style="font-size:0.85em">${obs.first_seen ? new Date(obs.first_seen).toLocaleDateString() : '—'}</div>
</div>
</div>
<div class="mono" style="font-size:0.75em;color:var(--text-muted);margin-bottom:20px;word-break:break-all">
ID: ${obs.id}
</div>
<div class="obs-charts" style="display:grid;grid-template-columns:repeat(auto-fit,minmax(400px,1fr));gap:16px">
<div class="chart-card" style="padding:12px">
<h3 style="margin:0 0 8px;font-size:0.95em">Packets Over Time</h3>
<canvas id="obsTimeChart"></canvas>
</div>
<div class="chart-card" style="padding:12px">
<h3 style="margin:0 0 8px;font-size:0.95em">Packet Types</h3>
<div style="max-width:280px;margin:0 auto"><canvas id="obsTypeChart"></canvas></div>
</div>
<div class="chart-card" style="padding:12px">
<h3 style="margin:0 0 8px;font-size:0.95em">Unique Nodes Heard</h3>
<canvas id="obsNodesChart"></canvas>
</div>
<div class="chart-card" style="padding:12px">
<h3 style="margin:0 0 8px;font-size:0.95em">SNR Distribution</h3>
<canvas id="obsSnrChart"></canvas>
</div>
</div>
<div style="margin-top:20px">
<h3 style="font-size:0.95em">Recent Packets</h3>
<div id="obsRecentPackets"><div class="text-muted">Loading…</div></div>
</div>`;
// Render charts
if (analytics.timeline && analytics.timeline.length > 0) {
renderTimelineChart(analytics.timeline);
}
if (analytics.packetTypes) {
renderTypeChart(analytics.packetTypes);
}
if (analytics.nodesTimeline && analytics.nodesTimeline.length > 0) {
renderNodesChart(analytics.nodesTimeline);
}
if (analytics.snrDistribution && analytics.snrDistribution.length > 0) {
renderSnrChart(analytics.snrDistribution);
}
if (analytics.recentPackets) {
renderRecentPackets(analytics.recentPackets);
}
}
function renderTimelineChart(timeline) {
const ctx = document.getElementById('obsTimeChart');
if (!ctx) return;
const c = new Chart(ctx, {
type: 'bar',
data: {
labels: timeline.map(t => t.label),
datasets: [{
label: 'Packets',
data: timeline.map(t => t.count),
backgroundColor: CHART_COLORS[0] + '80',
borderColor: CHART_COLORS[0],
borderWidth: 1,
}]
},
options: {
responsive: true, maintainAspectRatio: true,
plugins: { legend: { display: false } },
scales: {
x: { ticks: { maxRotation: 45, autoSkip: true, maxTicksLimit: 12 } },
y: { beginAtZero: true, ticks: { precision: 0 } }
}
}
});
charts.push(c);
}
function renderTypeChart(types) {
const ctx = document.getElementById('obsTypeChart');
if (!ctx) return;
const labels = Object.keys(types).map(k => PAYLOAD_LABELS[k] || 'Type ' + k);
const values = Object.values(types);
const c = new Chart(ctx, {
type: 'doughnut',
data: {
labels: labels,
datasets: [{ data: values, backgroundColor: CHART_COLORS.slice(0, labels.length) }]
},
options: {
responsive: true, maintainAspectRatio: true,
plugins: { legend: { position: 'bottom', labels: { boxWidth: 12 } } }
}
});
charts.push(c);
}
function renderNodesChart(timeline) {
const ctx = document.getElementById('obsNodesChart');
if (!ctx) return;
const c = new Chart(ctx, {
type: 'line',
data: {
labels: timeline.map(t => t.label),
datasets: [{
label: 'Unique Nodes',
data: timeline.map(t => t.count),
borderColor: CHART_COLORS[2],
backgroundColor: CHART_COLORS[2] + '20',
fill: true, tension: 0.3, pointRadius: 2,
}]
},
options: {
responsive: true, maintainAspectRatio: true,
plugins: { legend: { display: false } },
scales: {
x: { ticks: { maxRotation: 45, autoSkip: true, maxTicksLimit: 12 } },
y: { beginAtZero: true, ticks: { precision: 0 } }
}
}
});
charts.push(c);
}
function renderSnrChart(distribution) {
const ctx = document.getElementById('obsSnrChart');
if (!ctx) return;
const c = new Chart(ctx, {
type: 'bar',
data: {
labels: distribution.map(d => d.range),
datasets: [{
label: 'Packets',
data: distribution.map(d => d.count),
backgroundColor: CHART_COLORS[3] + '80',
borderColor: CHART_COLORS[3],
borderWidth: 1,
}]
},
options: {
responsive: true, maintainAspectRatio: true,
plugins: { legend: { display: false } },
scales: {
x: { title: { display: true, text: 'SNR (dB)' } },
y: { beginAtZero: true, ticks: { precision: 0 } }
}
}
});
charts.push(c);
}
function renderRecentPackets(packets) {
const el = document.getElementById('obsRecentPackets');
if (!el || !packets.length) { if (el) el.innerHTML = '<div class="text-muted">No recent packets.</div>'; return; }
el.innerHTML = `<table class="data-table" style="font-size:0.85em">
<thead><tr><th>Time</th><th>Type</th><th>Hash</th><th>SNR</th><th>RSSI</th><th>Hops</th></tr></thead>
<tbody>${packets.map(p => {
const decoded = typeof p.decoded_json === 'string' ? JSON.parse(p.decoded_json) : (p.decoded_json || {});
const hops = typeof p.path_json === 'string' ? JSON.parse(p.path_json) : (p.path_json || []);
const typeName = PAYLOAD_LABELS[p.payload_type] || 'Type ' + p.payload_type;
return `<tr style="cursor:pointer" onclick="location.hash='#/packet/${p.id}'">
<td>${timeAgo(p.timestamp)}</td>
<td>${typeName}</td>
<td class="mono" style="font-size:0.85em">${(p.hash || '').substring(0, 10)}</td>
<td>${p.snr != null ? p.snr.toFixed(1) : '—'}</td>
<td>${p.rssi != null ? p.rssi : '—'}</td>
<td>${hops.length}</td>
</tr>`;
}).join('')}</tbody>
</table>`;
}
registerPage('observer-detail', { init, destroy });
})();
+4 -5
View File
@@ -38,7 +38,7 @@
async function loadObservers() {
try {
const data = await api('/observers');
const data = await api('/observers', { ttl: CLIENT_TTL.observers });
observers = data.observers || [];
render();
} catch (e) {
@@ -69,10 +69,9 @@
}
function sparkBar(count, max) {
const aria = `role="meter" aria-valuenow="${count}" aria-valuemin="0" aria-valuemax="${max}" aria-label="Packet rate"`;
if (max === 0) return `<div class="spark-bar" ${aria}><div class="spark-fill" style="width:0"></div></div>`;
if (max === 0) return `<span class="text-muted">0/hr</span>`;
const pct = Math.min(100, Math.round((count / max) * 100));
return `<div class="spark-bar" ${aria}><div class="spark-fill" style="width:${pct}%"></div><span class="spark-label">${count}/hr</span></div>`;
return `<span style="display:inline-flex;align-items:center;gap:6px;white-space:nowrap"><span style="display:inline-block;width:60px;height:12px;background:var(--border);border-radius:3px;overflow:hidden;vertical-align:middle"><span style="display:block;height:100%;width:${pct}%;background:linear-gradient(90deg,#3b82f6,#60a5fa);border-radius:3px"></span></span><span style="font-size:11px">${count}/hr</span></span>`;
}
function render() {
@@ -107,7 +106,7 @@
<tbody>${observers.map(o => {
const h = healthStatus(o.last_seen);
const shape = h.cls === 'health-green' ? '●' : h.cls === 'health-yellow' ? '▲' : '✕';
return `<tr>
return `<tr style="cursor:pointer" onclick="location.hash='#/observers/${encodeURIComponent(o.id)}'">
<td><span class="health-dot ${h.cls}" title="${h.label}">${shape}</span> ${h.label}</td>
<td class="mono">${o.name || o.id}</td>
<td>${o.iata ? `<span class="badge-region">${o.iata}</span>` : '—'}</td>
+189 -38
View File
@@ -3,6 +3,13 @@
(function () {
let packets = [];
// Resolve observer_id to friendly name from loaded observers list
function obsName(id) {
if (!id) return '—';
const o = observers.find(ob => ob.id === id);
return o?.name || id;
}
let selectedId = null;
let groupByHash = true;
let filters = {};
@@ -182,9 +189,81 @@
} catch {}
}
wsHandler = debouncedOnWS(function (msgs) {
if (msgs.some(function (m) { return m.type === 'packet'; })) {
loadPackets();
const newPkts = msgs
.filter(m => m.type === 'packet' && m.data?.packet)
.map(m => m.data.packet);
if (!newPkts.length) return;
// Check if new packets pass current filters
const filtered = newPkts.filter(p => {
if (filters.type !== undefined && filters.type !== '' && p.payload_type !== Number(filters.type)) return false;
if (filters.observer && p.observer_id !== filters.observer) return false;
if (filters.hash && p.hash !== filters.hash) return false;
if (filters.region) {
const obs = observers.find(o => o.id === p.observer_id);
if (!obs || obs.iata !== filters.region) return false;
}
if (filters.node && !(p.decoded_json || '').includes(filters.node)) return false;
return true;
});
if (!filtered.length) return;
// Resolve any new hops, then update and re-render
const newHops = new Set();
for (const p of filtered) {
try { JSON.parse(p.path_json || '[]').forEach(h => { if (!(h in hopNameCache)) newHops.add(h); }); } catch {}
}
(newHops.size ? resolveHops([...newHops]) : Promise.resolve()).then(() => {
if (groupByHash) {
// Update existing groups or create new ones
for (const p of filtered) {
const h = p.hash;
const existing = packets.find(g => g.hash === h);
if (existing) {
existing.count = (existing.count || 1) + 1;
existing.observation_count = (existing.observation_count || 1) + 1;
existing.latest = p.timestamp > existing.latest ? p.timestamp : existing.latest;
// Track unique observers
if (p.observer_id && p.observer_id !== existing.observer_id) {
existing.observer_count = (existing.observer_count || 1) + 1;
}
// Keep longest path
if (p.path_json && (!existing.path_json || p.path_json.length > existing.path_json.length)) {
existing.path_json = p.path_json;
existing.raw_hex = p.raw_hex;
}
// Update decoded_json to latest
if (p.decoded_json) existing.decoded_json = p.decoded_json;
// Update expanded children if this group is expanded
if (expandedHashes.has(h) && existing._children) {
existing._children.unshift(p);
}
} else {
// New group
packets.unshift({
hash: h,
count: 1,
observer_count: 1,
latest: p.timestamp,
observer_id: p.observer_id,
observer_name: p.observer_name,
path_json: p.path_json,
payload_type: p.payload_type,
raw_hex: p.raw_hex,
decoded_json: p.decoded_json,
});
}
}
// Re-sort by latest DESC, cap size
packets.sort((a, b) => (b.latest || '').localeCompare(a.latest || ''));
packets = packets.slice(0, 200);
} else {
// Flat mode: prepend
packets = filtered.concat(packets).slice(0, 200);
}
totalCount += filtered.length;
renderTableRows();
});
});
}
@@ -207,7 +286,7 @@
async function loadObservers() {
try {
const data = await api('/observers');
const data = await api('/observers', { ttl: CLIENT_TTL.observers });
observers = data.observers || [];
} catch {}
}
@@ -278,6 +357,7 @@
</div>
</div>
<div class="filter-bar" id="pktFilters">
<button class="btn filter-toggle-btn" id="filterToggleBtn">Filters ▾</button>
<input type="text" placeholder="Packet hash…" id="fHash" aria-label="Filter by packet hash">
<div class="node-filter-wrap" style="position:relative">
<input type="text" placeholder="Node name…" id="fNode" autocomplete="off" role="combobox" aria-expanded="false" aria-owns="fNodeDropdown" aria-activedescendant="" aria-autocomplete="list">
@@ -310,7 +390,7 @@
const obsSel = document.getElementById('fObserver');
for (const o of observers) {
obsSel.innerHTML += `<option value="${o.id}" ${filters.observer === o.id ? 'selected' : ''}>${o.id}</option>`;
obsSel.innerHTML += `<option value="${o.id}" ${filters.observer === o.id ? 'selected' : ''}>${o.name || o.id}</option>`;
}
const typeSel = document.getElementById('fType');
@@ -318,6 +398,13 @@
typeSel.innerHTML += `<option value="${k}" ${String(filters.type) === k ? 'selected' : ''}>${v}</option>`;
}
// Filter toggle button for mobile
document.getElementById('filterToggleBtn').addEventListener('click', function() {
const bar = document.getElementById('pktFilters');
bar.classList.toggle('filters-expanded');
this.textContent = bar.classList.contains('filters-expanded') ? 'Filters ▴' : 'Filters ▾';
});
// Filter event listeners
document.getElementById('fHash').value = filters.hash || '';
document.getElementById('fHash').addEventListener('input', debounce((e) => { filters.hash = e.target.value || undefined; loadPackets(); }, 300));
@@ -343,7 +430,8 @@
{ key: 'rpt', label: 'Rpt' },
{ key: 'details', label: 'Details' },
];
const defaultHidden = ['region'];
const isMobile = window.innerWidth <= 640;
const defaultHidden = isMobile ? ['region', 'hash', 'observer', 'path', 'rpt', 'size'] : ['region'];
let visibleCols;
try {
visibleCols = JSON.parse(localStorage.getItem('packets-visible-cols'));
@@ -490,7 +578,7 @@
makeColumnsResizable('#pktTable', 'meshcore-pkt-col-widths');
}
function renderTableRows() {
async function renderTableRows() {
const tbody = document.getElementById('pktBody');
if (!tbody) return;
@@ -500,24 +588,21 @@
const groupBtn = document.getElementById('fGroup');
if (groupBtn) groupBtn.classList.toggle('active', groupByHash);
// Filter to claimed/favorited nodes if toggle is on
// Filter to claimed/favorited nodes if toggle is on — use server-side multi-node lookup
let displayPackets = packets;
if (filters.myNodes) {
const myNodes = JSON.parse(localStorage.getItem('meshcore-my-nodes') || '[]');
const myKeys = new Set(myNodes.map(n => n.pubkey));
const myKeys = myNodes.map(n => n.pubkey).filter(Boolean);
const favs = getFavorites();
const allKeys = new Set([...myKeys, ...favs]);
displayPackets = packets.filter(p => {
const allKeys = [...new Set([...myKeys, ...favs])];
if (allKeys.length > 0) {
try {
const d = JSON.parse(p.decoded_json || '{}');
const pathHops = JSON.parse(p.path_json || '[]');
// Check if any node key in decoded data or path matches
return (d.pubkey && allKeys.has(d.pubkey)) ||
(d.to && allKeys.has(d.to)) ||
(d.from && allKeys.has(d.from)) ||
pathHops.some(h => allKeys.has(h));
} catch { return false; }
});
const myData = await api('/packets?nodes=' + allKeys.join(',') + '&limit=500');
displayPackets = myData.packets || [];
} catch { displayPackets = []; }
} else {
displayPackets = [];
}
}
if (!displayPackets.length) {
@@ -544,9 +629,9 @@
<td class="mono col-hash">${truncate(p.hash || '—', 8)}</td>
<td class="col-size">${groupSize ? groupSize + 'B' : '—'}</td>
<td class="col-type">${p.payload_type != null ? `<span class="badge badge-${groupTypeClass}">${groupTypeName}</span>` : '—'}</td>
<td class="col-observer">${isSingle ? truncate(p.observer_name || p.observer_id || '—', 16) : truncate(p.observer_name || p.observer_id || '—', 10) + (p.observer_count > 1 ? ' +' + (p.observer_count - 1) : '')}</td>
<td class="col-observer">${isSingle ? truncate(obsName(p.observer_id), 16) : truncate(obsName(p.observer_id), 10) + (p.observer_count > 1 ? ' +' + (p.observer_count - 1) : '')}</td>
<td class="col-path"><span class="path-hops">${groupPathStr}</span></td>
<td class="col-rpt">${isSingle ? '' : p.count}</td>
<td class="col-rpt">${p.observation_count > 1 ? '<span class="badge badge-obs" title="Seen ' + p.observation_count + ' times">👁 ' + p.observation_count + '</span>' : (isSingle ? '' : p.count)}</td>
<td class="col-details">${getDetailPreview((() => { try { return JSON.parse(p.decoded_json || '{}'); } catch { return {}; } })())}</td>
</tr>`;
// Child rows (loaded async when expanded)
@@ -565,7 +650,7 @@
<td class="mono col-hash">${truncate(c.hash || '', 8)}</td>
<td class="col-size">${size}B</td>
<td class="col-type"><span class="badge badge-${typeClass}">${typeName}</span></td>
<td class="col-observer">${truncate(c.observer_name || c.observer_id || '—', 16)}</td>
<td class="col-observer">${truncate(obsName(c.observer_id), 16)}</td>
<td class="col-path"><span class="path-hops">${childPathStr}</span></td>
<td class="col-rpt"></td>
<td class="col-details">${getDetailPreview((() => { try { return JSON.parse(c.decoded_json); } catch { return {}; } })())}</td>
@@ -595,7 +680,7 @@
<td class="mono col-hash">${truncate(p.hash || String(p.id), 8)}</td>
<td class="col-size">${size}B</td>
<td class="col-type"><span class="badge badge-${typeClass}">${typeName}</span></td>
<td class="col-observer">${truncate(p.observer_name || p.observer_id || '—', 16)}</td>
<td class="col-observer">${truncate(obsName(p.observer_id), 16)}</td>
<td class="col-path"><span class="path-hops">${pathStr}</span></td>
<td class="col-rpt"></td>
<td class="col-details">${detail}</td>
@@ -632,11 +717,35 @@
async function selectPacket(id) {
selectedId = id;
history.replaceState(null, '', `#/packet/${id}`);
renderTableRows();
const panel = document.getElementById('pktRight');
panel.classList.remove('empty');
panel.innerHTML = '<div class="panel-resize-handle" id="pktResizeHandle"></div><div class="text-center text-muted" style="padding:40px">Loading…</div>';
initPanelResize();
const isMobileNow = window.innerWidth <= 640;
let panel;
if (isMobileNow) {
// Use mobile bottom sheet
let sheet = document.getElementById('mobileDetailSheet');
if (!sheet) {
sheet = document.createElement('div');
sheet.id = 'mobileDetailSheet';
sheet.className = 'mobile-detail-sheet';
sheet.innerHTML = '<div class="mobile-sheet-handle"></div><button class="mobile-sheet-close" id="mobileSheetClose">✕</button><div class="mobile-sheet-content"></div>';
document.body.appendChild(sheet);
sheet.querySelector('#mobileSheetClose').addEventListener('click', () => {
sheet.classList.remove('open');
});
sheet.querySelector('.mobile-sheet-handle').addEventListener('click', () => {
sheet.classList.remove('open');
});
}
panel = sheet.querySelector('.mobile-sheet-content');
panel.innerHTML = '<div class="text-center text-muted" style="padding:40px">Loading…</div>';
sheet.classList.add('open');
} else {
panel = document.getElementById('pktRight');
panel.classList.remove('empty');
panel.innerHTML = '<div class="panel-resize-handle" id="pktResizeHandle"></div><div class="text-center text-muted" style="padding:40px">Loading…</div>';
initPanelResize();
}
try {
const data = await api(`/packets/${id}`);
@@ -647,11 +756,11 @@
const newHops = hops.filter(h => !(h in hopNameCache));
if (newHops.length) await resolveHops(newHops);
} catch {}
panel.innerHTML = '<div class="panel-resize-handle" id="pktResizeHandle"></div>';
panel.innerHTML = isMobileNow ? '' : '<div class="panel-resize-handle" id="pktResizeHandle"></div>';
const content = document.createElement('div');
panel.appendChild(content);
renderDetail(content, data);
initPanelResize();
if (!isMobileNow) initPanelResize();
} catch (e) {
panel.innerHTML = `<div class="text-muted">Error: ${e.message}</div>`;
}
@@ -691,7 +800,7 @@
<div class="detail-hash">${pkt.hash || 'Packet #' + pkt.id}</div>
${messageHtml}
<dl class="detail-meta">
<dt>Observer</dt><dd>${pkt.observer_name || pkt.observer_id || '—'}</dd>
<dt>Observer</dt><dd>${obsName(pkt.observer_id)}</dd>
<dt>SNR / RSSI</dt><dd>${snr != null ? snr + ' dB' : '—'} / ${rssi != null ? rssi + ' dBm' : '—'}</dd>
<dt>Route Type</dt><dd>${routeTypeName(pkt.route_type)}</dd>
<dt>Payload Type</dt><dd><span class="badge badge-${payloadTypeColor(pkt.payload_type)}">${typeName}</span></dd>
@@ -699,6 +808,7 @@
<dt>Path</dt><dd>${pathHops.length ? renderPath(pathHops) : '—'}</dd>
</dl>
<div class="detail-actions">
<button class="copy-link-btn" data-packet-id="${pkt.id}" title="Copy link to this packet">🔗 Copy Link</button>
${pathHops.length ? `<button class="detail-map-link" id="viewRouteBtn">🗺️ View route on map</button>` : ''}
<button class="replay-live-btn" title="Replay this packet on the live map">▶ Replay</button>
</div>
@@ -709,6 +819,20 @@
${hasRawHex ? buildFieldTable(pkt, decoded, pathHops, ranges) : buildDecodedTable(decoded)}
`;
// Wire up copy link button
const copyLinkBtn = panel.querySelector('.copy-link-btn');
if (copyLinkBtn) {
copyLinkBtn.addEventListener('click', () => {
const url = `${location.origin}/#/packet/${copyLinkBtn.dataset.packetId}`;
navigator.clipboard.writeText(url).then(() => {
copyLinkBtn.textContent = '✅ Copied!';
setTimeout(() => { copyLinkBtn.textContent = '🔗 Copy Link'; }, 1500);
}).catch(() => {
prompt('Copy this link:', url);
});
});
}
// Wire up replay button
const replayBtn = panel.querySelector('.replay-live-btn');
if (replayBtn) {
@@ -717,7 +841,7 @@
id: pkt.id, hash: pkt.hash,
_ts: new Date(pkt.timestamp).getTime(),
decoded: { header: { payloadTypeName: typeName }, payload: decoded, path: { hops: pathHops } },
snr: pkt.snr, rssi: pkt.rssi, observer: pkt.observer_name
snr: pkt.snr, rssi: pkt.rssi, observer: obsName(pkt.observer_id)
};
sessionStorage.setItem('replay-packet', JSON.stringify(livePkt));
window.location.hash = '#/live';
@@ -729,7 +853,7 @@
if (routeBtn && pathHops.length) {
routeBtn.addEventListener('click', async () => {
try {
const obsId = pkt.observer_name || pkt.observer_id || '';
const obsId = obsName(pkt.observer_id);
const observerParam = obsId ? '&observer=' + encodeURIComponent(obsId) : '';
const resp = await fetch('/api/resolve-hops?hops=' + encodeURIComponent(pathHops.join(',')) + observerParam);
const data = await resp.json();
@@ -967,11 +1091,10 @@
return '<div class="byop-row"><span class="byop-key">' + key + '</span><span class="byop-val">' + val + '</span></div>';
}
// Load regions from config
// Load regions from config API
(async () => {
try {
// We'll use a simple approach - hardcode from config
regionMap = {"SJC":"San Jose, US","SFO":"San Francisco, US","OAK":"Oakland, US","MRY":"Monterey, US","LAR":"Los Angeles, US"};
regionMap = await api('/config/regions', { ttl: 3600 });
} catch {}
})();
@@ -982,11 +1105,12 @@
renderTableRows();
return;
}
// Load children for this hash
// Load children (observations) for this hash
try {
const data = await api(`/packets?hash=${hash}&limit=20`);
const data = await api(`/packets?hash=${hash}&limit=1&expand=observations`);
const pkt = (data.packets || [])[0];
const group = packets.find(p => p.hash === hash);
if (group) group._children = data.packets || [];
if (group && pkt) group._children = (pkt.observations || []).map(o => ({...pkt, ...o, _isObservation: true}));
// Resolve any new hops from children
const childHops = new Set();
for (const c of (group?._children || [])) {
@@ -1007,4 +1131,31 @@
}
registerPage('packets', { init, destroy });
// Standalone packet detail page: #/packet/123
registerPage('packet-detail', {
init: async (app, routeParam) => {
const id = Number(routeParam);
app.innerHTML = `<div style="max-width:800px;margin:0 auto;padding:20px"><div class="text-center text-muted" style="padding:40px">Loading packet #${id}…</div></div>`;
try {
const data = await api(`/packets/${id}`);
if (!data?.packet) { app.innerHTML = `<div style="max-width:800px;margin:0 auto;padding:40px;text-align:center"><h2>Packet not found</h2><p>Packet #${id} doesn't exist.</p><a href="#/packets">← Back to packets</a></div>`; return; }
const hops = [];
try { const ph = JSON.parse(data.packet.path_json || '[]'); hops.push(...ph); } catch {}
const newHops = hops.filter(h => !(h in hopNameCache));
if (newHops.length) await resolveHops(newHops);
const container = document.createElement('div');
container.style.cssText = 'max-width:800px;margin:0 auto;padding:20px';
container.innerHTML = `<div style="margin-bottom:16px"><a href="#/packets" style="color:var(--primary);text-decoration:none">← Back to packets</a></div>`;
const detail = document.createElement('div');
container.appendChild(detail);
renderDetail(detail, data);
app.innerHTML = '';
app.appendChild(container);
} catch (e) {
app.innerHTML = `<div style="max-width:800px;margin:0 auto;padding:40px;text-align:center"><h2>Error</h2><p>${e.message}</p><a href="#/packets">← Back to packets</a></div>`;
}
},
destroy: () => {}
});
})();
+144
View File
@@ -0,0 +1,144 @@
/* === MeshCore Analyzer — perf.js === */
'use strict';
(function () {
let interval = null;
async function render(app) {
app.innerHTML = '<div style="height:100%;overflow-y:auto;padding:16px 24px;"><h2>⚡ Performance Dashboard</h2><div id="perfContent">Loading...</div></div>';
await refresh();
}
async function refresh() {
const el = document.getElementById('perfContent');
if (!el) return;
try {
const [server, client] = await Promise.all([
fetch('/api/perf').then(r => r.json()),
Promise.resolve(window.apiPerf ? window.apiPerf() : null)
]);
// Also fetch health telemetry
const health = await fetch('/api/health').then(r => r.json()).catch(() => null);
let html = '';
// Server overview
html += `<div style="display:flex;gap:16px;flex-wrap:wrap;margin:16px 0;">
<div class="perf-card"><div class="perf-num">${server.totalRequests}</div><div class="perf-label">Total Requests</div></div>
<div class="perf-card"><div class="perf-num">${server.avgMs}ms</div><div class="perf-label">Avg Response</div></div>
<div class="perf-card"><div class="perf-num">${health ? health.uptimeHuman : Math.round(server.uptime / 60) + 'm'}</div><div class="perf-label">Uptime</div></div>
<div class="perf-card"><div class="perf-num">${server.slowQueries.length}</div><div class="perf-label">Slow (&gt;100ms)</div></div>
</div>`;
// System health (memory, event loop, WS)
if (health) {
const m = health.memory, el = health.eventLoop;
const elColor = el.p95Ms > 500 ? '#ef4444' : el.p95Ms > 100 ? '#f59e0b' : '#22c55e';
const memColor = m.heapUsed > m.heapTotal * 0.85 ? '#ef4444' : m.heapUsed > m.heapTotal * 0.7 ? '#f59e0b' : '#22c55e';
html += `<h3>System Health</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num" style="color:${memColor}">${m.heapUsed}MB</div><div class="perf-label">Heap Used / ${m.heapTotal}MB</div></div>
<div class="perf-card"><div class="perf-num">${m.rss}MB</div><div class="perf-label">RSS</div></div>
<div class="perf-card"><div class="perf-num" style="color:${elColor}">${el.p95Ms}ms</div><div class="perf-label">Event Loop p95</div></div>
<div class="perf-card"><div class="perf-num">${el.maxLagMs}ms</div><div class="perf-label">EL Max Lag</div></div>
<div class="perf-card"><div class="perf-num">${el.currentLagMs}ms</div><div class="perf-label">EL Current</div></div>
<div class="perf-card"><div class="perf-num">${health.websocket.clients}</div><div class="perf-label">WS Clients</div></div>
</div>`;
}
// Cache stats
if (server.cache) {
const c = server.cache;
const clientCache = _apiCache ? _apiCache.size : 0;
html += `<h3>Cache</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${c.size}</div><div class="perf-label">Server Entries</div></div>
<div class="perf-card"><div class="perf-num">${c.hits}</div><div class="perf-label">Server Hits</div></div>
<div class="perf-card"><div class="perf-num">${c.misses}</div><div class="perf-label">Server Misses</div></div>
<div class="perf-card"><div class="perf-num" style="color:${c.hitRate > 50 ? '#22c55e' : c.hitRate > 20 ? '#f59e0b' : '#ef4444'}">${c.hitRate}%</div><div class="perf-label">Server Hit Rate</div></div>
<div class="perf-card"><div class="perf-num">${c.staleHits || 0}</div><div class="perf-label">Stale Hits (SWR)</div></div>
<div class="perf-card"><div class="perf-num">${c.recomputes || 0}</div><div class="perf-label">Recomputes</div></div>
<div class="perf-card"><div class="perf-num">${clientCache}</div><div class="perf-label">Client Entries</div></div>
</div>`;
if (client) {
html += `<div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${client.cacheHits || 0}</div><div class="perf-label">Client Hits</div></div>
<div class="perf-card"><div class="perf-num">${client.cacheMisses || 0}</div><div class="perf-label">Client Misses</div></div>
<div class="perf-card"><div class="perf-num" style="color:${(client.cacheHitRate||0) > 50 ? '#22c55e' : '#f59e0b'}">${client.cacheHitRate || 0}%</div><div class="perf-label">Client Hit Rate</div></div>
</div>`;
}
}
// Packet Store stats
if (server.packetStore) {
const ps = server.packetStore;
html += `<h3>In-Memory Packet Store</h3><div style="display:flex;gap:16px;flex-wrap:wrap;margin:8px 0;">
<div class="perf-card"><div class="perf-num">${ps.inMemory.toLocaleString()}</div><div class="perf-label">Packets in RAM</div></div>
<div class="perf-card"><div class="perf-num">${ps.estimatedMB}MB</div><div class="perf-label">Memory Used</div></div>
<div class="perf-card"><div class="perf-num">${ps.maxMB}MB</div><div class="perf-label">Memory Limit</div></div>
<div class="perf-card"><div class="perf-num">${ps.queries.toLocaleString()}</div><div class="perf-label">Queries Served</div></div>
<div class="perf-card"><div class="perf-num">${ps.inserts.toLocaleString()}</div><div class="perf-label">Live Inserts</div></div>
<div class="perf-card"><div class="perf-num">${ps.evicted.toLocaleString()}</div><div class="perf-label">Evicted</div></div>
<div class="perf-card"><div class="perf-num">${ps.indexes.byHash.toLocaleString()}</div><div class="perf-label">Unique Hashes</div></div>
<div class="perf-card"><div class="perf-num">${ps.indexes.byObserver}</div><div class="perf-label">Observers</div></div>
<div class="perf-card"><div class="perf-num">${ps.indexes.byNode.toLocaleString()}</div><div class="perf-label">Indexed Nodes</div></div>
</div>`;
}
// Server endpoints table
const eps = Object.entries(server.endpoints);
if (eps.length) {
html += '<h3>Server Endpoints (sorted by total time)</h3>';
html += '<div style="overflow-x:auto"><table class="perf-table"><thead><tr><th>Endpoint</th><th>Count</th><th>Avg</th><th>P50</th><th>P95</th><th>Max</th><th>Total</th></tr></thead><tbody>';
for (const [path, s] of eps) {
const total = Math.round(s.count * s.avgMs);
const cls = s.p95Ms > 200 ? ' class="perf-slow"' : s.p95Ms > 50 ? ' class="perf-warn"' : '';
html += `<tr${cls}><td><code>${path}</code></td><td>${s.count}</td><td>${s.avgMs}ms</td><td>${s.p50Ms}ms</td><td>${s.p95Ms}ms</td><td>${s.maxMs}ms</td><td>${total}ms</td></tr>`;
}
html += '</tbody></table></div>';
}
// Client API calls
if (client && client.endpoints.length) {
html += '<h3>Client API Calls (this session)</h3>';
html += '<div style="overflow-x:auto"><table class="perf-table"><thead><tr><th>Endpoint</th><th>Count</th><th>Avg</th><th>Max</th><th>Total</th></tr></thead><tbody>';
for (const s of client.endpoints) {
const cls = s.maxMs > 500 ? ' class="perf-slow"' : s.avgMs > 200 ? ' class="perf-warn"' : '';
html += `<tr${cls}><td><code>${s.path}</code></td><td>${s.count}</td><td>${s.avgMs}ms</td><td>${s.maxMs}ms</td><td>${s.totalMs}ms</td></tr>`;
}
html += '</tbody></table></div>';
}
// Slow queries
if (server.slowQueries.length) {
html += '<h3>Recent Slow Queries (&gt;100ms)</h3>';
html += '<div style="overflow-x:auto"><table class="perf-table"><thead><tr><th>Time</th><th>Path</th><th>Duration</th><th>Status</th></tr></thead><tbody>';
for (const q of server.slowQueries.slice().reverse()) {
html += `<tr class="perf-slow"><td>${new Date(q.time).toLocaleTimeString()}</td><td><code>${q.path}</code></td><td>${q.ms}ms</td><td>${q.status}</td></tr>`;
}
html += '</tbody></table></div>';
}
html += `<div style="margin-top:16px"><button id="perfReset" style="padding:8px 16px;cursor:pointer">Reset Stats</button> <button id="perfRefresh" style="padding:8px 16px;cursor:pointer">Refresh</button></div>`;
el.innerHTML = html;
document.getElementById('perfReset')?.addEventListener('click', async () => {
await fetch('/api/perf/reset', { method: 'POST' });
if (window._apiPerf) { window._apiPerf = { calls: 0, totalMs: 0, log: [] }; }
refresh();
});
document.getElementById('perfRefresh')?.addEventListener('click', refresh);
} catch (err) {
el.innerHTML = `<p style="color:red">Error: ${err.message}</p>`;
}
}
registerPage('perf', {
init(app) {
render(app);
interval = setInterval(refresh, 5000);
},
destroy() {
if (interval) { clearInterval(interval); interval = null; }
}
});
})();
+127
View File
@@ -0,0 +1,127 @@
/* === MeshCore Analyzer — roles.js (shared config module) === */
'use strict';
/*
* Centralized roles, thresholds, tile URLs, and UI constants.
* Loaded BEFORE all page scripts via index.html.
* Defaults are set synchronously; server config overrides arrive via fetch.
*/
(function () {
// ─── Role definitions ───
window.ROLE_COLORS = {
repeater: '#dc2626', companion: '#2563eb', room: '#16a34a',
sensor: '#d97706', observer: '#8b5cf6', unknown: '#6b7280'
};
window.ROLE_LABELS = {
repeater: 'Repeaters', companion: 'Companions', room: 'Room Servers',
sensor: 'Sensors', observer: 'Observers'
};
window.ROLE_STYLE = {
repeater: { color: '#dc2626', shape: 'diamond', radius: 10, weight: 2 },
companion: { color: '#2563eb', shape: 'circle', radius: 8, weight: 2 },
room: { color: '#16a34a', shape: 'square', radius: 9, weight: 2 },
sensor: { color: '#d97706', shape: 'triangle', radius: 8, weight: 2 },
observer: { color: '#8b5cf6', shape: 'star', radius: 11, weight: 2 }
};
window.ROLE_EMOJI = {
repeater: '◆', companion: '●', room: '■', sensor: '▲', observer: '★'
};
window.ROLE_SORT = ['repeater', 'companion', 'room', 'sensor', 'observer'];
// ─── Health thresholds (ms) ───
window.HEALTH_THRESHOLDS = {
infraDegradedMs: 86400000, // 24h
infraSilentMs: 259200000, // 72h
nodeDegradedMs: 3600000, // 1h
nodeSilentMs: 86400000 // 24h
};
// Helper: get degraded/silent thresholds for a role
window.getHealthThresholds = function (role) {
var isInfra = role === 'repeater' || role === 'room';
return {
degradedMs: isInfra ? HEALTH_THRESHOLDS.infraDegradedMs : HEALTH_THRESHOLDS.nodeDegradedMs,
silentMs: isInfra ? HEALTH_THRESHOLDS.infraSilentMs : HEALTH_THRESHOLDS.nodeSilentMs
};
};
// ─── Tile URLs ───
window.TILE_DARK = 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
window.TILE_LIGHT = 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png';
window.getTileUrl = function () {
var isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' &&
window.matchMedia('(prefers-color-scheme: dark)').matches);
return isDark ? TILE_DARK : TILE_LIGHT;
};
// ─── SNR thresholds ───
window.SNR_THRESHOLDS = { excellent: 6, good: 0 };
// ─── Distance thresholds (km) ───
window.DIST_THRESHOLDS = { local: 50, regional: 200 };
// ─── MAX_HOP_DIST (degrees, ~200km ≈ 1.8°) ───
window.MAX_HOP_DIST = 1.8;
// ─── Result limits ───
window.LIMITS = {
topNodes: 15,
topPairs: 12,
topRingNodes: 8,
topSenders: 10,
topCollisionNodes: 10,
recentReplay: 8,
feedMax: 25
};
// ─── Performance thresholds ───
window.PERF_SLOW_MS = 100;
// ─── WebSocket reconnect delay (ms) ───
window.WS_RECONNECT_MS = 3000;
// ─── Cache invalidation debounce (ms) ───
window.CACHE_INVALIDATE_MS = 5000;
// ─── External URLs ───
window.EXTERNAL_URLS = {
flasher: 'https://flasher.meshcore.co.uk/'
};
// ─── Fetch server overrides ───
window.MeshConfigReady = fetch('/api/config/client').then(function (r) { return r.json(); }).then(function (cfg) {
if (cfg.roles) {
if (cfg.roles.colors) Object.assign(ROLE_COLORS, cfg.roles.colors);
if (cfg.roles.labels) Object.assign(ROLE_LABELS, cfg.roles.labels);
if (cfg.roles.style) {
for (var k in cfg.roles.style) ROLE_STYLE[k] = Object.assign(ROLE_STYLE[k] || {}, cfg.roles.style[k]);
}
if (cfg.roles.emoji) Object.assign(ROLE_EMOJI, cfg.roles.emoji);
if (cfg.roles.sort) window.ROLE_SORT = cfg.roles.sort;
}
if (cfg.healthThresholds) Object.assign(HEALTH_THRESHOLDS, cfg.healthThresholds);
if (cfg.tiles) {
if (cfg.tiles.dark) window.TILE_DARK = cfg.tiles.dark;
if (cfg.tiles.light) window.TILE_LIGHT = cfg.tiles.light;
}
if (cfg.snrThresholds) Object.assign(SNR_THRESHOLDS, cfg.snrThresholds);
if (cfg.distThresholds) Object.assign(DIST_THRESHOLDS, cfg.distThresholds);
if (cfg.maxHopDist != null) window.MAX_HOP_DIST = cfg.maxHopDist;
if (cfg.limits) Object.assign(LIMITS, cfg.limits);
if (cfg.perfSlowMs != null) window.PERF_SLOW_MS = cfg.perfSlowMs;
if (cfg.wsReconnectMs != null) window.WS_RECONNECT_MS = cfg.wsReconnectMs;
if (cfg.cacheInvalidateMs != null) window.CACHE_INVALIDATE_MS = cfg.cacheInvalidateMs;
if (cfg.externalUrls) Object.assign(EXTERNAL_URLS, cfg.externalUrls);
// Sync ROLE_STYLE colors with ROLE_COLORS
for (var role in ROLE_STYLE) {
if (ROLE_COLORS[role]) ROLE_STYLE[role].color = ROLE_COLORS[role];
}
}).catch(function () { /* use defaults */ });
})();
+77 -14
View File
@@ -226,6 +226,7 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
max-width: 0; /* forces td to respect table width instead of expanding to content */
}
.data-table td.col-details { white-space: normal; word-break: break-word; }
.data-table td:has(.spark-bar), .data-table td.col-spark { max-width: none; overflow: visible; min-width: 80px; }
.data-table tbody tr:nth-child(even) { background: var(--row-stripe); }
.data-table tbody tr:hover { background: var(--row-hover); cursor: pointer; }
.data-table tbody tr.selected { background: var(--selected-bg); }
@@ -262,6 +263,11 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
font-size: 10px; font-weight: 700; font-family: var(--mono);
background: var(--nav-bg); color: #fff; letter-spacing: .5px;
}
.badge-obs {
display: inline-block; padding: 1px 6px; border-radius: 10px;
font-size: 10px; font-weight: 600;
background: #ede9fe; color: #6d28d9;
}
/* === Monospace === */
.mono { font-family: var(--mono); font-size: 12px; }
@@ -681,7 +687,7 @@ button.ch-item.selected { background: var(--selected-bg); }
::-webkit-scrollbar-thumb:hover { background: #94a3b8; }
/* === Observers Page === */
.observers-page { padding: 20px; max-width: 1200px; margin: 0 auto; }
.observers-page { padding: 20px; max-width: 1200px; margin: 0 auto; overflow-y: auto; height: calc(100vh - 56px); }
.obs-summary { display: flex; gap: 20px; margin-bottom: 16px; flex-wrap: wrap; }
.obs-stat { display: flex; align-items: center; gap: 6px; font-size: 14px; color: var(--text-muted); }
.health-dot { width: 10px; height: 10px; border-radius: 50%; display: inline-block; flex-shrink: 0; }
@@ -689,6 +695,8 @@ button.ch-item.selected { background: var(--selected-bg); }
.health-dot.health-yellow { background: #eab308; box-shadow: 0 0 6px #eab30880; }
.health-dot.health-red { background: #ef4444; box-shadow: 0 0 6px #ef444480; }
.obs-table td:first-child { white-space: nowrap; }
.obs-table td:nth-child(6) { max-width: none; overflow: visible; }
.col-observer { min-width: 70px; max-width: none; }
.spark-bar { position: relative; min-width: 60px; max-width: 100px; flex: 1; height: 18px; background: var(--border); border-radius: 4px; overflow: hidden; display: inline-block; vertical-align: middle; }
@media (max-width: 640px) { .spark-bar { max-width: 60px; } }
.spark-fill { height: 100%; background: linear-gradient(90deg, #3b82f6, #60a5fa); border-radius: 4px; transition: width 0.3s; }
@@ -811,8 +819,8 @@ button.ch-item.selected { background: var(--selected-bg); }
/* Layouts: stack instead of side-by-side */
.split-layout { flex-direction: column; overflow-y: auto; }
.panel-left { padding: 10px; flex: none; min-height: 50vh; overflow-x: auto; }
.panel-right { width: 100%; min-width: 0; border-left: none; border-top: 1px solid var(--border); max-height: none; flex: none; }
.panel-left { padding: 6px; flex: 1; min-height: 0; overflow-x: auto; -webkit-overflow-scrolling: touch; }
.panel-right { display: none; }
/* Channels: Discord-style full screen toggle */
.ch-layout { flex-direction: row; position: relative; }
@@ -830,18 +838,21 @@ button.ch-item.selected { background: var(--selected-bg); }
.ch-back-btn { display: flex; }
.ch-main-header { display: flex; align-items: center; gap: 8px; }
/* Tables: smaller text, allow horizontal scroll */
.data-table { font-size: 12px; }
.data-table td { padding: 6px 6px; max-width: 120px; }
.data-table th { padding: 6px 6px; font-size: 11px; }
.panel-left { overflow-x: auto; -webkit-overflow-scrolling: touch; }
.data-table { min-width: 500px; }
/* Tables: smaller text for mobile */
.data-table { font-size: 11px; min-width: 0; }
.data-table td { padding: 5px 4px; max-width: 100px; }
.data-table th { padding: 5px 4px; font-size: 10px; }
.panel-left { overflow-x: auto; }
/* Filters: full width */
.filter-bar { flex-direction: column; gap: 4px; }
.filter-bar input { width: 100%; }
.filter-bar select { width: 100%; }
.filter-bar .btn { min-height: 44px; }
/* Filters: collapse on mobile */
.filter-bar { flex-direction: row; flex-wrap: wrap; gap: 4px; }
.filter-toggle-btn { display: inline-flex !important; }
.filter-bar > *:not(.filter-toggle-btn):not(.col-toggle-wrap) { display: none; }
.filter-bar.filters-expanded > * { display: inline-flex; }
.filter-bar.filters-expanded > .col-toggle-wrap { display: inline-block; }
.filter-bar.filters-expanded input { width: 100%; }
.filter-bar.filters-expanded select { width: 100%; }
.filter-bar .btn { min-height: 36px; }
.node-filter-wrap { width: 100%; }
/* Nodes */
@@ -1174,6 +1185,19 @@ button.ch-item.ch-item-encrypted .ch-badge { filter: grayscale(0.6); }
}
.detail-map-link:hover { background: rgba(245, 158, 11, 0.25); }
.copy-link-btn {
padding: 5px 12px;
background: rgba(59, 130, 246, 0.12);
border: 1px solid rgba(59, 130, 246, 0.25);
color: var(--primary, #3b82f6);
border-radius: 6px;
font-size: 0.78rem;
font-weight: 600;
cursor: pointer;
transition: background 0.15s;
}
.copy-link-btn:hover { background: rgba(59, 130, 246, 0.25); }
/* Route tooltip on map */
.route-tooltip {
background: rgba(0,0,0,0.8) !important;
@@ -1384,3 +1408,42 @@ tr[data-hops]:hover { background: rgba(59,130,246,0.1); }
.claimed-row { background: color-mix(in srgb, var(--accent) 8%, transparent) !important; border-left: 3px solid var(--accent); }
.claimed-row:hover { background: color-mix(in srgb, var(--accent) 14%, transparent) !important; }
.claimed-badge { color: var(--accent); font-size: 13px; margin-right: 2px; }
/* Filter toggle button — hidden on desktop */
.filter-toggle-btn { display: none; }
/* Mobile detail bottom sheet */
.mobile-detail-sheet {
display: none;
position: fixed; bottom: 0; left: 0; right: 0;
max-height: 70vh; background: var(--detail-bg);
border-top-left-radius: 16px; border-top-right-radius: 16px;
box-shadow: 0 -4px 24px rgba(0,0,0,.3);
z-index: 200; overflow-y: auto; padding: 8px 16px 24px;
transform: translateY(100%); transition: transform .25s ease;
}
.mobile-detail-sheet.open { display: block; transform: translateY(0); }
.mobile-sheet-handle {
width: 40px; height: 4px; background: var(--border);
border-radius: 2px; margin: 4px auto 8px; cursor: pointer;
}
.mobile-sheet-close {
position: absolute; top: 8px; right: 12px;
background: none; border: none; font-size: 20px;
color: var(--text-muted); cursor: pointer; z-index: 1;
}
.mobile-sheet-close:hover { color: var(--text); }
.mobile-sheet-content { padding-top: 4px; }
/* Perf dashboard */
.perf-card { background: var(--surface-1); border: 1px solid var(--border); border-radius: 8px; padding: 12px 20px; min-width: 120px; text-align: center; }
.perf-num { font-size: 24px; font-weight: 800; color: var(--text); font-variant-numeric: tabular-nums; }
.perf-label { font-size: 11px; color: var(--text-muted); text-transform: uppercase; letter-spacing: 0.5px; margin-top: 4px; }
.perf-table { width: 100%; border-collapse: collapse; font-size: 13px; }
.perf-table th { text-align: left; padding: 6px 10px; border-bottom: 2px solid var(--border); color: var(--text-muted); font-size: 11px; text-transform: uppercase; }
.perf-table td { padding: 5px 10px; border-bottom: 1px solid var(--border); font-variant-numeric: tabular-nums; }
.perf-table code { font-size: 12px; color: var(--text); }
.perf-table .perf-slow { background: rgba(239, 68, 68, 0.08); }
.perf-table .perf-slow td { color: #ef4444; }
.perf-table .perf-warn { background: rgba(251, 191, 36, 0.06); }
.perf-table .perf-warn td { color: #f59e0b; }
+137
View File
@@ -0,0 +1,137 @@
#!/usr/bin/env node
/**
* Milestone 1: Packet Dedup Schema Migration
*
* Creates `transmissions` and `observations` tables from the existing `packets` table.
* Idempotent drops and recreates new tables on each run.
* Does NOT touch the original `packets` table.
*
* Usage: node scripts/migrate-dedup.js <path-to-meshcore.db>
*/
const Database = require('better-sqlite3');
const path = require('path');
const dbPath = process.argv[2];
if (!dbPath) {
console.error('Usage: node scripts/migrate-dedup.js <path-to-meshcore.db>');
process.exit(1);
}
const start = Date.now();
const db = new Database(dbPath);
db.pragma('journal_mode = WAL');
db.pragma('foreign_keys = ON');
// --- Drop existing new tables (idempotent) ---
console.log('Dropping existing transmissions/observations tables if they exist...');
db.exec('DROP TABLE IF EXISTS observations');
db.exec('DROP TABLE IF EXISTS transmissions');
// --- Create new tables ---
console.log('Creating transmissions and observations tables...');
db.exec(`
CREATE TABLE transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
hash TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX idx_transmissions_hash ON transmissions(hash);
CREATE INDEX idx_transmissions_first_seen ON transmissions(first_seen);
CREATE INDEX idx_transmissions_payload_type ON transmissions(payload_type);
CREATE INDEX idx_observations_hash ON observations(hash);
CREATE INDEX idx_observations_transmission_id ON observations(transmission_id);
CREATE INDEX idx_observations_observer_id ON observations(observer_id);
CREATE INDEX idx_observations_timestamp ON observations(timestamp);
`);
// --- Read all packets ordered by timestamp ---
console.log('Reading packets...');
const packets = db.prepare('SELECT * FROM packets ORDER BY timestamp ASC').all();
const totalPackets = packets.length;
console.log(`Total packets: ${totalPackets}`);
// --- Group by hash and migrate ---
const insertTransmission = db.prepare(`
INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json)
VALUES (?, ?, ?, ?, ?, ?, ?)
`);
const insertObservation = db.prepare(`
INSERT INTO observations (transmission_id, hash, observer_id, observer_name, direction, snr, rssi, score, path_json, timestamp)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
const hashToTransmissionId = new Map();
let transmissionCount = 0;
const migrate = db.transaction(() => {
for (const pkt of packets) {
let txId = hashToTransmissionId.get(pkt.hash);
if (txId === undefined) {
const result = insertTransmission.run(
pkt.raw_hex, pkt.hash, pkt.timestamp,
pkt.route_type, pkt.payload_type, pkt.payload_version, pkt.decoded_json
);
txId = result.lastInsertRowid;
hashToTransmissionId.set(pkt.hash, txId);
transmissionCount++;
}
insertObservation.run(
txId, pkt.hash, pkt.observer_id, pkt.observer_name, pkt.direction,
pkt.snr, pkt.rssi, pkt.score, pkt.path_json, pkt.timestamp
);
}
});
migrate();
// --- Verify ---
const obsCount = db.prepare('SELECT COUNT(*) as c FROM observations').get().c;
const txCount = db.prepare('SELECT COUNT(*) as c FROM transmissions').get().c;
const distinctHash = db.prepare('SELECT COUNT(DISTINCT hash) as c FROM packets').get().c;
const elapsed = ((Date.now() - start) / 1000).toFixed(2);
console.log('\n=== Migration Stats ===');
console.log(`Total packets (source): ${totalPackets}`);
console.log(`Unique transmissions created: ${transmissionCount}`);
console.log(`Observations created: ${obsCount}`);
console.log(`Dedup ratio: ${(totalPackets / transmissionCount).toFixed(2)}x`);
console.log(`Time taken: ${elapsed}s`);
console.log('\n=== Verification ===');
const obsOk = obsCount === totalPackets;
const txOk = txCount === distinctHash;
console.log(`observations (${obsCount}) = packets (${totalPackets}): ${obsOk ? 'PASS ✓' : 'FAIL ✗'}`);
console.log(`transmissions (${txCount}) = distinct hashes (${distinctHash}): ${txOk ? 'PASS ✓' : 'FAIL ✗'}`);
if (!obsOk || !txOk) {
console.error('\nVerification FAILED!');
process.exit(1);
}
console.log('\nMigration complete!');
db.close();
+30
View File
@@ -0,0 +1,30 @@
#!/bin/sh
# Pre-push validation — catches common JS errors before they hit prod
set -e
echo "=== Syntax check ==="
node -c server.js
for f in public/*.js; do node -c "$f"; done
echo "✅ All JS files parse OK"
echo "=== Checking for undefined common references ==="
ERRORS=0
# esc() should only exist inside IIFEs that define it, not in files that don't
for f in public/live.js public/map.js public/home.js public/nodes.js public/channels.js public/observers.js; do
if grep -q '\besc(' "$f" 2>/dev/null && ! grep -q 'function esc' "$f" 2>/dev/null; then
REFS=$(grep -n '\besc(' "$f" | grep -v escapeHtml | grep -v "desc\|Esc\|resc\|safeEsc" || true)
if [ -n "$REFS" ]; then
echo "$f uses esc() but doesn't define it:"
echo "$REFS"
ERRORS=$((ERRORS + 1))
fi
fi
done
if [ "$ERRORS" -gt 0 ]; then
echo "$ERRORS validation error(s) found"
exit 1
fi
echo "✅ Validation passed"
+995 -185
View File
File diff suppressed because it is too large Load Diff