Compare commits

..

197 Commits

Author SHA1 Message Date
you
1050aa52d9 fix: pause button delegation on document instead of app element 2026-03-22 04:59:45 +00:00
you
e8d9aa6839 v2.4.1 — hotfix for ingestion, WS, and animation regressions 2026-03-22 04:57:29 +00:00
you
daa84fdd73 fix: null-guard animLayer/pathsLayer in animation cleanup
setInterval callbacks fire after navigating away from live map,
when layers are already destroyed.
2026-03-22 04:52:45 +00:00
you
01df3c70d8 fix: null-guard multi-select menu close handler
observerFilterWrap/typeFilterWrap don't exist when document click
fires before renderLeft completes.
2026-03-22 04:49:51 +00:00
you
368fef8713 Remove debug console.log from packets WS handler 2026-03-22 04:47:10 +00:00
you
657653dd3b fix: pause button crash killed WS handler registration
getElementById('pktPauseBtn') was null because button is rendered by
loadPackets() which runs async. Crash at line 218 prevented wsHandler
from being registered at line 260. Moved to data-action event delegation
which works regardless of render timing.
2026-03-22 04:45:04 +00:00
you
11828a321a debug: add WS handler logging to packets page 2026-03-22 04:42:41 +00:00
you
f09e338dfa fix: WS broadcast had null packet when observation was deduped
getById() returns null for deduped observations (not stored in byId).
Client filters on m.data.packet being truthy, so all deduped packets
were silently dropped from WS. Fallback to transmission or raw pktData.
2026-03-22 04:38:07 +00:00
you
87672797f9 docs: add ingestion regression to v2.4.0 changelog 2026-03-22 01:23:36 +00:00
you
b337da6461 fix: packet-store insert() returned undefined after insertPacket removal
Was returning undeclared 'id' variable. Now returns observationId or
transmissionId. This broke all MQTT packet ingestion on prod.
2026-03-22 01:22:21 +00:00
you
bd32aa9565 docs: tone down v2.4.0 description 2026-03-22 01:13:47 +00:00
you
60219ae02a v2.4.0 — The Observatory
Multi-select filters, time window selector, pause button, hex paths toggle,
legacy DB migration, pure client-side filtering, distance analytics,
observation drill-down, and 20+ bug fixes.

Full changelog: CHANGELOG.md
2026-03-22 01:12:20 +00:00
you
5e28fb15c9 Fix time window: remove label, self-descriptive options, restore saved value before first load 2026-03-22 01:10:32 +00:00
you
98f958641a Add pause/resume button to packets page for live WS updates 2026-03-22 01:08:19 +00:00
you
3a8d92e39b Replace packet count limit with time window selector
- Add time window dropdown (15min default, up to 24h or All)
- Use 'since' param instead of fixed limit=10000
- Persist selection to localStorage
- Keep limit=50000 as safety net
2026-03-22 01:06:46 +00:00
you
0f8d63636b Increase default packet limit from 100 to 10,000
Filtering is client-side now, so load everything upfront.
2026-03-22 01:01:21 +00:00
you
132ab60e06 Fix header row observer display when filtering by observer 2026-03-22 00:57:55 +00:00
you
c123ee731d Fix multi-select filters: client-side filtering, no unnecessary API calls
- Remove type/observer params from /api/packets calls (server ignores them)
- Add client-side type/observer filtering in renderTableRows()
- Change filter handlers to re-render instead of re-fetch
- Update displayed count to reflect filtered results
2026-03-22 00:57:11 +00:00
you
0fb586730b docs: update v2.4.0 release notes — multi-select filters, hex paths toggle, DB migration 2026-03-22 00:56:48 +00:00
you
7e304e60d5 fix: hex toggle calls renderTableRows (was undefined renderTable), rename to 'Hex Paths' with descriptive tooltip 2026-03-22 00:46:21 +00:00
you
9171eecc69 Drop legacy packets + paths tables on startup, remove dead code
Migration runs automatically on next startup — drops paths first (FK to
packets), then packets. Removes insertPacket(), insertPath(), all
prepared statements and references to both tables. Server-side type/
observer filtering also removed (client does it in-memory).

Saves ~2M rows (paths) + full packets table worth of disk.
2026-03-22 00:45:27 +00:00
you
552ba2f970 Add hex hash toggle to packets page filter bar 2026-03-22 00:34:46 +00:00
you
3ff29519b4 Fix multi-select filters (AND→OR) and add localStorage persistence
- Server: support comma-separated type filter values (OR logic)
- Server: add observer_id filtering to /api/packets endpoint
- Client: fix type and observer filters to use OR logic for multi-select
- Client: persist observer and type filter selections to localStorage
- Keys: meshcore-observer-filter, meshcore-type-filter
2026-03-22 00:34:38 +00:00
you
39c1782881 docs: tone down v2.4.0 release notes 2026-03-22 00:24:46 +00:00
you
cbd13c00d6 docs: v2.4.0 release notes — The Observatory 2026-03-22 00:22:32 +00:00
you
4b1c1c3f22 Convert Observer and Type filters to multi-select checkbox dropdowns
- Replace single-select dropdowns with multi-select checkbox menus
- Add .multi-select-* CSS classes (reusable, styled like region filter)
- Observer: comma-separated IDs, shows count/name/All Observers
- Type: comma-separated values, shows count/name/All Types
- Bug fix: filter group children by selected observer(s) when expanded
- Close dropdowns on outside click
2026-03-22 00:21:41 +00:00
you
0d3b12d40a fix: sort help tooltip renders below icon instead of above (behind navbar) 2026-03-22 00:20:02 +00:00
you
e000db438c fix: sort help tooltip uses real DOM element instead of CSS attr()
CSS content:attr() doesn't support newlines in any browser. Replaced
with a real <span> child element with white-space:pre-line, shown on
hover via .sort-help:hover .sort-help-tip { display: block }.
2026-03-22 00:17:04 +00:00
you
09f6f106ba fix: sort help uses CSS tooltip instead of title attribute
Native title tooltips are unreliable (delayed, sometimes not shown).
Replaced with CSS ::after pseudo-element tooltip using data-tip attr.
Shows immediately on hover with proper formatting.
2026-03-22 00:15:33 +00:00
you
a299798b31 fix: sort help tooltip — set title via JS so newlines render
HTML entities like &#10; don't work inside JS template literals
(inserted as literal text). Setting .title via JS with actual \n
newlines works correctly in browser tooltips.
2026-03-22 00:13:03 +00:00
you
181fddf196 fix: ungrouped mode flattens observations into individual rows
When Group by Hash is off, fetches all observations for multi-obs
packets and flattens them into individual rows showing each observer's
view. Previously just showed grouped transmissions without expand arrows.
2026-03-22 00:12:16 +00:00
you
2ae467bc72 fix: region box alignment, column checkboxes, sort help, dark mode active btn
- Region filter container: remove margin-bottom, use inline-flex align
- Column dropdown checkboxes: 14x14px to match region dropdown
- Sort help ⓘ: use &#10; for newlines in title (\n doesn't render)
- Dark mode: .filter-bar .btn.active now retains accent background
  (dark theme override was clobbering the active state)
2026-03-22 00:08:00 +00:00
you
a11ace77ac Polish filter bar: consistent sizing, logical grouping with separators, tooltips
- All filter-bar controls now exactly 34px tall with line-height:1 and border-radius:6px
- col-toggle-btn matched to same height/font-size as other controls
- Controls grouped into 4 logical sections (Filters, Display, Sort, Columns) with vertical separators
- Added title attributes with helpful descriptions to all controls
- Added sort help icon (ⓘ) with detailed tooltip explaining each sort mode
- Mobile responsive: separators hidden on small screens
2026-03-22 00:00:02 +00:00
you
b1c2e817fa fix: shrink region dropdown checkboxes to 14px 2026-03-21 23:55:18 +00:00
you
3d31afd0ec fix: resolve hops after sort changes header paths
When sort updates header row path_json, new hop hashes may not be in
hopNameCache. Now resolves unknown hops before re-rendering.
2026-03-21 23:54:49 +00:00
you
28ac094d83 fix: normalize filter bar heights + widen region dropdown
All filter bar controls now share: height 34px, font-size 13px,
border-radius 6px, same padding. Region dropdown trigger matches
other controls, menu widened to 220px with white-space:nowrap to
prevent text wrapping.
2026-03-21 23:53:36 +00:00
you
8c4b3f029f fix: sort change fetches observations for all visible groups
When switching to a non-Observer sort, batch-fetches observations for
all visible multi-observation groups that haven't been expanded yet.
Header rows update immediately without needing manual expand.
2026-03-21 23:52:20 +00:00
you
3b1e16a1d6 feat: sort header reflects first sorted child + asc/desc for path & time
- Header row (observer, path) updates to match the first child after sort
- Path sort: ascending (shortest first) and descending (longest first)
- Chronological: ascending (earliest first) and descending (latest first)
- Observer mode unchanged
2026-03-21 23:49:01 +00:00
you
fa7f1cf76a fix: region dropdown labels show 'IATA - Friendly Name' format
Dropdown items now display 'SJC - San Jose, US' instead of just
'San Jose, US'. Summary shows just IATA codes for brevity.
2026-03-21 23:47:45 +00:00
you
b800d77570 fix: apply observation sort on all code paths that set _children
Sort was only applied in pktToggleGroup and dropdown change handler.
Missing from: loadPackets restore (re-fetches children for expanded
groups) and WS update path (unshifts new observations). Now all
three paths call sortGroupChildren after modifying _children.
2026-03-21 23:46:30 +00:00
you
7e0cd455ae fix: move observation sort to filter bar dropdown, save to localStorage
- Removed per-group sort bar links (broken navigation)
- Added global 'Sort:' dropdown in filter toolbar
- Persists to localStorage across sessions
- Re-sorts all expanded groups on change
2026-03-21 23:41:24 +00:00
you
4f66e377d1 feat: add Chronological sort mode for expanded packet groups
Pure timestamp order, no grouping — shows propagation sequence.
2026-03-21 23:32:05 +00:00
you
1877c49adc feat: observation sort toggle — Observer (default) or Path length
Two sort modes for expanded packet groups:
- Observer: group by observer, earliest first, ascending time within
- Path length: shortest paths first, alphabetical observer within

Sort bar appears above expanded children with bold active mode.
2026-03-21 23:31:21 +00:00
you
461fb7ee68 fix: trace page accepts route param (#/traces/HASH) 2026-03-21 23:17:32 +00:00
you
679f9a552f fix: deeplink uses observation id (not observer_id) + add trace link
- Deeplinks now use ?obs=<observation_id> which is unique per row,
  fixing cases where same observer has multiple paths
- Added '🔍 Trace' link in detail pane actions
2026-03-21 23:15:42 +00:00
you
f1e3a57fcf fix: set header observer/path to earliest observation on DB load
During _loadNormalized(), observations load in DESC order so the first
observation processed is the LATEST. tx.observer_id was set from this
latest observation. Added post-load pass that finds the earliest
observation by timestamp and sets tx.observer_id/path_json to match.
2026-03-21 23:10:10 +00:00
you
1dc5daab67 fix: stop client WS handler from replacing header path with longest path
The WS handler was overwriting the group's path_json with the longest
path from any new observation. Header should always show the first
observer's path — individual observation paths are in the expanded rows.
2026-03-21 23:06:36 +00:00
you
8892fb0f66 docs: update v2.4.0 release notes with latest fixes 2026-03-21 23:03:13 +00:00
you
98a6cbd3b4 revert: undo unauthorized version bump 2026-03-21 23:01:57 +00:00
you
ad6a796b35 Disable auto-seeding on empty DB - require --seed flag or SEED_DB=true
Previously db.seed() ran unconditionally on startup and would populate
a fresh database with fake test data. Now seeding only triggers when
explicitly requested via --seed CLI flag or SEED_DB=true env var.

The seed functionality remains available for developers:
  node server.js --seed
  SEED_DB=true node server.js
  node db.js  (direct run still seeds)
2026-03-21 23:00:01 +00:00
you
1b2f28cb5f Add observation-level deeplinks to packet detail page
When viewing a specific observation, the URL now includes ?obs=OBSERVER_ID.
Opening such a link auto-expands the group and selects the observation.
Copy Link button includes the obs parameter when an observation is selected.
2026-03-21 22:59:07 +00:00
you
2894c38435 fix: header row shows first observer's path, not longest path
Removed server-side longest-path override in /api/packets/:id that
replaced the transmission's path_json with the longest observation
path. The header should always reflect the first observer's path.
Individual observation paths are available in the observations array.
2026-03-21 22:58:37 +00:00
you
97be64353a fix: detail pane shows clicked observation data, not parent packet
When expanding a grouped packet and clicking a child observation row,
the detail pane now shows that observation's observer, SNR/RSSI, path,
and timestamp instead of the parent packet's data.

Child rows use a new 'select-observation' action that builds a synthetic
packet object by overlaying observation-specific fields onto the parent
packet data (no extra API fetch needed).
2026-03-21 22:47:01 +00:00
you
8ae7d7710b docs: v2.4.0 release notes 2026-03-21 22:44:31 +00:00
you
b61b71635b fix: update observer_id, observer_name, path_json when first_seen moves earlier
When a later observation has an earlier timestamp, the transmission's
first_seen was updated but observer_id and path_json were not. This
caused the header row to show the wrong observer and path — whichever
MQTT message arrived first, not whichever observation was actually
earliest.
2026-03-21 22:43:18 +00:00
you
04a138eba3 tweak: bump chan-tag size from 0.8em to 0.9em with more padding 2026-03-21 22:33:14 +00:00
you
f51db66775 feat: show channel name tag in packet detail column for CHAN messages 2026-03-21 22:27:51 +00:00
you
75d76cf68e fix: use correct field names for observation sort (observer_name, timestamp)
Subagent used observer/rx_at/created_at but API returns
observer_name/timestamp. Sort was comparing empty strings.
2026-03-21 22:26:39 +00:00
you
1e61e021ec cleanup: remove cracker package files and temp docs accidentally committed by subagent 2026-03-21 22:24:12 +00:00
you
c4d6fb7cd3 Fix expanded packet group observation sorting: group by observer, earliest first 2026-03-21 22:22:44 +00:00
you
1158161e1a Fix channel list timeAgo counters: use monotonic lastActivityMs instead of ISO strings
- Track lastActivityMs (Date.now()) on each channel object instead of ISO lastActivity
- 1s interval iterates channels[] array and updates DOM text only (no re-render)
- Uses data-channel-hash attribute to find time elements after DOM rebuilds
- Simple formatSecondsAgo: <60s→Xs, <3600s→Xm, <86400s→Xh, else Xd
- Seed lastActivityMs from API ISO string on initial load
- WS handler sets lastActivityMs = Date.now() on receipt
- Bump channels.js cache buster
2026-03-21 22:21:28 +00:00
you
81f284e952 Fix duplicate observations in expanded packet group view
The insert() method had a second code path (for building rows from
packetData) that pushed observations without checking for duplicates.
Added the same dedup check (observer_id + path_json) that exists in
the other insert path and in the load-from-DB path.
2026-03-21 22:18:04 +00:00
you
6f64ed6b9b fix: tick channel timeAgo labels every 1s instead of re-rendering every 30s 2026-03-21 22:15:25 +00:00
you
d51c7a780c fix: use current time for channel lastActivity on WS updates
packet.timestamp is first_seen — when the transmission was originally
observed. When multiple observers re-see the same old packet, the
broadcast carries the original (stale) first_seen. For channel list
display, what matters is 'activity happened now', not 'packet was
first seen 10h ago'.
2026-03-21 22:12:56 +00:00
you
4a909fbd0b fix: deduplicate observations with NULL path_json
The UNIQUE index on (hash, observer_id, path_json) didn't prevent
duplicates when path_json was NULL because SQLite treats NULL != NULL
for uniqueness. Fixed by:

1. Using COALESCE(path_json, '') in the UNIQUE index expression
2. Adding migration to clean up existing duplicate rows
3. Adding NULL-safe dedup checks in PacketStore load and insert paths
2026-03-21 22:06:43 +00:00
you
105f8546b1 debug: log channel WS timestamp values 2026-03-21 22:05:52 +00:00
you
eca0c9bd61 fix: use packet hash instead of sender_timestamp for channel message dedup
Device clocks on MeshCore nodes are wildly inaccurate (off by hours or
epoch-near values like 4). The channel messages endpoint was using
sender_timestamp as part of the deduplication key, which could cause
messages to fail deduplication or incorrectly collide.

Changed dedupe key from sender:timestamp to sender:hash, which is the
correct unique identifier for a transmission.

Also added TIMESTAMP-AUDIT.md documenting all device timestamp usage.
2026-03-21 21:53:38 +00:00
you
3c12690ccb fix: use server timestamp for channel lastActivity, not device clock
WS messages from lincomatic bridge lack packet.timestamp, so the
code fell through to payload.sender_timestamp which reflects the
MeshCore device's clock (often wrong). Use current time as fallback.
2026-03-21 21:50:18 +00:00
you
ddb6dcb113 fix: tick channel list relative timestamps every 30s
timeAgo labels were computed once on render and never updated,
showing stale '11h ago' until next WS message triggered re-render.
Added 30s interval to re-render channel list, cleaned up on destroy.
2026-03-21 21:41:30 +00:00
you
746f5cf3b1 fix: dedup live channel messages by packet hash, not sender+timestamp
Multiple observers seeing the same packet triggered separate message
entries. Now deduplicates by packet hash — additional observations
increment repeats count and add to observers list. Channel messageCount
also only bumps once per unique packet.
2026-03-21 21:38:32 +00:00
you
1320e33bd6 feat: zero-API live updates on channels page via WS
Instead of re-fetching /api/channels and /api/channels/:hash/messages
on every WebSocket event, the channels page now processes WS messages
client-side:

- Extract sender, text, channel, timestamp from WS payload
- Append new messages directly to local messages[] array
- Update channel list entries (lastActivity, lastSender, messageCount)
- Create new channel entries for previously unseen channels
- Deduplicate repeated observations of the same message

API calls now only happen on:
- Initial page load (loadChannels)
- Channel selection (selectChannel)
- Region filter change

This eliminates all polling and WS-triggered re-fetches.
2026-03-21 21:35:30 +00:00
you
00ce8de7bc fix: stop channels page from spamming API requests
The channels WS handler was calling invalidateApiCache() before
loadChannels()/refreshMessages(), which nuked the cache and forced
network fetches. Combined with the global WS onmessage handler also
invalidating /channels every 5s, this created excessive API traffic
when sitting idle on the channels page.

Changes:
- channels.js: Remove invalidateApiCache calls from WS handler, use
  bust:true parameter instead to bypass cache only when WS triggers
- channels.js: Add bust parameter to loadChannels() and refreshMessages()
- app.js: Remove /channels from global WS cache invalidation (channels
  page manages its own cache busting via its dedicated WS handler)
2026-03-21 21:34:05 +00:00
you
32b897d8f3 fix: remove ADVERT timestamp validation — field isn't stored or used
Timestamp is decoded from the ADVERT but never persisted to the nodes
table. The validation was rejecting valid nodes with slightly-off clocks
(28h future) and nodes broadcasting timestamp=4. No reason to gate on it.
2026-03-21 21:31:25 +00:00
you
ebc72fa364 perf: reduce 3 API calls to 1 when expanding grouped packet row
Expanding a grouped row fired: packets?hash=X&expand=observations,
packets?hash=X&limit=1, and packets/HASH — all returning the same
data. Now uses single /packets/HASH call and passes prefetched data
to selectPacket() to skip redundant fetches.
2026-03-21 21:17:24 +00:00
you
6f350bb785 fix: invalidate channel cache before re-fetching on WS update
The WS handler's 250ms debounce fired loadChannels() before the
global 5s cache invalidation timer cleared the stale entry, so
the fetch returned cached data. Now channels.js invalidates its
own cache entries immediately before re-fetching.
2026-03-21 21:09:48 +00:00
you
775a45f9eb fix: build insert row from packetData instead of DB round-trip
packets_v view uses observation IDs, not transmission IDs, so
getPacket(transmissionId) returned null. Skip the DB lookup entirely
and construct the row directly from the incoming packetData object
which already has all needed fields.
2026-03-21 21:02:12 +00:00
you
9eb4bcc088 fix: use transmissionId for packet lookup after insert
packets_v view uses transmission IDs, not packets table IDs.
insertPacket returns a packets table ID which doesn't exist in
packets_v, so getPacket returned null and new packets never got
indexed in memory. Use transmissionId from insertTransmission instead.
2026-03-21 20:54:59 +00:00
you
e1590c6242 fix: re-export insertPacket from db.js (packet-store.js needs it)
Migration subagent removed it from exports but packet-store.js
calls dbModule.insertPacket() on every MQTT ingestion.
2026-03-21 20:46:15 +00:00
you
7d164f4a67 Move hop resolution to client side
Create public/hop-resolver.js that mirrors the server's disambiguateHops()
algorithm (prefix index, forward/backward pass, distance sanity check).

Replace all /api/resolve-hops fetch calls in packets.js with local
HopResolver.resolve() calls. The resolver lazily fetches and caches the
full nodes list via /api/nodes on first use.

The server endpoint is kept as fallback but no longer called by the UI,
eliminating 40+ HTTP requests per session.
2026-03-21 20:39:28 +00:00
you
e70dd8b2fa refactor: migrate all SQL queries from packets table to packets_v view (transmissions+observations)
- Create packets_v SQL view joining transmissions+observations to match old packets schema
- Replace all SELECT FROM packets with packets_v in db.js, packet-store.js, server.js
- Update countPackets/countRecentPackets to query observations directly
- Update seed() to use insertTransmission instead of insertPacket
- Remove insertPacket from exports (no longer called)
- Keep packets table schema intact (not dropped yet, pending testing)
2026-03-21 20:38:58 +00:00
you
a66cc8f126 perf: disable SQLite auto-checkpoint, use manual PASSIVE checkpoint
SQLite WAL auto-checkpoint (every 1000 pages/4MB) was causing 200ms+
event loop spikes on a 381MB database. This is synchronous I/O that
blocks the Node.js event loop unpredictably.

Fix: disable auto-checkpoint, run PASSIVE (non-blocking) checkpoint
every 5 minutes. PASSIVE won't stall readers or writers — it only
checkpoints pages that aren't currently in use.
2026-03-21 20:19:22 +00:00
you
7a3a3a5ea0 perf: eliminate synchronous blocking during startup pre-warm
Previous pre-warm called computeAllSubpaths() synchronously (500ms+)
directly in the setTimeout callback, then sequentially warmed 8 more
endpoints. Any browser request arriving during this 1.5s+ window
waited the full duration (user saw 4816ms for a 3-hop resolve-hops).

Fix: ALL pre-warm now goes through self-HTTP-requests which yield the
event loop between each computation. Delayed to 5s so initial page
load requests complete first (they populate cache on-demand).

Removed the sync computeAllSubpaths() call and inline subpath cache
population — the /api/analytics/subpaths endpoint handles this itself.
2026-03-21 20:07:27 +00:00
you
cf14701592 perf: node paths endpoint uses disambiguateHops with prefix index
Replaced inline resolveHopsInternal (allNodes.filter per hop) with
shared disambiguateHops (prefix-indexed). Also uses _parsedPath cache
and per-request disambig cache. /api/nodes/:pubkey/paths was 560ms cold,
should now be much faster.
2026-03-21 19:59:11 +00:00
you
7e841d89c1 perf: revert 200ms gap (made it worse), warm at 100ms instead of 1s
200ms gaps meant clients hit cold caches themselves (worse).
Pre-warm should be fast and immediate — start at 100ms after listen,
setImmediate between endpoints to yield but not delay.
2026-03-21 19:54:10 +00:00
you
795be6996f perf: 200ms gap between pre-warm requests to drain client queue
setImmediate wasn't enough — each analytics computation blocks for
200-400ms synchronously. Adding a 200ms setTimeout between pre-warm
requests gives pending client requests a window to complete between
the heavy computations.
2026-03-21 19:53:02 +00:00
you
187a2ac536 perf: shared cached node list + cached path JSON parse
- 8 separate 'SELECT * FROM nodes' queries replaced with getCachedNodes()
  (refreshes every 30s, prefix index built once and reused)
- Region-filtered subpaths + master subpaths use _parsedPath cache
- Eliminates repeated SQLite queries + prefix index rebuilds across
  back-to-back analytics endpoint calls
2026-03-21 19:49:54 +00:00
you
2c38d3c7d6 perf: yield event loop between pre-warm requests via setImmediate 2026-03-21 19:38:59 +00:00
you
9120985ab1 perf: add observers, nodes, distance to startup pre-warm
These endpoints were missing from the sequential pre-warm,
causing cold cache hits when clients connect before warm completes.
observers was 3s cold, distance was 600ms cold.
2026-03-21 19:36:46 +00:00
you
cc55e5733d Optimize analytics endpoints: prefix maps, cached JSON.parse, reduced scans
- Topology: replace O(N) allNodes.filter with prefix map + hop cache for resolveHop
- Topology: use _parsedPath cached JSON.parse for path_json (3 call sites)
- Topology: build observer map from already-filtered packets instead of second full scan
- Hash-sizes: prefix map for hop resolution instead of allNodes.find per hop
- Hash-sizes: use _parsedPath and _parsedDecoded cached parses
- Channels: use _parsedDecoded cached parse for decoded_json
2026-03-21 19:35:06 +00:00
you
6b78b3c5e4 Optimize /api/analytics/distance cold cache performance
- Build prefix map for O(1) hop resolution instead of O(N) linear scan per hop
- Cache resolved hops to avoid re-resolving same hex prefix across packets
- Pre-compute repeater set for O(1) role lookups
- Cache parsed path_json/decoded_json on packet objects (_parsedPath/_parsedDecoded)
2026-03-21 19:22:37 +00:00
you
75af7c3094 perf: precompute hash_size map at startup, update incrementally
The hash_size computation was scanning all 19K+ packets with JSON.parse
on every /api/nodes request, blocking the event loop for hundreds of ms.
Event loop p95 was 236ms, max 1732ms.

Now computed once at startup and updated incrementally on each new packet.
/api/nodes just does a Map.get per node instead of full scan.
2026-03-21 19:00:28 +00:00
you
e087156d90 fix: hash_size must use newest ADVERT, not oldest
Packets array is sorted newest-first. The previous 'last-wins'
approach (unconditional set) gave the OLDEST packet's hash_size.
Switched to first-wins (!has guard) which correctly uses the
newest ADVERT since we iterate newest-first.

Verified: Kpa Roof Solar has 1-byte ADVERTs (old firmware) and
2-byte ADVERTs (new firmware) interleaved. Newest are 2-byte.
2026-03-21 18:52:06 +00:00
you
fe552c8a4b fix: Pass 2 hash_size was overwriting ADVERT-derived values
Pass 1 correctly uses last-wins for ADVERT packets. But Pass 2
(fallback for nodes without ADVERTs) was also unconditionally
overwriting, so a stale 1-byte non-ADVERT packet would clobber
the correct 2-byte value from Pass 1.

Restored the !hashSizeMap.has() guard on Pass 2 only — it should
only fill gaps, never override ADVERT-derived hash_size.
2026-03-21 18:50:50 +00:00
you
3edb02a829 Fix hash_size using first-seen-wins instead of last-seen-wins
The hashSizeMap was guarded by !hashSizeMap.has(pk), meaning the oldest
ADVERT determined a node's hash_size permanently. If a node upgraded
firmware from 1-byte to 2-byte hash prefix, the stale value persisted.

Remove the guard so newer packets overwrite older ones (last-seen-wins).
2026-03-21 18:43:48 +00:00
you
7f1c735981 security: require API key for POST /api/packets and /api/perf/reset
- New config.apiKey field — when set, POST endpoints require X-Api-Key header
- If apiKey not configured, endpoints remain open (dev/local mode)
- GET endpoints and /api/decode (read-only) remain public
- Closes the packet injection attack surface
2026-03-21 18:40:06 +00:00
you
5bb17bbe60 feat: View on Map buttons for distance leaderboard hops and paths
- 🗺️ button on each top hop row → opens map with from/to markers + line
- 🗺️ button on each top path row → opens map with full multi-hop route
- Server now includes fromPk/toPk in topPaths hops for map resolution
- Uses existing drawPacketRoute() via sessionStorage handoff
2026-03-21 17:56:44 +00:00
you
a892691dd3 fix: cap max hop distance at 300km, link to node detail not analytics
- 1000km filter was too generous for LoRa (record ~250km)
- Uhuru kwa watu 📡 ↔ Bay Area hops at 880km were obviously wrong
- Node links in leaderboard now go to #/nodes/:pk (detail) not /analytics
2026-03-21 17:45:51 +00:00
you
fd57790d36 fix: remove dead histogram(null) call crashing distance tab
The subagent left a stray histogram(null, 0, ...) call that fell through
to the legacy path which does Math.min(...null) → Symbol.iterator error.
2026-03-21 17:41:24 +00:00
you
7d46d96563 Add Distance/Range analytics tab
New /api/analytics/distance endpoint that:
- Resolves path hops to nodes with valid GPS coordinates
- Calculates haversine distances between consecutive hops
- Separates stats by link type: R↔R, C↔R, C↔C
- Returns top longest hops, longest paths, category stats, histogram, time series
- Filters out invalid GPS (null, 0/0) and sanity-checks >1000km
- Supports region filtering and caching

New Distance tab in analytics UI with:
- Summary cards (total hops, paths, avg/max distance)
- Link type breakdown table
- Distance histogram
- Average distance over time sparkline
- Top 20 longest hops leaderboard
- Top 10 longest multi-hop paths table
2026-03-21 17:33:33 +00:00
you
47fa32f982 fix: region filter nodes by ADVERT observers, not data packets
The previous approach matched nodes via data packet hashes seen by
regional observers — but mesh packets propagate everywhere, so nearly
every node matched every region (550/558).

New approach: _advertByObserver index tracks which observers saw each
node's ADVERT packets. ADVERTs are local broadcasts that indicate
physical presence, so they're the correct signal for geographic filtering.

Also fixes role counts to reflect filtered results, not global totals.
2026-03-21 08:31:55 +00:00
you
b3599694c6 fix: stack overflow in /analytics/rf — replace Math.min/max spread
Math.min(...arr) and Math.max(...arr) blow the call stack when arr
has tens of thousands of elements. Replaced with simple for-loop
arrMin/arrMax helpers.
2026-03-21 08:26:34 +00:00
you
2f50bc0c89 fix: region dropdown matches btn-icon style, says 'All Regions'
- border-radius 6px (rectangle) instead of 16px (pill)
- Same padding/font-size as btn-icon
- Removed redundant 'Region:' label from dropdown mode
- Default label: 'All Regions' instead of 'All'
2026-03-21 08:24:36 +00:00
you
77fe834eb6 fix: btn-icon contrast — add color: var(--text)
BYOP button text was nearly invisible on dark background due to
missing explicit color on .btn-icon class.
2026-03-21 08:23:15 +00:00
you
9f4a9c6506 fix: use dropdown for region filter on packets page
Pills look out of place in the dense toolbar. Added { dropdown: true }
option to RegionFilter.init() to force dropdown mode regardless of
region count.
2026-03-21 08:22:12 +00:00
you
8bf5e64b1d fix: bump nodes.js cache buster so region filter loads
RegionFilter was added to nodes page but cache buster wasn't bumped,
so browsers served the old cached nodes.js without the filter.
2026-03-21 08:09:33 +00:00
you
f1fd34f47a fix: BYOP button accessibility compliance
- Add aria-label and aria-haspopup='dialog' to BYOP trigger button
- Add aria-label to close button and textarea
- Add role='status' and aria-live='polite' to result container
- Add role='alert' to error messages for screen reader announcement
- Fix textarea focus style: visible outline instead of outline:none
- Update cache busters
2026-03-21 08:06:37 +00:00
you
013e33253d fix: analytics RF stats respect region filter
- Separate region filtering from SNR filtering in /api/analytics/rf
- totalAllPackets now shows regional observation count (was global)
- Add totalTransmissions (unique hashes in regional set)
- Payload types and packet sizes use all regional data, not just SNR-filtered
- Signal stats (SNR, RSSI, scatter) use SNR-filtered subset
- Handle empty SNR/RSSI arrays gracefully (no Infinity/-Infinity)
2026-03-21 08:01:30 +00:00
you
bbf17b69ef packets: replace single-select region dropdown with shared RegionFilter component
- Fixes empty region dropdown (was populated before async regionMap loaded)
- Now uses multi-select RegionFilter component consistent with other pages
- Loads regions from /api/observers with proper async handling
- Supports multi-region filtering
2026-03-21 08:00:44 +00:00
you
3270e389c5 fix: null safety for analytics stats when region has no signal data
rf.snr.min/max/avg etc can be null when a region filter excludes all
packets with signal data. Added sf() helper for safe toFixed.
2026-03-21 07:43:14 +00:00
you
95443abd3e fix: network-status missing ? in region query string
/nodes/network-status + &region=X was producing an invalid URL.
Now correctly uses ? as separator when it's the first param.
2026-03-21 07:42:07 +00:00
you
35313c57d4 fix: revert broken SQL region filtering for nodes — use in-memory index
The subagent used a non-existent column (sender_key) in the SQL join.
Reverted to the same byObserver + _nodeHashIndex approach used by
bulk-health and network-status endpoints.
2026-03-21 07:15:22 +00:00
you
013cbaf5c4 fix: use SQL for region filtering on nodes page
The previous approach used pktStore._nodeHashIndex which only tracks
nodes appearing as sender/dest in decoded packet JSON. Most nodes only
send ADVERTs, so they had no entries in _nodeHashIndex and were filtered
out when a region was selected (showing 0 results).

Now uses a direct SQL join between observations and transmissions to find
all sender_keys observed by regional observers, which correctly includes
ADVERT-only nodes.
2026-03-21 07:10:56 +00:00
you
49d4841862 Fix region filtering in Route Patterns, Nodes, and Network Status tabs
- Add RegionFilter.regionQueryString() to all API calls in renderSubpaths and renderNodesTab
- Add region filtering to /api/analytics/subpaths (filter packets by regional observer hashes)
- Add region filtering to /api/nodes/bulk-health (filter nodes by regional presence)
- Add region filtering to /api/nodes/network-status (filter node counts by region)
- Add region param to nodes lookup in hash collision tab
- Update cache keys to include region param for proper cache separation
2026-03-21 07:10:38 +00:00
you
eaf0e621af fix: remove unnecessary region filter from packet trace screen 2026-03-21 07:10:09 +00:00
you
99dde1fc31 Fix region filter resetting analytics tab to overview
Track current active tab in _currentTab variable so that
loadAnalytics() re-renders the current tab instead of always
resetting to 'overview' when region filter changes.
2026-03-21 07:00:29 +00:00
you
cff995e00c region filter: add label, ARIA, dropdown mode for >4 regions
- Add 'Region:' label before filter controls
- ARIA roles: group with aria-label, checkbox roles on pills, aria-checked
- When >4 regions: render multi-select dropdown with checkboxes
  - Trigger shows summary (All / selected names / N selected)
  - All option at top, click-outside closes
- Pill bar mode unchanged for ≤4 regions (just added label + ARIA)
2026-03-21 06:57:48 +00:00
you
ab163227f8 chore: remove bare channel names from rainbow table — only #channels are valid 2026-03-21 06:48:36 +00:00
you
9664d6089c channels: only show decrypted messages, hide encrypted garbage
- Filter on decoded.type === 'CHAN' (successful decryption) only
- Skip GRP_TXT packets (failed decryption) entirely
- Channel key = decoded channel name instead of hash byte
- Remove channelHashNames lookup, encrypted field, isCollision logic
- Remove encrypted UI badges/indicators from frontend
- Channels with 0 decrypted messages no longer appear
2026-03-21 06:45:45 +00:00
you
606d4e134f Include channel-rainbow.json in Docker image 2026-03-21 06:38:02 +00:00
you
da475e9c13 Add rainbow table of pre-computed channel keys for common MeshCore channels
- channel-rainbow.json: 592 pre-computed SHA256-derived keys for common
  channel names (cities, topics, ham radio, emergency, etc.)
- server.js: Load rainbow table at startup as lowest-priority key source
- config.example.json: Add #LongFast to hashChannels list

Key derivation verified against MeshCore source: SHA256('#name')[:16bytes].
Rainbow table boosted decryption from ~48% to ~88% in testing.
2026-03-21 06:35:14 +00:00
you
739d4480a1 Use hashChannels for derived keys, keep only hardcoded public key in channelKeys
Channel keys for #test, #sf, #wardrive, #yo, #bot, #queer, #bookclub, #shtf
are all SHA256(channelName).slice(0,32) — no need to hardcode them. Move to
hashChannels array for auto-derivation. Only the MeshCore default public key
(8b3387e9c5cdea6ac9e5edbaa115cd72) needs explicit specification since it's
not derived from its channel name.
2026-03-21 06:22:05 +00:00
you
212990a295 fix: simplify channel key scheme + add CHAN packet detail renderer
- Remove composite key scheme (ch_/unk_ prefixes) that broke URL routing
  due to # in channel names. Use plain numeric channelHash as key instead.
- All packets with same hash byte go in one bucket; name is set from
  first successful decryption.
- Add packet detail renderer for decoded CHAN type showing channel name,
  sender, and sender timestamp.
- Update cache buster for packets.js.
2026-03-21 06:11:01 +00:00
you
ce190886ff fix: normalize packet hash case for deeplink lookups 2026-03-21 05:50:29 +00:00
you
9f53a059c7 fix: regional filters — proper indexed queries + frontend integration
Fixes Kpa-clawbot/meshcore-analyzer#111
2026-03-21 05:48:54 +00:00
you
c6d72e828d fix: restore channel message decryption — correct hash matching in API
The /api/channels endpoint was returning simple numeric hash (e.g. '45') while
/api/channels/:hash/messages was using composite keys (e.g. 'ch_#LongFast',
'unk_45') internally. This mismatch meant no channel ever matched, so all
messages appeared encrypted.

Fix: return the composite key as the hash field from /api/channels so the
frontend passes the correct identifier. Also add encodeURIComponent() to
channel API calls in the frontend since composite keys can contain '#'.
2026-03-21 05:47:51 +00:00
you
2834cfccba fix: rewrite favorites filter — correct packet matching + feed list filtering 2026-03-21 05:43:52 +00:00
you
d555ea26be feat: add regional filters to all tabs
Fixes Kpa-clawbot/meshcore-analyzer#111
2026-03-21 05:41:02 +00:00
you
133f267c4c fix: favorites filter only affects packet animations, not node markers 2026-03-21 05:37:22 +00:00
you
bd5171cf95 fix: validate ADVERT data to prevent corrupted node entries
Fixes Kpa-clawbot/meshcore-analyzer#112
2026-03-21 05:34:57 +00:00
you
dc6df38c9a fix: add favorites filter to live map
Fixes Kpa-clawbot/meshcore-analyzer#106
2026-03-21 05:34:51 +00:00
you
0ef1eb2595 Make map default center/zoom configurable via config.json
Adds mapDefaults config option with center and zoom properties.
New /api/config/map endpoint serves the defaults. live.js and map.js
fetch the config with fallback to hardcoded Bay Area defaults.

Fixes Kpa-clawbot/meshcore-analyzer#115
2026-03-21 05:29:05 +00:00
you
4bfe1ec363 Fix channel name resolution to use decryption key, not just hash
Channels sharing the same hash prefix but with different keys (e.g. #startrek
and #ai-bot both with hash 2d) now display the correct name by keying on the
actual channel name from decryption rather than just the hash byte.

Fixes Kpa-clawbot/meshcore-analyzer#108
2026-03-21 05:27:02 +00:00
you
290508d67c Display channel hash as hex in analytics pane
Fixes Kpa-clawbot/meshcore-analyzer#103
2026-03-21 05:25:54 +00:00
you
36ad6c8f75 fix: add hashChannels to config.example.json 2026-03-21 03:52:50 +00:00
you
071acd1561 fix: add paths section to mobile full-screen node view (loadFullNode) 2026-03-21 03:47:32 +00:00
you
c72f014f99 feat: add paths-through section to live map node detail panel 2026-03-21 03:38:42 +00:00
you
558687051e feat: label deconfliction on route view markers 2026-03-21 03:33:17 +00:00
you
5f291bdaa7 fix: skip animations when tab is backgrounded, resume cleanly on return 2026-03-21 02:36:51 +00:00
you
31a8f707b6 fix: dedup observations - UNIQUE(hash,observer_id,path_json) + INSERT OR IGNORE
~26% of observations were duplicates from multi-broker MQTT ingestion.
Added UNIQUE index to prevent future dupes, INSERT OR IGNORE to skip
silently, and in-memory dedup check in packet-store.
2026-03-21 02:31:51 +00:00
you
78ea581fc5 feat: improved route view - hide default markers, show origin node 2026-03-21 02:23:53 +00:00
you
1b737519bc feat: paths through node section on repeater detail page 2026-03-21 02:22:59 +00:00
you
0848a6c634 fix: anchor hop disambiguation from sender origin, not just observer
resolve-hops now accepts originLat/originLon params. Forward pass
starts from sender position so first ambiguous hop resolves to the
nearest node to the sender, not the observer.
2026-03-21 02:19:20 +00:00
you
926a68959b fix: raise feed panel bottom to 68px to clear VCR bar, revert z-index 2026-03-21 02:02:55 +00:00
you
67a90a3a33 fix: feed detail card z-index above VCR bar (1100 > 1000) 2026-03-21 01:50:46 +00:00
you
be1bcbf733 feat: replay sends all observations, uses realistic propagation on live map 2026-03-21 01:49:31 +00:00
you
f6fb024a20 feat: realistic packet propagation mode on live map 2026-03-21 01:41:55 +00:00
you
4395ff348c feat: show packet propagation time in detail pane (spread across observers) 2026-03-21 01:32:00 +00:00
lincomatic
0c97e4e980 Fix require statement for db module 2026-03-21 01:29:01 +00:00
lincomatic
78cc5edbb4 add hashChannels
(cherry picked from commit e35794c4531f3c16ceeb246fbde6912c7d831671)
2026-03-21 01:29:01 +00:00
you
c7e528331c remove self-referential hashtag channel key seeding from DB 2026-03-21 01:28:17 +00:00
Kpa-clawbot
e9b2dc7c00 Merge pull request #109 from lincomatic/prgraceful
graceful shutdown
2026-03-21 01:25:17 +00:00
Kpa-clawbot
5a36b8bf2e Merge pull request #105 from lincomatic/https
add https support
2026-03-21 01:25:15 +00:00
you
6bff9ce5e7 fix: move hash prefix labels toggle to Display section 2026-03-21 01:17:57 +00:00
you
92c258dabc fix: callout lines more visible — red, thicker, with dot at true position 2026-03-21 00:37:24 +00:00
you
5eacce1b40 feat: show hash prefix in node popup (bold, with byte size) 2026-03-21 00:36:39 +00:00
you
5f3e5a6ad1 fix: deconfliction applies to ALL markers (not just labels), size-aware bounding boxes 2026-03-21 00:35:37 +00:00
you
975abade32 fix: rename toggle to 'Hash prefix labels' 2026-03-21 00:34:06 +00:00
you
cf9d5e3325 fix: map labels show short hash ID (e.g. 5B, BEEF), better deconfliction with spiral offsets 2026-03-21 00:30:25 +00:00
you
003f5b1477 Fix hash size labels and add label overlap prevention
- Show '?' with grey background for nodes with null hash_size instead of '1B'
- Add collision detection to offset overlapping repeater labels
- Draw callout lines from offset labels back to true position
- Re-deconflict labels on zoom change
2026-03-21 00:25:55 +00:00
you
1adc3ca41d Add hash size labels for repeater markers on map
- Compute hash_size from ADVERT packets in /api/nodes response
- Show colored rectangle markers with hash size (e.g. '2B') for repeaters
- Add 'Hash size labels' toggle in map controls (default ON, saved to localStorage)
- Non-repeater markers unchanged
2026-03-21 00:19:15 +00:00
you
ab22b98f48 fix: switch all user-facing URLs to hash-based for stability across restarts
After dedup migration, packet IDs from the legacy 'packets' table differ
from transmission IDs in the 'transmissions' table. URLs using numeric IDs
became invalid after restart when _loadNormalized() assigned different IDs.

Changes:
- All packet URLs now use 16-char hex hashes instead of numeric IDs
  (#/packets/HASH instead of #/packet/ID)
- selectPacket() accepts hash parameter, uses hash-based URLs
- Copy Link generates hash-based URLs
- Search results link to hash-based URLs
- /api/packets/:id endpoint accepts both numeric IDs and 16-char hashes
- insert() now calls insertTransmission() to get stable transmission IDs
- Added db.getTransmission() for direct transmission table lookup
- Removed redundant byTransmission map (identical to byHash)
- All byTransmission references replaced with byHash
2026-03-21 00:18:11 +00:00
you
607eef2d06 Make ADVERT node names clickable links to node detail page 2026-03-21 00:09:30 +00:00
you
8c2d5d770f fix: Advertised Hash Size field shows byte offset 1 (path byte), not payload offset 2026-03-21 00:03:55 +00:00
you
9d65f041d4 fix: check transmission ID before observation ID in packet lookup 2026-03-20 23:58:23 +00:00
you
8ebce8087b fix: packet detail lookup uses transmission ID index
After dedup, table rows have transmission IDs but getById() maps
observation IDs. Added byTxId index so /api/packets/:id resolves
both observation and transmission IDs correctly.
2026-03-20 23:55:38 +00:00
you
e7f2613d1e packets: show Advertised Hash Size prominently in ADVERT field table 2026-03-20 23:43:40 +00:00
you
15e80e56f1 fix: packet links use hash instead of observation ID
After dedup migration, transmission IDs != old packet IDs.
Hash-based links (#/packets/HASH) are stable across the migration.
Affected: node detail, channel messages, live page packet cards.
2026-03-20 23:40:30 +00:00
you
84bf7fb545 packets: show hash_size in detail meta + fix blank hash_size in field table 2026-03-20 23:32:43 +00:00
you
9713e972f9 traces: clickable observer names, path graph, remove redundant observer table
- API returns observer_name + path_json per observation
- Timeline shows resolved names with links to #/observers/<id>
- Path Visualization replaced with SVG graph showing all observed paths
- Removed redundant Observer Details table (same data as timeline)
2026-03-20 23:19:37 +00:00
you
1a87e7410e v2.3.0: Packet Deduplication — normalized storage with transmissions + observations
CI: add setup-node to deploy workflow
2026-03-20 23:02:58 +00:00
you
91a6a2c525 fix: migration handles concurrent dual-write with INSERT OR IGNORE 2026-03-20 22:52:12 +00:00
you
e5a609bbfc Merge dedup-normalize: packet deduplication (M1-M5)
Normalized packet storage: transmissions + observations tables.
- M1: Schema migration script
- M2: Dual-write ingest
- M3: In-memory store restructured around transmissions
- M4: API responses include observation_count, totalTransmissions
- M5: Frontend badges, deeplinks, dedup-aware UI
2026-03-20 22:49:13 +00:00
you
95b59d1792 fix: recent packets deeplink uses route path not query param 2026-03-20 22:34:00 +00:00
you
e7aa4246ac fix: live node panel deeplinks — full detail, observer names, recent packets
- Full Detail links to #/nodes/<pubkey> (was #/nodes?selected= which just showed list)
- Heard By observer names link to #/observers/<id>
- Recent Packets link to #/packets?hash=<hash>
2026-03-20 22:17:13 +00:00
you
f1aa6caf93 fix: packet expand shows observations, heard-by uses correct field name
- pktToggleGroup fetches ?expand=observations and maps them as children
- Live page heard-by uses o.packetCount (was o.count → undefined)
2026-03-20 22:02:54 +00:00
you
a882aae681 M5: Frontend updates for dedup — observation_count badges, totalTransmissions
- packets.js: Show observation_count badge (👁 N) on grouped rows
- nodes.js: Use totalTransmissions (fallback totalPackets), show observation badges on recent packets
- home.js: Use totalTransmissions for network stats
- node-analytics.js: Use totalTransmissions for throughput display
- analytics.js: Use totalTransmissions for overview stats and node rankings
- live.js: Use totalTransmissions in node detail, show observation badges in feed and recent packets
- style.css: Add .badge-obs style for observation count badges
- index.html: Bump cache busters on all changed JS/CSS files

All changes have backward compat fallbacks to totalPackets.
2026-03-20 21:31:10 +00:00
you
aa35164252 M4: API response changes for dedup-normalize
- GET /api/packets: returns transmissions with observation_count, strip
  observations[] by default (use ?expand=observations to include)
- GET /api/packets/🆔 includes observation_count and observations[]
- GET /api/nodes/:pubkey/health: stats.totalTransmissions + totalObservations
  (totalPackets kept for backward compat)
- GET /api/nodes/bulk-health: same transmission/observation split
- WebSocket broadcast: includes observation_count
- db.js getStats(): adds totalTransmissions count
- All backward-compatible: old field names preserved alongside new ones
2026-03-20 20:49:34 +00:00
you
84f33aef7b M3: Restructure in-memory store around transmissions
- load() reads from transmissions JOIN observations (with legacy fallback)
- byHash now maps hash → single transmission object (1:1)
- byNode maps pubkey → [transmissions] (deduped, no inflated observations)
- byTransmission is the primary data structure
- byId maps observation IDs for backward-compat packet detail links
- byObserver still maps observer_id → [observations]
- getSiblings() returns observations from transmission
- findPacketsForNode() returns unique transmissions
- query()/queryGrouped() work with transmission-centric model
- All returned objects maintain backward-compatible fields
- SQLite-only fallback path (NO_MEMORY_STORE=1) unchanged
- Tested: 11.6K transmissions from 37.5K observations (3.2x dedup)
2026-03-20 20:44:32 +00:00
you
baa60cac0f M2: Dual-write ingest to transmissions/observations tables
- Add transmissions and observations schema to db.js init
- Add insertTransmission() function: upsert transmission by hash,
  always insert observation row
- All 6 pktStore.insert() call sites in server.js now also call
  db.insertTransmission() with try/catch (non-fatal on error)
- packet-store.js: add byTransmission Map index (hash → transmission
  with observations array) for future M3 query migration
- Existing insertPacket() and all read paths unchanged
2026-03-20 20:29:03 +00:00
you
d7e415daa7 fix: raw_hex NOT NULL in transmissions schema — deleted 4 junk test rows 2026-03-20 20:24:13 +00:00
you
2c6148fd2d Add dedup migration script (Milestone 1)
Creates transmissions and observations tables from existing packets table.
- Groups packets by hash → 1 transmission per unique hash
- Creates 1 observation per original packet row with FK to transmission
- Idempotent: drops and recreates new tables on each run
- Does NOT modify the original packets table
- Prints stats and verifies counts match

Tested on test DB: 33813 packets → 11530 transmissions (2.93x dedup ratio)
2026-03-20 20:22:30 +00:00
you
2feb2c5b94 fix: escapeHtml crashes on numbers — String(s) before .replace() 2026-03-20 19:25:48 +00:00
you
10b11106f6 ci: add pre-deploy JS validation — syntax check + undefined reference detection
Validation runs BEFORE docker build. If it fails, deployment is blocked.
No more broken code reaching production.
2026-03-20 19:24:11 +00:00
you
326d411c4a fix: esc is not defined — use escapeHtml in live.js node detail 2026-03-20 19:23:20 +00:00
you
15a93d5ea4 feat: clickable nodes on live map with slide-in detail panel
- Click any node marker to see name, role, status, location, stats
- Heard By observers and recent packets shown
- Links to full node detail and analytics pages
- Slide-in panel from right with blur background, matches live page style
- Uses shared ROLE_COLORS and HEALTH_THRESHOLDS
2026-03-20 19:21:30 +00:00
you
055467ca43 fix: live map legend uses shared ROLE_COLORS instead of hardcoded wrong colors 2026-03-20 19:17:57 +00:00
lincomatic
ac8a6a4dc3 graceful shutdown 2026-03-20 10:58:37 -07:00
you
4f7b02a91c fix: centralize hardcoded values — roles, thresholds, colors, tiles, limits — closes #104
- New public/roles.js shared module: ROLE_COLORS, ROLE_LABELS, ROLE_STYLE,
  ROLE_EMOJI, ROLE_SORT, HEALTH_THRESHOLDS, TILE_DARK/LIGHT, SNR_THRESHOLDS,
  DIST_THRESHOLDS, MAX_HOP_DIST, LIMITS — all configurable via /api/config/roles
- Removed duplicate ROLE_COLORS from map.js, nodes.js, live.js, analytics.js
- Removed duplicate health thresholds from nodes.js, home.js, observer-detail.js
- Deduplicated CartoDB tile URLs (3 copies → 1 in roles.js)
- Removed hardcoded region names from map.js and packets.js
- channels.js uses ROLE_EMOJI/ROLE_LABELS instead of hardcoded emoji chains
- server.js reads healthThresholds from config.json with defaults
- Unknown roles get gray circle fallback instead of crashing
2026-03-20 17:36:41 +00:00
lincomatic
209e17fcd4 add https support 2026-03-20 09:21:02 -07:00
you
f0db317051 fix: observer locations case-insensitive match, regions from API not hardcoded
- Observer ID is uppercase, node pubkey is lowercase — added COLLATE NOCASE
- New /api/config/regions endpoint merges config regions + observed IATAs
- map.js and packets.js fetch regions from API instead of hardcoded maps
2026-03-20 16:10:16 +00:00
you
9bf78bd28d feat: add MRY (Monterey) to lincomatic MQTT topics 2026-03-20 15:29:37 +00:00
you
5fe275b3f8 fix: use region-specific MQTT topics instead of wildcards — saves bandwidth 2026-03-20 15:28:54 +00:00
you
74a08d99b0 fix: observer location from nodes table direct join — observer ID = node pubkey 2026-03-20 15:11:57 +00:00
you
76d63ffe75 feat: observers as map markers (purple stars) with computed locations
- Removed dead 'MQTT Connected Only' checkbox (never worked)
- Added 'observer' role type with purple star marker
- Observer locations computed from average of nodes they've seen
- Observer popup with name, IATA, packets, link to detail page
- Role filter checkbox includes observers with count
2026-03-20 14:56:08 +00:00
you
157dc9a979 fix: spark bars use inline spans instead of div — immune to max-width:0 table crush
The div-based spark bar was always getting crushed to 0px by
table-layout:auto + max-width:0 on td. Inline spans with fixed
width survive because they participate in text flow, not block layout.
2026-03-20 14:48:15 +00:00
you
2f07ae2e5c fix: spark bar z-index so it renders above adjacent packet count cell 2026-03-20 14:39:48 +00:00
you
1f9cd3ead1 fix: add config.json mount to CI deploy workflow 2026-03-20 14:29:32 +00:00
33 changed files with 5104 additions and 836 deletions

View File

@@ -14,6 +14,13 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '22'
- name: Validate JS
run: sh scripts/validate.sh
- name: Build and deploy
run: |
set -e
@@ -24,6 +31,7 @@ jobs:
--restart unless-stopped \
-p 80:80 -p 443:443 -p 1883:1883 \
-v $HOME/meshcore-data:/app/data \
-v $HOME/meshcore-config.json:/app/config.json:ro \
-v $HOME/caddy-data:/data/caddy \
-v $HOME/meshcore-analyzer/Caddyfile:/etc/caddy/Caddyfile \
meshcore-analyzer

View File

@@ -1,45 +1,125 @@
# Changelog
## v2.1.1 — Multi-Broker MQTT & Observer Detail (2026-03-20)
## [2.4.1] — 2026-03-22
### 🆕 New Features
Hotfix release for regressions introduced in v2.4.0.
- **Multi-Broker MQTT** — Connect to multiple MQTT brokers simultaneously via `mqttSources` config array. Each source gets its own connection, topics, credentials, TLS settings, and optional IATA region filter. Legacy `mqtt` config still works.
- **IATA Region Filtering** — `mqttSources[].iataFilter` restricts accepted regions per source (e.g. only accept SJC/SFO/OAK packets from a shared feed).
- **Observer Detail Pages** — Click any observer row for a full detail page with status, radio info, battery/uptime/noise floor, packet type donut chart, timeline, unique nodes chart, SNR distribution, and recent packets table.
- **Observer Status Topic Parsing** — `meshcore/<region>/<id>/status` messages populate model, firmware, client_version, radio config, battery, uptime, and noise floor. 7 new columns in the observers table with auto-migration.
- **Channel Key Auto-Derivation** — Hashtag channel keys (`#channel`) are automatically derived as `SHA256("#channelname")` first 16 bytes on startup. Only non-hashtag keys (like `public`) need manual config.
- **Map Dark/Light Mode** — Map page now uses CartoDB dark/light tiles that swap automatically with the theme toggle (same as live page).
- **Shareable URLs** — Copy Link button on packet detail, standalone packet page at `#/packet/ID`, deep links to channels and observer detail pages.
- **Multi-Node Packet Filter** — "My Nodes" toggle in packets view now uses server-side `findPacketsForNode()` to find ALL packet types (messages, acks, traces), not just ADVERTs.
### Fixed
- Packet ingestion broken: `insert()` returned undefined after legacy table removal, causing all MQTT packets to fail silently
- Live packet updates not working: pause button `addEventListener` on null element crashed `init()`, preventing WS handler registration
- WS broadcast had null packet data when observation was deduped (2nd+ observer of same packet)
- Multi-select filter menu close handler crashed on null `observerFilterWrap`/`typeFilterWrap` elements
- Live map animation cleanup crashed with null `animLayer`/`pathsLayer` after navigating away (setInterval kept firing)
### 🐛 Bug Fixes
## [2.4.0] — 2026-03-22
- **Observer name resolution** — MQTT packets now pass `msg.origin` (friendly name) to both packet records and observer upserts. Previously only the status handler used it.
- **Observer analytics ordering** — Fixed `recentPackets` returning oldest instead of newest (wrong slice direction). Sorted observer analytics packets explicitly.
- **Spark bars visible** — Fixed `.data-table td { max-width: 0 }` crushing spark bar cells to zero width with inline style override.
- **My Nodes filter field names** — Fixed `pubkey``pubKey`, `to`/`from``srcPubKey`/`destPubKey`/`srcHash`/`destHash`.
- **Duplicate pin buttons** — Live page destroy now removes the nav pin button; init guards against duplicates.
- **Packets page crash** — Fixed non-async `renderTableRows` using `await` (syntax error prevented entire page from loading).
- **Node search all packet types** — Search by node name now returns messages, acks, and traces — not just ADVERTs.
- **Node packet count accuracy** — `findPacketsForNode()` is now single source of truth for all node packet lookups.
- **Health endpoint recentPackets** — Changed from `slice(-10).reverse()` to `slice(0, 20)` — 20 newest DESC instead of 10 oldest.
- **RF analytics total packets** — Added `totalAllPackets` field so frontend shows both total and signal-filtered counts.
- **Duplicate `const crypto` crash** — Removed duplicate `require('crypto')` that crashed prod for ~2 minutes.
- **PII scrubbed from git history** — Removed real names and coordinates from seed data across all commits.
UI polish, client-side filtering, time window selector, DB cleanup, and bug fixes.
### 🏗️ Infrastructure
### Added
- Observation-level deeplinks (`#/packets/HASH?obs=OBSERVER_ID`)
- Observation detail pane (click any child row for its specific data)
- Observation sort: Observer / Path ↑↓ / Time ↑↓ with persistent preference
- Ungrouped mode flattens all observations into individual rows
- Sort help tooltip (ⓘ) explaining each mode
- Distance/Range analytics tab with haversine calculations
- View on Map buttons for distance leaderboard entries
- Realistic packet propagation mode on live map
- Packet propagation time in detail pane
- Replay sends all observations with realistic animation
- Paths-through section on node detail (desktop + mobile)
- Regional filters on all tabs (shared RegionFilter component)
- Favorites filter on live map (packet-level, not node markers)
- Configurable map defaults via `config.json`
- Hash prefix labels on map with spiral deconfliction + callout lines
- Channel rainbow table (pre-computed keys for common names)
- Zero-API live channel updates via WebSocket
- Channel message dedup by packet hash
- Channel name tags (blue pill) in packet detail column
- Shareable channel URLs (`#/channels/HASH`)
- API key required for POST endpoints
- HTTPS support (lincomatic PR #105)
- Graceful shutdown (lincomatic PR #109)
- Filter bar: logical grouping, consistent 34px height, help tooltips
- Multi-select Observer and Type filters (checkbox dropdowns, OR logic)
- Hex Paths toggle: show raw hex hash prefixes vs resolved node names
- Time window selector (15min/30min/1h/3h/6h/12h/24h/All) replaces fixed packet count limit
- Pause/resume button (⏸/▶) for live WebSocket updates with buffered packet count
- localStorage persistence for all filter/view preferences
- **Docker container deployed to Azure VM** — Live at `https://analyzer.00id.net` with automatic Let's Encrypt TLS via Caddy.
- **`deploy.sh` fixed** — Config mount (`-v config.json:/app/config.json:ro`) was missing, causing every deploy to fall back to placeholder credentials. Added `|| true` to stop/rm to prevent chain failures.
- **CI/CD via GitHub Actions** — Self-hosted runner on VM, auto-deploys on push to master.
### Changed
- Channel keys: plain `String(channelHash)`, `hashChannels` for auto-derived SHA256
- Node region filtering uses ADVERT-based index (accurate local presence vs mesh-wide routing)
- Header row reflects first sorted observation's data
- Max hop distance filter: 1000km → 300km (LoRa record ~250km)
- Route view labels use deconflicted divIcons
- Channels page hides encrypted messages, shows only decrypted
- Dark mode: active filter buttons retain accent styling
- Region dropdown: `IATA - Friendly Name` format, proper sizing
- Observer/Type filters are pure client-side (no API calls on filter change)
- Packet loading: time-window based (`since`) instead of fixed count limit
- Header row shows matching observer when observer filter is active
---
### Removed
- Legacy `packets` and `paths` database tables (auto-migrated on startup)
- Redundant server-side type/observer filtering (client filters in-memory)
## v2.0.1 — Mobile Packets (2026-03-18)
### Fixed
- Header row showed longest path instead of first observer's path
- Observer/path mismatch when earlier observation arrives later
- Auto-seeding fake data on empty DB (now requires `--seed` flag)
- Channel "10h ago" bug (used stale `first_seen` instead of current time)
- Stale UI: wrong ID type for packet lookup after insert
- ADVERT timestamp validation rejecting valid nodes
- Channels page API spam on every WS update
- Duplicate observations in expanded view
- Analytics RF 500 error (stack overflow with 193K observations)
- Region filter SQL using non-existent column
- Channel hash: decimal→hex, keyed by decrypted name
- Corrupted repeater entries (ADVERT validation at ingestion)
- Hash_size: uses newest ADVERT, precomputed at startup
- Tab backgrounding: skip animations, resume cleanly
- Feed panel position (obscured by VCR bar)
- Hop disambiguation anchored from sender origin
- Packet hash case normalization for deeplinks
- Critical: packet ingestion broken after legacy table removal (`insert()` returned undefined)
- Sort help tooltip rendering (CSS pseudo-elements don't support newlines)
See [v2.0.1 release](https://github.com/Kpa-clawbot/meshcore-analyzer/releases/tag/v2.0.1).
### Performance
- `/api/analytics/distance`: 3s → 630ms
- `/api/analytics/topology`: 289ms → 193ms
- `/api/observers`: 3s → 130ms
- `/api/nodes`: 50ms → 2ms (precomputed hash_size)
- Event loop max: 3.2s → 903ms (startup only)
- Pre-warm yields event loop via `setImmediate`
- Client-side hop resolution
- SQLite manual PASSIVE checkpointing
- Single API call for packet expand (was 3)
## v2.0.0 — Live Trace Map & VCR Playback (2026-03-17)
## [2.3.0] - 2026-03-20
See [v2.0.0 release](https://github.com/Kpa-clawbot/meshcore-analyzer/releases/tag/v2.0.0).
### Added
- **Packet Deduplication**: Normalized storage with `transmissions` and `observations` tables — packets seen by multiple observers are stored once with linked observation records
- **Observation count badges**: Packets page shows 👁 badge indicating how many observers saw each transmission
- **`?expand=observations`**: API query param to include full observation details on packet responses
- **`totalTransmissions` / `totalObservations`**: Health and analytics APIs return both deduped and raw counts
- **Migration script**: `scripts/migrate-dedup.js` for converting existing packet data to normalized schema
- **Live map deeplinks**: Node detail panel links to full node detail, observer detail, and filtered packets
- **CI validation**: `setup-node` added to deploy workflow for JS syntax checking
### Changed
- In-memory packet store restructured around transmissions (primary) with observation indexes
- Packets API returns unique transmissions by default (was returning inflated observation rows)
- Home page shows "Transmissions" instead of "Packets" for network stats
- Analytics overview uses transmission counts for throughput metrics
- Node health stats include `totalTransmissions` alongside legacy `totalPackets`
- WebSocket broadcasts include `observation_count`
### Fixed
- Packet expand showing only the collapsed row instead of individual observations
- Live page "Heard By" showing "undefined pkts" (wrong field name)
- Recent packets deeplink using query param instead of route path
- Migration script handling concurrent dual-write during live deployment
### Performance
- **8.19× dedup ratio on production** (117K observations → 14K transmissions)
- RAM usage reduced proportionally — store loads transmissions, not inflated observations

93
DEDUP-DESIGN.md Normal file
View File

@@ -0,0 +1,93 @@
# Packet Deduplication Design
## The Problem
A single physical RF transmission gets recorded as N rows in the DB, where N = number of observers that heard it. Each row has the same `hash` but different `path_json` and `observer_id`.
### Example
```
Pkt 1 repeat 1: Path: A→B→C→D→E (observer E)
Pkt 1 repeat 2: Path: A→B→F→G (observer G)
Pkt 1 repeat 3: Path: A→C→H→J→K (observer K)
```
- Repeater A sent 1 packet, not 3
- Repeater B sent 1 packet, not 2 (C and F both heard the same broadcast)
- The hash is identical across all 3 rows
### Why the hash works
`computeContentHash()` = `SHA256(header_byte + payload)`, skipping path hops. Two observations of the same original packet through different paths produce the same hash. This is the dedup key.
## What's inflated (and what's not)
| Context | Current (inflated?) | Correct behavior |
|---------|-------------------|------------------|
| Node "total packets" | COUNT(*) — inflated | COUNT(DISTINCT hash) for transmissions |
| Packets/hour on observer page | Raw count | Correct — each observer DID receive it |
| Node analytics throughput | Inflated | DISTINCT hash |
| Live map animations | N animations per physical packet | 1 animation? Or 1 per path? TBD |
| "Heard By" table | Observations per observer | Correct as-is |
| RF analytics (SNR/RSSI) | Mixes observations | Each observation has its own SNR — all valid |
| Topology/path analysis | All paths shown | All paths are valuable — don't discard |
| Packet list (grouped mode) | Groups by hash already | Probably fine |
| Packet list (ungrouped) | Shows every observation | Maybe show distinct, expand for repeats? |
## Key Principle
**Observations are valuable data — never discard them.** The paths tell you about mesh topology, coverage, and redundancy. But **counts displayed to users should reflect reality** (1 transmission = 1 count).
## Design Decisions Needed
1. **What does "packets" mean in node detail?** Unique transmissions? Total observations? Both?
2. **Live map**: 1 animation with multiple path lines? Or 1 per observation?
3. **Analytics charts**: Should throughput charts show transmissions or observations?
4. **Packet list default view**: Group by hash by default?
5. **New metric: "observation ratio"?** — avg observations per transmission tells you about mesh redundancy/coverage
## Work Items
- [ ] **DB/API: Add distinct counts**`findPacketsForNode()` and health endpoint should return both `totalTransmissions` (DISTINCT hash) and `totalObservations` (COUNT(*))
- [ ] **Node detail UI** — show "X transmissions seen Y times" or similar
- [ ] **Bulk health / network status** — use distinct hash counts
- [ ] **Node analytics charts** — throughput should use distinct hashes
- [ ] **Packets page default** — consider grouping by hash by default
- [ ] **Live map** — decide on animation strategy for repeated observations
- [ ] **Observer page** — observation count is correct, but could add "unique packets" column
- [ ] **In-memory store** — add hash→[packets] index if not already there (check `pktStore.byHash`)
- [ ] **API: packet siblings**`/api/packets/:id/siblings` or `?groupByHash=true` (may already exist)
- [ ] **RF analytics** — keep all observations for SNR/RSSI (each is a real measurement) but label counts correctly
- [ ] **"Coverage ratio" metric** — avg(observations per unique hash) per node/observer — measures mesh redundancy
## Live Map Animation Design
### Current behavior
Every observation triggers a separate animation. Same packet heard by 3 observers = 3 independent route animations. Looks like 3 packets when it was 1.
### Options considered
**Option A: Single animation, all paths simultaneously (PREFERRED)**
When a hash first arrives, buffer briefly (500ms-2s) for sibling observations, then animate all paths at once. One pulse from origin, multiple route lines fanning out simultaneously. Most accurate — this IS what physically happened: one RF burst propagating through the mesh along multiple paths at once.
Timing challenge: observations don't arrive simultaneously (seconds apart). Need to buffer the first observation, wait for siblings, then render all together. Adds slight latency to "live" feel.
**Option B: Single animation, "best" path only** — REJECTED
Pick shortest/highest-SNR path, animate only that. Clean but loses coverage/redundancy info.
**Option C: Single origin pulse, staggered path reveals** — REJECTED
Origin pulses once, paths draw in sequence with delay. Dramatic but busy, and doesn't reflect reality (the propagation is simultaneous).
**Option D: Animate first, suppress siblings** — REJECTED (pragmatic but inaccurate)
First observation gets animation, subsequent same-hash observations silently logged. Simple but you never see alternate paths on the live map.
### Implementation notes (for when we build this)
- Need a client-side hash buffer: `Map<hash, {timer, packets[]}>`
- On first WS packet with new hash: start timer (configurable, ~1-2s)
- On subsequent packets with same hash: add to buffer, reset/extend timer
- On timer expiry: animate all buffered paths for that hash simultaneously
- Feed sidebar could show consolidated entry: "1 packet, 3 paths" with expand
- Buffer window should be configurable (config.json)
## Status
**Discussion phase** — no code changes yet. Iavor wants to finalize design before implementation. Live map changes tabled for later.

236
DEDUP-MIGRATION-PLAN.md Normal file
View File

@@ -0,0 +1,236 @@
# Packet Deduplication — Normalized Schema Migration Plan
## Overview
Split the monolithic `packets` table into two tables:
- **`packets`** — one row per unique physical transmission (keyed by content hash)
- **`observations`** — one row per observer sighting (SNR, RSSI, path, observer, timestamp)
This fixes inflated packet counts across the entire app and enables proper "1 transmission seen N times" semantics.
## Current State
**`packets` table**: 1 row per observation. ~61MB, ~30K+ rows. Same hash appears N times (once per observer). Fields mix transmission data (raw_hex, payload_type, decoded_json, hash) with observation data (observer_id, snr, rssi, path_json).
**`packet-store.js`**: In-memory mirror of packets table. Indexes: `byId`, `byHash` (hash → [packets]), `byObserver`, `byNode`. All reads served from RAM. SQLite is write-only for packets.
**Touch surface**: ~66 SQL queries across db.js/server.js/packet-store.js. ~12 frontend files consume packet data.
---
## Milestone 1: Schema Migration (Backend Only)
**Goal**: New tables exist, data migrated, old table preserved as backup. No behavioral changes yet.
### Tasks
1. **Create new schema** in `db.js` init:
```sql
CREATE TABLE IF NOT EXISTS transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
hash TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX idx_transmissions_hash ON transmissions(hash);
CREATE INDEX idx_transmissions_first_seen ON transmissions(first_seen);
CREATE INDEX idx_transmissions_payload_type ON transmissions(payload_type);
CREATE INDEX idx_observations_hash ON observations(hash);
CREATE INDEX idx_observations_transmission_id ON observations(transmission_id);
CREATE INDEX idx_observations_observer_id ON observations(observer_id);
CREATE INDEX idx_observations_timestamp ON observations(timestamp);
```
2. **Write migration script** (`scripts/migrate-dedup.js`):
- Read all rows from `packets` ordered by timestamp
- Group by hash
- For each unique hash: INSERT into `transmissions` (use first observation's raw_hex, decoded_json, etc.)
- For each row: INSERT into `observations` with foreign key to transmission
- Verify counts: `SELECT COUNT(*) FROM observations` = old packets count
- Verify: `SELECT COUNT(*) FROM transmissions` < observations count
- **Do NOT drop old `packets` table** — rename to `packets_backup`
3. **Print migration stats**: total packets, unique transmissions, dedup ratio, time taken
### Validation
- `COUNT(*) FROM observations` = `COUNT(*) FROM packets_backup`
- `COUNT(*) FROM transmissions` = `COUNT(DISTINCT hash) FROM packets_backup`
- Spot-check: pick 5 known multi-observer packets, verify transmission + observations match
### Risk: LOW — additive only, old data preserved
---
## Milestone 2: Dual-Write Ingest
**Goal**: New packets written to both old and new tables. Read path unchanged. Zero downtime.
### Tasks
1. **Update `db.js` `insertPacket()`**:
- On new packet: check if `transmissions` row exists for hash
- If not: INSERT into `transmissions`, get id
- If yes: UPDATE `first_seen` if this timestamp is earlier
- INSERT into `observations` with transmission_id
- **Still also write to old `packets` table** (dual-write for safety)
2. **Update `packet-store.js` `insert()`**: Mirror the dual-write in memory model
- Maintain both old flat array AND new `byTransmission` Map
### Validation
- Send test packets, verify they appear in both old and new tables
- Verify multi-observer packet creates 1 transmission + N observations
### Risk: LOW — old read path still works as fallback
---
## Milestone 3: In-Memory Store Restructure
**Goal**: `packet-store.js` switches from flat packet array to transmission-centric model.
### Tasks
1. **New in-memory data model**:
```
transmissions: Map<hash, {id, raw_hex, hash, first_seen, payload_type, decoded_json, observations: []}>
```
Each observation: `{id, observer_id, observer_name, snr, rssi, path_json, timestamp}`
2. **Update indexes**:
- `byHash`: hash → transmission object (1:1 instead of 1:N)
- `byObserver`: observer_id → [observation references]
- `byNode`: pubkey → [transmission references] (deduped!)
- `byId`: observation.id → observation (for backward compat with packet detail links)
3. **Update `load()`**: Read from `transmissions` JOIN `observations` instead of `packets`
4. **Update query methods**:
- `findPackets()` — returns transmissions by default, with `.observations` attached
- `findPacketsForNode()` — returns transmissions where node appears in ANY observation's path/decoded_json
- `getSiblings()` — becomes `getObservations(hash)` — trivial, just return `transmission.observations`
- `countForNode()` — returns `{transmissions: N, observations: M}`
### Validation
- All existing API endpoints return valid data
- Packet counts decrease (correctly!) for multi-observer nodes
- `/api/perf` shows no regression
### Risk: MEDIUM — core read path changes. Test thoroughly.
---
## Milestone 4: API Response Changes
**Goal**: APIs return deduped data with observation counts.
### Tasks
1. **`GET /api/packets`**:
- Default: return transmissions (1 row per unique packet)
- Each transmission includes `observation_count` and optionally `observations[]`
- `?expand=observations` to include full observation list
- `?groupByHash` becomes the default behavior (deprecate param)
- Preserve `observer` filter: return transmissions where at least one observation matches
2. **`GET /api/nodes/:pubkey/health`**:
- `stats.totalPackets` → `stats.totalTransmissions` (distinct hashes)
- Add `stats.totalObservations` (old count, for reference)
- `recentPackets` → returns transmissions with observation_count
3. **`GET /api/nodes/bulk-health`**: Same changes as health
4. **`GET /api/nodes/network-status`**: Use transmission counts
5. **`GET /api/nodes/:pubkey/analytics`**: All throughput charts use transmission counts
6. **WebSocket broadcast**: Include `observation_count` when sibling observations exist for same hash
### Backward Compatibility
- Add `?legacy=1` param that returns old-style flat observations (for any external consumers)
- Include both `totalTransmissions` and `totalObservations` in health responses during transition
### Risk: MEDIUM — frontend expects certain shapes. May need coordinated deploy with Milestone 5.
---
## Milestone 5: Frontend Updates
**Goal**: UI shows correct counts and leverages observation data.
### Tasks
1. **Packets page**:
- Default view shows transmissions (already has groupByHash mode — make it default)
- Expand row to see individual observations with their paths/SNR/RSSI
- Badge: "×3 observers" on grouped rows
2. **Node detail panel** (nodes.js + live.js):
- Show "X transmissions" not "X packets"
- Or "X packets (seen Y times)" to show both
3. **Home page**: Network stats use transmission counts
4. **Node analytics**: Throughput charts use transmissions
5. **Observer detail**: Keep observation counts (correct metric for observers)
6. **Analytics page**: Topology/RF analysis uses all observations (SNR per observation is valid data)
### Risk: LOW-MEDIUM — mostly display changes
---
## Milestone 6: Cleanup
**Goal**: Remove dual-write, drop old table, clean up.
### Tasks
1. Remove dual-write from `insertPacket()`
2. Drop `packets_backup` table (after confirming everything works for 1+ week)
3. Remove `?legacy=1` support if unused
4. Update DEDUP-DESIGN.md → mark as complete
5. VACUUM the database
6. Tag release (v2.3.0?)
### Risk: LOW — cleanup only, all functional changes already proven
---
## Estimated Scope
| Milestone | Files Modified | Complexity | Can Deploy Independently? |
|-----------|---------------|------------|--------------------------|
| 1. Schema Migration | db.js, new script | Low | Yes — additive only |
| 2. Dual-Write | db.js, packet-store.js | Low | Yes — old reads unchanged |
| 3. Memory Store | packet-store.js | Medium | No — must deploy with M4 |
| 4. API Changes | server.js, db.js | Medium | No — must deploy with M5 |
| 5. Frontend | 8+ public/*.js files | Medium | No — must deploy with M4 |
| 6. Cleanup | db.js, server.js | Low | Yes — after bake period |
**Milestones 1-2**: Safe to deploy independently, no user-visible changes.
**Milestones 3-5**: Must ship together (API shape changes + frontend expects new shape).
**Milestone 6**: Ship after 1 week bake.
## Open Questions
1. **Table naming**: `transmissions` + `observations`? Or keep `packets` + add `observations`? The word "transmission" is more accurate but "packet" is what the whole UI calls them.
2. **Packet detail URLs**: Currently `#/packet/123` uses the observation ID. Keep observation IDs as the URL key? Or switch to hash?
3. **Path dedup in paths table**: The `paths` table also has per-observation entries. Normalize that too, or leave as-is?
4. **Migration on prod**: Run migration script before deploying new code, or make new code handle both old and new schema?

View File

@@ -9,7 +9,7 @@ COPY package.json package-lock.json ./
RUN npm ci --production
# Copy application
COPY *.js config.example.json ./
COPY *.js config.example.json channel-rainbow.json ./
COPY public/ ./public/
# Supervisor + Mosquitto + Caddy config

View File

@@ -148,6 +148,10 @@ Edit `config.json`:
```json
{
"port": 3000,
"https": {
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
},
"mqtt": {
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
@@ -178,6 +182,7 @@ Edit `config.json`:
| Field | Description |
|-------|-------------|
| `port` | HTTP server port (default: 3000) |
| `https.cert` / `https.key` | Optional PEM cert/key paths to enable native HTTPS (falls back to HTTP if omitted or unreadable) |
| `mqtt.broker` | Local MQTT broker URL. Set to `""` to disable |
| `mqtt.topic` | MQTT topic pattern for packet ingestion |
| `mqttSources` | Array of external MQTT broker connections (optional) |

298
channel-rainbow.json Normal file
View File

@@ -0,0 +1,298 @@
{
"#LongFast": "2cc3d22840e086105ad73443da2cacb8",
"#MediumSlow": "99aa7084b6312841eb9b79b3a146bea4",
"#ShortFast": "18267412b697cb98344c4a44b044d04d",
"#ShortSlow": "8dffe23f9ed28b7d617fc587bdb19ec0",
"#LongSlow": "7f8722cce459fc6d452db4f5be59ba5e",
"#MediumFast": "7a5d6b6c3977df0e9a0929cd6fe98f5f",
"#LongModerate": "ca954bbfd33831fa3a2bb7018d3ab654",
"#ShortTurbo": "efe09f21c232292838d5c657a0ecb814",
"#test": "9cd8fcf22a47333b591d96a2b848b73f",
"#public": "8b4b705b080c0d943b1c80f6b3ef6b6d",
"#general": "4c49f3f24629f5ee4ad5b3965db47985",
"#chat": "d0bdd6d71538138ed979eec00d98ad97",
"#local": "d2d35fa76be9875ed254db80397483a5",
"#emergency": "e1ad578d25108e344808f30dfdaaf926",
"#help": "dcc67fae2067046832af7b2b0b743165",
"#info": "ce51a275a0a0507c43d1651d78292320",
"#news": "ecadb1a7d803db8958bea1302ca6e8be",
"#weather": "88f502554fee92a1625cfb311546e7cb",
"#admin": "889334b7e486938c776dbdde120da9de",
"#mod": "8e238c8f71e508c849fa1743783359c9",
"#ops": "3b644de377c32c78793605a25aa915bf",
"#dev": "d41bcba61e9dca7177c7b8533d23a0bc",
"#debug": "fd7a60ed4796efcd1965c2de466105cb",
"#bot": "eb50a1bcb3e4e5d7bf69a57c9dada211",
"#bots": "0d24f5830b449668b8c221759b6c50d2",
"#sf": "a32c1fcfda0def959c305e4cd803def1",
"#bayarea": "7f9a5fd3070ad14e337ba100ca53a89b",
"#sfo": "1ecce5970716c9415b0411bf190944b1",
"#oakland": "c5a2f1d9f4433d041881447bf443084b",
"#sanjose": "ce964c28170b1c043d06073e6fcd83a7",
"#socal": "f4018307615ac79d2e5ef17bb44654d4",
"#la": "21349a74e68588be435f33abed117d84",
"#losangeles": "3dd9373dcea0294bd05ab067cc58e9d4",
"#sandiego": "08623bbd90a96ecdc1f5c34e7292b35f",
"#sacramento": "a6d6927f0b48762cf1346e2ae95cec14",
"#nyc": "6e6554655f84ca26fbc09d81d15d6b96",
"#newyork": "82a78024dd7edf6c9be298c919632e25",
"#brooklyn": "bcc6e13acd87570dcee4d5d87ac711e6",
"#manhattan": "56c90afc93b5bc55d1e3bdb1003dc2ef",
"#queens": "55ff7df317b63d269d878e51d125ced3",
"#seattle": "ef627a9bbbb549347fdb76bf0cd3bc14",
"#portland": "45c6bc719c15b9fb809f48f594359877",
"#pdx": "e75d6c892ff4d085e66701548c97acec",
"#denver": "b24355a0d22ed2bf393ec530d75810b4",
"#boulder": "eaa379d95ac9bbf857f499019ed0e8a5",
"#colorado": "ef61c9e5a3286053746f7603044bcb08",
"#austin": "b2e6f9af95d959734d71cdf90ca62533",
"#dallas": "5b2efa4e2ad0a2b83c5486ca4dd244de",
"#houston": "c001fbacad2d97676395ca37e2576345",
"#texas": "c4a214e133de5e9ad276f99fdbd7216f",
"#dfw": "5b7dc809ba579affccab5462c537244e",
"#chicago": "c1c289b131e5222370cbc2048445844b",
"#detroit": "bd01f26bf7d8c90952753157a41c61ac",
"#minneapolis": "3283cca82b7b0ac50e8014c344cb8a86",
"#stlouis": "f366422991a19e745f65096e59b43d51",
"#boston": "9587d847a7208da684c89cc1f525bc03",
"#philly": "9ff9182dd800e0be620dead724cbdf88",
"#philadelphia": "963ad5382e8d910ce0872958c5b36e6a",
"#dc": "0f3aa71fed514f5c16ebaf265ef05b2e",
"#washingtondc": "0067d451cc79f26bc9924b3fc53f28d7",
"#baltimore": "c5e649cacdc8fe661d5910d00c7c95ac",
"#atlanta": "3c8f15665f99a349a97427e7c312ee0e",
"#miami": "d81c566a5d337d588ebd250df6fc1b63",
"#tampa": "8d10508c39d5d8e6c5e3e8fab41d1c09",
"#orlando": "3553bdbd9b3a54da760624a27dcda156",
"#florida": "c44bd74eac2c81dcb6dfb217727b05cd",
"#phoenix": "027850d9410fa98809819c96644ec04c",
"#tucson": "cf989ccea881cf5ddcf40d87bbfc441e",
"#arizona": "8017183d8f9c01b660cd3663b8972e9d",
"#vegas": "d45435b6467e7ded33c28f6796bf5183",
"#lasvegas": "b82b823d6dcc845ead47cee8cfa758b3",
"#nevada": "9dc06c13ed0875b4a1acd545653fef33",
"#hawaii": "3fd57495820328594e1641d14583faa6",
"#honolulu": "81a4f1ae399448b8c10f8e761ba4e216",
"#maui": "cd08692b0cfbecfc06248bf8b8f10463",
"#alaska": "2a5841192b151422baec71537b0b5238",
"#anchorage": "2d87885e753b3231d33fd57dc53b5d69",
"#london": "9881d2b7ab9105a41a8d0f6ba449447e",
"#uk": "22b2eed34b5cc429ce1dc5e88635ff84",
"#berlin": "8bffd7b0bf481d92afc625e409b88a16",
"#germany": "0a834b4687fd4e09f72f6eeb3191ee25",
"#paris": "d0fc2b1ec400eb8669010ce0311a00fb",
"#france": "edcb362b38e74b99025a7e551d925d20",
"#amsterdam": "d768f5a0aa65f8c54e4ea521bd49eb4f",
"#netherlands": "cfc0a6c4004324e8adc99dfec1501943",
"#tokyo": "c574cca64e441dc3a414ef8047e8054d",
"#japan": "6c3db58db1c49d7e974971f675a66c89",
"#sydney": "57fe5284a5b905835193868d9bdbe1e9",
"#australia": "eadb84fa1da64c44b77c40fd11b9d78e",
"#melbourne": "4d73731d9450ccb9673eb923c0f40af2",
"#brisbane": "e4bd09784621decac600d0d4d857e3b2",
"#toronto": "e0d3774dd1da4dc5c55d8bd731555334",
"#vancouver": "16d6034be448ab86d11858cfa4c57c9c",
"#canada": "8373bb1055f34164c6dd2663927cb6d9",
"#montreal": "0c4c03b5fbea5b80f89e2a2a16ed3f40",
"#calgary": "bbbb1ad23fe1cdd7739667418204a57d",
"#mexico": "1e3806b6eccf8114ff7fc27ed8f84b0b",
"#cdmx": "042268f5d791d342d8c50c065ef0c50b",
"#brasil": "47d6242d2c1dd0f1e0eee4f0df64b2c1",
"#brazil": "aac6e31471f27feac8da78793bed9690",
"#ham": "5270db3979da687fa133fec6684cd952",
"#radio": "266f225baced1b2a868dcc8e9c69a304",
"#mesh": "5b664cde0b08b220612113db980650f3",
"#lora": "0749ea267c6be7b54ca1dbbae7dba0aa",
"#meshtastic": "73a2e13ff0dea9ed19b24b2ab753650f",
"#meshcore": "2fa78a5aef618e7c2a78f0ab5c8869b3",
"#offgrid": "aaa26662bebf122262692d0bca61dcef",
"#prepper": "1eec1a7df7080a392cb490473c4a9920",
"#preppers": "4877173813de9668b9ef33adbc1b8f37",
"#survival": "e1f465ea51df09fc901389758f1c5f01",
"#shtf": "9321638017bd7f42ca4468726cd06893",
"#emcomm": "a9a49340642dfc4c562d7849b7c8a258",
"#ares": "44419e394cde859e45710f288db939a8",
"#cert": "828d37872695b8b47e537164fb1570cf",
"#skywarn": "46e8175d5a3b373985eb471a3ad479f8",
"#hike": "a8f9964431372ed34db9088d03362f6f",
"#hiking": "2370a013053e384e5f18918bc2b26baf",
"#camping": "c011b7c2abf33748cd9bd5a78c2b4955",
"#outdoors": "b4c10e8ecfe10ef66cae6299ae29d488",
"#trail": "9b7f5ade7e2bff2eaaeebfd0333401a5",
"#bike": "22a682f2c0f3011aab4510c533278413",
"#cycling": "795980fbfe059133a2fc47c2c210f127",
"#wardrive": "4076c315c1ef385fa93f066027320fe5",
"#wardriving": "e3c26491e9cd321e3a6be50d57d54acf",
"#security": "b7123ea6c2dcbf7332a198ddd6612da9",
"#infosec": "364aa6797508df1cf248c8983ec968bd",
"#cyber": "bc435860170598cb9b1c7cc6938c8be5",
"#hacker": "98010d08107a5bc0b7f41ce3c93cba99",
"#hackers": "bb6c2edeb9a25b4f77398a687acfdb85",
"#maker": "04cf78295deab8c76fa3236f8a87612e",
"#makers": "4d8febe1979910912d7f8d11ca8ce689",
"#diy": "2947bea0668269732f29808d2b9c8fed",
"#electronics": "34d8645bda0ba7e4a7c78a1f9f3ed1ac",
"#arduino": "ce8c596922eff9274f3b1e19ad296754",
"#raspberry": "469bc476d8263b64b6f90ca868b04b91",
"#esp32": "9f8aac4c48973b07bbdc47acbaea3e8e",
"#linux": "02c74eda5d8ec8b9bb945049fb2f55c6",
"#unix": "a4c8cd0f130f2f9801d1afa2d0a52a2f",
"#tech": "5177b749eefbfab3c90a21b4e2518c5f",
"#code": "2a8a567349cce15a48fd5d81709424f4",
"#coding": "becdff6841b6f36610e97ac89c3a40ea",
"#programming": "bce55c792ec60d79530a2eec9a8ede14",
"#yo": "51f93a1e79f96333fe5d0c8eb3bed7c3",
"#sup": "153ec8634ac55727e006e37c39b29f16",
"#hello": "dc9fe3652402447479779b06609a22a5",
"#hi": "e411034bd14b17b5e39a76b5a5b4f348",
"#hey": "3972b5f0fe9a438f260ffc1d125eefdb",
"#random": "bda30db2910b90b1bc5a608a1b5f0ee4",
"#offtopic": "4bfd513d655d6e435bd8f4d6b863e500",
"#memes": "5b36bb8722a8c1741127266299439cdd",
"#fun": "ecfceab52a3d730051d2ce6adea30f78",
"#music": "b025ab29b0a5f56fb68a474741d7a2cc",
"#gaming": "3802c79121f195ea50ad9ab2aa2c402a",
"#games": "ddffe2cfbf037caed279d02c41b74f5f",
"#sports": "e8ee81f3aabf105d9ba2d2d4bd94fe4a",
"#food": "7b4f27d6b5bbf5f8eb3bfc6f43770fdd",
"#beer": "8fbe47b032102949554ac78fcd583560",
"#coffee": "82cd4bd9e7dda8cae0854281246cc64f",
"#queer": "5754476f162d93bbee3de0efba136860",
"#lgbtq": "7d71d54a2bb4bbd7352322a59126d7fb",
"#pride": "c732b12b15a5bca3cff2e39b7480baca",
"#bookclub": "b803ab3fbb867737ab5b7d32914d7e67",
"#books": "b2fdc9c9313dad4515527cade7d67021",
"#reading": "46d531186b6148726a97a62324f8a356",
"#film": "35ed875bfa83f29298e81f83c3c56c1e",
"#movies": "15cea29e9e62887c767cabec8c601ebe",
"#dogs": "e956e2b054b795c129a5daab4aad0be1",
"#cats": "4aeaf541d243ea9f84abc406b7eba360",
"#pets": "6f36ecba946c2eba3a7a9c694c53f23a",
"#1": "0b0fa0b2280a09639e2059e56c8fa932",
"#2": "b6b52c0e41dd6e18a31636deb586175b",
"#3": "ea46599a238699be9409316928670559",
"#4": "7e3cfd9c828a75671a34898112caa743",
"#5": "7aff87c72ca0f13d288b3418c89f67e9",
"#42": "2bccaf40951009e4203e2065b2a4bde0",
"#69": "0285b036b9837b5babac54668e623ce0",
"#420": "d4346dd20f9a85256cff48f33f46de0a",
"#1337": "817d01d43c5960c1ceaf9a2467182675",
"#a": "302d59c4f9e75750166105ea1d8b3673",
"#b": "dd883973c3c017ed51c9e10fba7bca0b",
"#c": "3732c64f873466d50e0badb3f8d79faf",
"#aa": "e147f36926b7b509af9b41b65304dc30",
"#ab": "7ee8192c80507041253e255dcc7e6f87",
"#abc": "00e16e1d31c0ba2b3b3d17583bb2ac3b",
"#norcal": "f2438510715f5d9d55eb4370664330f5",
"#nocal": "819836f0049f5dca9596cd681d0cbab8",
"#bay": "80e5fadb907564764eb09d2667a02638",
"#eastbay": "4c2e48f600e4952346441278ac363432",
"#southbay": "3c9e372b38917334d1091419f32bed8a",
"#northbay": "8dfb7427cc1e5abab25bc16e8ae4373c",
"#peninsula": "7f2ce5480359431d2f3ca259bf8bef68",
"#marin": "a084727e9d2d2afcc73f49055c6f7764",
"#pnw": "98059014c708581fbf0a398cfe8a486d",
"#pacnw": "49cc714305cf0a61817a01114414d490",
"#cascadia": "1313c4078af5c36040bec10115c04806",
"#midwest": "cf7910dacee35da8b90da21a0e37fea7",
"#northeast": "3b1ca0ae6003193eb9f91984eefef5dc",
"#southeast": "7b4392ed3c3fdde98cdb167c2f0f2c8b",
"#southwest": "e17c45726653035f7a90a6b57b5a0d57",
"#socalmesh": "df9e74198a7c334964f18f200a065e33",
"#sfmesh": "89454fcff893b5a2ecc16d886e9cf3b3",
"#nycmesh": "021be2e194650cc5d3ea77618eea817f",
"#atlmesh": "50b8266c71b3d3ee0253d462b34f6b2b",
"#wx": "472dd8595b8fd0ab542b3e86a379a620",
"#fire": "3d74a070077293ab66baf3aa724349ff",
"#earthquake": "0c7082c04a1a90502a5f32fc6a9f6524",
"#flood": "f2a0fd0abe4c9fc6865b5f8eebf319d6",
"#tsunami": "153ddc83452935d8486ab3d34dd6d313",
"#storm": "113761b9e31a5c30e0ee4fc78dc7310b",
"#alert": "678c7d2e08c019e113ace03eaaa128ad",
"#alerts": "b8212240d8b433b54db46906738e2094",
"#sos": "9ed2c78bcd68ac7ce2a2fc3bb4045114",
"#rescue": "8fefdc46995fd86cf1265a96e25e9be1",
"#missing": "bb627c3e98fb103e54b203a11d2c1a8c",
"#evac": "c20a9bf0eabefaf3dd3553bb5df19b61",
"#shelter": "b11ebee14de380147b2f0b613c82ee84",
"#default": "66e7fd8b7b4caa5dc98e752d43044d30",
"#main": "512ba51f98c27b93cd2ff6fbc2c0fad1",
"#primary": "3e417c7ce555fa7dcb705a94cbb358e4",
"#secondary": "69abb13534f87ebfedc0d92797da1fbe",
"#backup": "1fa50c46f15c60363567cae7982b8394",
"#private": "bd072d2fbd62a89db08b2a9e6976cc36",
"#secret": "be657c0527e122bf93bc735999cd7e0d",
"#hidden": "f78143af9f1168c243729b1bf6bb3235",
"#invite": "d46e531ee7d3b591fcf2dfc9e23e63e1",
"#repeater": "289991a3077903263f2d31493887c651",
"#repeaters": "89db441e2814dccf0dbd2e8cc5f501a3",
"#node": "85cdc068443a7bf5b9435423c40dbefd",
"#nodes": "d2b5d06216710d4dbef3de9d08168a53",
"#gateway": "73feacb0c27f83b3d2db143823efb891",
"#server": "11c4e843fc066c8bfe01719cfca1fb1e",
"#gisborne": "d8b45a1eb52d0bd45655d5f2a72d571f",
"#wellington": "c2da57d74f78996ad71bbe3e22446f16",
"#auckland": "7c6f1c71a5a3d6823a1d7bfd2a349ac8",
"#nz": "eb87ee8817ba71315ac7be9c733b523a",
"#newzealand": "1870a961e4aa06f62b02b835efcacb71",
"#spain": "49217e19fa0d5c28bd02fd6b688dd11d",
"#madrid": "8886480d4b99f6882328d9068d0c6235",
"#barcelona": "919a4a5d9522320ca9c95dceb92a5544",
"#italy": "f8d4dd36f6b9476eb4cec18a1536b17d",
"#rome": "4c3729aca56d948088bd25750e8d7d33",
"#milan": "71cd9f729336b7a1110f77bb93c43e9e",
"#sweden": "d6bcdf00baf5d2d981846d2b47ba5b42",
"#stockholm": "889483e6b54ca5bf8cbb2af23e1ab7f0",
"#norway": "0ed3e16f327a787d5ff4c4496bb4c4a9",
"#oslo": "1a06f287bc45a8cf14a304f898cc1fea",
"#finland": "cc012bb6718f447824c8ba7cd81b7fbd",
"#helsinki": "6bd215dc2f6f8833a309eba8d4ba57d7",
"#denmark": "7bb81ff9d3eb29f091d1d64e044b2a79",
"#copenhagen": "76c1331a0d13d081988a379c60ad59bb",
"#poland": "e63d27631c0f74aca88a6b91efcc7067",
"#warsaw": "86afa156c7f0567c97ea03df07482888",
"#czech": "0a15067478a2b8e74177c8fc68e4001e",
"#prague": "564cb31b3895393f22f0f50354389334",
"#austria": "faaa5ef01081222e319a8205357321f4",
"#vienna": "8e52cd2b9ff13fb0030fd41714edc95e",
"#switzerland": "8ad1ce57ad257627090ed28413c1f0b7",
"#zurich": "95a7261009cdcb13d22e8f8d532f3ba5",
"#portugal": "11f13d9d06c892574a277337967a7267",
"#lisbon": "6c49a07fd953c5856df538d5dce0b19e",
"#ireland": "1b2a12acc5db1517d9d407946756b1da",
"#dublin": "8792638977132bc05a1f72d6bb913694",
"#scotland": "f4fce403cfd56f7089920d065718f29c",
"#edinburgh": "72d70a8e87b0e7f8072239d68ffccc9e",
"#wales": "809573d8134fa262d284400a788f63d9",
"#india": "bedb569c4d55038e801985e87bc311cb",
"#mumbai": "408b4cecfb253c8150cadd5da8925b1f",
"#delhi": "4974162f580211f17187d1a16cab2514",
"#bangalore": "f87f1fcf72618fbb5d36642847859df0",
"#korea": "38ea37fe7de4691145f8e200a3fc6976",
"#seoul": "d6e3685ad1ce9d943c34594321eb3d75",
"#taiwan": "489ff602625eb18ae5f457fb70e149cd",
"#taipei": "257f2ea07b93df20b8ef8a69459cc541",
"#singapore": "524771d953b40e8880e00d7250f02c42",
"#malaysia": "a51ef9684d30551eb7fe4faa00c4dd64",
"#thailand": "6ddc1b15fd53b35a8c1a7e8bb720d5ac",
"#bangkok": "68b4467f7e1a99705143a6a5ebcfb8e8",
"#vietnam": "9a6514b712cbc8a1be0663591c6a6e13",
"#indonesia": "2be2d4470d8cc641ce69dfe6497a2842",
"#jakarta": "5b931a38c29a24ec8395c23421248138",
"#philippines": "74d160fb0ad7867295730d41351dd21a",
"#manila": "48a840132b292a2dcde0ef0d10c3149b",
"#southafrica": "35bb5bcec03c0c7ba256bdc948108a1c",
"#capetown": "a16101e1fa482e43dcb4b35ab836b3f3",
"#nairobi": "44cf6c94cdcb61d576c8855935997260",
"#kenya": "5bde28964c3008a6741f593d3b70c78e",
"#nigeria": "9a20897b3b223ee02ba9eebc43ac2300",
"#lagos": "c1d000aa764a45ba0d992d0289137991",
"#argentina": "22304ee269f9623972776a4d1d306afd",
"#buenosaires": "43ece797139ce4051cf62568a6a28c2b",
"#chile": "15f352f255947e485b845652791f3354",
"#santiago": "9d7f9df716281124aa16d27a45b2ff5f",
"#colombia": "bea223a8c1d13ed9638ee000ea3a6aca",
"#bogota": "6d0864985b64350ce4cbfebf4979e970",
"#peru": "7e6fc347bf29a4c128ac3156865bd521",
"#lima": "5f167ce354eca08ab742463df10ef255"
}

View File

@@ -1,5 +1,10 @@
{
"port": 3000,
"apiKey": "your-secret-api-key-here",
"https": {
"cert": "/path/to/cert.pem",
"key": "/path/to/key.pem"
},
"mqtt": {
"broker": "mqtt://localhost:1883",
"topic": "meshcore/+/+/packets"
@@ -8,7 +13,10 @@
{
"name": "local",
"broker": "mqtt://localhost:1883",
"topics": ["meshcore/+/+/packets", "meshcore/#"]
"topics": [
"meshcore/+/+/packets",
"meshcore/#"
]
},
{
"name": "lincomatic",
@@ -16,22 +24,42 @@
"username": "your-username",
"password": "your-password",
"rejectUnauthorized": false,
"topics": ["meshcore/+/+/packets", "meshcore/+/+/status"],
"iataFilter": ["SJC", "SFO", "OAK"]
"topics": [
"meshcore/SJC/#",
"meshcore/SFO/#",
"meshcore/OAK/#",
"meshcore/MRY/#"
],
"iataFilter": [
"SJC",
"SFO",
"OAK",
"MRY"
]
}
],
"channelKeys": {
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72",
"#test": "9cd8fcf22a47333b591d96a2b848b73f",
"#sf": "a32c1fcfda0def959c305e4cd803def1",
"#wardrive": "4076c315c1ef385fa93f066027320fe5",
"#yo": "51f93a1e79f96333fe5d0c8eb3bed7c3",
"#bot": "eb50a1bcb3e4e5d7bf69a57c9dada211",
"#queer": "5754476f162d93bbee3de0efba136860",
"#bookclub": "b803ab3fbb867737ab5b7d32914d7e67",
"#shtf": "9321638017bd7f42ca4468726cd06893"
"public": "8b3387e9c5cdea6ac9e5edbaa115cd72"
},
"hashChannels": [
"#LongFast",
"#test",
"#sf",
"#wardrive",
"#yo",
"#bot",
"#queer",
"#bookclub",
"#shtf"
],
"defaultRegion": "SJC",
"mapDefaults": {
"center": [
37.45,
-122.0
],
"zoom": 9
},
"regions": {
"SJC": "San Jose, US",
"SFO": "San Francisco, US",
@@ -59,6 +87,10 @@
"invalidationDebounce": 30,
"_comment": "All values in seconds. Server uses these directly. Client fetches via /api/config/cache."
},
"liveMap": {
"propagationBufferMs": 5000,
"_comment": "How long (ms) to buffer incoming observations of the same packet before animating. Mesh packets propagate through multiple paths and arrive at different observers over several seconds. This window collects all observations of a single transmission so the live map can animate them simultaneously as one realistic propagation event. Set higher for wide meshes with many observers, lower for snappier animations. 5000ms captures ~95% of observations for a typical mesh."
},
"packetStore": {
"maxMemoryMB": 1024,
"estimatedPacketBytes": 450,

238
db.js
View File

@@ -10,28 +10,21 @@ if (!fs.existsSync(dataDir)) fs.mkdirSync(dataDir, { recursive: true });
const db = new Database(dbPath);
db.pragma('journal_mode = WAL');
db.pragma('foreign_keys = ON');
db.pragma('wal_autocheckpoint = 0'); // Disable auto-checkpoint — manual checkpoint on timer to avoid random event loop spikes
// --- Migration: drop legacy tables (replaced by transmissions + observations in v2.3.0) ---
// Drop paths first (has FK to packets)
const legacyTables = ['paths', 'packets'];
for (const t of legacyTables) {
const exists = db.prepare(`SELECT name FROM sqlite_master WHERE type='table' AND name=?`).get(t);
if (exists) {
console.log(`[migration] Dropping legacy table: ${t}`);
db.exec(`DROP TABLE IF EXISTS ${t}`);
}
}
// --- Schema ---
db.exec(`
CREATE TABLE IF NOT EXISTS packets (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
timestamp TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
hash TEXT,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
path_json TEXT,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS nodes (
public_key TEXT PRIMARY KEY,
name TEXT,
@@ -59,18 +52,58 @@ db.exec(`
noise_floor INTEGER
);
CREATE TABLE IF NOT EXISTS paths (
id INTEGER PRIMARY KEY AUTOINCREMENT,
packet_id INTEGER REFERENCES packets(id),
hop_index INTEGER,
node_hash TEXT
);
CREATE INDEX IF NOT EXISTS idx_packets_timestamp ON packets(timestamp);
CREATE INDEX IF NOT EXISTS idx_packets_hash ON packets(hash);
CREATE INDEX IF NOT EXISTS idx_packets_payload_type ON packets(payload_type);
CREATE INDEX IF NOT EXISTS idx_nodes_last_seen ON nodes(last_seen);
CREATE INDEX IF NOT EXISTS idx_observers_last_seen ON observers(last_seen);
CREATE TABLE IF NOT EXISTS transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
hash TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_transmissions_hash ON transmissions(hash);
CREATE INDEX IF NOT EXISTS idx_transmissions_first_seen ON transmissions(first_seen);
CREATE INDEX IF NOT EXISTS idx_transmissions_payload_type ON transmissions(payload_type);
CREATE INDEX IF NOT EXISTS idx_observations_hash ON observations(hash);
CREATE INDEX IF NOT EXISTS idx_observations_transmission_id ON observations(transmission_id);
CREATE INDEX IF NOT EXISTS idx_observations_observer_id ON observations(observer_id);
CREATE INDEX IF NOT EXISTS idx_observations_timestamp ON observations(timestamp);
DROP INDEX IF EXISTS idx_observations_dedup;
CREATE UNIQUE INDEX IF NOT EXISTS idx_observations_dedup ON observations(hash, observer_id, COALESCE(path_json, ''));
-- Clean up legacy duplicates (same hash+observer+path, keep lowest id)
DELETE FROM observations WHERE id NOT IN (
SELECT MIN(id) FROM observations GROUP BY hash, observer_id, COALESCE(path_json, '')
);
CREATE VIEW IF NOT EXISTS packets_v AS
SELECT o.id, t.raw_hex, o.timestamp, o.observer_id, o.observer_name,
o.direction, o.snr, o.rssi, o.score, t.hash, t.route_type,
t.payload_type, t.payload_version, o.path_json, t.decoded_json,
t.created_at
FROM observations o
JOIN transmissions t ON t.id = o.transmission_id;
`);
// --- Migrations for existing DBs ---
@@ -83,13 +116,21 @@ for (const col of ['model', 'firmware', 'client_version', 'radio', 'battery_mv',
}
}
// --- Cleanup corrupted nodes on startup ---
// Remove nodes with obviously invalid data (short pubkeys, control chars in names, etc.)
{
const cleaned = db.prepare(`
DELETE FROM nodes WHERE
length(public_key) < 16
OR public_key GLOB '*[^0-9a-fA-F]*'
OR (lat IS NOT NULL AND (lat < -90 OR lat > 90))
OR (lon IS NOT NULL AND (lon < -180 OR lon > 180))
`).run();
if (cleaned.changes > 0) console.log(`[cleanup] Removed ${cleaned.changes} corrupted node(s) from DB`);
}
// --- Prepared statements ---
const stmts = {
insertPacket: db.prepare(`
INSERT INTO packets (raw_hex, timestamp, observer_id, observer_name, direction, snr, rssi, score, hash, route_type, payload_type, payload_version, path_json, decoded_json)
VALUES (@raw_hex, @timestamp, @observer_id, @observer_name, @direction, @snr, @rssi, @score, @hash, @route_type, @payload_type, @payload_version, @path_json, @decoded_json)
`),
insertPath: db.prepare(`INSERT INTO paths (packet_id, hop_index, node_hash) VALUES (?, ?, ?)`),
upsertNode: db.prepare(`
INSERT INTO nodes (public_key, name, role, lat, lon, last_seen, first_seen, advert_count)
VALUES (@public_key, @name, @role, @lat, @lon, @last_seen, @first_seen, 1)
@@ -132,49 +173,72 @@ const stmts = {
uptime_secs = COALESCE(@uptime_secs, uptime_secs),
noise_floor = COALESCE(@noise_floor, noise_floor)
`),
getPacket: db.prepare(`SELECT * FROM packets WHERE id = ?`),
getPathsForPacket: db.prepare(`SELECT * FROM paths WHERE packet_id = ? ORDER BY hop_index`),
getPacket: db.prepare(`SELECT * FROM packets_v WHERE id = ?`),
getNode: db.prepare(`SELECT * FROM nodes WHERE public_key = ?`),
getRecentPacketsForNode: db.prepare(`
SELECT * FROM packets WHERE decoded_json LIKE ? OR decoded_json LIKE ? OR decoded_json LIKE ? OR decoded_json LIKE ?
SELECT * FROM packets_v WHERE decoded_json LIKE ? OR decoded_json LIKE ? OR decoded_json LIKE ? OR decoded_json LIKE ?
ORDER BY timestamp DESC LIMIT 20
`),
getObservers: db.prepare(`SELECT * FROM observers ORDER BY last_seen DESC`),
countPackets: db.prepare(`SELECT COUNT(*) as count FROM packets`),
countPackets: db.prepare(`SELECT COUNT(*) as count FROM observations`),
countNodes: db.prepare(`SELECT COUNT(*) as count FROM nodes`),
countObservers: db.prepare(`SELECT COUNT(*) as count FROM observers`),
countRecentPackets: db.prepare(`SELECT COUNT(*) as count FROM packets WHERE timestamp > ?`),
countRecentPackets: db.prepare(`SELECT COUNT(*) as count FROM observations WHERE timestamp > ?`),
getTransmissionByHash: db.prepare(`SELECT id, first_seen FROM transmissions WHERE hash = ?`),
insertTransmission: db.prepare(`
INSERT INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json)
VALUES (@raw_hex, @hash, @first_seen, @route_type, @payload_type, @payload_version, @decoded_json)
`),
updateTransmissionFirstSeen: db.prepare(`UPDATE transmissions SET first_seen = @first_seen WHERE id = @id`),
insertObservation: db.prepare(`
INSERT OR IGNORE INTO observations (transmission_id, hash, observer_id, observer_name, direction, snr, rssi, score, path_json, timestamp)
VALUES (@transmission_id, @hash, @observer_id, @observer_name, @direction, @snr, @rssi, @score, @path_json, @timestamp)
`),
};
// --- Helper functions ---
function insertPacket(data) {
const d = {
raw_hex: data.raw_hex,
timestamp: data.timestamp || new Date().toISOString(),
function insertTransmission(data) {
const hash = data.hash;
if (!hash) return null; // Can't deduplicate without a hash
const timestamp = data.timestamp || new Date().toISOString();
let transmissionId;
const existing = stmts.getTransmissionByHash.get(hash);
if (existing) {
transmissionId = existing.id;
// Update first_seen if this observation is earlier
if (timestamp < existing.first_seen) {
stmts.updateTransmissionFirstSeen.run({ id: transmissionId, first_seen: timestamp });
}
} else {
const result = stmts.insertTransmission.run({
raw_hex: data.raw_hex || '',
hash,
first_seen: timestamp,
route_type: data.route_type ?? null,
payload_type: data.payload_type ?? null,
payload_version: data.payload_version ?? null,
decoded_json: data.decoded_json || null,
});
transmissionId = result.lastInsertRowid;
}
const obsResult = stmts.insertObservation.run({
transmission_id: transmissionId,
hash,
observer_id: data.observer_id || null,
observer_name: data.observer_name || null,
direction: data.direction || null,
snr: data.snr ?? null,
rssi: data.rssi ?? null,
score: data.score ?? null,
hash: data.hash || null,
route_type: data.route_type ?? null,
payload_type: data.payload_type ?? null,
payload_version: data.payload_version ?? null,
path_json: data.path_json || null,
decoded_json: data.decoded_json || null,
};
return stmts.insertPacket.run(d).lastInsertRowid;
}
function insertPath(packetId, hops) {
const tx = db.transaction((hops) => {
for (let i = 0; i < hops.length; i++) {
stmts.insertPath.run(packetId, i, hops[i]);
}
timestamp,
});
tx(hops);
return { transmissionId, observationId: obsResult.lastInsertRowid };
}
function upsertNode(data) {
@@ -234,15 +298,20 @@ function getPackets({ limit = 50, offset = 0, type, route, hash, since } = {}) {
if (hash) { where.push('hash = @hash'); params.hash = hash; }
if (since) { where.push('timestamp > @since'); params.since = since; }
const clause = where.length ? 'WHERE ' + where.join(' AND ') : '';
const rows = db.prepare(`SELECT * FROM packets ${clause} ORDER BY timestamp DESC LIMIT @limit OFFSET @offset`).all({ ...params, limit, offset });
const total = db.prepare(`SELECT COUNT(*) as count FROM packets ${clause}`).get(params).count;
const rows = db.prepare(`SELECT * FROM packets_v ${clause} ORDER BY timestamp DESC LIMIT @limit OFFSET @offset`).all({ ...params, limit, offset });
const total = db.prepare(`SELECT COUNT(*) as count FROM packets_v ${clause}`).get(params).count;
return { rows, total };
}
function getTransmission(id) {
try {
return db.prepare('SELECT * FROM transmissions WHERE id = ?').get(id) || null;
} catch { return null; }
}
function getPacket(id) {
const packet = stmts.getPacket.get(id);
if (!packet) return null;
packet.paths = stmts.getPathsForPacket.all(id);
return packet;
}
@@ -276,8 +345,15 @@ function getObservers() {
function getStats() {
const oneHourAgo = new Date(Date.now() - 3600000).toISOString();
// Try to get transmission count from normalized schema
let totalTransmissions = null;
try {
totalTransmissions = db.prepare('SELECT COUNT(*) as count FROM transmissions').get().count;
} catch {}
return {
totalPackets: stmts.countPackets.get().count,
totalPackets: totalTransmissions || stmts.countPackets.get().count,
totalTransmissions,
totalObservations: stmts.countPackets.get().count,
totalNodes: stmts.countNodes.get().count,
totalObservers: stmts.countObservers.get().count,
packetsLastHour: stmts.countRecentPackets.get(oneHourAgo).count,
@@ -291,7 +367,7 @@ function seed() {
upsertObserver({ id: 'obs-seed-001', name: 'Seed Observer', iata: 'UNK', last_seen: now, first_seen: now });
const pktId = insertPacket({
insertTransmission({
raw_hex: rawHex,
timestamp: now,
observer_id: 'obs-seed-001',
@@ -308,8 +384,6 @@ function seed() {
decoded_json: JSON.stringify({ type: 'ADVERT', name: 'Test Repeater', role: 'repeater', lat: 0, lon: 0 }),
});
insertPath(pktId, ['A1B2', 'C3D4']);
upsertNode({
public_key: 'seed-test-pubkey',
name: 'Test Repeater',
@@ -359,7 +433,7 @@ function getNodeHealth(pubkey) {
const observers = db.prepare(`
SELECT observer_id, observer_name,
AVG(snr) as avgSnr, AVG(rssi) as avgRssi, COUNT(*) as packetCount
FROM packets
FROM packets_v
WHERE ${whereClause} AND observer_id IS NOT NULL
GROUP BY observer_id
ORDER BY packetCount DESC
@@ -367,20 +441,20 @@ function getNodeHealth(pubkey) {
// Stats
const packetsToday = db.prepare(`
SELECT COUNT(*) as count FROM packets WHERE ${whereClause} AND timestamp > @since
SELECT COUNT(*) as count FROM packets_v WHERE ${whereClause} AND timestamp > @since
`).get({ ...params, since: todayISO }).count;
const avgStats = db.prepare(`
SELECT AVG(snr) as avgSnr FROM packets WHERE ${whereClause}
SELECT AVG(snr) as avgSnr FROM packets_v WHERE ${whereClause}
`).get(params);
const lastHeard = db.prepare(`
SELECT MAX(timestamp) as lastHeard FROM packets WHERE ${whereClause}
SELECT MAX(timestamp) as lastHeard FROM packets_v WHERE ${whereClause}
`).get(params).lastHeard;
// Avg hops from path_json
const pathRows = db.prepare(`
SELECT path_json FROM packets WHERE ${whereClause} AND path_json IS NOT NULL
SELECT path_json FROM packets_v WHERE ${whereClause} AND path_json IS NOT NULL
`).all(params);
let totalHops = 0, hopCount = 0;
@@ -393,12 +467,12 @@ function getNodeHealth(pubkey) {
const avgHops = hopCount > 0 ? Math.round(totalHops / hopCount) : 0;
const totalPackets = db.prepare(`
SELECT COUNT(*) as count FROM packets WHERE ${whereClause}
SELECT COUNT(*) as count FROM packets_v WHERE ${whereClause}
`).get(params).count;
// Recent 10 packets
const recentPackets = db.prepare(`
SELECT * FROM packets WHERE ${whereClause} ORDER BY timestamp DESC LIMIT 10
SELECT * FROM packets_v WHERE ${whereClause} ORDER BY timestamp DESC LIMIT 10
`).all(params);
return {
@@ -429,31 +503,31 @@ function getNodeAnalytics(pubkey, days) {
// Activity timeline
const activityTimeline = db.prepare(`
SELECT strftime('%Y-%m-%dT%H:00:00Z', timestamp) as bucket, COUNT(*) as count
FROM packets WHERE ${timeWhere} GROUP BY bucket ORDER BY bucket
FROM packets_v WHERE ${timeWhere} GROUP BY bucket ORDER BY bucket
`).all(params);
// SNR trend
const snrTrend = db.prepare(`
SELECT timestamp, snr, rssi, observer_id, observer_name
FROM packets WHERE ${timeWhere} AND snr IS NOT NULL ORDER BY timestamp
FROM packets_v WHERE ${timeWhere} AND snr IS NOT NULL ORDER BY timestamp
`).all(params);
// Packet type breakdown
const packetTypeBreakdown = db.prepare(`
SELECT payload_type, COUNT(*) as count FROM packets WHERE ${timeWhere} GROUP BY payload_type
SELECT payload_type, COUNT(*) as count FROM packets_v WHERE ${timeWhere} GROUP BY payload_type
`).all(params);
// Observer coverage
const observerCoverage = db.prepare(`
SELECT observer_id, observer_name, COUNT(*) as packetCount,
AVG(snr) as avgSnr, AVG(rssi) as avgRssi, MIN(timestamp) as firstSeen, MAX(timestamp) as lastSeen
FROM packets WHERE ${timeWhere} AND observer_id IS NOT NULL
FROM packets_v WHERE ${timeWhere} AND observer_id IS NOT NULL
GROUP BY observer_id ORDER BY packetCount DESC
`).all(params);
// Hop distribution
const pathRows = db.prepare(`
SELECT path_json FROM packets WHERE ${timeWhere} AND path_json IS NOT NULL
SELECT path_json FROM packets_v WHERE ${timeWhere} AND path_json IS NOT NULL
`).all(params);
const hopCounts = {};
@@ -475,7 +549,7 @@ function getNodeAnalytics(pubkey, days) {
// Peer interactions from decoded_json
const decodedRows = db.prepare(`
SELECT decoded_json, timestamp FROM packets WHERE ${timeWhere} AND decoded_json IS NOT NULL
SELECT decoded_json, timestamp FROM packets_v WHERE ${timeWhere} AND decoded_json IS NOT NULL
`).all(params);
const peerMap = {};
@@ -501,11 +575,11 @@ function getNodeAnalytics(pubkey, days) {
const uptimeHeatmap = db.prepare(`
SELECT CAST(strftime('%w', timestamp) AS INTEGER) as dayOfWeek,
CAST(strftime('%H', timestamp) AS INTEGER) as hour, COUNT(*) as count
FROM packets WHERE ${timeWhere} GROUP BY dayOfWeek, hour
FROM packets_v WHERE ${timeWhere} GROUP BY dayOfWeek, hour
`).all(params);
// Computed stats
const totalPackets = db.prepare(`SELECT COUNT(*) as count FROM packets WHERE ${timeWhere}`).get(params).count;
const totalPackets = db.prepare(`SELECT COUNT(*) as count FROM packets_v WHERE ${timeWhere}`).get(params).count;
const uniqueObservers = observerCoverage.length;
const uniquePeers = peerInteractions.length;
const avgPacketsPerDay = days > 0 ? Math.round(totalPackets / days * 10) / 10 : totalPackets;
@@ -517,7 +591,7 @@ function getNodeAnalytics(pubkey, days) {
// Longest silence
const timestamps = db.prepare(`
SELECT timestamp FROM packets WHERE ${timeWhere} ORDER BY timestamp
SELECT timestamp FROM packets_v WHERE ${timeWhere} ORDER BY timestamp
`).all(params).map(r => new Date(r.timestamp).getTime());
let longestSilenceMs = 0, longestSilenceStart = null;
@@ -557,4 +631,4 @@ function getNodeAnalytics(pubkey, days) {
};
}
module.exports = { db, insertPacket, insertPath, upsertNode, upsertObserver, updateObserverStatus, getPackets, getPacket, getNodes, getNode, getObservers, getStats, seed, searchNodes, getNodeHealth, getNodeAnalytics };
module.exports = { db, insertTransmission, upsertNode, upsertObserver, updateObserverStatus, getPackets, getPacket, getTransmission, getNodes, getNode, getObservers, getStats, seed, searchNodes, getNodeHealth, getNodeAnalytics };

View File

@@ -265,7 +265,58 @@ function decodePacket(hexString, channelKeys) {
};
}
module.exports = { decodePacket, ROUTE_TYPES, PAYLOAD_TYPES };
// --- ADVERT validation ---
const VALID_ROLES = new Set(['repeater', 'companion', 'room', 'sensor']);
/**
* Validate decoded ADVERT data before upserting into the DB.
* Returns { valid: true } or { valid: false, reason: string }.
*/
function validateAdvert(advert) {
if (!advert || advert.error) return { valid: false, reason: advert?.error || 'null advert' };
// pubkey must be at least 16 hex chars (8 bytes) and not all zeros
const pk = advert.pubKey || '';
if (pk.length < 16) return { valid: false, reason: `pubkey too short (${pk.length} hex chars)` };
if (/^0+$/.test(pk)) return { valid: false, reason: 'pubkey is all zeros' };
// lat/lon must be in valid ranges if present
if (advert.lat != null) {
if (!Number.isFinite(advert.lat) || advert.lat < -90 || advert.lat > 90) {
return { valid: false, reason: `invalid lat: ${advert.lat}` };
}
}
if (advert.lon != null) {
if (!Number.isFinite(advert.lon) || advert.lon < -180 || advert.lon > 180) {
return { valid: false, reason: `invalid lon: ${advert.lon}` };
}
}
// name must not contain control chars (except space) or be garbage
if (advert.name != null) {
// eslint-disable-next-line no-control-regex
if (/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]/.test(advert.name)) {
return { valid: false, reason: 'name contains control characters' };
}
// Reject names that are mostly non-printable or suspiciously long
if (advert.name.length > 64) {
return { valid: false, reason: `name too long (${advert.name.length} chars)` };
}
}
// role derivation check — flags byte should produce a known role
if (advert.flags) {
const role = advert.flags.repeater ? 'repeater' : advert.flags.room ? 'room' : advert.flags.sensor ? 'sensor' : 'companion';
if (!VALID_ROLES.has(role)) return { valid: false, reason: `unknown role: ${role}` };
}
// timestamp: decoded but not currently used for node storage — skip validation
return { valid: true };
}
module.exports = { decodePacket, validateAdvert, ROUTE_TYPES, PAYLOAD_TYPES, VALID_ROLES };
// --- Tests ---
if (require.main === module) {

View File

@@ -1,6 +1,6 @@
{
"name": "meshcore-analyzer",
"version": "2.1.0",
"version": "2.4.1",
"description": "Community-run alternative to the closed-source `analyzer.letsmesh.net`. MQTT packet collection + open-source web analyzer for the Bay Area MeshCore mesh.",
"main": "index.js",
"scripts": {

View File

@@ -1,13 +1,14 @@
'use strict';
/**
* In-memory packet store — loads all packets from SQLite on startup,
* In-memory packet store — loads transmissions + observations from SQLite on startup,
* serves reads from RAM, writes to both RAM + SQLite.
* M3: Restructured around transmissions (deduped by hash) with observations.
* Caps memory at configurable limit (default 1GB).
*/
class PacketStore {
constructor(dbModule, config = {}) {
this.dbModule = dbModule; // The full db module (has .db, .insertPacket, .getPacket)
this.dbModule = dbModule; // The full db module (has .db, .insertTransmission, .getPacket)
this.db = dbModule.db; // Raw better-sqlite3 instance for queries
this.maxBytes = (config.maxMemoryMB || 1024) * 1024 * 1024;
this.estPacketBytes = config.estimatedPacketBytes || 450;
@@ -16,16 +17,23 @@ class PacketStore {
// SQLite-only mode: skip RAM loading, all reads go to DB
this.sqliteOnly = process.env.NO_MEMORY_STORE === '1';
// Core storage: array sorted by timestamp DESC (newest first)
// Primary storage: transmissions sorted by first_seen DESC (newest first)
// Each transmission looks like a packet for backward compat
this.packets = [];
// Indexes
this.byId = new Map();
this.byHash = new Map(); // hash → [packet, ...]
this.byObserver = new Map(); // observer_id → [packet, ...]
this.byNode = new Map(); // pubkey → [packet, ...]
this.byId = new Map(); // observation_id → observation object (backward compat for packet detail links)
this.byTxId = new Map(); // transmission_id → transmission object
this.byHash = new Map(); // hash → transmission object (1:1)
this.byObserver = new Map(); // observer_id → [observation objects]
this.byNode = new Map(); // pubkey → [transmission objects] (deduped)
// Track which hashes are indexed per node pubkey (avoid dupes in byNode)
this._nodeHashIndex = new Map(); // pubkey → Set<hash>
this._advertByObserver = new Map(); // pubkey → Set<observer_id> (ADVERT-only, for region filtering)
this.loaded = false;
this.stats = { totalLoaded: 0, evicted: 0, inserts: 0, queries: 0 };
this.stats = { totalLoaded: 0, totalObservations: 0, evicted: 0, inserts: 0, queries: 0 };
}
/** Load all packets from SQLite into memory */
@@ -35,61 +43,411 @@ class PacketStore {
this.loaded = true;
return this;
}
const t0 = Date.now();
const rows = this.db.prepare(
'SELECT * FROM packets ORDER BY timestamp DESC'
).all();
for (const row of rows) {
if (this.packets.length >= this.maxPackets) break;
this._index(row);
this.packets.push(row);
const t0 = Date.now();
// Check if normalized schema exists
const hasTransmissions = this.db.prepare(
"SELECT name FROM sqlite_master WHERE type='table' AND name='transmissions'"
).get();
if (hasTransmissions) {
this._loadNormalized();
} else {
this._loadLegacy();
}
this.stats.totalLoaded = this.packets.length;
this.loaded = true;
const elapsed = Date.now() - t0;
console.log(`[PacketStore] Loaded ${this.packets.length} packets in ${elapsed}ms (${Math.round(this.packets.length * this.estPacketBytes / 1024 / 1024)}MB est)`);
console.log(`[PacketStore] Loaded ${this.packets.length} transmissions (${this.stats.totalObservations} observations) in ${elapsed}ms (${Math.round(this.packets.length * this.estPacketBytes / 1024 / 1024)}MB est)`);
return this;
}
/** Index a packet into all lookup maps */
_index(pkt) {
this.byId.set(pkt.id, pkt);
/** Load from normalized transmissions + observations tables */
_loadNormalized() {
const rows = this.db.prepare(`
SELECT t.id AS transmission_id, t.raw_hex, t.hash, t.first_seen, t.route_type,
t.payload_type, t.payload_version, t.decoded_json,
o.id AS observation_id, o.observer_id, o.observer_name, o.direction,
o.snr, o.rssi, o.score, o.path_json, o.timestamp AS obs_timestamp
FROM transmissions t
LEFT JOIN observations o ON o.transmission_id = t.id
ORDER BY t.first_seen DESC, o.timestamp DESC
`).all();
if (pkt.hash) {
if (!this.byHash.has(pkt.hash)) this.byHash.set(pkt.hash, []);
this.byHash.get(pkt.hash).push(pkt);
for (const row of rows) {
if (this.packets.length >= this.maxPackets && !this.byHash.has(row.hash)) break;
let tx = this.byHash.get(row.hash);
if (!tx) {
tx = {
id: row.transmission_id,
raw_hex: row.raw_hex,
hash: row.hash,
first_seen: row.first_seen,
timestamp: row.first_seen,
route_type: row.route_type,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
observations: [],
observation_count: 0,
// Filled from first observation for backward compat
observer_id: null,
observer_name: null,
snr: null,
rssi: null,
path_json: null,
direction: null,
};
this.byHash.set(row.hash, tx);
this.byHash.set(row.hash, tx);
this.packets.push(tx);
this.byTxId.set(tx.id, tx);
this._indexByNode(tx);
}
if (row.observation_id != null) {
const obs = {
id: row.observation_id,
observer_id: row.observer_id,
observer_name: row.observer_name,
direction: row.direction,
snr: row.snr,
rssi: row.rssi,
score: row.score,
path_json: row.path_json,
timestamp: row.obs_timestamp,
// Carry transmission fields for backward compat
hash: row.hash,
raw_hex: row.raw_hex,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
route_type: row.route_type,
};
// Dedup: skip if same observer + same path already loaded
const isDupeLoad = tx.observations.some(o => o.observer_id === obs.observer_id && (o.path_json || '') === (obs.path_json || ''));
if (isDupeLoad) continue;
tx.observations.push(obs);
tx.observation_count++;
// Fill first observation data into transmission for backward compat
if (tx.observer_id == null && obs.observer_id) {
tx.observer_id = obs.observer_id;
tx.observer_name = obs.observer_name;
tx.snr = obs.snr;
tx.rssi = obs.rssi;
tx.path_json = obs.path_json;
tx.direction = obs.direction;
}
// byId maps observation IDs for packet detail links
this.byId.set(obs.id, obs);
// byObserver
if (obs.observer_id) {
if (!this.byObserver.has(obs.observer_id)) this.byObserver.set(obs.observer_id, []);
this.byObserver.get(obs.observer_id).push(obs);
}
this.stats.totalObservations++;
}
}
// Post-load: set each transmission's observer/path to the EARLIEST observation
for (const tx of this.packets) {
if (tx.observations.length > 0) {
let earliest = tx.observations[0];
for (let i = 1; i < tx.observations.length; i++) {
if (tx.observations[i].timestamp < earliest.timestamp) earliest = tx.observations[i];
}
tx.observer_id = earliest.observer_id;
tx.observer_name = earliest.observer_name;
tx.snr = earliest.snr;
tx.rssi = earliest.rssi;
tx.path_json = earliest.path_json;
tx.direction = earliest.direction;
}
}
// Post-load: build ADVERT-by-observer index (needs all observations loaded first)
for (const tx of this.packets) {
if (tx.payload_type === 4 && tx.decoded_json) {
try {
const d = JSON.parse(tx.decoded_json);
if (d.pubKey) this._indexAdvertObservers(d.pubKey, tx);
} catch {}
}
}
console.log(`[PacketStore] ADVERT observer index: ${this._advertByObserver.size} nodes tracked`);
}
/** Fallback: load from legacy packets table */
_loadLegacy() {
const rows = this.db.prepare(
'SELECT * FROM packets_v ORDER BY timestamp DESC'
).all();
for (const row of rows) {
if (this.packets.length >= this.maxPackets) break;
this._indexLegacy(row);
}
}
/** Index a legacy packet row (old flat structure) — builds transmission + observation */
_indexLegacy(pkt) {
let tx = this.byHash.get(pkt.hash);
if (!tx) {
tx = {
id: pkt.id,
raw_hex: pkt.raw_hex,
hash: pkt.hash,
first_seen: pkt.timestamp,
timestamp: pkt.timestamp,
route_type: pkt.route_type,
payload_type: pkt.payload_type,
decoded_json: pkt.decoded_json,
observations: [],
observation_count: 0,
observer_id: pkt.observer_id,
observer_name: pkt.observer_name,
snr: pkt.snr,
rssi: pkt.rssi,
path_json: pkt.path_json,
direction: pkt.direction,
};
this.byHash.set(pkt.hash, tx);
this.byHash.set(pkt.hash, tx);
this.packets.push(tx);
this.byTxId.set(tx.id, tx);
this._indexByNode(tx);
}
if (pkt.timestamp < tx.first_seen) {
tx.first_seen = pkt.timestamp;
tx.timestamp = pkt.timestamp;
tx.observer_id = pkt.observer_id;
tx.observer_name = pkt.observer_name;
tx.path_json = pkt.path_json;
}
const obs = {
id: pkt.id,
observer_id: pkt.observer_id,
observer_name: pkt.observer_name,
direction: pkt.direction,
snr: pkt.snr,
rssi: pkt.rssi,
score: pkt.score,
path_json: pkt.path_json,
timestamp: pkt.timestamp,
hash: pkt.hash,
raw_hex: pkt.raw_hex,
payload_type: pkt.payload_type,
decoded_json: pkt.decoded_json,
route_type: pkt.route_type,
};
// Dedup: skip if same observer + same path already recorded for this transmission
const isDupe = tx.observations.some(o => o.observer_id === obs.observer_id && (o.path_json || '') === (obs.path_json || ''));
if (isDupe) return tx;
tx.observations.push(obs);
tx.observation_count++;
this.byId.set(pkt.id, obs);
if (pkt.observer_id) {
if (!this.byObserver.has(pkt.observer_id)) this.byObserver.set(pkt.observer_id, []);
this.byObserver.get(pkt.observer_id).push(pkt);
this.byObserver.get(pkt.observer_id).push(obs);
}
// Index by node pubkeys mentioned in decoded_json
this._indexByNode(pkt);
this.stats.totalObservations++;
}
/** Extract node pubkeys/names from decoded_json and index */
_indexByNode(pkt) {
if (!pkt.decoded_json) return;
/** Extract node pubkeys from decoded_json and index transmission in byNode */
_indexByNode(tx) {
if (!tx.decoded_json) return;
try {
const decoded = JSON.parse(pkt.decoded_json);
const decoded = JSON.parse(tx.decoded_json);
const keys = new Set();
if (decoded.pubKey) keys.add(decoded.pubKey);
if (decoded.destPubKey) keys.add(decoded.destPubKey);
if (decoded.srcPubKey) keys.add(decoded.srcPubKey);
for (const k of keys) {
if (!this._nodeHashIndex.has(k)) this._nodeHashIndex.set(k, new Set());
if (this._nodeHashIndex.get(k).has(tx.hash)) continue;
this._nodeHashIndex.get(k).add(tx.hash);
if (!this.byNode.has(k)) this.byNode.set(k, []);
this.byNode.get(k).push(pkt);
this.byNode.get(k).push(tx);
}
} catch {}
}
/** Track which observers saw an ADVERT from a given pubkey */
_indexAdvertObservers(pubkey, tx) {
if (!this._advertByObserver.has(pubkey)) this._advertByObserver.set(pubkey, new Set());
const s = this._advertByObserver.get(pubkey);
for (const obs of tx.observations) {
if (obs.observer_id) s.add(obs.observer_id);
}
}
/** Get node pubkeys whose ADVERTs were seen by any of the given observer IDs */
getNodesByAdvertObservers(observerIds) {
const result = new Set();
for (const [pubkey, observers] of this._advertByObserver) {
for (const obsId of observerIds) {
if (observers.has(obsId)) { result.add(pubkey); break; }
}
}
return result;
}
/** Remove oldest transmissions when over memory limit */
_evict() {
while (this.packets.length > this.maxPackets) {
const old = this.packets.pop();
this.byHash.delete(old.hash);
this.byHash.delete(old.hash);
this.byTxId.delete(old.id);
// Remove observations from byId and byObserver
for (const obs of old.observations) {
this.byId.delete(obs.id);
if (obs.observer_id && this.byObserver.has(obs.observer_id)) {
const arr = this.byObserver.get(obs.observer_id).filter(o => o.id !== obs.id);
if (arr.length) this.byObserver.set(obs.observer_id, arr); else this.byObserver.delete(obs.observer_id);
}
}
// Skip node index cleanup (expensive, low value)
this.stats.evicted++;
}
}
/** Insert a new packet (to both memory and SQLite) */
insert(packetData) {
// Write to normalized tables and get the transmission ID
const txResult = this.dbModule.insertTransmission ? this.dbModule.insertTransmission(packetData) : null;
const transmissionId = txResult ? txResult.transmissionId : null;
const observationId = txResult ? txResult.observationId : null;
// Build row directly from packetData — avoids view ID mismatch issues
const row = {
id: observationId,
raw_hex: packetData.raw_hex,
hash: packetData.hash,
timestamp: packetData.timestamp,
route_type: packetData.route_type,
payload_type: packetData.payload_type,
payload_version: packetData.payload_version,
decoded_json: packetData.decoded_json,
observer_id: packetData.observer_id,
observer_name: packetData.observer_name,
snr: packetData.snr,
rssi: packetData.rssi,
path_json: packetData.path_json,
direction: packetData.direction,
};
if (!this.sqliteOnly) {
// Update or create transmission in memory
let tx = this.byHash.get(row.hash);
if (!tx) {
tx = {
id: transmissionId || row.id,
raw_hex: row.raw_hex,
hash: row.hash,
first_seen: row.timestamp,
timestamp: row.timestamp,
route_type: row.route_type,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
observations: [],
observation_count: 0,
observer_id: row.observer_id,
observer_name: row.observer_name,
snr: row.snr,
rssi: row.rssi,
path_json: row.path_json,
direction: row.direction,
};
this.byHash.set(row.hash, tx);
this.byHash.set(row.hash, tx);
this.packets.unshift(tx); // newest first
this.byTxId.set(tx.id, tx);
this._indexByNode(tx);
} else {
// Update first_seen if earlier — also update observer + path to match
if (row.timestamp < tx.first_seen) {
tx.first_seen = row.timestamp;
tx.timestamp = row.timestamp;
tx.observer_id = row.observer_id;
tx.observer_name = row.observer_name;
tx.path_json = row.path_json;
}
}
// Add observation
const obs = {
id: row.id,
observer_id: row.observer_id,
observer_name: row.observer_name,
direction: row.direction,
snr: row.snr,
rssi: row.rssi,
score: row.score,
path_json: row.path_json,
timestamp: row.timestamp,
hash: row.hash,
raw_hex: row.raw_hex,
payload_type: row.payload_type,
decoded_json: row.decoded_json,
route_type: row.route_type,
};
// Dedup: skip if same observer + same path already recorded for this transmission
const isDupe = tx.observations.some(o => o.observer_id === obs.observer_id && (o.path_json || '') === (obs.path_json || ''));
if (!isDupe) {
tx.observations.push(obs);
tx.observation_count++;
}
// Update transmission's display fields if this is first observation
if (tx.observations.length === 1) {
tx.observer_id = obs.observer_id;
tx.observer_name = obs.observer_name;
tx.snr = obs.snr;
tx.rssi = obs.rssi;
tx.path_json = obs.path_json;
}
this.byId.set(obs.id, obs);
if (obs.observer_id) {
if (!this.byObserver.has(obs.observer_id)) this.byObserver.set(obs.observer_id, []);
this.byObserver.get(obs.observer_id).push(obs);
}
this.stats.totalObservations++;
// Update ADVERT observer index for live ingestion
if (tx.payload_type === 4 && obs.observer_id && tx.decoded_json) {
try {
const d = JSON.parse(tx.decoded_json);
if (d.pubKey) {
if (!this._advertByObserver.has(d.pubKey)) this._advertByObserver.set(d.pubKey, new Set());
this._advertByObserver.get(d.pubKey).add(obs.observer_id);
}
} catch {}
}
this._evict();
this.stats.inserts++;
}
return observationId || transmissionId;
}
/**
* Find ALL packets referencing a node — by pubkey index + name + pubkey text search.
* Single source of truth for "get packets for node X".
* Returns unique transmissions (deduped).
* @param {string} nodeIdOrName - pubkey or friendly name
* @param {Array} [fromPackets] - packet array to filter (defaults to this.packets)
* @returns {{ packets: Array, pubkey: string, nodeName: string }}
@@ -104,49 +462,24 @@ class PacketStore {
if (row) { pubkey = row.public_key; nodeName = row.name || nodeIdOrName; }
} catch {}
// Combine: index hits + text search by both name and pubkey
// Combine: index hits + text search
const indexed = this.byNode.get(pubkey);
const idSet = indexed ? new Set(indexed.map(p => p.id)) : new Set();
const hashSet = indexed ? new Set(indexed.map(t => t.hash)) : new Set();
const source = fromPackets || this.packets;
const packets = source.filter(p =>
idSet.has(p.id) ||
(p.decoded_json && (p.decoded_json.includes(nodeName) || p.decoded_json.includes(pubkey)))
const packets = source.filter(t =>
hashSet.has(t.hash) ||
(t.decoded_json && (t.decoded_json.includes(nodeName) || t.decoded_json.includes(pubkey)))
);
return { packets, pubkey, nodeName };
}
/** Remove oldest packets when over memory limit */
_evict() {
while (this.packets.length > this.maxPackets) {
const old = this.packets.pop();
this.byId.delete(old.id);
// Remove from hash index
if (old.hash && this.byHash.has(old.hash)) {
const arr = this.byHash.get(old.hash).filter(p => p.id !== old.id);
if (arr.length) this.byHash.set(old.hash, arr); else this.byHash.delete(old.hash);
}
// Remove from observer index
if (old.observer_id && this.byObserver.has(old.observer_id)) {
const arr = this.byObserver.get(old.observer_id).filter(p => p.id !== old.id);
if (arr.length) this.byObserver.set(old.observer_id, arr); else this.byObserver.delete(old.observer_id);
}
// Skip node index cleanup for eviction (expensive, low value)
this.stats.evicted++;
}
}
/** Insert a new packet (to both memory and SQLite) */
insert(packetData) {
const id = this.dbModule.insertPacket(packetData);
const row = this.dbModule.getPacket(id);
if (row) {
this.packets.unshift(row); // newest first
this._index(row);
this._evict();
this.stats.inserts++;
}
return id;
/** Count transmissions and observations for a node */
countForNode(pubkey) {
const txs = this.byNode.get(pubkey) || [];
let observations = 0;
for (const tx of txs) observations += tx.observation_count;
return { transmissions: txs.length, observations };
}
/** Query packets with filters — all from memory (or SQLite in fallback mode) */
@@ -159,9 +492,11 @@ class PacketStore {
// Use indexes for single-key filters when possible
if (hash && !type && !route && !region && !observer && !since && !until && !node) {
results = this.byHash.get(hash) || [];
const tx = this.byHash.get(hash);
results = tx ? [tx] : [];
} else if (observer && !type && !route && !region && !hash && !since && !until && !node) {
results = this.byObserver.get(observer) || [];
// For observer filter, find unique transmissions where any observation matches
results = this._transmissionsForObserver(observer);
} else if (node && !type && !route && !region && !observer && !hash && !since && !until) {
results = this.findPacketsForNode(node).packets;
} else {
@@ -174,18 +509,23 @@ class PacketStore {
const r = Number(route);
results = results.filter(p => p.route_type === r);
}
if (observer) results = results.filter(p => p.observer_id === observer);
if (hash) results = results.filter(p => p.hash === hash);
if (observer) results = this._transmissionsForObserver(observer, results);
if (hash) {
const h = hash.toLowerCase();
const tx = this.byHash.get(h);
results = tx ? results.filter(p => p.hash === h) : [];
}
if (since) results = results.filter(p => p.timestamp > since);
if (until) results = results.filter(p => p.timestamp < until);
if (region) {
// Need to look up observers for this region
const regionObservers = new Set();
try {
const obs = this.db.prepare('SELECT id FROM observers WHERE iata = ?').all(region);
obs.forEach(o => regionObservers.add(o.id));
} catch {}
results = results.filter(p => regionObservers.has(p.observer_id));
results = results.filter(p =>
p.observations.some(o => regionObservers.has(o.observer_id))
);
}
if (node) {
results = this.findPacketsForNode(node, results).packets;
@@ -209,52 +549,52 @@ class PacketStore {
return { packets: paginated, total };
}
/** Query with groupByHash — aggregate packets by content hash */
/** Find unique transmissions that have at least one observation from given observer */
_transmissionsForObserver(observerId, fromTransmissions) {
if (fromTransmissions) {
return fromTransmissions.filter(tx =>
tx.observations.some(o => o.observer_id === observerId)
);
}
// Use byObserver index: get observations, then unique transmissions
const obs = this.byObserver.get(observerId) || [];
const seen = new Set();
const result = [];
for (const o of obs) {
if (!seen.has(o.hash)) {
seen.add(o.hash);
const tx = this.byHash.get(o.hash);
if (tx) result.push(tx);
}
}
return result;
}
/** Query with groupByHash — now trivial since packets ARE transmissions */
queryGrouped({ limit = 50, offset = 0, type, route, region, observer, hash, since, until, node } = {}) {
this.stats.queries++;
if (this.sqliteOnly) return this._queryGroupedSQLite({ limit, offset, type, route, region, observer, hash, since, until, node });
// Get filtered results first
// Get filtered transmissions
const { packets: filtered, total: filteredTotal } = this.query({
limit: 999999, offset: 0, type, route, region, observer, hash, since, until, node
});
// Group by hash
const groups = new Map();
for (const p of filtered) {
const h = p.hash || p.id.toString();
if (!groups.has(h)) {
groups.set(h, {
hash: p.hash,
observer_count: new Set(),
count: 0,
latest: p.timestamp,
observer_id: p.observer_id,
observer_name: p.observer_name,
path_json: p.path_json,
payload_type: p.payload_type,
raw_hex: p.raw_hex,
decoded_json: p.decoded_json,
});
}
const g = groups.get(h);
g.count++;
if (p.observer_id) g.observer_count.add(p.observer_id);
if (p.timestamp > g.latest) {
g.latest = p.timestamp;
}
// Keep longest path
if (p.path_json && (!g.path_json || p.path_json.length > g.path_json.length)) {
g.path_json = p.path_json;
g.raw_hex = p.raw_hex;
}
}
// Sort by latest DESC, paginate
const sorted = [...groups.values()]
.map(g => ({ ...g, observer_count: g.observer_count.size }))
.sort((a, b) => b.latest.localeCompare(a.latest));
// Already grouped by hash — just format for backward compat
const sorted = filtered.map(tx => ({
hash: tx.hash,
count: tx.observation_count,
observer_count: new Set(tx.observations.map(o => o.observer_id).filter(Boolean)).size,
latest: tx.observations.length ? tx.observations.reduce((max, o) => o.timestamp > max ? o.timestamp : max, tx.observations[0].timestamp) : tx.timestamp,
observer_id: tx.observer_id,
observer_name: tx.observer_name,
path_json: tx.path_json,
payload_type: tx.payload_type,
raw_hex: tx.raw_hex,
decoded_json: tx.decoded_json,
observation_count: tx.observation_count,
})).sort((a, b) => b.latest.localeCompare(a.latest));
const total = sorted.length;
const paginated = sorted.slice(Number(offset), Number(offset) + Number(limit));
@@ -264,7 +604,7 @@ class PacketStore {
/** Get timestamps for sparkline */
getTimestamps(since) {
if (this.sqliteOnly) {
return this.db.prepare('SELECT timestamp FROM packets WHERE timestamp > ? ORDER BY timestamp ASC').all(since).map(r => r.timestamp);
return this.db.prepare('SELECT timestamp FROM packets_v WHERE timestamp > ? ORDER BY timestamp ASC').all(since).map(r => r.timestamp);
}
const results = [];
for (const p of this.packets) {
@@ -274,27 +614,35 @@ class PacketStore {
return results.reverse();
}
/** Get a single packet by ID */
/** Get a single packet by ID — checks observation IDs first (backward compat) */
getById(id) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets WHERE id = ?').get(id) || null;
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets_v WHERE id = ?').get(id) || null;
return this.byId.get(id) || null;
}
/** Get all siblings of a packet (same hash) */
getSiblings(hash) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets WHERE hash = ? ORDER BY timestamp DESC').all(hash);
return this.byHash.get(hash) || [];
/** Get a transmission by its transmission table ID */
getByTxId(id) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM transmissions WHERE id = ?').get(id) || null;
return this.byTxId.get(id) || null;
}
/** Get all packets (raw array reference — do not mutate) */
/** Get all siblings of a packet (same hash) — returns observations array */
getSiblings(hash) {
const h = hash.toLowerCase();
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets_v WHERE hash = ? ORDER BY timestamp DESC').all(h);
const tx = this.byHash.get(h);
return tx ? tx.observations : [];
}
/** Get all transmissions (backward compat — returns packets array) */
all() {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets ORDER BY timestamp DESC').all();
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets_v ORDER BY timestamp DESC').all();
return this.packets;
}
/** Get all packets matching a filter function */
/** Get all transmissions matching a filter function */
filter(fn) {
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets ORDER BY timestamp DESC').all().filter(fn);
if (this.sqliteOnly) return this.db.prepare('SELECT * FROM packets_v ORDER BY timestamp DESC').all().filter(fn);
return this.packets.filter(fn);
}
@@ -311,6 +659,7 @@ class PacketStore {
byHash: this.byHash.size,
byObserver: this.byObserver.size,
byNode: this.byNode.size,
advertByObserver: this._advertByObserver.size,
}
};
}
@@ -321,14 +670,14 @@ class PacketStore {
if (type !== undefined) { where.push('payload_type = ?'); params.push(Number(type)); }
if (route !== undefined) { where.push('route_type = ?'); params.push(Number(route)); }
if (observer) { where.push('observer_id = ?'); params.push(observer); }
if (hash) { where.push('hash = ?'); params.push(hash); }
if (hash) { where.push('hash = ?'); params.push(hash.toLowerCase()); }
if (since) { where.push('timestamp > ?'); params.push(since); }
if (until) { where.push('timestamp < ?'); params.push(until); }
if (region) { where.push('observer_id IN (SELECT id FROM observers WHERE iata = ?)'); params.push(region); }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('(decoded_json LIKE ? OR id IN (SELECT packet_id FROM paths WHERE node_hash = ?))'); params.push('%' + pk + '%', pk.substring(0, 8)); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('decoded_json LIKE ?'); params.push('%' + pk + '%'); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
const w = where.length ? 'WHERE ' + where.join(' AND ') : '';
const total = this.db.prepare(`SELECT COUNT(*) as c FROM packets ${w}`).get(...params).c;
const packets = this.db.prepare(`SELECT * FROM packets ${w} ORDER BY timestamp ${order === 'ASC' ? 'ASC' : 'DESC'} LIMIT ? OFFSET ?`).all(...params, limit, offset);
const total = this.db.prepare(`SELECT COUNT(*) as c FROM packets_v ${w}`).get(...params).c;
const packets = this.db.prepare(`SELECT * FROM packets_v ${w} ORDER BY timestamp ${order === 'ASC' ? 'ASC' : 'DESC'} LIMIT ? OFFSET ?`).all(...params, limit, offset);
return { packets, total };
}
@@ -338,21 +687,21 @@ class PacketStore {
if (type !== undefined) { where.push('payload_type = ?'); params.push(Number(type)); }
if (route !== undefined) { where.push('route_type = ?'); params.push(Number(route)); }
if (observer) { where.push('observer_id = ?'); params.push(observer); }
if (hash) { where.push('hash = ?'); params.push(hash); }
if (hash) { where.push('hash = ?'); params.push(hash.toLowerCase()); }
if (since) { where.push('timestamp > ?'); params.push(since); }
if (until) { where.push('timestamp < ?'); params.push(until); }
if (region) { where.push('observer_id IN (SELECT id FROM observers WHERE iata = ?)'); params.push(region); }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('(decoded_json LIKE ? OR id IN (SELECT packet_id FROM paths WHERE node_hash = ?))'); params.push('%' + pk + '%', pk.substring(0, 8)); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
if (node) { try { const nr = this.db.prepare('SELECT public_key FROM nodes WHERE public_key = ? OR name = ? LIMIT 1').get(node, node); const pk = nr ? nr.public_key : node; where.push('decoded_json LIKE ?'); params.push('%' + pk + '%'); } catch(e) { where.push('decoded_json LIKE ?'); params.push('%' + node + '%'); } }
const w = where.length ? 'WHERE ' + where.join(' AND ') : '';
const sql = `SELECT hash, COUNT(*) as count, COUNT(DISTINCT observer_id) as observer_count,
MAX(timestamp) as latest, MIN(observer_id) as observer_id, MIN(observer_name) as observer_name,
MIN(path_json) as path_json, MIN(payload_type) as payload_type, MIN(raw_hex) as raw_hex,
MIN(decoded_json) as decoded_json
FROM packets ${w} GROUP BY hash ORDER BY latest DESC LIMIT ? OFFSET ?`;
FROM packets_v ${w} GROUP BY hash ORDER BY latest DESC LIMIT ? OFFSET ?`;
const packets = this.db.prepare(sql).all(...params, limit, offset);
const countSql = `SELECT COUNT(DISTINCT hash) as c FROM packets ${w}`;
const countSql = `SELECT COUNT(DISTINCT hash) as c FROM packets_v ${w}`;
const total = this.db.prepare(countSql).get(...params).c;
return { packets, total };
}

View File

@@ -3,6 +3,7 @@
(function () {
let _analyticsData = {};
const sf = (v, d) => (v != null ? v.toFixed(d) : ''); // safe toFixed
function esc(s) { return s ? String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;') : ''; }
// --- SVG helpers ---
@@ -65,6 +66,7 @@
<div class="analytics-header">
<h2>📊 Mesh Analytics</h2>
<p class="text-muted">Deep dive into your mesh network data</p>
<div id="analyticsRegionFilter" class="region-filter-container"></div>
<div class="analytics-tabs" id="analyticsTabs">
<button class="tab-btn active" data-tab="overview">Overview</button>
<button class="tab-btn" data-tab="rf">RF / Signal</button>
@@ -74,6 +76,7 @@
<button class="tab-btn" data-tab="collisions">Hash Collisions</button>
<button class="tab-btn" data-tab="subpaths">Route Patterns</button>
<button class="tab-btn" data-tab="nodes">Nodes</button>
<button class="tab-btn" data-tab="distance">Distance</button>
</div>
</div>
<div id="analyticsContent" class="analytics-content">
@@ -89,9 +92,13 @@
if (!btn) return;
document.querySelectorAll('.tab-btn').forEach(b => b.classList.remove('active'));
btn.classList.add('active');
renderTab(btn.dataset.tab);
_currentTab = btn.dataset.tab;
renderTab(_currentTab);
});
RegionFilter.init(document.getElementById('analyticsRegionFilter'));
RegionFilter.onChange(function () { loadAnalytics(); });
// Delegated click/keyboard handler for clickable table rows
const analyticsContent = document.getElementById('analyticsContent');
if (analyticsContent) {
@@ -106,16 +113,24 @@
analyticsContent.addEventListener('keydown', handler);
}
loadAnalytics();
}
let _currentTab = 'overview';
async function loadAnalytics() {
try {
_analyticsData = {};
const rqs = RegionFilter.regionQueryString();
const sep = rqs ? '?' + rqs.slice(1) : '';
const [hashData, rfData, topoData, chanData] = await Promise.all([
api('/analytics/hash-sizes', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/rf', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/topology', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/channels', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/hash-sizes' + sep, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/rf' + sep, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/topology' + sep, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/channels' + sep, { ttl: CLIENT_TTL.analyticsRF }),
]);
_analyticsData = { hashData, rfData, topoData, chanData };
renderTab('overview');
renderTab(_currentTab);
} catch (e) {
document.getElementById('analyticsContent').innerHTML =
`<div class="text-muted" role="alert" aria-live="polite" style="padding:40px">Failed to load: ${e.message}</div>`;
@@ -134,6 +149,7 @@
case 'collisions': await renderCollisionTab(el, d.hashData); break;
case 'subpaths': await renderSubpaths(el); break;
case 'nodes': await renderNodesTab(el); break;
case 'distance': await renderDistanceTab(el); break;
}
// Auto-apply column resizing to all analytics tables
requestAnimationFrame(() => {
@@ -150,30 +166,30 @@
el.innerHTML = `
<div class="stats-grid">
<div class="stat-card">
<div class="stat-value">${(rf.totalAllPackets || rf.totalPackets).toLocaleString()}</div>
<div class="stat-label">Total Packets</div>
<div class="stat-value">${(rf.totalTransmissions || rf.totalAllPackets || rf.totalPackets).toLocaleString()}</div>
<div class="stat-label">Total Transmissions</div>
<div class="stat-spark">${sparkSvg(rf.packetsPerHour.map(h=>h.count), 'var(--accent)')}</div>
</div>
<div class="stat-card">
<div class="stat-value">${rf.totalPackets.toLocaleString()}</div>
<div class="stat-label">With Signal Data</div>
<div class="stat-label">Observations with Signal</div>
</div>
<div class="stat-card">
<div class="stat-value">${topo.uniqueNodes}</div>
<div class="stat-label">Unique Nodes</div>
</div>
<div class="stat-card">
<div class="stat-value">${rf.snr.avg.toFixed(1)} dB</div>
<div class="stat-value">${sf(rf.snr.avg, 1)} dB</div>
<div class="stat-label">Avg SNR</div>
<div class="stat-detail">${rf.snr.min.toFixed(1)} to ${rf.snr.max.toFixed(1)}</div>
<div class="stat-detail">${sf(rf.snr.min, 1)} to ${sf(rf.snr.max, 1)}</div>
</div>
<div class="stat-card">
<div class="stat-value">${rf.rssi.avg.toFixed(0)} dBm</div>
<div class="stat-value">${sf(rf.rssi.avg, 0)} dBm</div>
<div class="stat-label">Avg RSSI</div>
<div class="stat-detail">${rf.rssi.min} to ${rf.rssi.max}</div>
</div>
<div class="stat-card">
<div class="stat-value">${topo.avgHops.toFixed(1)}</div>
<div class="stat-value">${sf(topo.avgHops, 1)}</div>
<div class="stat-label">Avg Hops</div>
<div class="stat-detail">max ${topo.maxHops}</div>
</div>
@@ -241,11 +257,11 @@
<p class="text-muted">Signal-to-Noise Ratio (higher = cleaner signal)</p>
${snrHist.svg}
<div class="rf-stats">
<span>Min: <strong>${rf.snr.min.toFixed(1)} dB</strong></span>
<span>Mean: <strong>${rf.snr.avg.toFixed(1)} dB</strong></span>
<span>Median: <strong>${rf.snr.median.toFixed(1)} dB</strong></span>
<span>Max: <strong>${rf.snr.max.toFixed(1)} dB</strong></span>
<span>σ: <strong>${rf.snr.stddev.toFixed(1)} dB</strong></span>
<span>Min: <strong>${sf(rf.snr.min, 1)} dB</strong></span>
<span>Mean: <strong>${sf(rf.snr.avg, 1)} dB</strong></span>
<span>Median: <strong>${sf(rf.snr.median, 1)} dB</strong></span>
<span>Max: <strong>${sf(rf.snr.max, 1)} dB</strong></span>
<span>σ: <strong>${sf(rf.snr.stddev, 1)} dB</strong></span>
</div>
</div>
<div class="analytics-card flex-1">
@@ -254,10 +270,10 @@
${rssiHist.svg}
<div class="rf-stats">
<span>Min: <strong>${rf.rssi.min} dBm</strong></span>
<span>Mean: <strong>${rf.rssi.avg.toFixed(0)} dBm</strong></span>
<span>Mean: <strong>${sf(rf.rssi.avg, 0)} dBm</strong></span>
<span>Median: <strong>${rf.rssi.median} dBm</strong></span>
<span>Max: <strong>${rf.rssi.max} dBm</strong></span>
<span>σ: <strong>${rf.rssi.stddev.toFixed(1)} dBm</strong></span>
<span>σ: <strong>${sf(rf.rssi.stddev, 1)} dBm</strong></span>
</div>
</div>
</div>
@@ -357,9 +373,9 @@
html += `<tr>
<td><strong>${t.name}</strong></td>
<td>${t.count}</td>
<td><strong>${t.avg.toFixed(1)} dB</strong></td>
<td>${t.min.toFixed(1)}</td>
<td>${t.max.toFixed(1)}</td>
<td><strong>${sf(t.avg, 1)} dB</strong></td>
<td>${sf(t.min, 1)}</td>
<td>${sf(t.max, 1)}</td>
<td><div class="hash-bar-track" style="height:14px"><div class="hash-bar-fill" style="width:${barPct}%;background:${color};height:100%"></div></div></td>
</tr>`;
});
@@ -408,7 +424,7 @@
<p class="text-muted">Number of repeater hops per packet</p>
${barChart(topo.hopDistribution.map(h=>h.count), topo.hopDistribution.map(h=>h.hops), ['#3b82f6'])}
<div class="rf-stats">
<span>Avg: <strong>${topo.avgHops.toFixed(1)} hops</strong></span>
<span>Avg: <strong>${sf(topo.avgHops, 1)} hops</strong></span>
<span>Median: <strong>${topo.medianHops}</strong></span>
<span>Max: <strong>${topo.maxHops}</strong></span>
<span>1-hop direct: <strong>${topo.hopDistribution[0]?.count || 0}</strong></span>
@@ -604,7 +620,7 @@
<tbody>
${ch.channels.map(c => `<tr class="clickable-row" data-action="navigate" data-value="#/channels?ch=${c.hash}" tabindex="0" role="row">
<td><strong>${esc(c.name || 'Unknown')}</strong></td>
<td class="mono">${c.hash}</td>
<td class="mono">${typeof c.hash === 'number' ? '0x' + c.hash.toString(16).toUpperCase().padStart(2, '0') : c.hash}</td>
<td>${c.messages}</td>
<td>${c.senders}</td>
<td>${timeAgo(c.lastActivity)}</td>
@@ -759,7 +775,7 @@
</div>
`;
let allNodes = [];
try { const nd = await api('/nodes?limit=2000', { ttl: CLIENT_TTL.nodeList }); allNodes = nd.nodes || []; } catch {}
try { const nd = await api('/nodes?limit=2000' + RegionFilter.regionQueryString(), { ttl: CLIENT_TTL.nodeList }); allNodes = nd.nodes || []; } catch {}
renderHashMatrix(data.topHops, allNodes);
renderCollisions(data.topHops, allNodes);
}
@@ -949,11 +965,12 @@
async function renderSubpaths(el) {
el.innerHTML = '<div class="text-center text-muted" style="padding:40px">Analyzing route patterns…</div>';
try {
const rq = RegionFilter.regionQueryString();
const [d2, d3, d4, d5] = await Promise.all([
api('/analytics/subpaths?minLen=2&maxLen=2&limit=50', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=3&maxLen=3&limit=30', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=4&maxLen=4&limit=20', { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=5&maxLen=8&limit=15', { ttl: CLIENT_TTL.analyticsRF })
api('/analytics/subpaths?minLen=2&maxLen=2&limit=50' + rq, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=3&maxLen=3&limit=30' + rq, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=4&maxLen=4&limit=20' + rq, { ttl: CLIENT_TTL.analyticsRF }),
api('/analytics/subpaths?minLen=5&maxLen=8&limit=15' + rq, { ttl: CLIENT_TTL.analyticsRF })
]);
function renderTable(data, title) {
@@ -1129,7 +1146,7 @@
// Render minimap
if (hasMap && typeof L !== 'undefined') {
const map = L.map('subpathMap', { zoomControl: false, attributionControl: false });
L.tileLayer('https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png', { maxZoom: 18 }).addTo(map);
L.tileLayer(getTileUrl(), { maxZoom: 18 }).addTo(map);
const latlngs = [];
nodesWithLoc.forEach((n, i) => {
@@ -1152,10 +1169,11 @@
async function renderNodesTab(el) {
el.innerHTML = '<div style="padding:40px;text-align:center;color:var(--text-muted)">Loading node analytics…</div>';
try {
const rq = RegionFilter.regionQueryString();
const [nodesResp, bulkHealth, netStatus] = await Promise.all([
api('/nodes?limit=200&sortBy=lastSeen', { ttl: CLIENT_TTL.nodeList }),
api('/nodes/bulk-health?limit=50', { ttl: CLIENT_TTL.analyticsRF }),
api('/nodes/network-status', { ttl: CLIENT_TTL.analyticsRF })
api('/nodes?limit=200&sortBy=lastSeen' + rq, { ttl: CLIENT_TTL.nodeList }),
api('/nodes/bulk-health?limit=50' + rq, { ttl: CLIENT_TTL.analyticsRF }),
api('/nodes/network-status' + (rq ? '?' + rq.slice(1) : ''), { ttl: CLIENT_TTL.analyticsRF })
]);
const nodes = nodesResp.nodes || nodesResp;
const myNodes = JSON.parse(localStorage.getItem('meshcore-my-nodes') || '[]');
@@ -1167,7 +1185,7 @@
const enriched = nodes.filter(n => healthMap[n.public_key]).map(n => ({ ...n, health: { stats: healthMap[n.public_key].stats, observers: healthMap[n.public_key].observers } }));
// Compute rankings
const byPackets = [...enriched].sort((a, b) => (b.health.stats.totalPackets || 0) - (a.health.stats.totalPackets || 0));
const byPackets = [...enriched].sort((a, b) => (b.health.stats.totalTransmissions || b.health.stats.totalPackets || 0) - (a.health.stats.totalTransmissions || a.health.stats.totalPackets || 0));
const bySnr = [...enriched].filter(n => n.health.stats.avgSnr != null).sort((a, b) => b.health.stats.avgSnr - a.health.stats.avgSnr);
const byObservers = [...enriched].sort((a, b) => (b.health.observers?.length || 0) - (a.health.observers?.length || 0));
const byRecent = [...enriched].filter(n => n.health.stats.lastHeard).sort((a, b) => new Date(b.health.stats.lastHeard) - new Date(a.health.stats.lastHeard));
@@ -1182,7 +1200,7 @@
return myKeys.has(n.public_key) ? ' <span style="color:var(--accent);font-size:10px">★ MINE</span>' : '';
}
const ROLE_COLORS = { repeater: '#dc2626', companion: '#2563eb', room: '#16a34a', sensor: '#d97706' };
// ROLE_COLORS from shared roles.js
el.innerHTML = `
<div class="analytics-section">
@@ -1223,7 +1241,7 @@
return `<tr>
<td>${nodeLink(n)}</td>
<td><span class="badge" style="background:${(ROLE_COLORS[n.role]||'#6b7280')}20;color:${ROLE_COLORS[n.role]||'#6b7280'}">${n.role}</span></td>
<td>${s.totalPackets || 0}</td>
<td>${s.totalTransmissions || s.totalPackets || 0}</td>
<td>${s.avgSnr != null ? s.avgSnr.toFixed(1) + ' dB' : '—'}</td>
<td>${n.health.observers?.length || 0}</td>
<td>${s.lastHeard ? timeAgo(s.lastHeard) : '—'}</td>
@@ -1240,7 +1258,7 @@
<td>${i + 1}</td>
<td>${nodeLink(n)}${claimedBadge(n)}</td>
<td><span class="badge" style="background:${(ROLE_COLORS[n.role]||'#6b7280')}20;color:${ROLE_COLORS[n.role]||'#6b7280'}">${n.role}</span></td>
<td>${n.health.stats.totalPackets || 0}</td>
<td>${n.health.stats.totalTransmissions || n.health.stats.totalPackets || 0}</td>
<td>${n.health.stats.packetsToday || 0}</td>
<td><a href="#/nodes/${encodeURIComponent(n.public_key)}/analytics" class="analytics-link">📊</a></td>
</tr>`).join('')}
@@ -1296,6 +1314,92 @@
}
}
async function renderDistanceTab(el) {
try {
const rqs = RegionFilter.regionQueryString();
const sep = rqs ? '?' + rqs.slice(1) : '';
const data = await api('/analytics/distance' + sep, { ttl: CLIENT_TTL.analyticsRF });
const s = data.summary;
let html = `<div class="analytics-grid">
<div class="stat-card"><div class="stat-value">${s.totalHops.toLocaleString()}</div><div class="stat-label">Total Hops Analyzed</div></div>
<div class="stat-card"><div class="stat-value">${s.totalPaths.toLocaleString()}</div><div class="stat-label">Paths Analyzed</div></div>
<div class="stat-card"><div class="stat-value">${s.avgDist} km</div><div class="stat-label">Avg Hop Distance</div></div>
<div class="stat-card"><div class="stat-value">${s.maxDist} km</div><div class="stat-label">Max Hop Distance</div></div>
</div>`;
// Category stats
const cats = data.catStats;
html += `<div class="analytics-section"><h3>Distance by Link Type</h3><table class="data-table"><thead><tr><th>Type</th><th>Count</th><th>Avg (km)</th><th>Median (km)</th><th>Min (km)</th><th>Max (km)</th></tr></thead><tbody>`;
for (const [cat, st] of Object.entries(cats)) {
if (!st.count) continue;
html += `<tr><td><strong>${esc(cat)}</strong></td><td>${st.count.toLocaleString()}</td><td>${st.avg}</td><td>${st.median}</td><td>${st.min}</td><td>${st.max}</td></tr>`;
}
html += `</tbody></table></div>`;
// Histogram
if (data.distHistogram && data.distHistogram.bins) {
const buckets = data.distHistogram.bins.map(b => b.count);
const labels = data.distHistogram.bins.map(b => b.x.toFixed(1));
html += `<div class="analytics-section"><h3>Hop Distance Distribution</h3>${barChart(buckets, labels, '#22c55e')}</div>`;
}
// Distance over time
if (data.distOverTime && data.distOverTime.length > 1) {
html += `<div class="analytics-section"><h3>Average Distance Over Time</h3>${sparkSvg(data.distOverTime.map(d => d.avg), 'var(--accent)', 800, 120)}</div>`;
}
// Top hops leaderboard
html += `<div class="analytics-section"><h3>🏆 Top 20 Longest Hops</h3><table class="data-table"><thead><tr><th>#</th><th>From</th><th>To</th><th>Distance (km)</th><th>Type</th><th>SNR</th><th>Packet</th><th></th></tr></thead><tbody>`;
const top20 = data.topHops.slice(0, 20);
top20.forEach((h, i) => {
const fromLink = h.fromPk ? `<a href="#/nodes/${encodeURIComponent(h.fromPk)}" class="analytics-link">${esc(h.fromName)}</a>` : esc(h.fromName || '?');
const toLink = h.toPk ? `<a href="#/nodes/${encodeURIComponent(h.toPk)}" class="analytics-link">${esc(h.toName)}</a>` : esc(h.toName || '?');
const snr = h.snr != null ? h.snr + ' dB' : '<span class="text-muted">—</span>';
const pktLink = h.hash ? `<a href="#/packet/${encodeURIComponent(h.hash)}" class="analytics-link mono" style="font-size:0.85em">${esc(h.hash.slice(0, 12))}…</a>` : '—';
const mapBtn = h.fromPk && h.toPk ? `<button class="btn-icon dist-map-hop" data-from="${esc(h.fromPk)}" data-to="${esc(h.toPk)}" title="View on map">🗺️</button>` : '';
html += `<tr><td>${i+1}</td><td>${fromLink}</td><td>${toLink}</td><td><strong>${h.dist}</strong></td><td>${esc(h.type)}</td><td>${snr}</td><td>${pktLink}</td><td>${mapBtn}</td></tr>`;
});
html += `</tbody></table></div>`;
// Top paths
if (data.topPaths.length) {
html += `<div class="analytics-section"><h3>🛤️ Top 10 Longest Multi-Hop Paths</h3><table class="data-table"><thead><tr><th>#</th><th>Total Distance (km)</th><th>Hops</th><th>Route</th><th>Packet</th><th></th></tr></thead><tbody>`;
data.topPaths.slice(0, 10).forEach((p, i) => {
const route = p.hops.map(h => esc(h.fromName)).concat(esc(p.hops[p.hops.length-1].toName)).join(' → ');
const pktLink = p.hash ? `<a href="#/packet/${encodeURIComponent(p.hash)}" class="analytics-link mono" style="font-size:0.85em">${esc(p.hash.slice(0, 12))}…</a>` : '—';
// Collect all unique pubkeys in path order
const pathPks = [];
p.hops.forEach(h => { if (h.fromPk && !pathPks.includes(h.fromPk)) pathPks.push(h.fromPk); });
if (p.hops.length && p.hops[p.hops.length-1].toPk) { const last = p.hops[p.hops.length-1].toPk; if (!pathPks.includes(last)) pathPks.push(last); }
const mapBtn = pathPks.length >= 2 ? `<button class="btn-icon dist-map-path" data-hops='${JSON.stringify(pathPks)}' title="View on map">🗺️</button>` : '';
html += `<tr><td>${i+1}</td><td><strong>${p.totalDist}</strong></td><td>${p.hopCount}</td><td style="font-size:0.9em">${route}</td><td>${pktLink}</td><td>${mapBtn}</td></tr>`;
});
html += `</tbody></table></div>`;
}
el.innerHTML = html;
// Wire up map buttons
el.querySelectorAll('.dist-map-hop').forEach(btn => {
btn.addEventListener('click', () => {
sessionStorage.setItem('map-route-hops', JSON.stringify({ hops: [btn.dataset.from, btn.dataset.to] }));
window.location.hash = '#/map?route=1';
});
});
el.querySelectorAll('.dist-map-path').forEach(btn => {
btn.addEventListener('click', () => {
try {
const hops = JSON.parse(btn.dataset.hops);
sessionStorage.setItem('map-route-hops', JSON.stringify({ hops }));
window.location.hash = '#/map?route=1';
} catch {}
});
});
} catch (e) {
el.innerHTML = `<div style="padding:40px;text-align:center;color:#ff6b6b">Failed to load distance analytics: ${esc(e.message)}</div>`;
}
}
function destroy() { _analyticsData = {}; }
registerPage('analytics', { init, destroy });

View File

@@ -215,7 +215,6 @@ function connectWS() {
api._invalidateTimer = null;
invalidateApiCache('/stats');
invalidateApiCache('/nodes');
invalidateApiCache('/channels');
}, 5000);
}
wsListeners.forEach(fn => fn(msg));
@@ -228,8 +227,8 @@ function offWS(fn) { wsListeners = wsListeners.filter(f => f !== fn); }
/* Global escapeHtml — used by multiple pages */
function escapeHtml(s) {
if (!s) return '';
return s.replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
if (s == null) return '';
return String(s).replace(/&/g,'&amp;').replace(/</g,'&lt;').replace(/>/g,'&gt;').replace(/"/g,'&quot;');
}
/* Global debounce */
@@ -383,7 +382,7 @@ window.addEventListener('DOMContentLoaded', () => {
try {
const h = await api('/nodes/' + pk + '/health', { ttl: CLIENT_TTL.nodeHealth });
const age = h.stats.lastHeard ? Date.now() - new Date(h.stats.lastHeard).getTime() : null;
const status = age === null ? '🔴' : age < 3600000 ? '🟢' : age < 86400000 ? '🟡' : '🔴';
const status = age === null ? '🔴' : age < HEALTH_THRESHOLDS.nodeDegradedMs ? '🟢' : age < HEALTH_THRESHOLDS.nodeSilentMs ? '🟡' : '🔴';
return '<a href="#/nodes/' + pk + '" class="fav-dd-item" data-key="' + pk + '">'
+ '<span class="fav-dd-status">' + status + '</span>'
+ '<span class="fav-dd-name">' + (h.node.name || truncate(pk, 12)) + '</span>'
@@ -455,7 +454,7 @@ window.addEventListener('DOMContentLoaded', () => {
const pktList = packets.packets || packets;
if (Array.isArray(pktList)) {
for (const p of pktList.slice(0, 5)) {
html += `<div class="search-result-item" onclick="location.hash='#/packets?id=${p.id}';document.getElementById('searchOverlay').classList.add('hidden')">
html += `<div class="search-result-item" onclick="location.hash='#/packets/${p.packet_hash || p.hash || p.id}';document.getElementById('searchOverlay').classList.add('hidden')">
<span class="search-result-type">Packet</span>${truncate(p.packet_hash || '', 16)}${payloadTypeName(p.payload_type)}</div>`;
}
}

View File

@@ -40,7 +40,8 @@
tip.id = 'chNodeTooltip';
tip.className = 'ch-node-tooltip';
tip.setAttribute('role', 'tooltip');
const role = node.is_repeater ? '📡 Repeater' : node.is_room ? '🏠 Room' : node.is_sensor ? '🌡 Sensor' : '📻 Companion';
const roleKey = node.role || (node.is_repeater ? 'repeater' : node.is_room ? 'room' : node.is_sensor ? 'sensor' : 'companion');
const role = (ROLE_EMOJI[roleKey] || '●') + ' ' + (ROLE_LABELS[roleKey] || roleKey);
const lastSeen = node.last_seen ? timeAgo(node.last_seen) : 'unknown';
tip.innerHTML = `<div class="ch-tooltip-name">${escapeHtml(node.name)}</div>
<div class="ch-tooltip-role">${role}</div>
@@ -113,7 +114,8 @@
const detail = await api('/nodes/' + encodeURIComponent(node.public_key), { ttl: CLIENT_TTL.nodeDetail });
const n = detail.node;
const adverts = detail.recentAdverts || [];
const role = n.is_repeater ? '📡 Repeater' : n.is_room ? '🏠 Room' : n.is_sensor ? '🌡 Sensor' : '📻 Companion';
const roleKey = n.role || (n.is_repeater ? 'repeater' : n.is_room ? 'room' : n.is_sensor ? 'sensor' : 'companion');
const role = (ROLE_EMOJI[roleKey] || '●') + ' ' + (ROLE_LABELS[roleKey] || roleKey);
const lastSeen = n.last_seen ? timeAgo(n.last_seen) : 'unknown';
panel.innerHTML = `<div class="ch-node-panel-header">
@@ -203,6 +205,14 @@
return str.length > len ? str.slice(0, len) + '…' : str;
}
function formatSecondsAgo(sec) {
if (sec < 0) sec = 0;
if (sec < 60) return sec + 's ago';
if (sec < 3600) return Math.floor(sec / 60) + 'm ago';
if (sec < 86400) return Math.floor(sec / 3600) + 'h ago';
return Math.floor(sec / 86400) + 'd ago';
}
function highlightMentions(text) {
if (!text) return '';
return escapeHtml(text).replace(/@\[([^\]]+)\]/g, function(_, name) {
@@ -211,12 +221,15 @@
});
}
let regionChangeHandler = null;
function init(app, routeParam) {
app.innerHTML = `<div class="ch-layout">
<div class="ch-sidebar" aria-label="Channel list">
<div class="ch-sidebar-header">
<div class="ch-sidebar-title"><span class="ch-icon">💬</span> Channels</div>
</div>
<div id="chRegionFilter" class="region-filter-container" style="padding:0 8px"></div>
<div class="ch-channel-list" id="chList" role="listbox" aria-label="Channels">
<div class="ch-loading">Loading channels…</div>
</div>
@@ -235,6 +248,9 @@
</div>
</div>`;
RegionFilter.init(document.getElementById('chRegionFilter'));
regionChangeHandler = RegionFilter.onChange(function () { loadChannels(); });
loadChannels().then(() => {
if (routeParam) selectChannel(routeParam);
});
@@ -365,21 +381,140 @@
});
wsHandler = debouncedOnWS(function (msgs) {
var dominated = msgs.some(function (m) {
var dominated = msgs.filter(function (m) {
return m.type === 'message' || (m.type === 'packet' && m.data?.decoded?.header?.payloadTypeName === 'GRP_TXT');
});
if (dominated) {
loadChannels(true);
if (selectedHash) {
refreshMessages();
if (!dominated.length) return;
var channelListDirty = false;
var messagesDirty = false;
var seenHashes = new Set();
for (var i = 0; i < dominated.length; i++) {
var m = dominated[i];
var payload = m.data?.decoded?.payload;
if (!payload) continue;
var channelName = payload.channel || 'unknown';
var rawText = payload.text || '';
var sender = payload.sender || null;
var displayText = rawText;
// Parse "sender: message" format
if (rawText && !sender) {
var colonIdx = rawText.indexOf(': ');
if (colonIdx > 0 && colonIdx < 50) {
sender = rawText.slice(0, colonIdx);
displayText = rawText.slice(colonIdx + 2);
}
} else if (rawText && sender) {
var colonIdx2 = rawText.indexOf(': ');
if (colonIdx2 > 0 && colonIdx2 < 50) {
displayText = rawText.slice(colonIdx2 + 2);
}
}
if (!sender) sender = 'Unknown';
var ts = new Date().toISOString();
var pktHash = m.data?.hash || m.data?.packet?.hash || null;
var pktId = m.data?.id || null;
var snr = m.data?.snr ?? m.data?.packet?.snr ?? payload.SNR ?? null;
var observer = m.data?.packet?.observer_name || m.data?.observer || null;
// Update channel list entry — only once per unique packet hash
var isFirstObservation = pktHash && !seenHashes.has(pktHash + ':' + channelName);
if (pktHash) seenHashes.add(pktHash + ':' + channelName);
var ch = channels.find(function (c) { return c.hash === channelName; });
if (ch) {
if (isFirstObservation) ch.messageCount = (ch.messageCount || 0) + 1;
ch.lastActivityMs = Date.now();
ch.lastSender = sender;
ch.lastMessage = truncate(displayText, 100);
channelListDirty = true;
} else if (isFirstObservation) {
// New channel we haven't seen
channels.push({
hash: channelName,
name: channelName,
messageCount: 1,
lastActivityMs: Date.now(),
lastSender: sender,
lastMessage: truncate(displayText, 100),
});
channelListDirty = true;
}
// If this message is for the selected channel, append to messages
if (selectedHash && channelName === selectedHash) {
// Deduplicate by packet hash — same message seen by multiple observers
var existing = pktHash ? messages.find(function (msg) { return msg.packetHash === pktHash; }) : null;
if (existing) {
existing.repeats = (existing.repeats || 1) + 1;
if (observer && existing.observers && existing.observers.indexOf(observer) === -1) {
existing.observers.push(observer);
}
} else {
messages.push({
sender: sender,
text: displayText,
timestamp: ts,
sender_timestamp: payload.sender_timestamp || null,
packetId: pktId,
packetHash: pktHash,
repeats: 1,
observers: observer ? [observer] : [],
hops: payload.path_len || 0,
snr: snr,
});
}
messagesDirty = true;
}
}
if (channelListDirty) {
channels.sort(function (a, b) { return (b.lastActivityMs || 0) - (a.lastActivityMs || 0); });
renderChannelList();
}
if (messagesDirty) {
renderMessages();
// Update header count
var ch2 = channels.find(function (c) { return c.hash === selectedHash; });
var header = document.getElementById('chHeader');
if (header && ch2) {
header.querySelector('.ch-header-text').textContent = (ch2.name || 'Channel ' + selectedHash) + ' — ' + messages.length + ' messages';
}
var msgEl = document.getElementById('chMessages');
if (msgEl && autoScroll) scrollToBottom();
else {
document.getElementById('chScrollBtn')?.classList.remove('hidden');
var liveEl = document.getElementById('chAriaLive');
if (liveEl) liveEl.textContent = 'New message received';
}
}
});
// Tick relative timestamps every 1s — iterates channels array, updates DOM text only
timeAgoTimer = setInterval(function () {
var now = Date.now();
for (var i = 0; i < channels.length; i++) {
var ch = channels[i];
if (!ch.lastActivityMs) continue;
var el = document.querySelector('.ch-item-time[data-channel-hash="' + ch.hash + '"]');
if (el) el.textContent = formatSecondsAgo(Math.floor((now - ch.lastActivityMs) / 1000));
}
}, 1000);
}
var timeAgoTimer = null;
function destroy() {
if (wsHandler) offWS(wsHandler);
wsHandler = null;
if (timeAgoTimer) clearInterval(timeAgoTimer);
timeAgoTimer = null;
if (regionChangeHandler) RegionFilter.offChange(regionChangeHandler);
regionChangeHandler = null;
channels = [];
messages = [];
selectedHash = null;
@@ -391,8 +526,13 @@
async function loadChannels(silent) {
try {
const data = await api('/channels', { ttl: CLIENT_TTL.channels });
channels = (data.channels || []).sort((a, b) => (b.lastActivity || '').localeCompare(a.lastActivity || ''));
const rp = RegionFilter.getRegionParam();
const qs = rp ? '?region=' + encodeURIComponent(rp) : '';
const data = await api('/channels' + qs, { ttl: CLIENT_TTL.channels });
channels = (data.channels || []).map(ch => {
ch.lastActivityMs = ch.lastActivity ? new Date(ch.lastActivity).getTime() : 0;
return ch;
}).sort((a, b) => (b.lastActivityMs || 0) - (a.lastActivityMs || 0));
renderChannelList();
} catch (e) {
if (!silent) {
@@ -407,30 +547,27 @@
if (!el) return;
if (channels.length === 0) { el.innerHTML = '<div class="ch-empty">No channels found</div>'; return; }
// Sort: decrypted first (by message count desc), then encrypted (by message count desc)
// Sort by message count desc
const sorted = [...channels].sort((a, b) => {
if (a.encrypted !== b.encrypted) return a.encrypted ? 1 : -1;
return (b.messageCount || 0) - (a.messageCount || 0);
});
el.innerHTML = sorted.map(ch => {
const name = ch.name || `Channel ${ch.hash}`;
const color = getChannelColor(ch.hash);
const time = ch.lastActivity ? timeAgo(ch.lastActivity) : '';
const time = ch.lastActivityMs ? formatSecondsAgo(Math.floor((Date.now() - ch.lastActivityMs) / 1000)) : '';
const preview = ch.lastSender && ch.lastMessage
? `${ch.lastSender}: ${truncate(ch.lastMessage, 28)}`
: ch.encrypted ? `🔒 ${ch.messageCount} encrypted` : `${ch.messageCount} messages`;
: `${ch.messageCount} messages`;
const sel = selectedHash === ch.hash ? ' selected' : '';
const lockIcon = ch.encrypted ? ' 🔒' : '';
const encClass = ch.encrypted ? ' ch-item-encrypted' : '';
const abbr = name.startsWith('#') ? name.slice(0, 3) : name.slice(0, 2).toUpperCase();
return `<button class="ch-item${sel}${encClass}" data-hash="${ch.hash}" type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}">
return `<button class="ch-item${sel}" data-hash="${ch.hash}" type="button" role="option" aria-selected="${selectedHash === ch.hash ? 'true' : 'false'}" aria-label="${escapeHtml(name)}">
<div class="ch-badge" style="background:${color}" aria-hidden="true">${escapeHtml(abbr)}</div>
<div class="ch-item-body">
<div class="ch-item-top">
<span class="ch-item-name">${escapeHtml(name)}${lockIcon}</span>
<span class="ch-item-time">${time}</span>
<span class="ch-item-name">${escapeHtml(name)}</span>
<span class="ch-item-time" data-channel-hash="${ch.hash}">${time}</span>
</div>
<div class="ch-item-preview">${escapeHtml(preview)}</div>
</div>
@@ -440,7 +577,7 @@
async function selectChannel(hash) {
selectedHash = hash;
history.replaceState(null, '', `#/channels/${hash}`);
history.replaceState(null, '', `#/channels/${encodeURIComponent(hash)}`);
renderChannelList();
const ch = channels.find(c => c.hash === hash);
const name = ch?.name || `Channel ${hash}`;
@@ -454,7 +591,7 @@
msgEl.innerHTML = '<div class="ch-loading">Loading messages…</div>';
try {
const data = await api(`/channels/${hash}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
const data = await api(`/channels/${encodeURIComponent(hash)}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
messages = data.messages || [];
renderMessages();
scrollToBottom();
@@ -469,7 +606,7 @@
if (!msgEl) return;
const wasAtBottom = msgEl.scrollHeight - msgEl.scrollTop - msgEl.clientHeight < 60;
try {
const data = await api(`/channels/${selectedHash}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
const data = await api(`/channels/${encodeURIComponent(selectedHash)}/messages?limit=200`, { ttl: CLIENT_TTL.channelMessages });
const newMsgs = data.messages || [];
// #92: Use message ID/hash for change detection instead of count + timestamp
var _getLastId = function (arr) { var m = arr.length ? arr[arr.length - 1] : null; return m ? (m.id || m.packetId || m.timestamp || '') : ''; };
@@ -497,11 +634,7 @@
const senderLetter = sender.replace(/[^\w]/g, '').charAt(0).toUpperCase() || '?';
let displayText;
if (msg.encrypted) {
displayText = '<span class="mono ch-encrypted-text">🔒 encrypted</span>';
} else {
displayText = highlightMentions(msg.text || '');
}
displayText = highlightMentions(msg.text || '');
const time = msg.timestamp ? new Date(msg.timestamp).toLocaleTimeString([], { hour: '2-digit', minute: '2-digit' }) : '';
const date = msg.timestamp ? new Date(msg.timestamp).toLocaleDateString() : '';
@@ -519,7 +652,7 @@
<div class="ch-msg-content">
<div class="ch-msg-sender ch-sender-link ch-tappable" style="color:${senderColor}" tabindex="0" role="button" data-node="${safeId}">${escapeHtml(sender)}</div>
<div class="ch-msg-bubble">${displayText}</div>
<div class="ch-msg-meta">${meta.join(' · ')}${msg.packetId ? ` · <a href="#/packets/id/${msg.packetId}" class="ch-analyze-link">View packet →</a>` : ''}</div>
<div class="ch-msg-meta">${meta.join(' · ')}${msg.packetHash ? ` · <a href="#/packets/${msg.packetHash}" class="ch-analyze-link">View packet →</a>` : ''}</div>
</div>
</div>`;
}).join('');

View File

@@ -253,7 +253,7 @@
const obs = h.observers || [];
const age = stats.lastHeard ? Date.now() - new Date(stats.lastHeard).getTime() : null;
const status = age === null ? 'silent' : age < 3600000 ? 'healthy' : age < 86400000 ? 'degraded' : 'silent';
const status = age === null ? 'silent' : age < HEALTH_THRESHOLDS.nodeDegradedMs ? 'healthy' : age < HEALTH_THRESHOLDS.nodeSilentMs ? 'degraded' : 'silent';
const statusDot = status === 'healthy' ? '🟢' : status === 'degraded' ? '🟡' : '🔴';
const statusText = status === 'healthy' ? 'Active' : status === 'degraded' ? 'Degraded' : 'Silent';
const name = node.name || mn.name || truncate(mn.pubkey, 12);
@@ -373,7 +373,7 @@
const el = document.getElementById('homeStats');
if (!el) return;
el.innerHTML = `
<div class="home-stat"><div class="val">${s.totalPackets ?? '—'}</div><div class="lbl">Packets</div></div>
<div class="home-stat"><div class="val">${s.totalTransmissions ?? s.totalPackets ?? '—'}</div><div class="lbl">Transmissions</div></div>
<div class="home-stat"><div class="val">${s.totalNodes ?? '—'}</div><div class="lbl">Nodes</div></div>
<div class="home-stat"><div class="val">${s.totalObservers ?? '—'}</div><div class="lbl">Observers</div></div>
<div class="home-stat"><div class="val">${s.packetsLast24h ?? '—'}</div><div class="lbl">Last 24h</div></div>
@@ -403,8 +403,8 @@
if (stats.lastHeard) {
const ageMs = Date.now() - new Date(stats.lastHeard).getTime();
const ago = timeAgo(stats.lastHeard);
if (ageMs < 3600000) { status = 'healthy'; color = 'green'; statusMsg = `Last heard ${ago}`; }
else if (ageMs < 86400000) { status = 'degraded'; color = 'yellow'; statusMsg = `Last heard ${ago}`; }
if (ageMs < HEALTH_THRESHOLDS.nodeDegradedMs) { status = 'healthy'; color = 'green'; statusMsg = `Last heard ${ago}`; }
else if (ageMs < HEALTH_THRESHOLDS.nodeSilentMs) { status = 'degraded'; color = 'yellow'; statusMsg = `Last heard ${ago}`; }
else { statusMsg = `Last heard ${ago}`; }
}

142
public/hop-resolver.js Normal file
View File

@@ -0,0 +1,142 @@
/**
* Client-side hop resolver — eliminates /api/resolve-hops HTTP requests.
* Mirrors the server's disambiguateHops() logic from server.js.
*/
window.HopResolver = (function() {
'use strict';
const MAX_HOP_DIST = 1.8; // ~200km in degrees
let prefixIdx = {}; // lowercase hex prefix → [node, ...]
let nodesList = [];
function dist(lat1, lon1, lat2, lon2) {
return Math.sqrt((lat1 - lat2) ** 2 + (lon1 - lon2) ** 2);
}
/**
* Initialize (or rebuild) the prefix index from the full nodes list.
* @param {Array} nodes - Array of {public_key, name, lat, lon, ...}
*/
function init(nodes) {
nodesList = nodes || [];
prefixIdx = {};
for (const n of nodesList) {
if (!n.public_key) continue;
const pk = n.public_key.toLowerCase();
for (let len = 1; len <= 3; len++) {
const p = pk.slice(0, len * 2);
if (!prefixIdx[p]) prefixIdx[p] = [];
prefixIdx[p].push(n);
}
}
}
/**
* Resolve an array of hex hop prefixes to node info.
* Returns a map: { hop: {name, pubkey, lat, lon, ambiguous, unreliable} }
*
* @param {string[]} hops - Hex prefixes
* @param {number|null} originLat - Sender latitude (forward anchor)
* @param {number|null} originLon - Sender longitude (forward anchor)
* @param {number|null} observerLat - Observer latitude (backward anchor)
* @param {number|null} observerLon - Observer longitude (backward anchor)
* @returns {Object} resolved map keyed by hop prefix
*/
function resolve(hops, originLat, originLon, observerLat, observerLon) {
if (!hops || !hops.length) return {};
const resolved = {};
const hopPositions = {};
// First pass: find candidates
for (const hop of hops) {
const h = hop.toLowerCase();
const candidates = prefixIdx[h] || [];
if (candidates.length === 0) {
resolved[hop] = { name: null, candidates: [] };
} else if (candidates.length === 1) {
resolved[hop] = { name: candidates[0].name, pubkey: candidates[0].public_key, candidates: [{ name: candidates[0].name, pubkey: candidates[0].public_key }] };
} else {
resolved[hop] = { name: candidates[0].name, pubkey: candidates[0].public_key, ambiguous: true, candidates: candidates.map(c => ({ name: c.name, pubkey: c.public_key, lat: c.lat, lon: c.lon })) };
}
}
// Build initial positions for unambiguous hops
for (const hop of hops) {
const r = resolved[hop];
if (r && !r.ambiguous && r.pubkey) {
const node = nodesList.find(n => n.public_key === r.pubkey);
if (node && node.lat && node.lon && !(node.lat === 0 && node.lon === 0)) {
hopPositions[hop] = { lat: node.lat, lon: node.lon };
}
}
}
// Forward pass
let lastPos = (originLat != null && originLon != null) ? { lat: originLat, lon: originLon } : null;
for (let i = 0; i < hops.length; i++) {
const hop = hops[i];
if (hopPositions[hop]) { lastPos = hopPositions[hop]; continue; }
const r = resolved[hop];
if (!r || !r.ambiguous) continue;
const withLoc = r.candidates.filter(c => c.lat && c.lon && !(c.lat === 0 && c.lon === 0));
if (!withLoc.length) continue;
let anchor = lastPos;
if (!anchor && i === hops.length - 1 && observerLat != null) {
anchor = { lat: observerLat, lon: observerLon };
}
if (anchor) {
withLoc.sort((a, b) => dist(a.lat, a.lon, anchor.lat, anchor.lon) - dist(b.lat, b.lon, anchor.lat, anchor.lon));
}
r.name = withLoc[0].name;
r.pubkey = withLoc[0].pubkey;
hopPositions[hop] = { lat: withLoc[0].lat, lon: withLoc[0].lon };
lastPos = hopPositions[hop];
}
// Backward pass
let nextPos = (observerLat != null && observerLon != null) ? { lat: observerLat, lon: observerLon } : null;
for (let i = hops.length - 1; i >= 0; i--) {
const hop = hops[i];
if (hopPositions[hop]) { nextPos = hopPositions[hop]; continue; }
const r = resolved[hop];
if (!r || !r.ambiguous) continue;
const withLoc = r.candidates.filter(c => c.lat && c.lon && !(c.lat === 0 && c.lon === 0));
if (!withLoc.length || !nextPos) continue;
withLoc.sort((a, b) => dist(a.lat, a.lon, nextPos.lat, nextPos.lon) - dist(b.lat, b.lon, nextPos.lat, nextPos.lon));
r.name = withLoc[0].name;
r.pubkey = withLoc[0].pubkey;
hopPositions[hop] = { lat: withLoc[0].lat, lon: withLoc[0].lon };
nextPos = hopPositions[hop];
}
// Sanity check: drop hops impossibly far from neighbors
for (let i = 0; i < hops.length; i++) {
const pos = hopPositions[hops[i]];
if (!pos) continue;
const prev = i > 0 ? hopPositions[hops[i - 1]] : null;
const next = i < hops.length - 1 ? hopPositions[hops[i + 1]] : null;
if (!prev && !next) continue;
const dPrev = prev ? dist(pos.lat, pos.lon, prev.lat, prev.lon) : 0;
const dNext = next ? dist(pos.lat, pos.lon, next.lat, next.lon) : 0;
const tooFarPrev = prev && dPrev > MAX_HOP_DIST;
const tooFarNext = next && dNext > MAX_HOP_DIST;
if ((tooFarPrev && tooFarNext) || (tooFarPrev && !next) || (tooFarNext && !prev)) {
const r = resolved[hops[i]];
if (r) r.unreliable = true;
delete hopPositions[hops[i]];
}
}
return resolved;
}
/**
* Check if the resolver has been initialized with nodes.
*/
function ready() {
return nodesList.length > 0;
}
return { init: init, resolve: resolve, ready: ready };
})();

View File

@@ -22,9 +22,9 @@
<meta name="twitter:title" content="MeshCore Analyzer">
<meta name="twitter:description" content="Real-time MeshCore LoRa mesh network analyzer — live packet visualization, node tracking, channel decryption, and route analysis.">
<meta name="twitter:image" content="https://raw.githubusercontent.com/Kpa-clawbot/meshcore-analyzer/master/public/og-image.png">
<link rel="stylesheet" href="style.css?v=1773998477">
<link rel="stylesheet" href="style.css?v=1774138896">
<link rel="stylesheet" href="home.css">
<link rel="stylesheet" href="live.css?v=1773966856">
<link rel="stylesheet" href="live.css?v=1774058575">
<link rel="stylesheet" href="https://unpkg.com/leaflet@1.9.4/dist/leaflet.css"
integrity="sha256-p4NxAoJBhIIN+hmNHrzRCf9tD/miZyoHS5obTRR9BMY="
crossorigin="anonymous">
@@ -79,18 +79,21 @@
<main id="app" role="main"></main>
<script src="vendor/qrcode.js"></script>
<script src="app.js?v=1773993532"></script>
<script src="home.js?v=1773977027"></script>
<script src="packets.js?v=1773999188"></script>
<script src="map.js?v=1773998477" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1773977027" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1773977027" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1773972187" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1773996158" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1773998477" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1773998477" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1773993532" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1773996158" onerror="console.error('Failed to load:', this.src)"></script>
<script src="roles.js?v=1774290000"></script>
<script src="region-filter.js?v=1774136865"></script>
<script src="hop-resolver.js?v=1774126708"></script>
<script src="app.js?v=1774126708"></script>
<script src="home.js?v=1774042199"></script>
<script src="packets.js?v=1774155585"></script>
<script src="map.js?v=1774126708" onerror="console.error('Failed to load:', this.src)"></script>
<script src="channels.js?v=1774331200" onerror="console.error('Failed to load:', this.src)"></script>
<script src="nodes.js?v=1774126708" onerror="console.error('Failed to load:', this.src)"></script>
<script src="traces.js?v=1774135052" onerror="console.error('Failed to load:', this.src)"></script>
<script src="analytics.js?v=1774126708" onerror="console.error('Failed to load:', this.src)"></script>
<script src="live.js?v=1774155165" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observers.js?v=1774290000" onerror="console.error('Failed to load:', this.src)"></script>
<script src="observer-detail.js?v=1774028201" onerror="console.error('Failed to load:', this.src)"></script>
<script src="node-analytics.js?v=1774126708" onerror="console.error('Failed to load:', this.src)"></script>
<script src="perf.js?v=1773985649" onerror="console.error('Failed to load:', this.src)"></script>
</body>
</html>

View File

@@ -100,6 +100,26 @@
background: color-mix(in srgb, var(--text) 14%, transparent);
}
/* ---- Node Detail Panel ---- */
.live-node-detail {
top: 60px;
right: 12px;
width: 320px;
max-height: calc(100vh - 140px);
overflow-y: auto;
background: color-mix(in srgb, var(--surface-1) 95%, transparent);
backdrop-filter: blur(12px);
border-radius: 10px;
border: 1px solid var(--border);
box-shadow: 0 4px 24px rgba(0, 0, 0, 0.5);
transition: transform 0.2s ease, opacity 0.2s ease;
}
.live-node-detail.hidden {
transform: translateX(340px);
opacity: 0;
pointer-events: none;
}
/* ---- Feed ---- */
.live-feed {
bottom: 12px;
@@ -622,7 +642,7 @@
.vcr-prompt-btn:hover { background: rgba(59,130,246,0.3); }
/* Adjust feed position to not overlap VCR bar */
.live-feed { bottom: 58px; }
.live-feed { bottom: 68px; }
.feed-show-btn { bottom: 68px !important; }
/* Mobile VCR */

View File

@@ -11,6 +11,9 @@
let audioCtx = null;
let soundEnabled = false;
let showGhostHops = localStorage.getItem('live-ghost-hops') !== 'false';
let realisticPropagation = localStorage.getItem('live-realistic-propagation') === 'true';
let showOnlyFavorites = localStorage.getItem('live-favorites-only') === 'true';
const propagationBuffer = new Map(); // hash -> {timer, packets[]}
let _onResize = null;
let _navCleanup = null;
let _timelineRefreshInterval = null;
@@ -30,10 +33,7 @@
timelineFetchedScope: 0, // last fetched scope to avoid redundant fetches
};
const ROLE_COLORS = {
repeater: '#3b82f6', companion: '#06b6d4', room: '#a855f7',
sensor: '#f59e0b', unknown: '#6b7280'
};
// ROLE_COLORS loaded from shared roles.js (includes 'unknown')
const TYPE_COLORS = {
ADVERT: '#22c55e', GRP_TXT: '#3b82f6', TXT_MSG: '#f59e0b', ACK: '#6b7280',
@@ -426,6 +426,21 @@
}
// Buffer a packet from WS
let _tabHidden = false;
document.addEventListener('visibilitychange', () => {
if (document.hidden) {
_tabHidden = true;
} else {
// Tab restored — skip animating anything that queued while away
_tabHidden = false;
// Clear any pending propagation buffers so they don't all fire at once
for (const [hash, entry] of propagationBuffer) {
clearTimeout(entry.timer);
}
propagationBuffer.clear();
}
});
function bufferPacket(pkt) {
pkt._ts = Date.now();
const entry = { ts: pkt._ts, pkt };
@@ -440,7 +455,26 @@
}
if (VCR.mode === 'LIVE') {
animatePacket(pkt);
// Skip animations when tab is backgrounded — just buffer for VCR timeline
if (_tabHidden) {
updateTimeline();
return;
}
if (realisticPropagation && pkt.hash) {
const hash = pkt.hash;
if (propagationBuffer.has(hash)) {
propagationBuffer.get(hash).packets.push(pkt);
} else {
const entry = { packets: [pkt], timer: setTimeout(() => {
const buffered = propagationBuffer.get(hash);
propagationBuffer.delete(hash);
if (buffered) animateRealisticPropagation(buffered.packets);
}, PROPAGATION_BUFFER_MS) };
propagationBuffer.set(hash, entry);
}
} else {
animatePacket(pkt);
}
updateTimeline();
} else if (VCR.mode === 'PAUSED') {
VCR.missedCount++;
@@ -595,12 +629,20 @@
<span id="heatDesc" class="sr-only">Overlay a density heat map on the mesh nodes</span>
<label><input type="checkbox" id="liveGhostToggle" checked aria-describedby="ghostDesc"> Ghosts</label>
<span id="ghostDesc" class="sr-only">Show interpolated ghost markers for unknown hops</span>
<label><input type="checkbox" id="liveRealisticToggle" aria-describedby="realisticDesc"> Realistic</label>
<span id="realisticDesc" class="sr-only">Buffer packets by hash and animate all paths simultaneously</span>
<label><input type="checkbox" id="liveFavoritesToggle" aria-describedby="favDesc"> ⭐ Favorites</label>
<span id="favDesc" class="sr-only">Show only favorited and claimed nodes</span>
</div>
</div>
<div class="live-overlay live-feed" id="liveFeed">
<button class="feed-hide-btn" id="feedHideBtn" title="Hide feed">✕</button>
</div>
<button class="feed-show-btn hidden" id="feedShowBtn" title="Show feed">📋</button>
<div class="live-overlay live-node-detail hidden" id="liveNodeDetail">
<button class="feed-hide-btn" id="nodeDetailClose" title="Close">✕</button>
<div id="nodeDetailContent"></div>
</div>
<button class="legend-toggle-btn hidden" id="legendToggleBtn" aria-label="Show legend" title="Show legend">🎨</button>
<div class="live-overlay live-legend" id="liveLegend" role="region" aria-label="Map legend">
<h3 class="legend-title">PACKET TYPES</h3>
@@ -612,12 +654,7 @@
<li><span class="live-dot" style="background:#ec4899" aria-hidden="true"></span> Trace — Route trace</li>
</ul>
<h3 class="legend-title" style="margin-top:8px">NODE ROLES</h3>
<ul class="legend-list">
<li><span class="live-dot" style="background:#3b82f6" aria-hidden="true"></span> Repeater</li>
<li><span class="live-dot" style="background:#06b6d4" aria-hidden="true"></span> Companion</li>
<li><span class="live-dot" style="background:#a855f7" aria-hidden="true"></span> Room</li>
<li><span class="live-dot" style="background:#f59e0b" aria-hidden="true"></span> Sensor</li>
</ul>
<ul class="legend-list" id="roleLegendList"></ul>
</div>
<!-- VCR Bar -->
@@ -649,22 +686,29 @@
</div>
</div>`;
// Fetch configurable map defaults (#115)
let mapCenter = [37.45, -122.0];
let mapZoom = 9;
try {
const mapCfg = await (await fetch('/api/config/map')).json();
if (Array.isArray(mapCfg.center) && mapCfg.center.length === 2) mapCenter = mapCfg.center;
if (typeof mapCfg.zoom === 'number') mapZoom = mapCfg.zoom;
} catch {}
map = L.map('liveMap', {
zoomControl: false, attributionControl: false,
zoomAnimation: true, markerZoomAnimation: true
}).setView([37.45, -122.0], 9);
}).setView(mapCenter, mapZoom);
const isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
const DARK_TILES = 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
const LIGHT_TILES = 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png';
let tileLayer = L.tileLayer(isDark ? DARK_TILES : LIGHT_TILES, { maxZoom: 19 }).addTo(map);
let tileLayer = L.tileLayer(isDark ? TILE_DARK : TILE_LIGHT, { maxZoom: 19 }).addTo(map);
// Swap tiles when theme changes
const _themeObs = new MutationObserver(function () {
const dark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
tileLayer.setUrl(dark ? DARK_TILES : LIGHT_TILES);
tileLayer.setUrl(dark ? TILE_DARK : TILE_LIGHT);
});
_themeObs.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
L.control.zoom({ position: 'topright' }).addTo(map);
@@ -680,14 +724,27 @@
initResizeHandler();
startRateCounter();
// Check for single packet replay from packets page
// Check for packet replay from packets page (single or array of observations)
const replayData = sessionStorage.getItem('replay-packet');
if (replayData) {
sessionStorage.removeItem('replay-packet');
try {
const pkt = JSON.parse(replayData);
const parsed = JSON.parse(replayData);
const packets = Array.isArray(parsed) ? parsed : [parsed];
vcrPause(); // suppress live packets
setTimeout(() => animatePacket(pkt), 1500);
if (packets.length > 1 && packets[0].hash) {
// Multiple observations — use realistic propagation (animate all paths at once)
setTimeout(() => {
if (typeof animateRealisticPropagation === 'function') {
animateRealisticPropagation(packets);
} else {
// Fallback: stagger animations
packets.forEach((p, i) => setTimeout(() => animatePacket(p), i * 400));
}
}, 1500);
} else {
setTimeout(() => animatePacket(packets[0]), 1500);
}
} catch {}
} else {
replayRecent();
@@ -714,6 +771,21 @@
localStorage.setItem('live-ghost-hops', showGhostHops);
});
const realisticToggle = document.getElementById('liveRealisticToggle');
realisticToggle.checked = realisticPropagation;
realisticToggle.addEventListener('change', (e) => {
realisticPropagation = e.target.checked;
localStorage.setItem('live-realistic-propagation', realisticPropagation);
});
const favoritesToggle = document.getElementById('liveFavoritesToggle');
favoritesToggle.checked = showOnlyFavorites;
favoritesToggle.addEventListener('change', (e) => {
showOnlyFavorites = e.target.checked;
localStorage.setItem('live-favorites-only', showOnlyFavorites);
applyFavoritesFilter();
});
// Feed show/hide
const feedEl = document.getElementById('liveFeed');
// Keyboard support for feed items (event delegation)
@@ -749,6 +821,23 @@
});
}
// Populate role legend from shared roles.js
const roleLegendList = document.getElementById('roleLegendList');
if (roleLegendList) {
for (const role of (window.ROLE_SORT || ['repeater', 'companion', 'room', 'sensor', 'observer'])) {
const li = document.createElement('li');
li.innerHTML = `<span class="live-dot" style="background:${ROLE_COLORS[role] || '#6b7280'}" aria-hidden="true"></span> ${(ROLE_LABELS[role] || role).replace(/s$/, '')}`;
roleLegendList.appendChild(li);
}
}
// Node detail panel
const nodeDetailPanel = document.getElementById('liveNodeDetail');
const nodeDetailContent = document.getElementById('nodeDetailContent');
document.getElementById('nodeDetailClose').addEventListener('click', () => {
nodeDetailPanel.classList.add('hidden');
});
// Feed panel resize handle (#27)
const savedFeedWidth = localStorage.getItem('live-feed-width');
if (savedFeedWidth) feedEl.style.width = savedFeedWidth + 'px';
@@ -966,6 +1055,116 @@
}, 2000);
}
async function showNodeDetail(pubkey) {
const panel = document.getElementById('liveNodeDetail');
const content = document.getElementById('nodeDetailContent');
panel.classList.remove('hidden');
content.innerHTML = '<div style="padding:20px;color:var(--text-muted)">Loading…</div>';
try {
const [data, healthData] = await Promise.all([
api('/nodes/' + encodeURIComponent(pubkey), { ttl: 30 }),
api('/nodes/' + encodeURIComponent(pubkey) + '/health', { ttl: 30 }).catch(() => null)
]);
const n = data.node;
const h = healthData || {};
const stats = h.stats || {};
const observers = h.observers || [];
const recent = h.recentPackets || [];
const roleColor = ROLE_COLORS[n.role] || '#6b7280';
const roleLabel = (ROLE_LABELS[n.role] || n.role || 'unknown').replace(/s$/, '');
const hasLoc = n.lat != null && n.lon != null;
const lastSeen = n.last_seen ? timeAgo(n.last_seen) : '—';
const thresholds = window.getHealthThresholds ? getHealthThresholds(n.role) : { degradedMs: 3600000, silentMs: 86400000 };
const ageMs = n.last_seen ? Date.now() - new Date(n.last_seen).getTime() : Infinity;
const statusDot = ageMs < thresholds.degradedMs ? 'health-green' : ageMs < thresholds.silentMs ? 'health-yellow' : 'health-red';
const statusLabel = ageMs < thresholds.degradedMs ? 'Online' : ageMs < thresholds.silentMs ? 'Degraded' : 'Offline';
let html = `
<div style="padding:16px;">
<div style="display:flex;align-items:center;gap:8px;margin-bottom:12px;">
<span class="${statusDot}" style="font-size:18px">●</span>
<h3 style="margin:0;font-size:16px;font-weight:700;">${escapeHtml(n.name || 'Unknown')}</h3>
</div>
<div style="margin-bottom:12px;">
<span style="display:inline-block;padding:2px 10px;border-radius:12px;font-size:11px;font-weight:600;background:${roleColor};color:#fff;">${roleLabel.toUpperCase()}</span>
<span style="color:var(--text-muted);font-size:12px;margin-left:8px;">${statusLabel}</span>
</div>
<div style="font-size:12px;color:var(--text-muted);margin-bottom:8px;">
<code style="font-size:10px;word-break:break-all;">${escapeHtml(n.public_key)}</code>
</div>
<table style="font-size:12px;width:100%;border-collapse:collapse;">
<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Last Seen</td><td>${lastSeen}</td></tr>
<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Adverts</td><td>${n.advert_count || 0}</td></tr>
${hasLoc ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Location</td><td>${n.lat.toFixed(5)}, ${n.lon.toFixed(5)}</td></tr>` : ''}
${stats.avgSnr != null ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Avg SNR</td><td>${stats.avgSnr.toFixed(1)} dB</td></tr>` : ''}
${stats.avgHops != null ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Avg Hops</td><td>${stats.avgHops.toFixed(1)}</td></tr>` : ''}
${stats.totalTransmissions || stats.totalPackets ? `<tr><td style="color:var(--text-muted);padding:4px 8px 4px 0;">Total Packets</td><td>${stats.totalTransmissions || stats.totalPackets}</td></tr>` : ''}
</table>`;
if (observers.length) {
html += `<h4 style="font-size:12px;margin:12px 0 6px;color:var(--text-muted);">Heard By</h4>
<div style="font-size:11px;">` +
observers.map(o => `<div style="padding:2px 0;"><a href="#/observers/${encodeURIComponent(o.observer_id)}" style="color:var(--accent);text-decoration:none;">${escapeHtml(o.observer_name || o.observer_id.slice(0, 12))}</a> — ${o.packetCount || o.count || 0} pkts</div>`).join('') +
'</div>';
}
if (recent.length) {
html += `<h4 style="font-size:12px;margin:12px 0 6px;color:var(--text-muted);">Recent Packets</h4>
<div style="font-size:11px;max-height:200px;overflow-y:auto;">` +
recent.slice(0, 10).map(p => `<div style="padding:2px 0;display:flex;justify-content:space-between;">
<a href="#/packets/${encodeURIComponent(p.hash || '')}" style="color:var(--accent);text-decoration:none;">${escapeHtml(p.payload_type || '?')}${p.observation_count > 1 ? ' <span class="badge badge-obs" style="font-size:9px">👁 ' + p.observation_count + '</span>' : ''}</a>
<span style="color:var(--text-muted)">${p.timestamp ? timeAgo(p.timestamp) : '—'}</span>
</div>`).join('') +
'</div>';
}
html += `<div id="liveNodePaths" style="margin-top:8px;"><div style="font-size:11px;color:var(--text-muted);padding:4px 0;"><span class="spinner" style="font-size:10px"></span> Loading paths…</div></div>`;
html += `<div style="margin-top:12px;display:flex;gap:8px;">
<a href="#/nodes/${encodeURIComponent(n.public_key)}" style="font-size:12px;color:var(--accent);">Full Detail →</a>
<a href="#/nodes/${encodeURIComponent(n.public_key)}/analytics" style="font-size:12px;color:var(--accent);">📊 Analytics</a>
</div></div>`;
content.innerHTML = html;
// Fetch paths asynchronously
api('/nodes/' + encodeURIComponent(n.public_key) + '/paths', { ttl: 300 }).then(pathData => {
const pathEl = document.getElementById('liveNodePaths');
if (!pathEl) return;
if (!pathData || !pathData.paths || !pathData.paths.length) {
pathEl.innerHTML = '';
return;
}
const COLLAPSE = 5;
function renderPathList(paths) {
return paths.map(p => {
const chain = p.hops.map(h => {
const isThis = h.pubkey === n.public_key || (h.prefix && n.public_key.toLowerCase().startsWith(h.prefix.toLowerCase()));
const name = escapeHtml(h.name || h.prefix);
if (isThis) return `<strong style="color:var(--accent)">${name}</strong>`;
return h.pubkey ? `<a href="#/nodes/${h.pubkey}" style="color:var(--text-primary);text-decoration:none">${name}</a>` : name;
}).join(' → ');
return `<div style="padding:3px 0;font-size:11px;line-height:1.4">${chain} <span style="color:var(--text-muted)">(${p.count}×)</span></div>`;
}).join('');
}
pathEl.innerHTML = `<h4 style="font-size:12px;margin:8px 0 4px;color:var(--text-muted);">Paths Through (${pathData.totalPaths})</h4>` +
`<div id="livePathsList" style="max-height:200px;overflow-y:auto;">` +
renderPathList(pathData.paths.slice(0, COLLAPSE)) +
(pathData.paths.length > COLLAPSE ? `<button id="showMorePaths" style="font-size:11px;color:var(--accent);background:none;border:none;cursor:pointer;padding:4px 0;">Show all ${pathData.paths.length} paths</button>` : '') +
'</div>';
const moreBtn = document.getElementById('showMorePaths');
if (moreBtn) moreBtn.addEventListener('click', () => {
document.getElementById('livePathsList').innerHTML = renderPathList(pathData.paths);
});
}).catch(() => {
const pathEl = document.getElementById('liveNodePaths');
if (pathEl) pathEl.innerHTML = '';
});
} catch (e) {
content.innerHTML = `<div style="padding:20px;color:var(--text-muted);">Error: ${e.message}</div>`;
}
}
async function loadNodes(beforeTs) {
try {
const url = beforeTs
@@ -993,6 +1192,74 @@
if (heatLayer) { map.removeLayer(heatLayer); heatLayer = null; }
}
function getFavoritePubkeys() {
let favs = [];
try { favs = favs.concat(JSON.parse(localStorage.getItem('meshcore-favorites') || '[]')); } catch {}
try { favs = favs.concat(JSON.parse(localStorage.getItem('meshcore-my-nodes') || '[]').map(n => n.pubkey)); } catch {}
return favs.filter(Boolean);
}
function packetInvolvesFavorite(pkt) {
const favs = getFavoritePubkeys();
if (favs.length === 0) return false;
const decoded = pkt.decoded || {};
const payload = decoded.payload || {};
const hops = decoded.path?.hops || [];
// Full pubkeys: sender
if (payload.pubKey && favs.some(f => f === payload.pubKey)) return true;
// Observer: may be name or pubkey
const obs = pkt.observer_name || pkt.observer || '';
if (obs) {
if (favs.some(f => f === obs)) return true;
for (const nd of Object.values(nodeData)) {
if ((nd.name === obs || nd.public_key === obs) && favs.some(f => f === nd.public_key)) return true;
}
}
// Hops are truncated hex prefixes — match by prefix in either direction
for (const hop of hops) {
const h = (hop.id || hop.public_key || hop).toString().toLowerCase();
if (favs.some(f => f.toLowerCase().startsWith(h) || h.startsWith(f.toLowerCase()))) return true;
}
return false;
}
function isNodeFavorited(pubkey) {
return getFavoritePubkeys().some(f => f === pubkey);
}
function rebuildFeedList() {
const feed = document.getElementById('liveFeed');
if (!feed) return;
// Remove all feed items but keep the hide button and resize handle
feed.querySelectorAll('.live-feed-item').forEach(el => el.remove());
// Re-add from VCR buffer (most recent first, up to 25)
const entries = VCR.buffer.slice(-100).reverse();
let count = 0;
for (const entry of entries) {
if (count >= 25) break;
const pkt = entry.pkt;
if (showOnlyFavorites && !packetInvolvesFavorite(pkt)) continue;
const decoded = pkt.decoded || {};
const header = decoded.header || {};
const payload = decoded.payload || {};
const typeName = header.payloadTypeName || 'UNKNOWN';
const icon = PAYLOAD_ICONS[typeName] || '📦';
const hops = decoded.path?.hops || [];
const color = TYPE_COLORS[typeName] || '#6b7280';
addFeedItemDOM(icon, typeName, payload, hops, color, pkt, feed);
count++;
}
}
function applyFavoritesFilter() {
// Node markers always stay visible — only rebuild the feed list
rebuildFeedList();
}
function addNodeMarker(n) {
if (nodeMarkers[n.public_key]) return nodeMarkers[n.public_key];
const color = ROLE_COLORS[n.role] || ROLE_COLORS.unknown;
@@ -1014,6 +1281,8 @@
permanent: false, direction: 'top', offset: [0, -10], className: 'live-tooltip'
});
marker.on('click', () => showNodeDetail(n.public_key));
marker._glowMarker = glow;
marker._baseColor = color;
marker._baseSize = size;
@@ -1059,7 +1328,7 @@
if (msg.type === 'packet') bufferPacket(msg.data);
} catch {}
};
ws.onclose = () => setTimeout(connectWS, 3000);
ws.onclose = () => setTimeout(connectWS, WS_RECONNECT_MS);
ws.onerror = () => {};
}
@@ -1079,6 +1348,9 @@
playSound(typeName);
addFeedItem(icon, typeName, payload, hops, color, pkt);
// Favorites filter: skip animation if packet doesn't involve a favorited node
if (showOnlyFavorites && !packetInvolvesFavorite(pkt)) return;
// If ADVERT, ensure node appears on map
if (typeName === 'ADVERT' && payload.pubKey) {
const key = payload.pubKey;
@@ -1096,6 +1368,94 @@
animatePath(hopPositions, typeName, color);
}
function animateRealisticPropagation(packets) {
if (!packets.length) return;
const first = packets[0];
const decoded = first.decoded || {};
const header = decoded.header || {};
const typeName = header.payloadTypeName || 'UNKNOWN';
const color = TYPE_COLORS[typeName] || '#6b7280';
const icon = PAYLOAD_ICONS[typeName] || '📦';
const payload = decoded.payload || {};
packetCount += packets.length;
pktTimestamps.push(Date.now());
const _el = document.getElementById('livePktCount'); if (_el) _el.textContent = packetCount;
// Favorites filter: skip if none of the packets involve a favorite
if (showOnlyFavorites && !packets.some(p => packetInvolvesFavorite(p))) return;
playSound(typeName);
// Ensure ADVERT nodes appear
for (const pkt of packets) {
const d = pkt.decoded || {};
const h = d.header || {};
const p = d.payload || {};
if (h.payloadTypeName === 'ADVERT' && p.pubKey) {
const key = p.pubKey;
if (!nodeMarkers[key] && p.lat != null && p.lon != null && !(p.lat === 0 && p.lon === 0)) {
const n = { public_key: key, name: p.name || key.slice(0,8), role: p.role || 'unknown', lat: p.lat, lon: p.lon };
nodeData[key] = n;
addNodeMarker(n);
}
}
}
const _el2 = document.getElementById('liveNodeCount'); if (_el2) _el2.textContent = Object.keys(nodeMarkers).length;
// Resolve all unique paths
const allPaths = [];
const seenPathKeys = new Set();
const observers = new Set();
for (const pkt of packets) {
const d = pkt.decoded || {};
const p = d.payload || {};
const hops = d.path?.hops || [];
if (pkt.observer) observers.add(pkt.observer);
const pathKey = hops.join(',');
if (seenPathKeys.has(pathKey)) continue;
seenPathKeys.add(pathKey);
const hopPositions = resolveHopPositions(hops, p);
if (hopPositions.length >= 2) allPaths.push(hopPositions);
}
// Consolidated feed item
const hops0 = decoded.path?.hops || [];
const text = payload.text || payload.name || '';
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const feed = document.getElementById('liveFeed');
if (feed) {
const item = document.createElement('div');
item.className = 'live-feed-item live-feed-enter';
item.setAttribute('tabindex', '0');
item.setAttribute('role', 'button');
item.style.cursor = 'pointer';
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
<span class="feed-hops">${allPaths.length}${observers.size}👁</span>
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${new Date(first._ts || Date.now()).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit',second:'2-digit'})}</span>
`;
item.addEventListener('click', () => showFeedCard(item, first, color));
feed.prepend(item);
requestAnimationFrame(() => { requestAnimationFrame(() => item.classList.remove('live-feed-enter')); });
while (feed.children.length > 25) feed.removeChild(feed.lastChild);
}
if (allPaths.length === 0) {
// Single hop or unresolvable — just pulse origin if possible
const hp0 = resolveHopPositions(decoded.path?.hops || [], payload);
if (hp0.length >= 1) pulseNode(hp0[0].key, hp0[0].pos, typeName);
return;
}
// Animate all paths simultaneously
for (const hopPositions of allPaths) {
animatePath(hopPositions, typeName, color);
}
}
function resolveHopPositions(hops, payload) {
const known = Object.values(nodeData);
@@ -1154,7 +1514,7 @@
// Sanity check: drop hops that are impossibly far from both neighbors (>200km ≈ 1.8°)
// These are almost certainly 1-byte prefix collisions with distant nodes
const MAX_HOP_DIST = 1.8;
// MAX_HOP_DIST from shared roles.js
for (let i = 0; i < raw.length; i++) {
if (!raw[i].known || !raw[i].pos) continue;
const prev = i > 0 && raw[i-1].known && raw[i-1].pos ? raw[i-1].pos : null;
@@ -1325,13 +1685,12 @@
if (step >= steps) {
clearInterval(interval);
animLayer.removeLayer(dot);
if (animLayer) animLayer.removeLayer(dot);
recentPaths.push({ line, glowLine: contrail, time: Date.now() });
while (recentPaths.length > 5) {
const old = recentPaths.shift();
pathsLayer.removeLayer(old.line);
pathsLayer.removeLayer(old.glowLine);
if (pathsLayer) { pathsLayer.removeLayer(old.line); pathsLayer.removeLayer(old.glowLine); }
}
setTimeout(() => {
@@ -1340,8 +1699,7 @@
fadeOp -= 0.1;
if (fadeOp <= 0) {
clearInterval(fi);
pathsLayer.removeLayer(line);
pathsLayer.removeLayer(contrail);
if (pathsLayer) { pathsLayer.removeLayer(line); pathsLayer.removeLayer(contrail); }
recentPaths = recentPaths.filter(p => p.line !== line);
} else {
line.setStyle({ opacity: fadeOp });
@@ -1380,13 +1738,38 @@
if (heatLayer) { map.removeLayer(heatLayer); heatLayer = null; }
}
function addFeedItemDOM(icon, typeName, payload, hops, color, pkt, feed) {
const text = payload.text || payload.name || '';
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = pkt.observation_count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${pkt.observation_count}</span>` : '';
const item = document.createElement('div');
item.className = 'live-feed-item';
item.setAttribute('tabindex', '0');
item.setAttribute('role', 'button');
item.style.cursor = 'pointer';
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${new Date(pkt._ts || Date.now()).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit',second:'2-digit'})}</span>
`;
item.addEventListener('click', () => showFeedCard(item, pkt, color));
feed.appendChild(item);
}
function addFeedItem(icon, typeName, payload, hops, color, pkt) {
const feed = document.getElementById('liveFeed');
if (!feed) return;
// Favorites filter: skip feed item if packet doesn't involve a favorite
if (showOnlyFavorites && !packetInvolvesFavorite(pkt)) return;
const text = payload.text || payload.name || '';
const preview = text ? ' ' + (text.length > 35 ? text.slice(0, 35) + '…' : text) : '';
const hopStr = hops.length ? `<span class="feed-hops">${hops.length}⇢</span>` : '';
const obsBadge = pkt.observation_count > 1 ? `<span class="badge badge-obs" style="font-size:10px;margin-left:4px">👁 ${pkt.observation_count}</span>` : '';
const item = document.createElement('div');
item.className = 'live-feed-item live-feed-enter';
@@ -1396,7 +1779,7 @@
item.innerHTML = `
<span class="feed-icon" style="color:${color}">${icon}</span>
<span class="feed-type" style="color:${color}">${typeName}</span>
${hopStr}
${hopStr}${obsBadge}
<span class="feed-text">${escapeHtml(preview)}</span>
<span class="feed-time">${new Date(pkt._ts || Date.now()).toLocaleTimeString([], {hour:'2-digit',minute:'2-digit',second:'2-digit'})}</span>
`;
@@ -1437,7 +1820,7 @@
${rssi != null ? `<span>📡 ${rssi} dBm</span>` : ''}
${observer ? `<span>👁 ${escapeHtml(observer)}</span>` : ''}
</div>
${pktId ? `<a class="fdc-link" href="#/packets/id/${pktId}">View in packets →</a>` : ''}
${pkt.hash ? `<a class="fdc-link" href="#/packets/${pkt.hash.toLowerCase()}">View in packets →</a>` : ''}
<button class="fdc-replay">↻ Replay</button>
`;
card.querySelector('.fdc-close').addEventListener('click', (e) => { e.stopPropagation(); card.remove(); });

View File

@@ -8,7 +8,7 @@
let clusterGroup = null;
let nodes = [];
let observers = [];
let filters = { repeater: true, companion: true, room: true, sensor: true, lastHeard: '30d', mqttOnly: false, neighbors: false, clusters: false };
let filters = { repeater: true, companion: true, room: true, sensor: true, observer: true, lastHeard: '30d', neighbors: false, clusters: false, hashLabels: localStorage.getItem('meshcore-map-hash-labels') !== 'false' };
let wsHandler = null;
let heatLayer = null;
let userHasMoved = false;
@@ -17,16 +17,7 @@
// Safe escape — falls back to identity if app.js hasn't loaded yet
const safeEsc = (typeof esc === 'function') ? esc : function (s) { return s; };
// Distinct shapes + high-contrast WCAG AA colors for each role
const ROLE_STYLE = {
repeater: { color: '#dc2626', shape: 'diamond', radius: 10, weight: 2 }, // red diamond
companion: { color: '#2563eb', shape: 'circle', radius: 8, weight: 2 }, // blue circle
room: { color: '#16a34a', shape: 'square', radius: 9, weight: 2 }, // green square
sensor: { color: '#d97706', shape: 'triangle', radius: 8, weight: 2 }, // amber triangle
};
const ROLE_LABELS = { repeater: 'Repeaters', companion: 'Companions', room: 'Room Servers', sensor: 'Sensors' };
const ROLE_COLORS = { repeater: '#dc2626', companion: '#2563eb', room: '#16a34a', sensor: '#d97706' };
// Roles loaded from shared roles.js (ROLE_STYLE, ROLE_LABELS, ROLE_COLORS globals)
function makeMarkerIcon(role) {
const s = ROLE_STYLE[role] || ROLE_STYLE.companion;
@@ -43,6 +34,19 @@
case 'triangle':
path = `<polygon points="${c},2 ${size-2},${size-2} 2,${size-2}" fill="${s.color}" stroke="#fff" stroke-width="2"/>`;
break;
case 'star': {
// 5-pointed star
const cx = c, cy = c, outer = c - 1, inner = outer * 0.4;
let pts = '';
for (let i = 0; i < 5; i++) {
const aOuter = (i * 72 - 90) * Math.PI / 180;
const aInner = ((i * 72) + 36 - 90) * Math.PI / 180;
pts += `${cx + outer * Math.cos(aOuter)},${cy + outer * Math.sin(aOuter)} `;
pts += `${cx + inner * Math.cos(aInner)},${cy + inner * Math.sin(aInner)} `;
}
path = `<polygon points="${pts.trim()}" fill="${s.color}" stroke="#fff" stroke-width="1.5"/>`;
break;
}
default: // circle
path = `<circle cx="${c}" cy="${c}" r="${c-2}" fill="${s.color}" stroke="#fff" stroke-width="2"/>`;
}
@@ -56,7 +60,24 @@
});
}
function init(container) {
function makeRepeaterLabelIcon(node) {
var s = ROLE_STYLE['repeater'] || ROLE_STYLE.companion;
var hs = node.hash_size || 1;
// Show the short mesh hash ID (first N bytes of pubkey, uppercased)
var shortHash = node.public_key ? node.public_key.slice(0, hs * 2).toUpperCase() : '??';
var bgColor = node.hash_size ? s.color : '#888';
var html = '<div style="background:' + bgColor + ';color:#fff;font-weight:bold;font-size:11px;padding:2px 5px;border-radius:3px;border:2px solid #fff;box-shadow:0 1px 3px rgba(0,0,0,0.4);text-align:center;line-height:1.2;white-space:nowrap;">' +
shortHash + '</div>';
return L.divIcon({
html: html,
className: 'meshcore-marker meshcore-label-marker',
iconSize: null,
iconAnchor: [14, 12],
popupAnchor: [0, -12],
});
}
async function init(container) {
container.innerHTML = `
<div id="map-wrap" style="position:relative;width:100%;height:100%;">
<div id="leaflet-map" style="width:100%;height:100%;"></div>
@@ -71,10 +92,10 @@
<legend class="mc-label">Display</legend>
<label for="mcClusters"><input type="checkbox" id="mcClusters"> Show clusters</label>
<label for="mcHeatmap"><input type="checkbox" id="mcHeatmap"> Heat map</label>
<label for="mcHashLabels"><input type="checkbox" id="mcHashLabels"> Hash prefix labels</label>
</fieldset>
<fieldset class="mc-section">
<legend class="mc-label">Filters</legend>
<label for="mcMqtt"><input type="checkbox" id="mcMqtt"> MQTT Connected Only</label>
<label for="mcNeighbors"><input type="checkbox" id="mcNeighbors"> Show direct neighbors</label>
</fieldset>
<fieldset class="mc-section">
@@ -95,9 +116,14 @@
</div>
</div>`;
// Init Leaflet — restore saved position or default to Bay Area
const defaultCenter = [37.6, -122.1];
const defaultZoom = 9;
// Init Leaflet — restore saved position or use configurable defaults (#115)
let defaultCenter = [37.6, -122.1];
let defaultZoom = 9;
try {
const mapCfg = await (await fetch('/api/config/map')).json();
if (Array.isArray(mapCfg.center) && mapCfg.center.length === 2) defaultCenter = mapCfg.center;
if (typeof mapCfg.zoom === 'number') defaultZoom = mapCfg.zoom;
} catch {}
let initCenter = defaultCenter;
let initZoom = defaultZoom;
const savedView = localStorage.getItem('map-view');
@@ -106,18 +132,16 @@
}
map = L.map('leaflet-map', { zoomControl: true }).setView(initCenter, initZoom);
const DARK_TILES = 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
const LIGHT_TILES = 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png';
const isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
const tileLayer = L.tileLayer(isDark ? DARK_TILES : LIGHT_TILES, {
const tileLayer = L.tileLayer(isDark ? TILE_DARK : TILE_LIGHT, {
attribution: '© OpenStreetMap © CartoDB',
maxZoom: 19,
}).addTo(map);
const _mapThemeObs = new MutationObserver(function () {
const dark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' && window.matchMedia('(prefers-color-scheme: dark)').matches);
tileLayer.setUrl(dark ? DARK_TILES : LIGHT_TILES);
tileLayer.setUrl(dark ? TILE_DARK : TILE_LIGHT);
});
_mapThemeObs.observe(document.documentElement, { attributes: true, attributeFilter: ['data-theme'] });
@@ -128,6 +152,10 @@
userHasMoved = true;
});
map.on('zoomend', () => {
if (filters.hashLabels && !_renderingMarkers) renderMarkers();
});
markerLayer = L.layerGroup().addTo(map);
routeLayer = L.layerGroup().addTo(map);
@@ -152,8 +180,14 @@
// Bind controls
document.getElementById('mcClusters').addEventListener('change', e => { filters.clusters = e.target.checked; renderMarkers(); });
document.getElementById('mcHeatmap').addEventListener('change', e => { toggleHeatmap(e.target.checked); });
document.getElementById('mcMqtt').addEventListener('change', e => { filters.mqttOnly = e.target.checked; renderMarkers(); });
document.getElementById('mcNeighbors').addEventListener('change', e => { filters.neighbors = e.target.checked; renderMarkers(); });
// Hash Labels toggle
const hashLabelEl = document.getElementById('mcHashLabels');
if (hashLabelEl) {
hashLabelEl.checked = filters.hashLabels;
hashLabelEl.addEventListener('change', e => { filters.hashLabels = e.target.checked; localStorage.setItem('meshcore-map-hash-labels', filters.hashLabels); renderMarkers(); });
}
document.getElementById('mcLastHeard').addEventListener('change', e => { filters.lastHeard = e.target.value; loadNodes(); });
// WS for live advert updates
@@ -169,14 +203,42 @@
if (routeHopsJson) {
sessionStorage.removeItem('map-route-hops');
try {
const hopKeys = JSON.parse(routeHopsJson);
drawPacketRoute(hopKeys);
const parsed = JSON.parse(routeHopsJson);
// Support new format {origin, hops} and legacy plain array
if (Array.isArray(parsed)) {
drawPacketRoute(parsed, null);
} else {
drawPacketRoute(parsed.hops || [], parsed.origin || null);
}
} catch {}
}
});
}
function drawPacketRoute(hopKeys) {
function drawPacketRoute(hopKeys, origin) {
// Hide default markers so only the route is visible
if (markerLayer) map.removeLayer(markerLayer);
if (clusterGroup) map.removeLayer(clusterGroup);
if (heatLayer) map.removeLayer(heatLayer);
routeLayer.clearLayers();
// Add close route button
const closeBtn = L.control({ position: 'topright' });
closeBtn.onAdd = function () {
const div = L.DomUtil.create('div', 'leaflet-bar');
div.innerHTML = '<a href="#" title="Close route" style="font-size:18px;font-weight:bold;text-decoration:none;display:block;width:36px;height:36px;line-height:36px;text-align:center;background:var(--bg-secondary,#1e293b);color:var(--text-primary,#e2e8f0);border-radius:4px">✕</a>';
L.DomEvent.on(div, 'click', function (e) {
L.DomEvent.preventDefault(e);
routeLayer.clearLayers();
if (markerLayer) map.addLayer(markerLayer);
if (clusterGroup) map.addLayer(clusterGroup);
map.removeControl(closeBtn);
});
return div;
};
closeBtn.addTo(map);
// Resolve hop short hashes to node positions with geographic disambiguation
const raw = hopKeys.map(hop => {
const hopLower = hop.toLowerCase();
@@ -213,29 +275,52 @@
}
const positions = raw.filter(h => h && h.resolved);
// Resolve and prepend origin node
if (origin) {
let originPos = null;
if (origin.lat != null && origin.lon != null) {
originPos = { lat: origin.lat, lon: origin.lon, name: origin.name || 'Sender', pubkey: origin.pubkey, isOrigin: true };
} else if (origin.pubkey) {
const pk = origin.pubkey.toLowerCase();
const match = nodes.find(n => n.public_key.toLowerCase() === pk || n.public_key.toLowerCase().startsWith(pk));
if (match && match.lat != null && match.lon != null) {
originPos = { lat: match.lat, lon: match.lon, name: origin.name || match.name || 'Sender', pubkey: match.public_key, role: match.role, isOrigin: true };
}
}
if (originPos) positions.unshift(originPos);
}
if (positions.length < 1) return;
// Even a single node is worth showing (zoom to it)
const coords = positions.map(p => [p.lat, p.lon]);
if (positions.length >= 2) {
// Draw route polyline
L.polyline(coords, {
color: '#f59e0b', weight: 3, opacity: 0.8, dashArray: '8 4'
}).addTo(routeLayer);
}
// Add numbered markers at each hop
var labelItems = [];
positions.forEach((p, i) => {
const color = i === 0 ? '#22c55e' : i === positions.length - 1 ? '#ef4444' : '#f59e0b';
const label = i === 0 ? 'Origin' : i === positions.length - 1 ? 'Destination' : `Hop ${i}`;
const isOrigin = i === 0 && p.isOrigin;
const isLast = i === positions.length - 1 && positions.length > 1;
const color = isOrigin ? '#06b6d4' : isLast ? '#ef4444' : i === 0 ? '#22c55e' : '#f59e0b';
const radius = isOrigin ? 14 : 10;
const label = isOrigin ? 'Sender' : isLast ? 'Last Hop' : `Hop ${isOrigin ? i : i}`;
if (isOrigin) {
L.circleMarker([p.lat, p.lon], {
radius: radius + 4, fillColor: 'transparent', fillOpacity: 0, color: '#06b6d4', weight: 2, opacity: 0.6
}).addTo(routeLayer);
}
const marker = L.circleMarker([p.lat, p.lon], {
radius: 10, fillColor: color,
radius: radius, fillColor: color,
fillOpacity: 0.9, color: '#fff', weight: 2
}).addTo(routeLayer);
marker.bindTooltip(`${i + 1}. ${p.name}`, { permanent: true, direction: 'top', className: 'route-tooltip' });
const popupHtml = `<div style="font-size:12px;min-width:160px">
<div style="font-weight:700;margin-bottom:4px">${label}: ${safeEsc(p.name)}</div>
<div style="color:#9ca3af;font-size:11px;margin-bottom:4px">${p.role || 'unknown'}</div>
@@ -244,6 +329,19 @@
${p.pubkey ? `<div style="margin-top:6px"><a href="#/nodes/${p.pubkey}" style="color:var(--accent);font-size:11px">View Node →</a></div>` : ''}
</div>`;
marker.bindPopup(popupHtml, { className: 'route-popup' });
labelItems.push({ latLng: L.latLng(p.lat, p.lon), isLabel: true, text: `${i + 1}. ${p.name}` });
});
// Deconflict labels so overlapping hop names spread out
deconflictLabels(labelItems, map);
labelItems.forEach(function (m) {
var pos = m.adjustedLatLng || m.latLng;
var icon = L.divIcon({ className: 'route-tooltip', html: m.text, iconSize: [null, null], iconAnchor: [0, 0] });
L.marker(pos, { icon: icon, interactive: false }).addTo(routeLayer);
if (m.offset > 2) {
L.polyline([m.latLng, pos], { weight: 1, color: '#475569', opacity: 0.5, dashArray: '3 3' }).addTo(routeLayer);
}
});
// Fit map to route
@@ -256,13 +354,17 @@
async function loadNodes() {
try {
// Load regions from config + observed IATAs
try { REGION_NAMES = await api('/config/regions', { ttl: 3600 }); } catch {}
const data = await api(`/nodes?limit=10000&lastHeard=${filters.lastHeard}`, { ttl: CLIENT_TTL.nodeList });
nodes = data.nodes || [];
buildRoleChecks(data.counts || {});
// Load observers for jump buttons
// Load observers for jump buttons + map markers
const obsData = await api('/observers', { ttl: CLIENT_TTL.observers });
observers = obsData.observers || [];
buildRoleChecks(data.counts || {});
buildJumpButtons();
renderMarkers();
@@ -277,12 +379,14 @@
const el = document.getElementById('mcRoleChecks');
if (!el) return;
el.innerHTML = '';
for (const role of ['repeater', 'companion', 'room', 'sensor']) {
const count = counts[role + 's'] || 0;
const obsCount = observers.filter(o => o.lat && o.lon).length;
const roles = ['repeater', 'companion', 'room', 'sensor', 'observer'];
const shapeMap = { repeater: '◆', companion: '●', room: '■', sensor: '▲', observer: '★' };
for (const role of roles) {
const count = role === 'observer' ? obsCount : (counts[role + 's'] || 0);
const cbId = 'mcRole_' + role;
const lbl = document.createElement('label');
lbl.setAttribute('for', cbId);
const shapeMap = { repeater: '◆', companion: '●', room: '■', sensor: '▲' };
const shape = shapeMap[role] || '●';
lbl.innerHTML = `<input type="checkbox" id="${cbId}" data-role="${role}" ${filters[role] ? 'checked' : ''}> <span style="color:${ROLE_COLORS[role]};font-weight:600;" aria-hidden="true">${shape}</span> ${ROLE_LABELS[role]} <span style="color:var(--text-muted)">(${count})</span>`;
lbl.querySelector('input').addEventListener('change', e => {
@@ -293,7 +397,7 @@
}
}
const REGION_NAMES = { SJC: 'San Jose', SFO: 'San Francisco', OAK: 'Oakland', MTV: 'Mountain View', SCZ: 'Santa Cruz', MRY: 'Monterey', PAO: 'Palo Alto' };
let REGION_NAMES = {};
function buildJumpButtons() {
const el = document.getElementById('mcJumps');
@@ -339,7 +443,65 @@
map.fitBounds(bounds, { padding: [40, 40], maxZoom: 12 });
}
var _renderingMarkers = false;
var _lastDeconflictZoom = null;
function deconflictLabels(markers, mapRef) {
const placed = [];
const PAD = 4;
var overlaps = function(b) {
for (var k = 0; k < placed.length; k++) {
var p = placed[k];
if (b.x < p.x + p.w + PAD && b.x + b.w + PAD > p.x &&
b.y < p.y + p.h + PAD && b.y + b.h + PAD > p.y) return true;
}
return false;
};
// Spiral offsets — 6 rings, 8 directions, up to ~132px
var offsets = [];
for (var ring = 1; ring <= 6; ring++) {
var dist = ring * 22;
for (var angle = 0; angle < 360; angle += 45) {
var rad = angle * Math.PI / 180;
offsets.push([Math.round(Math.cos(rad) * dist), Math.round(Math.sin(rad) * dist)]);
}
}
for (var i = 0; i < markers.length; i++) {
var m = markers[i];
var w = m.isLabel ? 38 : 20;
var h = m.isLabel ? 24 : 20;
var pt = mapRef.latLngToLayerPoint(m.latLng);
var bestPt = pt;
var box = { x: pt.x - w / 2, y: pt.y - h / 2, w: w, h: h };
if (overlaps(box)) {
for (var j = 0; j < offsets.length; j++) {
var tryPt = L.point(pt.x + offsets[j][0], pt.y + offsets[j][1]);
var tryBox = { x: tryPt.x - w / 2, y: tryPt.y - h / 2, w: w, h: h };
if (!overlaps(tryBox)) {
bestPt = tryPt;
box = tryBox;
break;
}
}
}
placed.push(box);
m.adjustedLatLng = mapRef.layerPointToLatLng(bestPt);
m.offset = Math.sqrt(Math.pow(bestPt.x - pt.x, 2) + Math.pow(bestPt.y - pt.y, 2));
}
}
function renderMarkers() {
if (_renderingMarkers) return;
_renderingMarkers = true;
try { _renderMarkersInner(); } finally { _renderingMarkers = false; }
}
function _renderMarkersInner() {
markerLayer.clearLayers();
const filtered = nodes.filter(n => {
@@ -348,16 +510,72 @@
return true;
});
for (const node of filtered) {
const icon = makeMarkerIcon(node.role || 'companion');
const marker = L.marker([node.lat, node.lon], {
icon,
alt: `${node.name || 'Unknown'} (${node.role || 'node'})`,
});
const allMarkers = [];
marker.bindPopup(buildPopup(node), { maxWidth: 280 });
markerLayer.addLayer(marker);
for (const node of filtered) {
const useLabel = node.role === 'repeater' && filters.hashLabels;
const icon = useLabel ? makeRepeaterLabelIcon(node) : makeMarkerIcon(node.role || 'companion');
const latLng = L.latLng(node.lat, node.lon);
allMarkers.push({ latLng, node, icon, isLabel: useLabel, popupFn: function() { return buildPopup(node); }, alt: (node.name || 'Unknown') + ' (' + (node.role || 'node') + ')' });
}
// Add observer markers
if (filters.observer) {
for (const obs of observers) {
if (!obs.lat || !obs.lon) continue;
const icon = makeMarkerIcon('observer');
const latLng = L.latLng(obs.lat, obs.lon);
allMarkers.push({ latLng, node: obs, icon, isLabel: false, popupFn: function() { return buildObserverPopup(obs); }, alt: (obs.name || obs.id || 'Unknown') + ' (observer)' });
}
}
// Deconflict ALL markers
if (allMarkers.length > 0) {
deconflictLabels(allMarkers, map);
}
for (const m of allMarkers) {
const pos = m.adjustedLatLng || m.latLng;
const marker = L.marker(pos, { icon: m.icon, alt: m.alt });
marker.bindPopup(m.popupFn(), { maxWidth: 280 });
markerLayer.addLayer(marker);
if (m.offset > 10) {
const line = L.polyline([m.latLng, pos], {
color: '#ef4444', weight: 2, dashArray: '6,4', opacity: 0.85
});
markerLayer.addLayer(line);
// Small dot at true GPS position
const dot = L.circleMarker(m.latLng, {
radius: 3, fillColor: '#ef4444', fillOpacity: 0.9, stroke: true, color: '#fff', weight: 1
});
markerLayer.addLayer(dot);
}
}
}
function buildObserverPopup(obs) {
const name = safeEsc(obs.name || obs.id || 'Unknown');
const iata = obs.iata ? `<span class="badge-region">${safeEsc(obs.iata)}</span>` : '';
const lastSeen = obs.last_seen ? timeAgo(obs.last_seen) : '—';
const packets = (obs.packet_count || 0).toLocaleString();
const loc = `${obs.lat.toFixed(5)}, ${obs.lon.toFixed(5)}`;
const roleBadge = `<span style="display:inline-block;padding:2px 8px;border-radius:12px;font-size:11px;font-weight:600;background:${ROLE_COLORS.observer};color:#fff;">OBSERVER</span>`;
return `
<div class="map-popup" style="font-family:var(--font);min-width:180px;">
<h3 style="font-weight:700;font-size:14px;margin:0 0 4px;">${name}</h3>
${roleBadge} ${iata}
<dl style="margin-top:8px;font-size:12px;">
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Location</dt>
<dd style="margin-left:88px;padding:2px 0;">${loc}</dd>
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Last Seen</dt>
<dd style="margin-left:88px;padding:2px 0;">${lastSeen}</dd>
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Packets</dt>
<dd style="margin-left:88px;padding:2px 0;">${packets}</dd>
</dl>
<a href="#/observers/${encodeURIComponent(obs.id || obs.observer_id)}" style="display:block;margin-top:8px;font-size:12px;color:var(--accent);">View Detail →</a>
</div>`;
}
function buildPopup(node) {
@@ -365,12 +583,17 @@
const loc = (node.lat && node.lon) ? `${node.lat.toFixed(5)}, ${node.lon.toFixed(5)}` : '—';
const lastAdvert = node.last_seen ? timeAgo(node.last_seen) : '—';
const roleBadge = `<span style="display:inline-block;padding:2px 8px;border-radius:12px;font-size:11px;font-weight:600;background:${ROLE_COLORS[node.role] || '#4b5563'};color:#fff;">${(node.role || 'unknown').toUpperCase()}</span>`;
const hs = node.hash_size || 1;
const hashPrefix = node.public_key ? node.public_key.slice(0, hs * 2).toUpperCase() : '—';
const hashPrefixRow = `<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Hash Prefix</dt>
<dd style="font-family:var(--mono);font-size:11px;font-weight:700;margin-left:88px;padding:2px 0;">${safeEsc(hashPrefix)} <span style="font-weight:400;color:var(--text-muted);">(${hs}B)</span></dd>`;
return `
<div class="map-popup" style="font-family:var(--font);min-width:180px;">
<h3 style="font-weight:700;font-size:14px;margin:0 0 4px;">${safeEsc(node.name || 'Unknown')}</h3>
${roleBadge}
<dl style="margin-top:8px;font-size:12px;">
${hashPrefixRow}
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Key</dt>
<dd style="font-family:var(--mono);font-size:11px;margin-left:88px;padding:2px 0;">${safeEsc(key)}</dd>
<dt style="color:var(--text-muted);float:left;clear:left;width:80px;padding:2px 0;">Location</dt>

View File

@@ -55,7 +55,7 @@
<div style="margin-bottom:12px">
<a href="#/nodes/${encodeURIComponent(n.public_key)}" style="color:var(--accent);text-decoration:none;font-size:12px">← Back to ${nodeName}</a>
<h2 style="margin:4px 0 2px;font-size:18px">📊 ${nodeName} — Analytics</h2>
<div style="color:var(--text-muted);font-size:11px">${n.role || 'Unknown role'} · ${s.totalPackets} packets in ${days}d window</div>
<div style="color:var(--text-muted);font-size:11px">${n.role || 'Unknown role'} · ${s.totalTransmissions || s.totalPackets} packets in ${days}d window</div>
</div>
<div class="analytics-time-range" id="timeRangeBtns">

View File

@@ -24,7 +24,7 @@
let wsHandler = null;
let detailMap = null;
const ROLE_COLORS = { repeater: '#3b82f6', room: '#6b7280', companion: '#22c55e', sensor: '#f59e0b' };
// ROLE_COLORS loaded from shared roles.js
const TABS = [
{ key: 'all', label: 'All' },
{ key: 'repeater', label: 'Repeaters' },
@@ -35,6 +35,8 @@
let directNode = null; // set when navigating directly to #/nodes/:pubkey
let regionChangeHandler = null;
function init(app, routeParam) {
directNode = routeParam || null;
@@ -66,12 +68,16 @@
<input type="text" class="nodes-search" id="nodeSearch" placeholder="Search nodes by name…" aria-label="Search nodes by name">
<div class="nodes-counts" id="nodeCounts"></div>
</div>
<div id="nodesRegionFilter" class="region-filter-container"></div>
<div class="split-layout">
<div class="panel-left" id="nodesLeft"></div>
<div class="panel-right empty" id="nodesRight"><span>Select a node to view details</span></div>
</div>
</div>`;
RegionFilter.init(document.getElementById('nodesRegionFilter'));
regionChangeHandler = RegionFilter.onChange(function () { loadNodes(); });
document.getElementById('nodeSearch').addEventListener('input', debounce(e => {
search = e.target.value;
loadNodes();
@@ -107,9 +113,7 @@
// Repeaters/rooms: flood advert every 12-24h, so degraded after 24h, silent after 72h
// Companions/sensors: user-initiated adverts, shorter thresholds
const role = (n.role || '').toLowerCase();
const isInfra = role === 'repeater' || role === 'room';
const degradedMs = isInfra ? 86400000 : 3600000; // 24h : 1h
const silentMs = isInfra ? 259200000 : 86400000; // 72h : 24h
const { degradedMs, silentMs } = getHealthThresholds(role);
const statusLabel = statusAge < degradedMs ? '🟢 Active' : statusAge < silentMs ? '🟡 Degraded' : '🔴 Silent';
body.innerHTML = `
@@ -130,7 +134,7 @@
<dl class="detail-meta">
<dt>Last Heard</dt><dd>${lastHeard ? timeAgo(lastHeard) : (n.last_seen ? timeAgo(n.last_seen) : '—')}</dd>
<dt>First Seen</dt><dd>${n.first_seen ? new Date(n.first_seen).toLocaleString() : '—'}</dd>
<dt>Total Packets</dt><dd>${stats.totalPackets || n.advert_count || 0}</dd>
<dt>Total Packets</dt><dd>${stats.totalTransmissions || stats.totalPackets || n.advert_count || 0}${stats.totalObservations && stats.totalObservations !== (stats.totalTransmissions || stats.totalPackets) ? ' <span class="text-muted" style="font-size:0.85em">(seen ' + stats.totalObservations + '×)</span>' : ''}</dd>
<dt>Packets Today</dt><dd>${stats.packetsToday || 0}</dd>
${stats.avgSnr != null ? `<dt>Avg SNR</dt><dd>${stats.avgSnr.toFixed(1)} dB</dd>` : ''}
${stats.avgHops ? `<dt>Avg Hops</dt><dd>${stats.avgHops}</dd>` : ''}
@@ -153,6 +157,11 @@
</table>
</div>` : ''}
<div class="node-full-card" id="fullPathsSection">
<h4>Paths Through This Node</h4>
<div id="fullPathsContent"><div class="text-muted" style="padding:8px"><span class="spinner"></span> Loading paths…</div></div>
</div>
<div class="node-full-card">
<h4>Recent Packets (${adverts.length})</h4>
<div class="node-activity-list">
@@ -163,10 +172,11 @@
const obs = p.observer_name || p.observer_id;
const snr = p.snr != null ? ` · SNR ${p.snr}dB` : '';
const rssi = p.rssi != null ? ` · RSSI ${p.rssi}dBm` : '';
const obsBadge = p.observation_count > 1 ? ` <span class="badge badge-obs" title="Seen ${p.observation_count} times">👁 ${p.observation_count}</span>` : '';
return `<div class="node-activity-item">
<span class="node-activity-time">${timeAgo(p.timestamp)}</span>
<span>${typeLabel}${detail}${obs ? ' via ' + escapeHtml(obs) : ''}${snr}${rssi}</span>
<a href="#/packets/id/${p.id}" class="ch-analyze-link" style="margin-left:8px;font-size:0.8em">Analyze </a>
<span>${typeLabel}${detail}${obsBadge}${obs ? ' via ' + escapeHtml(obs) : ''}${snr}${rssi}</span>
<a href="#/packets/${p.hash}" class="ch-analyze-link" style="margin-left:8px;font-size:0.8em">Analyze →</a>
</div>`;
}).join('') : '<div class="text-muted">No recent packets</div>'}
</div>
@@ -209,15 +219,55 @@
} catch {}
}
// Fetch paths through this node (full-screen view)
api('/nodes/' + encodeURIComponent(n.public_key) + '/paths', { ttl: CLIENT_TTL.nodeDetail }).then(pathData => {
const el = document.getElementById('fullPathsContent');
if (!el) return;
if (!pathData || !pathData.paths || !pathData.paths.length) {
el.innerHTML = '<div class="text-muted" style="padding:8px">No paths observed through this node</div>';
return;
}
document.querySelector('#fullPathsSection h4').textContent = `Paths Through This Node (${pathData.totalPaths} unique, ${pathData.totalTransmissions} transmissions)`;
const COLLAPSE_LIMIT = 10;
function renderPaths(paths) {
return paths.map(p => {
const chain = p.hops.map(h => {
const isThis = h.pubkey === n.public_key;
const name = escapeHtml(h.name || h.prefix);
const link = h.pubkey ? `<a href="#/nodes/${encodeURIComponent(h.pubkey)}" style="${isThis ? 'font-weight:700;color:var(--accent, #3b82f6)' : ''}">${name}</a>` : `<span>${name}</span>`;
return link;
}).join(' → ');
return `<div style="padding:6px 0;border-bottom:1px solid var(--border);font-size:12px">
<div>${chain}</div>
<div style="color:var(--text-muted);margin-top:2px">${p.count}× · last ${timeAgo(p.lastSeen)} · <a href="#/packets/${p.sampleHash}" class="ch-analyze-link">Analyze →</a></div>
</div>`;
}).join('');
}
if (pathData.paths.length <= COLLAPSE_LIMIT) {
el.innerHTML = renderPaths(pathData.paths);
} else {
el.innerHTML = renderPaths(pathData.paths.slice(0, COLLAPSE_LIMIT)) +
`<button id="showAllFullPaths" class="btn-primary" style="margin-top:8px;font-size:11px;padding:4px 12px">Show all ${pathData.paths.length} paths</button>`;
document.getElementById('showAllFullPaths').addEventListener('click', function() {
el.innerHTML = renderPaths(pathData.paths);
});
}
}).catch(() => {
const el = document.getElementById('fullPathsContent');
if (el) el.innerHTML = '<div class="text-muted" style="padding:8px">Failed to load paths</div>';
});
} catch (e) {
body.innerHTML = `<div class="text-muted" style="padding:40px">Failed to load node: ${e.message}</div>`;
}
}
function destroy() {
function destroy() {
if (wsHandler) offWS(wsHandler);
wsHandler = null;
if (detailMap) { detailMap.remove(); detailMap = null; }
if (regionChangeHandler) RegionFilter.offChange(regionChangeHandler);
regionChangeHandler = null;
nodes = [];
selectedKey = null;
}
@@ -228,10 +278,19 @@
if (activeTab !== 'all') params.set('role', activeTab);
if (search) params.set('search', search);
if (lastHeard) params.set('lastHeard', lastHeard);
const rp = RegionFilter.getRegionParam();
if (rp) params.set('region', rp);
const data = await api('/nodes?' + params, { ttl: CLIENT_TTL.nodeList });
nodes = data.nodes || [];
counts = data.counts || {};
// Defensive filter: hide nodes with obviously corrupted data
nodes = nodes.filter(n => {
if (n.public_key && n.public_key.length < 16) return false;
if (!n.name && !n.advert_count) return false;
return true;
});
// Ensure claimed nodes are always present even if not in current page
const myNodes = JSON.parse(localStorage.getItem('meshcore-my-nodes') || '[]');
const existingKeys = new Set(nodes.map(n => n.public_key));
@@ -426,11 +485,9 @@
const lastHeard = stats.lastHeard;
const statusAge = lastHeard ? (Date.now() - new Date(lastHeard).getTime()) : Infinity;
const role = (n.role || '').toLowerCase();
const isInfra = role === 'repeater' || role === 'room';
const degradedMs = isInfra ? 86400000 : 3600000;
const silentMs = isInfra ? 259200000 : 86400000;
const { degradedMs, silentMs } = getHealthThresholds(role);
const statusLabel = statusAge < degradedMs ? '🟢 Active' : statusAge < silentMs ? '🟡 Degraded' : '🔴 Silent';
const totalPackets = stats.totalPackets || n.advert_count || 0;
const totalPackets = stats.totalTransmissions || stats.totalPackets || n.advert_count || 0;
panel.innerHTML = `
<div class="node-detail">
@@ -470,6 +527,11 @@
</div>
</div>` : ''}
<div class="node-detail-section" id="pathsSection">
<h4>Paths Through This Node</h4>
<div id="pathsContent"><div class="text-muted" style="padding:8px"><span class="spinner"></span> Loading paths…</div></div>
</div>
<div class="node-detail-section">
<h4>Recent Packets (${adverts.length})</h4>
<div id="advertTimeline">
@@ -484,9 +546,10 @@
<span class="advert-dot" style="background:${roleColor}"></span>
<div class="advert-info">
<strong>${timeAgo(a.timestamp)}</strong> ${icon} ${pType}${detail}
${a.observation_count > 1 ? ' <span class="badge badge-obs">👁 ' + a.observation_count + '</span>' : ''}
${obs ? ' via ' + escapeHtml(obs) : ''}
${a.snr != null ? ` · SNR ${a.snr}dB` : ''}${a.rssi != null ? ` · RSSI ${a.rssi}dBm` : ''}
<br><a href="#/packets/id/${a.id}" class="ch-analyze-link">Analyze </a>
<br><a href="#/packets/${a.hash}" class="ch-analyze-link">Analyze </a>
</div>
</div>`;
}).join('') : '<div class="text-muted" style="padding:8px">No recent packets</div>'}
@@ -530,6 +593,46 @@
setTimeout(() => btn.textContent = '📋 Copy URL', 2000);
}).catch(() => {});
});
// Fetch paths through this node
api('/nodes/' + encodeURIComponent(n.public_key) + '/paths', { ttl: CLIENT_TTL.nodeDetail }).then(pathData => {
const el = document.getElementById('pathsContent');
if (!el) return;
if (!pathData || !pathData.paths || !pathData.paths.length) {
el.innerHTML = '<div class="text-muted" style="padding:8px">No paths observed through this node</div>';
document.querySelector('#pathsSection h4').textContent = 'Paths Through This Node';
return;
}
document.querySelector('#pathsSection h4').textContent = `Paths Through This Node (${pathData.totalPaths} unique path${pathData.totalPaths !== 1 ? 's' : ''}, ${pathData.totalTransmissions} transmissions)`;
const COLLAPSE_LIMIT = 10;
const showAll = pathData.paths.length <= COLLAPSE_LIMIT;
function renderPaths(paths) {
return paths.map(p => {
const chain = p.hops.map(h => {
const isThis = h.pubkey === n.public_key;
const name = escapeHtml(h.name || h.prefix);
const link = h.pubkey ? `<a href="#/nodes/${encodeURIComponent(h.pubkey)}" style="${isThis ? 'font-weight:700;color:var(--accent, #3b82f6)' : ''}">${name}</a>` : `<span>${name}</span>`;
return link;
}).join(' → ');
return `<div style="padding:6px 0;border-bottom:1px solid var(--border);font-size:12px">
<div>${chain}</div>
<div style="color:var(--text-muted);margin-top:2px">${p.count}× · last ${timeAgo(p.lastSeen)} · <a href="#/packets/${p.sampleHash}" class="ch-analyze-link">Analyze →</a></div>
</div>`;
}).join('');
}
if (showAll) {
el.innerHTML = renderPaths(pathData.paths);
} else {
el.innerHTML = renderPaths(pathData.paths.slice(0, COLLAPSE_LIMIT)) +
`<button id="showAllPaths" class="btn-primary" style="margin-top:8px;font-size:11px;padding:4px 12px">Show all ${pathData.paths.length} paths</button>`;
document.getElementById('showAllPaths').addEventListener('click', function() {
el.innerHTML = renderPaths(pathData.paths);
});
}
}).catch(() => {
const el = document.getElementById('pathsContent');
if (el) el.innerHTML = '<div class="text-muted" style="padding:8px">Failed to load paths</div>';
});
}
registerPage('nodes', { init, destroy });

View File

@@ -97,8 +97,8 @@
// Health status
const ago = obs.last_seen ? Date.now() - new Date(obs.last_seen).getTime() : Infinity;
const statusCls = ago < 600000 ? 'health-green' : ago < 3600000 ? 'health-yellow' : 'health-red';
const statusLabel = ago < 600000 ? 'Online' : ago < 3600000 ? 'Stale' : 'Offline';
const statusCls = ago < 600000 ? 'health-green' : ago < HEALTH_THRESHOLDS.nodeDegradedMs ? 'health-yellow' : 'health-red';
const statusLabel = ago < 600000 ? 'Online' : ago < HEALTH_THRESHOLDS.nodeDegradedMs ? 'Stale' : 'Offline';
el.innerHTML = `
<div class="obs-info-grid" style="display:grid;grid-template-columns:repeat(auto-fit,minmax(200px,1fr));gap:12px;margin-bottom:20px">

View File

@@ -5,6 +5,7 @@
let observers = [];
let wsHandler = null;
let refreshTimer = null;
let regionChangeHandler = null;
function init(app) {
app.innerHTML = `
@@ -13,8 +14,11 @@
<h2>Observer Status</h2>
<button class="btn-icon" data-action="obs-refresh" title="Refresh" aria-label="Refresh observers">🔄</button>
</div>
<div id="obsRegionFilter" class="region-filter-container"></div>
<div id="obsContent"><div class="text-center text-muted" style="padding:40px">Loading…</div></div>
</div>`;
RegionFilter.init(document.getElementById('obsRegionFilter'));
regionChangeHandler = RegionFilter.onChange(function () { render(); });
loadObservers();
// Event delegation for data-action buttons
app.addEventListener('click', function (e) {
@@ -33,6 +37,8 @@
wsHandler = null;
if (refreshTimer) clearInterval(refreshTimer);
refreshTimer = null;
if (regionChangeHandler) RegionFilter.offChange(regionChangeHandler);
regionChangeHandler = null;
observers = [];
}
@@ -69,34 +75,39 @@
}
function sparkBar(count, max) {
const aria = `role="meter" aria-valuenow="${count}" aria-valuemin="0" aria-valuemax="${max}" aria-label="Packet rate"`;
if (max === 0) return `<div class="spark-bar" ${aria}><div class="spark-fill" style="width:0"></div></div>`;
if (max === 0) return `<span class="text-muted">0/hr</span>`;
const pct = Math.min(100, Math.round((count / max) * 100));
return `<div class="spark-bar" ${aria}><div class="spark-fill" style="width:${pct}%"></div><span class="spark-label">${count}/hr</span></div>`;
return `<span style="display:inline-flex;align-items:center;gap:6px;white-space:nowrap"><span style="display:inline-block;width:60px;height:12px;background:var(--border);border-radius:3px;overflow:hidden;vertical-align:middle"><span style="display:block;height:100%;width:${pct}%;background:linear-gradient(90deg,#3b82f6,#60a5fa);border-radius:3px"></span></span><span style="font-size:11px">${count}/hr</span></span>`;
}
function render() {
const el = document.getElementById('obsContent');
if (!el) return;
if (observers.length === 0) {
// Apply region filter
const selectedRegions = RegionFilter.getSelected();
const filtered = selectedRegions
? observers.filter(o => o.iata && selectedRegions.includes(o.iata))
: observers;
if (filtered.length === 0) {
el.innerHTML = '<div class="text-center text-muted" style="padding:40px">No observers found.</div>';
return;
}
const maxPktsHr = Math.max(1, ...observers.map(o => o.packetsLastHour || 0));
const maxPktsHr = Math.max(1, ...filtered.map(o => o.packetsLastHour || 0));
// Summary counts
const online = observers.filter(o => healthStatus(o.last_seen).cls === 'health-green').length;
const stale = observers.filter(o => healthStatus(o.last_seen).cls === 'health-yellow').length;
const offline = observers.filter(o => healthStatus(o.last_seen).cls === 'health-red').length;
const online = filtered.filter(o => healthStatus(o.last_seen).cls === 'health-green').length;
const stale = filtered.filter(o => healthStatus(o.last_seen).cls === 'health-yellow').length;
const offline = filtered.filter(o => healthStatus(o.last_seen).cls === 'health-red').length;
el.innerHTML = `
<div class="obs-summary">
<span class="obs-stat"><span class="health-dot health-green">●</span> ${online} Online</span>
<span class="obs-stat"><span class="health-dot health-yellow">▲</span> ${stale} Stale</span>
<span class="obs-stat"><span class="health-dot health-red">✕</span> ${offline} Offline</span>
<span class="obs-stat">📡 ${observers.length} Total</span>
<span class="obs-stat">📡 ${filtered.length} Total</span>
</div>
<div class="obs-table-scroll"><table class="data-table obs-table" id="obsTable">
<caption class="sr-only">Observer status and statistics</caption>
@@ -104,7 +115,7 @@
<th>Status</th><th>Name</th><th>Region</th><th>Last Seen</th>
<th>Packets</th><th>Packets/Hour</th><th>Uptime</th>
</tr></thead>
<tbody>${observers.map(o => {
<tbody>${filtered.map(o => {
const h = healthStatus(o.last_seen);
const shape = h.cls === 'health-green' ? '●' : h.cls === 'health-yellow' ? '▲' : '✕';
return `<tr style="cursor:pointer" onclick="location.hash='#/observers/${encodeURIComponent(o.id)}'">
@@ -113,7 +124,7 @@
<td>${o.iata ? `<span class="badge-region">${o.iata}</span>` : '—'}</td>
<td>${timeAgo(o.last_seen)}</td>
<td>${(o.packet_count || 0).toLocaleString()}</td>
<td class="col-spark" style="max-width:none;overflow:visible;min-width:80px">${sparkBar(o.packetsLastHour || 0, maxPktsHr)}</td>
<td>${sparkBar(o.packetsLastHour || 0, maxPktsHr)}</td>
<td>${uptimeStr(o.first_seen)}</td>
</tr>`;
}).join('')}</tbody>

File diff suppressed because it is too large Load Diff

216
public/region-filter.js Normal file
View File

@@ -0,0 +1,216 @@
/* === MeshCore Analyzer — region-filter.js (shared region filter component) === */
'use strict';
(function () {
var LS_KEY = 'meshcore-region-filter';
var _regions = {}; // { code: label }
var _selected = null; // Set of selected region codes, null = all
var _listeners = [];
var _loaded = false;
function loadFromStorage() {
try {
var stored = JSON.parse(localStorage.getItem(LS_KEY));
if (Array.isArray(stored) && stored.length > 0) return new Set(stored);
} catch (e) { /* ignore */ }
return null; // null = all selected
}
function saveToStorage() {
if (!_selected) {
localStorage.removeItem(LS_KEY);
} else {
localStorage.setItem(LS_KEY, JSON.stringify(Array.from(_selected)));
}
}
_selected = loadFromStorage();
/** Fetch regions from server */
async function fetchRegions() {
if (_loaded) return _regions;
try {
var data = await fetch('/api/config/regions').then(function (r) { return r.json(); });
_regions = data || {};
_loaded = true;
// If stored selection has codes no longer valid, clean up
if (_selected) {
var codes = Object.keys(_regions);
var cleaned = new Set();
_selected.forEach(function (c) { if (codes.includes(c)) cleaned.add(c); });
_selected = cleaned.size > 0 ? cleaned : null;
saveToStorage();
}
} catch (e) {
_regions = {};
}
return _regions;
}
/** Get selected regions as array, or null if all */
function getSelected() {
if (!_selected || _selected.size === 0) return null;
return Array.from(_selected);
}
/** Get region query param string for API calls: "SJC,SFO" or empty */
function getRegionParam() {
var sel = getSelected();
return sel ? sel.join(',') : '';
}
/** Build query string fragment: "&region=SJC,SFO" or "" */
function regionQueryString() {
var p = getRegionParam();
return p ? '&region=' + encodeURIComponent(p) : '';
}
/** Handle a region toggle (shared logic for both pill and dropdown modes) */
function toggleRegion(region, codes, container) {
if (region === '__all__') {
_selected = null;
} else {
if (!_selected) {
_selected = new Set([region]);
} else if (_selected.has(region)) {
_selected.delete(region);
if (_selected.size === 0) _selected = null;
} else {
_selected.add(region);
}
if (_selected && _selected.size === codes.length) _selected = null;
}
saveToStorage();
render(container);
_listeners.forEach(function (fn) { fn(getSelected()); });
}
/** Build summary label for dropdown trigger */
function dropdownLabel(codes) {
if (!_selected) return 'All Regions';
var sel = Array.from(_selected);
if (sel.length === 0) return 'All Regions';
if (sel.length <= 2) return sel.join(', ');
return sel.length + ' Regions';
}
/** Render pill bar mode (≤4 regions) */
function renderPills(container, codes) {
var allSelected = !_selected;
var html = '<div class="region-filter-bar" role="group" aria-label="Region filter">';
html += '<span class="region-filter-label" id="region-filter-label">Region:</span>';
html += '<button class="region-pill' + (allSelected ? ' region-pill-active' : '') +
'" data-region="__all__" role="checkbox" aria-checked="' + allSelected + '">All</button>';
codes.forEach(function (code) {
var label = _regions[code] || code;
var active = allSelected || (_selected && _selected.has(code));
html += '<button class="region-pill' + (active ? ' region-pill-active' : '') +
'" data-region="' + code + '" role="checkbox" aria-checked="' + !!active + '">' + label + '</button>';
});
html += '</div>';
container.innerHTML = html;
container.onclick = function (e) {
var btn = e.target.closest('[data-region]');
if (!btn) return;
toggleRegion(btn.dataset.region, codes, container);
};
}
/** Render dropdown mode (>4 regions) */
function renderDropdown(container, codes) {
var allSelected = !_selected;
var html = '<div class="region-dropdown-wrap" role="group" aria-label="Region filter">';
html += '<button class="region-dropdown-trigger" aria-haspopup="listbox" aria-expanded="false">' +
dropdownLabel(codes) + ' ▾</button>';
html += '<div class="region-dropdown-menu" role="listbox" aria-label="Select regions" hidden>';
html += '<label class="region-dropdown-item"><input type="checkbox" data-region="__all__"' +
(allSelected ? ' checked' : '') + '> <strong>All</strong></label>';
codes.forEach(function (code) {
var label = _regions[code] ? (code + ' - ' + _regions[code]) : code;
var active = allSelected || (_selected && _selected.has(code));
html += '<label class="region-dropdown-item"><input type="checkbox" data-region="' + code + '"' +
(active ? ' checked' : '') + '> ' + label + '</label>';
});
html += '</div></div>';
container.innerHTML = html;
var trigger = container.querySelector('.region-dropdown-trigger');
var menu = container.querySelector('.region-dropdown-menu');
trigger.onclick = function () {
var open = !menu.hidden;
menu.hidden = open;
trigger.setAttribute('aria-expanded', String(!open));
};
menu.onchange = function (e) {
var input = e.target;
if (!input.dataset.region) return;
toggleRegion(input.dataset.region, codes, container);
};
// Close on outside click
function onDocClick(e) {
if (!container.contains(e.target)) {
menu.hidden = true;
trigger.setAttribute('aria-expanded', 'false');
}
}
document.addEventListener('click', onDocClick, true);
container._regionCleanup = function () {
document.removeEventListener('click', onDocClick, true);
};
}
/** Render the filter bar into a container element */
function render(container) {
// Clean up previous outside-click listener if any
if (container._regionCleanup) { container._regionCleanup(); container._regionCleanup = null; }
var codes = Object.keys(_regions);
if (codes.length < 2) {
container.innerHTML = '';
container.style.display = 'none';
return;
}
container.style.display = '';
if (codes.length > 4 || container._forceDropdown) {
renderDropdown(container, codes);
} else {
renderPills(container, codes);
}
}
/** Subscribe to selection changes. Callback receives selected array or null */
function onChange(fn) {
_listeners.push(fn);
return fn;
}
/** Unsubscribe */
function offChange(fn) {
_listeners = _listeners.filter(function (f) { return f !== fn; });
}
/** Initialize filter in a container, fetch regions, render, return promise.
* Options: { dropdown: true } to force dropdown mode regardless of region count */
async function initFilter(container, opts) {
if (opts && opts.dropdown) container._forceDropdown = true;
await fetchRegions();
render(container);
}
// Expose globally
window.RegionFilter = {
init: initFilter,
render: render,
getSelected: getSelected,
getRegionParam: getRegionParam,
regionQueryString: regionQueryString,
onChange: onChange,
offChange: offChange,
fetchRegions: fetchRegions
};
})();

131
public/roles.js Normal file
View File

@@ -0,0 +1,131 @@
/* === MeshCore Analyzer — roles.js (shared config module) === */
'use strict';
/*
* Centralized roles, thresholds, tile URLs, and UI constants.
* Loaded BEFORE all page scripts via index.html.
* Defaults are set synchronously; server config overrides arrive via fetch.
*/
(function () {
// ─── Role definitions ───
window.ROLE_COLORS = {
repeater: '#dc2626', companion: '#2563eb', room: '#16a34a',
sensor: '#d97706', observer: '#8b5cf6', unknown: '#6b7280'
};
window.ROLE_LABELS = {
repeater: 'Repeaters', companion: 'Companions', room: 'Room Servers',
sensor: 'Sensors', observer: 'Observers'
};
window.ROLE_STYLE = {
repeater: { color: '#dc2626', shape: 'diamond', radius: 10, weight: 2 },
companion: { color: '#2563eb', shape: 'circle', radius: 8, weight: 2 },
room: { color: '#16a34a', shape: 'square', radius: 9, weight: 2 },
sensor: { color: '#d97706', shape: 'triangle', radius: 8, weight: 2 },
observer: { color: '#8b5cf6', shape: 'star', radius: 11, weight: 2 }
};
window.ROLE_EMOJI = {
repeater: '◆', companion: '●', room: '■', sensor: '▲', observer: '★'
};
window.ROLE_SORT = ['repeater', 'companion', 'room', 'sensor', 'observer'];
// ─── Health thresholds (ms) ───
window.HEALTH_THRESHOLDS = {
infraDegradedMs: 86400000, // 24h
infraSilentMs: 259200000, // 72h
nodeDegradedMs: 3600000, // 1h
nodeSilentMs: 86400000 // 24h
};
// Helper: get degraded/silent thresholds for a role
window.getHealthThresholds = function (role) {
var isInfra = role === 'repeater' || role === 'room';
return {
degradedMs: isInfra ? HEALTH_THRESHOLDS.infraDegradedMs : HEALTH_THRESHOLDS.nodeDegradedMs,
silentMs: isInfra ? HEALTH_THRESHOLDS.infraSilentMs : HEALTH_THRESHOLDS.nodeSilentMs
};
};
// ─── Tile URLs ───
window.TILE_DARK = 'https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png';
window.TILE_LIGHT = 'https://{s}.basemaps.cartocdn.com/light_all/{z}/{x}/{y}{r}.png';
window.getTileUrl = function () {
var isDark = document.documentElement.getAttribute('data-theme') === 'dark' ||
(document.documentElement.getAttribute('data-theme') !== 'light' &&
window.matchMedia('(prefers-color-scheme: dark)').matches);
return isDark ? TILE_DARK : TILE_LIGHT;
};
// ─── SNR thresholds ───
window.SNR_THRESHOLDS = { excellent: 6, good: 0 };
// ─── Distance thresholds (km) ───
window.DIST_THRESHOLDS = { local: 50, regional: 200 };
// ─── MAX_HOP_DIST (degrees, ~200km ≈ 1.8°) ───
window.MAX_HOP_DIST = 1.8;
// ─── Result limits ───
window.LIMITS = {
topNodes: 15,
topPairs: 12,
topRingNodes: 8,
topSenders: 10,
topCollisionNodes: 10,
recentReplay: 8,
feedMax: 25
};
// ─── Performance thresholds ───
window.PERF_SLOW_MS = 100;
// ─── WebSocket reconnect delay (ms) ───
window.WS_RECONNECT_MS = 3000;
// ─── Propagation buffer (ms) for realistic mode ───
window.PROPAGATION_BUFFER_MS = 5000;
// ─── Cache invalidation debounce (ms) ───
window.CACHE_INVALIDATE_MS = 5000;
// ─── External URLs ───
window.EXTERNAL_URLS = {
flasher: 'https://flasher.meshcore.co.uk/'
};
// ─── Fetch server overrides ───
window.MeshConfigReady = fetch('/api/config/client').then(function (r) { return r.json(); }).then(function (cfg) {
if (cfg.roles) {
if (cfg.roles.colors) Object.assign(ROLE_COLORS, cfg.roles.colors);
if (cfg.roles.labels) Object.assign(ROLE_LABELS, cfg.roles.labels);
if (cfg.roles.style) {
for (var k in cfg.roles.style) ROLE_STYLE[k] = Object.assign(ROLE_STYLE[k] || {}, cfg.roles.style[k]);
}
if (cfg.roles.emoji) Object.assign(ROLE_EMOJI, cfg.roles.emoji);
if (cfg.roles.sort) window.ROLE_SORT = cfg.roles.sort;
}
if (cfg.healthThresholds) Object.assign(HEALTH_THRESHOLDS, cfg.healthThresholds);
if (cfg.tiles) {
if (cfg.tiles.dark) window.TILE_DARK = cfg.tiles.dark;
if (cfg.tiles.light) window.TILE_LIGHT = cfg.tiles.light;
}
if (cfg.snrThresholds) Object.assign(SNR_THRESHOLDS, cfg.snrThresholds);
if (cfg.distThresholds) Object.assign(DIST_THRESHOLDS, cfg.distThresholds);
if (cfg.maxHopDist != null) window.MAX_HOP_DIST = cfg.maxHopDist;
if (cfg.limits) Object.assign(LIMITS, cfg.limits);
if (cfg.perfSlowMs != null) window.PERF_SLOW_MS = cfg.perfSlowMs;
if (cfg.wsReconnectMs != null) window.WS_RECONNECT_MS = cfg.wsReconnectMs;
if (cfg.cacheInvalidateMs != null) window.CACHE_INVALIDATE_MS = cfg.cacheInvalidateMs;
if (cfg.externalUrls) Object.assign(EXTERNAL_URLS, cfg.externalUrls);
if (cfg.propagationBufferMs != null) window.PROPAGATION_BUFFER_MS = cfg.propagationBufferMs;
// Sync ROLE_STYLE colors with ROLE_COLORS
for (var role in ROLE_STYLE) {
if (ROLE_COLORS[role]) ROLE_STYLE[role].color = ROLE_COLORS[role];
}
}).catch(function () { /* use defaults */ });
})();

View File

@@ -188,22 +188,34 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
display: flex; flex-wrap: wrap; gap: 6px; margin-bottom: 8px; align-items: center;
}
.filter-bar input, .filter-bar select {
padding: 4px 8px; border: 1px solid var(--border); border-radius: 4px;
font-size: 12px; background: var(--input-bg); color: var(--text); font-family: var(--font);
padding: 6px 10px; border: 1px solid var(--border); border-radius: 6px;
font-size: 13px; background: var(--input-bg); color: var(--text); font-family: var(--font);
height: 34px; box-sizing: border-box; line-height: 1;
}
.filter-bar input { width: 120px; }
.filter-bar select { min-width: 90px; }
.filter-bar .btn {
padding: 6px 14px; border: 1px solid var(--border); border-radius: 6px;
background: var(--input-bg); cursor: pointer; font-size: 13px; transition: all .15s;
font-family: var(--font); color: var(--text);
font-family: var(--font); color: var(--text); height: 34px; box-sizing: border-box; line-height: 1;
}
.filter-group { display: flex; gap: 6px; align-items: center; }
.filter-group + .filter-group { border-left: 1px solid var(--border); padding-left: 12px; margin-left: 6px; }
.sort-help { cursor: help; font-size: 14px; color: var(--text-muted, #888); position: relative; display: inline-block; }
.sort-help-tip {
display: none; position: absolute; top: 130%; left: 50%; transform: translateX(-50%);
background: var(--card-bg, #222); color: var(--text, #eee); border: 1px solid var(--border);
border-radius: 6px; padding: 8px 12px; font-size: 12px; line-height: 1.5;
white-space: pre-line; width: 260px; z-index: 100;
box-shadow: 0 4px 12px rgba(0,0,0,.3); pointer-events: none;
}
.sort-help:hover .sort-help-tip { display: block; }
.filter-bar .btn:hover { background: var(--row-hover); }
.filter-bar .btn.active { background: var(--accent); color: #fff; border-color: var(--accent); }
.btn-icon {
background: none; border: 1px solid var(--border); border-radius: 6px;
padding: 6px 10px; cursor: pointer; font-size: 14px; transition: all .15s;
color: var(--text); padding: 6px 10px; cursor: pointer; font-size: 14px; transition: all .15s;
}
.btn-icon:hover { background: var(--row-hover); }
@@ -263,6 +275,11 @@ a:focus-visible, button:focus-visible, input:focus-visible, select:focus-visible
font-size: 10px; font-weight: 700; font-family: var(--mono);
background: var(--nav-bg); color: #fff; letter-spacing: .5px;
}
.badge-obs {
display: inline-block; padding: 1px 6px; border-radius: 10px;
font-size: 10px; font-weight: 600;
background: #ede9fe; color: #6d28d9;
}
/* === Monospace === */
.mono { font-family: var(--mono); font-size: 12px; }
@@ -706,6 +723,7 @@ button.ch-item.selected { background: var(--selected-bg); }
[data-theme="dark"] .trace-search input,
[data-theme="dark"] .mc-jump-btn,
[data-theme="dark"] .filter-bar .btn { background: var(--input-bg); color: var(--text); border-color: var(--border); }
[data-theme="dark"] .filter-bar .btn.active { background: var(--accent); color: #fff; border-color: var(--accent); }
[data-theme="dark"] .ch-item.selected,
[data-theme="dark"] .data-table tbody tr.selected { background: var(--selected-bg); }
[data-theme="dark"] .tl-bar-container { background: #334155; }
@@ -847,6 +865,8 @@ button.ch-item.selected { background: var(--selected-bg); }
.filter-bar.filters-expanded > .col-toggle-wrap { display: inline-block; }
.filter-bar.filters-expanded input { width: 100%; }
.filter-bar.filters-expanded select { width: 100%; }
.filter-group { flex-wrap: wrap; }
.filter-group + .filter-group { border-left: none; padding-left: 0; margin-left: 0; }
.filter-bar .btn { min-height: 36px; }
.node-filter-wrap { width: 100%; }
@@ -914,7 +934,7 @@ button.ch-item.selected { background: var(--selected-bg); }
border: 1px solid var(--border); border-radius: 6px; background: var(--surface-1);
color: var(--text);
}
.byop-input:focus { border-color: var(--accent); outline: none; }
.byop-input:focus { border-color: var(--accent); outline: 2px solid var(--accent); outline-offset: 1px; }
.byop-err { color: #ef4444; font-size: .85rem; }
.byop-decoded { margin-top: 8px; }
.byop-section { margin-bottom: 14px; }
@@ -1265,10 +1285,11 @@ tr[data-hops]:hover { background: rgba(59,130,246,0.1); }
/* #71 — Column visibility toggle */
.col-toggle-wrap { position: relative; display: inline-block; }
.col-toggle-btn { font-size: .8rem; padding: 4px 8px; cursor: pointer; background: var(--input-bg); border: 1px solid var(--border); border-radius: 4px; color: var(--text); }
.col-toggle-btn { font-size: 13px; padding: 6px 10px; cursor: pointer; background: var(--input-bg); border: 1px solid var(--border); border-radius: 6px; color: var(--text); height: 34px; box-sizing: border-box; line-height: 1; }
.col-toggle-menu { display: none; position: absolute; top: 100%; left: 0; z-index: 50; background: var(--card-bg); border: 1px solid var(--border); border-radius: 6px; padding: 6px 0; min-width: 150px; box-shadow: 0 4px 12px rgba(0,0,0,.15); }
.col-toggle-menu.open { display: block; }
.col-toggle-menu label { display: flex; align-items: center; gap: 6px; padding: 4px 12px; font-size: .82rem; cursor: pointer; color: var(--text); }
.col-toggle-menu label input[type="checkbox"] { width: 14px; height: 14px; margin: 0; flex-shrink: 0; }
.col-toggle-menu label:hover { background: var(--row-hover); }
/* Column hide classes */
@@ -1442,3 +1463,70 @@ tr[data-hops]:hover { background: rgba(59,130,246,0.1); }
.perf-table .perf-slow td { color: #ef4444; }
.perf-table .perf-warn { background: rgba(251, 191, 36, 0.06); }
.perf-table .perf-warn td { color: #f59e0b; }
/* ─── Region filter bar ─── */
.region-filter-bar { display: flex; flex-wrap: wrap; gap: 6px; padding: 8px 0; }
.region-filter-container { margin: 0; padding: 0; display: inline-flex; align-items: center; }
.region-pill {
display: inline-flex; align-items: center; padding: 4px 12px; border-radius: 16px;
font-size: 12px; font-weight: 500; cursor: pointer; border: 1.5px solid var(--border);
background: transparent; color: var(--text-muted); transition: all 0.15s;
}
.region-pill:hover { border-color: var(--accent); color: var(--accent); }
.region-pill-active {
background: var(--accent); color: #fff; border-color: var(--accent);
}
.region-pill-active:hover { opacity: 0.85; }
.region-filter-label {
font-size: 12px; font-weight: 600; color: var(--text-muted); align-self: center;
margin-right: 2px; user-select: none;
}
.region-dropdown-wrap { position: relative; display: inline-flex; align-items: center; }
.region-dropdown-trigger {
display: inline-flex; align-items: center; padding: 6px 10px; border-radius: 6px;
font-size: 13px; font-weight: 500; cursor: pointer; border: 1px solid var(--border);
background: var(--input-bg); color: var(--text); transition: all 0.15s;
height: 34px; box-sizing: border-box; white-space: nowrap; line-height: 1;
}
.region-dropdown-trigger:hover { border-color: var(--accent); color: var(--accent); }
.region-dropdown-menu {
position: absolute; top: 100%; left: 0; z-index: 90;
min-width: 220px; max-height: 260px; overflow-y: auto;
background: var(--card-bg, #fff); border: 1px solid var(--border); border-radius: 8px;
box-shadow: 0 4px 16px rgba(0,0,0,0.12); padding: 4px 0;
}
.region-dropdown-item {
display: flex; align-items: center; gap: 6px; padding: 6px 12px;
font-size: 13px; cursor: pointer; color: var(--text); white-space: nowrap;
}
.region-dropdown-item input[type="checkbox"] {
width: 14px; height: 14px; margin: 0; flex-shrink: 0;
}
.region-dropdown-item:hover { background: var(--row-hover, #f5f5f5); }
/* Generic multi-select dropdown (Observer, Type filters) */
.multi-select-wrap { position: relative; display: inline-flex; align-items: center; }
.multi-select-trigger {
display: inline-flex; align-items: center; padding: 6px 10px; border-radius: 6px;
font-size: 13px; font-weight: 500; cursor: pointer; border: 1px solid var(--border);
background: var(--input-bg); color: var(--text); transition: all 0.15s;
height: 34px; box-sizing: border-box; white-space: nowrap; line-height: 1;
}
.multi-select-trigger:hover { border-color: var(--accent); color: var(--accent); }
.multi-select-menu {
position: absolute; top: 100%; left: 0; z-index: 90;
min-width: 220px; max-height: 260px; overflow-y: auto;
background: var(--card-bg, #fff); border: 1px solid var(--border); border-radius: 8px;
box-shadow: 0 4px 16px rgba(0,0,0,0.12); padding: 4px 0; display: none;
}
.multi-select-menu.open { display: block; }
.multi-select-item {
display: flex; align-items: center; gap: 6px; padding: 6px 12px;
font-size: 13px; cursor: pointer; color: var(--text); white-space: nowrap;
}
.multi-select-item input[type="checkbox"] {
width: 14px; height: 14px; margin: 0; flex-shrink: 0;
}
.multi-select-item:hover { background: var(--row-hover, #f5f5f5); }
.chan-tag { background: var(--accent, #3b82f6); color: #fff; padding: 2px 8px; border-radius: 4px; font-size: 0.9em; font-weight: 600; }

View File

@@ -5,11 +5,10 @@
let currentHash = null;
let traceData = [];
let packetMeta = null;
function init(app) {
// Check URL for pre-filled hash
function init(app, routeParam) {
// Check URL for pre-filled hash — support both route param and query param
const params = new URLSearchParams(location.hash.split('?')[1] || '');
const urlHash = params.get('hash') || '';
const urlHash = routeParam || params.get('hash') || '';
app.innerHTML = `
<div class="traces-page">
@@ -37,6 +36,16 @@
packetMeta = null;
}
function obsLabel(t) {
return t.observer_name || (t.observer && t.observer.length > 16 ? t.observer.slice(0, 12) + '…' : t.observer) || '—';
}
function obsLink(t) {
const label = escapeHtml(obsLabel(t));
if (!t.observer) return label;
return `<a href="#/observers/${encodeURIComponent(t.observer)}" style="color:var(--accent);text-decoration:none;" title="${escapeHtml(t.observer)}">${label}</a>`;
}
async function doTrace() {
const input = document.getElementById('traceHashInput');
const hash = input.value.trim();
@@ -60,14 +69,23 @@
return;
}
// Extract path from first packet that has it
let pathHops = [];
for (const p of packets) {
// Extract ALL unique paths from observations
const allPaths = [];
for (const t of traceData) {
try {
const hops = JSON.parse(p.path_json || '[]');
if (hops.length > 0) { pathHops = hops; break; }
const hops = JSON.parse(t.path_json || '[]');
if (hops.length > 0) allPaths.push({ hops, observer: obsLabel(t) });
} catch {}
}
// Fallback to packet-level path
if (allPaths.length === 0) {
for (const p of packets) {
try {
const hops = JSON.parse(p.path_json || '[]');
if (hops.length > 0) { allPaths.push({ hops, observer: 'packet' }); break; }
} catch {}
}
}
// Get packet type info from first packet
packetMeta = packets[0] || null;
@@ -76,13 +94,13 @@
try { decoded = JSON.parse(packetMeta.decoded_json); } catch {}
}
renderResults(results, pathHops, decoded);
renderResults(results, allPaths, decoded);
} catch (e) {
results.innerHTML = `<div class="trace-empty" style="color:#ef4444">Error: ${e.message}</div>`;
}
}
function renderResults(container, pathHops, decoded) {
function renderResults(container, allPaths, decoded) {
const uniqueObservers = [...new Set(traceData.map(t => t.observer))];
const typeName = packetMeta ? payloadTypeName(packetMeta.payload_type) : '—';
const typeClass = packetMeta ? payloadTypeColor(packetMeta.payload_type) : 'unknown';
@@ -118,31 +136,136 @@
</div>
</div>
${pathHops.length > 0 ? renderPathViz(pathHops) : ''}
${allPaths.length > 0 ? renderPathGraph(allPaths) : ''}
${traceData.length > 0 ? renderTimeline(t0, spreadMs) : ''}
${renderObserverTable()}
`;
makeColumnsResizable('#traceObsTable', 'meshcore-trace-col-widths');
}
function renderPathViz(hops) {
const arrows = hops.map(h => `<span class="trace-path-hop">${h}</span>`).join('<span class="trace-path-arrow">→</span>');
function renderPathGraph(allPaths) {
// Collect unique nodes and edges across all observed paths
const nodeSet = new Set();
const edgeMap = new Map(); // "from→to" => Set of observer labels
nodeSet.add('Origin');
nodeSet.add('Dest');
for (const { hops, observer } of allPaths) {
const chain = ['Origin', ...hops, 'Dest'];
for (let i = 0; i < chain.length - 1; i++) {
nodeSet.add(chain[i]);
nodeSet.add(chain[i + 1]);
const key = chain[i] + '→' + chain[i + 1];
if (!edgeMap.has(key)) edgeMap.set(key, new Set());
edgeMap.get(key).add(observer);
}
}
const nodes = [...nodeSet];
// Assign positions: lay out nodes left to right by their earliest appearance in any path
const order = new Map();
order.set('Origin', 0);
let maxCol = 0;
for (const { hops } of allPaths) {
const chain = ['Origin', ...hops, 'Dest'];
for (let i = 0; i < chain.length; i++) {
if (!order.has(chain[i])) {
order.set(chain[i], i);
}
maxCol = Math.max(maxCol, i);
}
}
order.set('Dest', maxCol);
// Group nodes by column for vertical stacking
const colGroups = new Map();
for (const [node, col] of order) {
if (!colGroups.has(col)) colGroups.set(col, []);
colGroups.get(col).push(node);
}
const colCount = maxCol + 1;
const svgW = Math.max(600, colCount * 140);
const maxRows = Math.max(...[...colGroups.values()].map(g => g.length));
const svgH = Math.max(120, maxRows * 60 + 40);
const colSpacing = svgW / (colCount + 1);
// Compute node positions
const nodePos = new Map();
for (const [col, group] of colGroups) {
const rowSpacing = svgH / (group.length + 1);
group.forEach((node, i) => {
nodePos.set(node, { x: (col + 1) * colSpacing, y: (i + 1) * rowSpacing });
});
}
// Colors for edges (cycle through)
const edgeColors = ['#3b82f6', '#10b981', '#f59e0b', '#ef4444', '#8b5cf6', '#ec4899', '#06b6d4', '#84cc16'];
const observerColorMap = new Map();
let colorIdx = 0;
for (const obsSet of edgeMap.values()) {
for (const obs of obsSet) {
if (!observerColorMap.has(obs)) {
observerColorMap.set(obs, edgeColors[colorIdx % edgeColors.length]);
colorIdx++;
}
}
}
// Build SVG
let edgesSvg = '';
for (const [key, observers] of edgeMap) {
const [from, to] = key.split('→');
const p1 = nodePos.get(from);
const p2 = nodePos.get(to);
if (!p1 || !p2) continue;
const obsArr = [...observers];
const thickness = Math.min(obsArr.length, 6);
// Use first observer's color, show count as tooltip
const color = observerColorMap.get(obsArr[0]) || '#6b7280';
const title = obsArr.length > 1 ? `${obsArr.length} observers: ${obsArr.join(', ')}` : obsArr[0];
edgesSvg += `<line x1="${p1.x}" y1="${p1.y}" x2="${p2.x}" y2="${p2.y}" stroke="${color}" stroke-width="${thickness}" stroke-opacity="0.6"><title>${escapeHtml(title)}</title></line>`;
// Arrowhead
const angle = Math.atan2(p2.y - p1.y, p2.x - p1.x);
const arrowLen = 8;
const ax = p2.x - 20 * Math.cos(angle);
const ay = p2.y - 20 * Math.sin(angle);
const a1x = ax - arrowLen * Math.cos(angle - 0.4);
const a1y = ay - arrowLen * Math.sin(angle - 0.4);
const a2x = ax - arrowLen * Math.cos(angle + 0.4);
const a2y = ay - arrowLen * Math.sin(angle + 0.4);
edgesSvg += `<polygon points="${ax},${ay} ${a1x},${a1y} ${a2x},${a2y}" fill="${color}" opacity="0.8"/>`;
}
let nodesSvg = '';
for (const [node, pos] of nodePos) {
const isEndpoint = node === 'Origin' || node === 'Dest';
const r = isEndpoint ? 18 : 14;
const fill = isEndpoint ? 'var(--primary, #3b82f6)' : 'var(--surface-2, #374151)';
const stroke = isEndpoint ? 'var(--primary, #3b82f6)' : 'var(--border, #4b5563)';
const label = isEndpoint ? node : node;
nodesSvg += `<circle cx="${pos.x}" cy="${pos.y}" r="${r}" fill="${fill}" stroke="${stroke}" stroke-width="2"/>`;
nodesSvg += `<text x="${pos.x}" y="${pos.y + 4}" text-anchor="middle" fill="white" font-size="${isEndpoint ? 10 : 9}" font-weight="${isEndpoint ? 700 : 500}">${escapeHtml(label)}</text>`;
}
// Legend: unique paths
const uniquePaths = [...new Set(allPaths.map(p => p.hops.join('→')))];
const legendHtml = uniquePaths.length > 1
? `<div class="trace-path-info" style="margin-top:8px">${uniquePaths.length} unique path${uniquePaths.length > 1 ? 's' : ''} observed by ${allPaths.length} observer${allPaths.length > 1 ? 's' : ''}</div>`
: `<div class="trace-path-info">${allPaths[0].hops.length} hop${allPaths[0].hops.length !== 1 ? 's' : ''} in relay path</div>`;
return `
<div class="trace-section">
<h3>Path Visualization</h3>
<div class="trace-path-viz">
<span class="trace-path-label">Origin</span>
<span class="trace-path-arrow">→</span>
${arrows}
<span class="trace-path-arrow">→</span>
<span class="trace-path-label">Dest</span>
<h3>Path Graph</h3>
<div style="overflow-x:auto;">
<svg width="${svgW}" height="${svgH}" style="display:block;margin:0 auto;">
${edgesSvg}
${nodesSvg}
</svg>
</div>
<div class="trace-path-info">${hops.length} hop${hops.length !== 1 ? 's' : ''} in relay path</div>
${legendHtml}
</div>`;
}
function renderTimeline(t0, spreadMs) {
// Build timeline bars
const barWidth = spreadMs > 0 ? spreadMs : 1;
const rows = traceData.map((t, i) => {
const time = new Date(t.time);
@@ -152,7 +275,7 @@
const delta = spreadMs > 0 ? `+${(offsetMs / 1000).toFixed(3)}s` : '';
return `<div class="tl-row">
<div class="tl-observer">${truncate(t.observer || '—', 20)}</div>
<div class="tl-observer">${obsLink(t)}</div>
<div class="tl-bar-container">
<div class="tl-marker" style="left:${pct}%" title="${time.toISOString()}"></div>
</div>
@@ -172,27 +295,5 @@
</div>`;
}
function renderObserverTable() {
const rows = traceData.map((t, i) => {
const snrClass = t.snr != null ? (t.snr >= 0 ? 'good' : t.snr >= -10 ? 'ok' : 'bad') : '';
return `<tr>
<td>${i + 1}</td>
<td class="mono">${t.observer || '—'}</td>
<td>${t.time ? new Date(t.time).toLocaleString() : '—'}</td>
<td class="tl-snr ${snrClass}">${t.snr != null ? t.snr.toFixed(1) + ' dB' : '—'}</td>
<td>${t.rssi != null ? t.rssi.toFixed(0) + ' dBm' : '—'}</td>
</tr>`;
});
return `
<div class="trace-section">
<h3>Observer Details</h3>
<table class="data-table" id="traceObsTable">
<thead><tr><th>#</th><th>Observer</th><th>Timestamp</th><th>SNR</th><th>RSSI</th></tr></thead>
<tbody>${rows.join('')}</tbody>
</table>
</div>`;
}
registerPage('traces', { init, destroy });
})();

144
scripts/migrate-dedup.js Normal file
View File

@@ -0,0 +1,144 @@
#!/usr/bin/env node
/**
* Milestone 1: Packet Dedup Schema Migration
*
* Creates `transmissions` and `observations` tables from the existing `packets` table.
* Idempotent — drops and recreates new tables on each run.
* Does NOT touch the original `packets` table.
*
* Usage: node scripts/migrate-dedup.js <path-to-meshcore.db>
*/
const Database = require('better-sqlite3');
const path = require('path');
const dbPath = process.argv[2];
if (!dbPath) {
console.error('Usage: node scripts/migrate-dedup.js <path-to-meshcore.db>');
process.exit(1);
}
const start = Date.now();
const db = new Database(dbPath);
db.pragma('journal_mode = WAL');
db.pragma('foreign_keys = ON');
// --- Drop existing new tables (idempotent) ---
console.log('Dropping existing transmissions/observations tables if they exist...');
db.exec('DROP TABLE IF EXISTS observations');
db.exec('DROP TABLE IF EXISTS transmissions');
// --- Create new tables ---
console.log('Creating transmissions and observations tables...');
db.exec(`
CREATE TABLE transmissions (
id INTEGER PRIMARY KEY AUTOINCREMENT,
raw_hex TEXT NOT NULL,
hash TEXT NOT NULL UNIQUE,
first_seen TEXT NOT NULL,
route_type INTEGER,
payload_type INTEGER,
payload_version INTEGER,
decoded_json TEXT,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE TABLE observations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
transmission_id INTEGER NOT NULL REFERENCES transmissions(id),
hash TEXT NOT NULL,
observer_id TEXT,
observer_name TEXT,
direction TEXT,
snr REAL,
rssi REAL,
score INTEGER,
path_json TEXT,
timestamp TEXT NOT NULL,
created_at TEXT DEFAULT (datetime('now'))
);
CREATE INDEX idx_transmissions_hash ON transmissions(hash);
CREATE INDEX idx_transmissions_first_seen ON transmissions(first_seen);
CREATE INDEX idx_transmissions_payload_type ON transmissions(payload_type);
CREATE INDEX idx_observations_hash ON observations(hash);
CREATE INDEX idx_observations_transmission_id ON observations(transmission_id);
CREATE INDEX idx_observations_observer_id ON observations(observer_id);
CREATE INDEX idx_observations_timestamp ON observations(timestamp);
`);
// --- Read all packets ordered by timestamp ---
console.log('Reading packets...');
const packets = db.prepare('SELECT * FROM packets ORDER BY timestamp ASC').all();
const totalPackets = packets.length;
console.log(`Total packets: ${totalPackets}`);
// --- Group by hash and migrate ---
const insertTransmission = db.prepare(`
INSERT OR IGNORE INTO transmissions (raw_hex, hash, first_seen, route_type, payload_type, payload_version, decoded_json)
VALUES (?, ?, ?, ?, ?, ?, ?)
`);
const insertObservation = db.prepare(`
INSERT INTO observations (transmission_id, hash, observer_id, observer_name, direction, snr, rssi, score, path_json, timestamp)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
`);
const hashToTransmissionId = new Map();
let transmissionCount = 0;
const lookupTransmission = db.prepare('SELECT id FROM transmissions WHERE hash = ?');
const migrate = db.transaction(() => {
for (const pkt of packets) {
let txId = hashToTransmissionId.get(pkt.hash);
if (txId === undefined) {
const result = insertTransmission.run(
pkt.raw_hex, pkt.hash, pkt.timestamp,
pkt.route_type, pkt.payload_type, pkt.payload_version, pkt.decoded_json
);
if (result.changes > 0) {
txId = result.lastInsertRowid;
} else {
// Already inserted by dual-write, look up existing
txId = lookupTransmission.get(pkt.hash).id;
}
hashToTransmissionId.set(pkt.hash, txId);
transmissionCount++;
}
insertObservation.run(
txId, pkt.hash, pkt.observer_id, pkt.observer_name, pkt.direction,
pkt.snr, pkt.rssi, pkt.score, pkt.path_json, pkt.timestamp
);
}
});
migrate();
// --- Verify ---
const obsCount = db.prepare('SELECT COUNT(*) as c FROM observations').get().c;
const txCount = db.prepare('SELECT COUNT(*) as c FROM transmissions').get().c;
const distinctHash = db.prepare('SELECT COUNT(DISTINCT hash) as c FROM packets').get().c;
const elapsed = ((Date.now() - start) / 1000).toFixed(2);
console.log('\n=== Migration Stats ===');
console.log(`Total packets (source): ${totalPackets}`);
console.log(`Unique transmissions created: ${transmissionCount}`);
console.log(`Observations created: ${obsCount}`);
console.log(`Dedup ratio: ${(totalPackets / transmissionCount).toFixed(2)}x`);
console.log(`Time taken: ${elapsed}s`);
console.log('\n=== Verification ===');
const obsOk = obsCount === totalPackets;
const txOk = txCount === distinctHash;
console.log(`observations (${obsCount}) = packets (${totalPackets}): ${obsOk ? 'PASS ✓' : 'FAIL ✗'}`);
console.log(`transmissions (${txCount}) = distinct hashes (${distinctHash}): ${txOk ? 'PASS ✓' : 'FAIL ✗'}`);
if (!obsOk || !txOk) {
console.error('\nVerification FAILED!');
process.exit(1);
}
console.log('\nMigration complete!');
db.close();

30
scripts/validate.sh Executable file
View File

@@ -0,0 +1,30 @@
#!/bin/sh
# Pre-push validation — catches common JS errors before they hit prod
set -e
echo "=== Syntax check ==="
node -c server.js
for f in public/*.js; do node -c "$f"; done
echo "✅ All JS files parse OK"
echo "=== Checking for undefined common references ==="
ERRORS=0
# esc() should only exist inside IIFEs that define it, not in files that don't
for f in public/live.js public/map.js public/home.js public/nodes.js public/channels.js public/observers.js; do
if grep -q '\besc(' "$f" 2>/dev/null && ! grep -q 'function esc' "$f" 2>/dev/null; then
REFS=$(grep -n '\besc(' "$f" | grep -v escapeHtml | grep -v "desc\|Esc\|resc\|safeEsc" || true)
if [ -n "$REFS" ]; then
echo "$f uses esc() but doesn't define it:"
echo "$REFS"
ERRORS=$((ERRORS + 1))
fi
fi
done
if [ "$ERRORS" -gt 0 ]; then
echo "$ERRORS validation error(s) found"
exit 1
fi
echo "✅ Validation passed"

1211
server.js

File diff suppressed because it is too large Load Diff